MetaPartitionManager (Apache Giraph Core 1.3.0-SNAPSHOT API)

java.lang.Object
- org.apache.giraph.ooc.data.MetaPartitionManager

```
public class MetaPartitionManager
extends Object
```
Class to keep meta-information about partition data, edge data, and message data of each partition on a worker.

Field Summary

Fields
Modifier and Type Field and Description

static int NO_PARTITION_TO_PROCESS
Flag representing no partitions is left to process in the current iteration cycle over all partitions.

Fields
Modifier and Type	Field and Description
`static int`	`NO_PARTITION_TO_PROCESS` Flag representing no partitions is left to process in the current iteration cycle over all partitions.

Constructor Summary

Constructors
Constructor and Description

MetaPartitionManager(int numIOThreads, OutOfCoreEngine oocEngine)
Constructor

Constructors
Constructor and Description
`MetaPartitionManager(int numIOThreads, OutOfCoreEngine oocEngine)` Constructor

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addPartition(int partitionId)` Add a partition
`void`	`doneLoadingPartition(int partitionId, long superstep)` Notify this meta store that load of a partition for a specific superstep is completed
`void`	`doneOffloadingBuffer(int partitionId)` Notify this meta store that offload of raw data buffers (vertex/edges/ messages) of a particular partition is completed.
`void`	`doneOffloadingMessages(int partitionId)` Notify this meta store that offload of messages for a particular partition is complete.
`void`	`doneOffloadingPartition(int partitionId)` Notify this meta store that offload of a partition (partition data and its current messages) is completed.
`double`	`getGraphFractionInMemory()` Since the statistics are based on estimates, we assume each partial partition is taking about half of the full partition in terms of memory footprint.
`Integer`	`getLoadPartitionId(int threadId)` Get id of a partition to load its data to memory.
`Integer`	`getNextPartition()` Return the id of an unprocessed partition in memory.
`int`	`getNumInMemoryPartitions()`
`int`	`getNumPartiallyInMemoryPartitions()`
`int`	`getNumPartitions()` Get total number of partitions
`Integer`	`getOffloadMessageBufferId(int threadId)` Get id of a partition to offload its incoming message buffers on disk
`Integer`	`getOffloadMessageId(int threadId)` Get id of a partition to offload its incoming message on disk.
`Integer`	`getOffloadPartitionBufferId(int threadId)` Get id of a partition to offload its vertex/edge buffers on disk
`Integer`	`getOffloadPartitionId(int threadId)` Get id of a partition to offload to disk.
`int`	`getOwnerThreadId(int partitionId)` Get the thread id that is responsible for a particular partition
`Iterable<Integer>`	`getPartitionIds()` Return the list of all available partitions as an iterable
`boolean`	`hasPartition(Integer partitionId)` Whether a given partition is available
`boolean`	`hasProcessedOnMemory()` Whether there is any processed partition stored in memory (excluding those that are prefetched to execute in the next superstep).
`boolean`	`isPartitionOnDisk(int partitionId)` Whether a partition is on disk (both its data and its current messages)
`boolean`	`isPartitionProcessed(Integer partitionId)` Whether a partition is processed in the current iteration cycle over partitions.
`void`	`markPartitionAsInProcess(int partitionId)` Mark a partition as being 'IN_PROCESS'
`void`	`removePartition(Integer partitionId)` Remove a partition.
`void`	`resetMessages()` Reset messages in the meta store.
`void`	`resetPartitions()` Reset the meta store for a new iteration cycle over all partitions.
`void`	`setPartitionIsProcessed(int partitionId)` Mark a partition as 'PROCESSED'
`boolean`	`startLoadingPartition(int partitionId, long superstep)` Notify this meta store that load of a partition for a specific superstep is about to start.
`boolean`	`startOffloadingBuffer(int partitionId)` Notify this meta store that offload of raw data buffers (vertex/edges/ messages) of a particular partition is about to start.
`boolean`	`startOffloadingMessages(int partitionId)` Notify this meta store that offload of messages for a particular partition is about to start.
`boolean`	`startOffloadingPartition(int partitionId)` Notify this meta store that offload of a partition (partition data and its current messages) is about to start.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - NO_PARTITION_TO_PROCESS
```
public static final int NO_PARTITION_TO_PROCESS
```
    Flag representing no partitions is left to process in the current iteration cycle over all partitions.
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - MetaPartitionManager
```
public MetaPartitionManager(int numIOThreads,
                            OutOfCoreEngine oocEngine)
```
    Constructor
    
    Parameters:
    
    numIOThreads - number of IO threads
    
    oocEngine - out-of-core engine
- Method Detail
  - getNumInMemoryPartitions
```
public int getNumInMemoryPartitions()
```
    Returns:
    
    number of partitions in memory
  - getNumPartiallyInMemoryPartitions
```
public int getNumPartiallyInMemoryPartitions()
```
    Returns:
    
    number of partitions that are partially in memory
  - getNumPartitions
```
public int getNumPartitions()
```
    Get total number of partitions
    
    Returns:
    
    total number of partitions
  - getGraphFractionInMemory
```
public double getGraphFractionInMemory()
```
    Since the statistics are based on estimates, we assume each partial partition is taking about half of the full partition in terms of memory footprint.
    
    Returns:
    
    estimate of fraction of graph in memory
  - hasPartition
```
public boolean hasPartition(Integer partitionId)
```
    Whether a given partition is available
    
    Parameters:
    
    partitionId - id of the partition to check if this worker owns it
    
    Returns:
    
    true if the worker owns the partition, false otherwise
  - getPartitionIds
```
public Iterable<Integer> getPartitionIds()
```
    Return the list of all available partitions as an iterable
    
    Returns:
    
    list of all available partitions
  - getOwnerThreadId
```
public int getOwnerThreadId(int partitionId)
```
    Get the thread id that is responsible for a particular partition
    
    Parameters:
    
    partitionId - id of the given partition
    
    Returns:
    
    id of the thread responsible for the given partition
  - addPartition
```
public void addPartition(int partitionId)
```
    Add a partition
    
    Parameters:
    
    partitionId - id of a partition to add
  - removePartition
```
public void removePartition(Integer partitionId)
```
    Remove a partition. This method assumes that the partition is already retrieved and is in memory)
    
    Parameters:
    
    partitionId - id of a partition to remove
  - getOffloadPartitionId
```
public Integer getOffloadPartitionId(int threadId)
```
    Get id of a partition to offload to disk. Prioritize offloading processed partitions over unprocessed partition. Also, prioritize offloading partitions partially in memory over partitions fully in memory.
    
    Parameters:
    
    threadId - id of the thread who is going to store the partition on disk
    
    Returns:
    
    id of the partition to offload on disk
  - getOffloadPartitionBufferId
```
public Integer getOffloadPartitionBufferId(int threadId)
```
    Get id of a partition to offload its vertex/edge buffers on disk
    
    Parameters:
    
    threadId - id of the thread who is going to store the buffers on disk
    
    Returns:
    
    id of the partition to offload its vertex/edge buffers on disk
  - getOffloadMessageBufferId
```
public Integer getOffloadMessageBufferId(int threadId)
```
    Get id of a partition to offload its incoming message buffers on disk
    
    Parameters:
    
    threadId - id of the thread who is going to store the buffers on disk
    
    Returns:
    
    id of the partition to offload its message buffer on disk
  - getOffloadMessageId
```
public Integer getOffloadMessageId(int threadId)
```
    Get id of a partition to offload its incoming message on disk. Prioritize offloading messages of partitions already on disk, and then partitions in-transit, over partitions in-memory. Also, prioritize processed partitions over unprocessed (processed partitions would go on disk with more chances that unprocessed partitions)
    
    Parameters:
    
    threadId - id of the thread who is going to store the incoming messages on disk
    
    Returns:
    
    id of the partition to offload its message on disk
  - getLoadPartitionId
```
public Integer getLoadPartitionId(int threadId)
```
    Get id of a partition to load its data to memory. Prioritize loading an unprocessed partition over loading processed partition. Also, prioritize loading a partition partially in memory over partitions entirely on disk.
    
    Parameters:
    
    threadId - id of the thread who is going to load the partition data
    
    Returns:
    
    id of the partition to load its data to memory
  - markPartitionAsInProcess
```
public void markPartitionAsInProcess(int partitionId)
```
    Mark a partition as being 'IN_PROCESS'
    
    Parameters:
    
    partitionId - id of the partition to mark
  - hasProcessedOnMemory
```
public boolean hasProcessedOnMemory()
```
    Whether there is any processed partition stored in memory (excluding those that are prefetched to execute in the next superstep).
    
    Returns:
    
    true iff there is any processed partition in memory
  - isPartitionProcessed
```
public boolean isPartitionProcessed(Integer partitionId)
```
    Whether a partition is *processed* in the current iteration cycle over partitions.
    
    Parameters:
    
    partitionId - id of the partition to check
    
    Returns:
    
    true iff processing the given partition is done
  - setPartitionIsProcessed
```
public void setPartitionIsProcessed(int partitionId)
```
    Mark a partition as 'PROCESSED'
    
    Parameters:
    
    partitionId - id of the partition to mark
  - startLoadingPartition
```
public boolean startLoadingPartition(int partitionId,
                                     long superstep)
```
    Notify this meta store that load of a partition for a specific superstep is about to start.
    
    Parameters:
    
    partitionId - id of the partition to load to memory
    
    superstep - superstep in which the partition is needed for
    
    Returns:
    
    true iff load of the given partition is viable
  - doneLoadingPartition
```
public void doneLoadingPartition(int partitionId,
                                 long superstep)
```
    Notify this meta store that load of a partition for a specific superstep is completed
    
    Parameters:
    
    partitionId - id of the partition for which the load is completed
    
    superstep - superstep in which the partition is loaded for
  - startOffloadingMessages
```
public boolean startOffloadingMessages(int partitionId)
```
    Notify this meta store that offload of messages for a particular partition is about to start.
    
    Parameters:
    
    partitionId - id of the partition that its messages is being offloaded
    
    Returns:
    
    true iff offload of messages of the given partition is viable
  - doneOffloadingMessages
```
public void doneOffloadingMessages(int partitionId)
```
    Notify this meta store that offload of messages for a particular partition is complete.
    
    Parameters:
    
    partitionId - id of the partition that its messages is offloaded to disk
  - startOffloadingBuffer
```
public boolean startOffloadingBuffer(int partitionId)
```
    Notify this meta store that offload of raw data buffers (vertex/edges/ messages) of a particular partition is about to start.
    
    Parameters:
    
    partitionId - id of the partition that its buffer is being offloaded
    
    Returns:
    
    true iff offload of buffers of the given partition is viable
  - doneOffloadingBuffer
```
public void doneOffloadingBuffer(int partitionId)
```
    Notify this meta store that offload of raw data buffers (vertex/edges/ messages) of a particular partition is completed.
    
    Parameters:
    
    partitionId - id of the partition that its buffer is offloaded
  - startOffloadingPartition
```
public boolean startOffloadingPartition(int partitionId)
```
    Notify this meta store that offload of a partition (partition data and its current messages) is about to start.
    
    Parameters:
    
    partitionId - id of the partition that its data is being offloaded
    
    Returns:
    
    true iff offload of the given partition is viable
  - doneOffloadingPartition
```
public void doneOffloadingPartition(int partitionId)
```
    Notify this meta store that offload of a partition (partition data and its current messages) is completed.
    
    Parameters:
    
    partitionId - id of the partition that its data is offloaded
  - resetPartitions
```
public void resetPartitions()
```
    Reset the meta store for a new iteration cycle over all partitions. Note: this is not thread-safe and should be called from a single thread.
  - resetMessages
```
public void resetMessages()
```
    Reset messages in the meta store. Note: this is not thread-safe and should be called from a single thread.
  - getNextPartition
```
public Integer getNextPartition()
```
    Return the id of an unprocessed partition in memory. If all partitions are processed, return an appropriate 'finisher signal'. If there are unprocessed partitions, but none are in memory, return null.
    
    Returns:
    
    id of the partition to be processed next.
  - isPartitionOnDisk
```
public boolean isPartitionOnDisk(int partitionId)
```
    Whether a partition is on disk (both its data and its current messages)
    
    Parameters:
    
    partitionId - id of the partition to check if it is on disk
    
    Returns:
    
    true if partition data or its current messages are on disk, false otherwise

Class MetaPartitionManager

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

NO_PARTITION_TO_PROCESS

Constructor Detail

MetaPartitionManager

Method Detail

getNumInMemoryPartitions

getNumPartiallyInMemoryPartitions

getNumPartitions

getGraphFractionInMemory

hasPartition

getPartitionIds

getOwnerThreadId

addPartition

removePartition

getOffloadPartitionId

getOffloadPartitionBufferId

getOffloadMessageBufferId

getOffloadMessageId

getLoadPartitionId

markPartitionAsInProcess

hasProcessedOnMemory

isPartitionProcessed

setPartitionIsProcessed

startLoadingPartition

doneLoadingPartition

startOffloadingMessages

doneOffloadingMessages

startOffloadingBuffer

doneOffloadingBuffer

startOffloadingPartition

doneOffloadingPartition

resetPartitions

resetMessages

getNextPartition

isPartitionOnDisk