AbstractEdgeStore (Apache Giraph Parent 1.4.0-SNAPSHOT API)

java.lang.Object
- org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>
- - org.apache.giraph.edge.AbstractEdgeStore<I,V,E,K,Et>

Type Parameters:

I - Vertex id

V - Vertex value

E - Edge value

K - Key corresponding to Vertex id

Et - Entry type

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>, EdgeStore<I,V,E>

Direct Known Subclasses:

IntEdgeStore, LongEdgeStore, SimpleEdgeStore
```
public abstract class AbstractEdgeStore
extends DefaultImmutableClassesGiraphConfigurable<I,V,E>
implements EdgeStore<I,V,E>
```
Basic implementation of edges store, extended this to easily define simple and primitive edge stores

Field Summary

Fields
Modifier and Type	Field and Description
`protected ImmutableClassesGiraphConfiguration<I,V,E>`	`configuration` Giraph configuration.
`static ThreadLocalProgressCounter`	`PROGRESS_COUNTER` Used to keep track of progress during the move-edges process
`protected org.apache.hadoop.util.Progressable`	`progressable` Progressable to report progress.
`protected boolean`	`reuseEdgeObjects` Whether the chosen `OutEdges` implementation allows for Edge reuse.
`protected CentralizedServiceWorker<I,V,E>`	`service` Service worker.
`protected ConcurrentMap<Integer,Map<K,OutEdges<I,E>>>`	`transientEdges` Map used to temporarily store incoming edges.
`protected boolean`	`useInputOutEdges` Whether the `OutEdges` class used during input is different from the one used during computation.

Constructor Summary

Constructors
Constructor and Description
`AbstractEdgeStore(CentralizedServiceWorker<I,V,E> service, ImmutableClassesGiraphConfiguration<I,V,E> configuration, org.apache.hadoop.util.Progressable progressable)` Constructor.

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addPartitionEdges(int partitionId, VertexIdEdges<I,E> edges)` Add edges belonging to a given partition on this worker.
`protected abstract I`	`createVertexId(Et entry)` Create vertexId from a given key
`protected abstract OutEdges<I,E>`	`getPartitionEdges(Et entry)` Return the OutEdges for a given partition
`protected abstract Map<K,OutEdges<I,E>>`	`getPartitionEdges(int partitionId)` Get OutEdges for a given partition
`protected abstract Iterator<Et>`	`getPartitionEdgesIterator(Map<K,OutEdges<I,E>> partitionEdges)` Get iterator for partition edges
`protected abstract I`	`getVertexId(Et entry, I representativeVertexId)` Get vertexId for a given key
`protected abstract OutEdges<I,E>`	`getVertexOutEdges(VertexIdEdgeIterator<I,E> vertexIdEdgeIterator, Map<K,OutEdges<I,E>> partitionEdgesIn)` Get out-edges for a given vertex
`boolean`	`hasEdgesForPartition(int partitionId)` Check if edge store has edge for a given partition
`void`	`moveEdgesToVertices()` Move all edges from temporary storage to their source vertices.
`void`	`readPartitionEdgeStore(int partitionId, DataInput input)` Serialize the edges of a given partition, and adds it to the partition store (assumes that the edge store does not have any edge from the partition already).
`protected abstract K`	`readVertexKey(DataInput input)` Reads the given key from the input
`void`	`writePartitionEdgeStore(int partitionId, DataOutput output)` Deserialize the edges of a given partition, and removes the associated data from the store.
`protected abstract void`	`writeVertexKey(K key, DataOutput output)` Writes the given key to the output

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable
getConf, setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
 - PROGRESS_COUNTER
```
public static final ThreadLocalProgressCounter PROGRESS_COUNTER
```
 Used to keep track of progress during the move-edges process
 - service
```
protected CentralizedServiceWorker service
```
 Service worker.
 - configuration
```
protected ImmutableClassesGiraphConfiguration configuration
```
 Giraph configuration.
 - progressable
```
protected org.apache.hadoop.util.Progressable progressable
```
 Progressable to report progress.
 - transientEdges
```
protected ConcurrentMap<Integer,Map<K,OutEdges>> transientEdges
```
 Map used to temporarily store incoming edges.
 - reuseEdgeObjects
```
protected boolean reuseEdgeObjects
```
 Whether the chosen OutEdges implementation allows for Edge reuse.
 - useInputOutEdges
```
protected boolean useInputOutEdges
```
 Whether the OutEdges class used during input is different from the one used during computation.
- Constructor Detail
 - AbstractEdgeStore
```
public AbstractEdgeStore(CentralizedServiceWorker<I,V,E> service,
 ImmutableClassesGiraphConfiguration<I,V,E> configuration,
 org.apache.hadoop.util.Progressable progressable)
```
 Constructor.
 
 Parameters:
 
 service - Service worker
 
 configuration - Configuration
 
 progressable - Progressable
- Method Detail
 - getVertexId
```
protected abstract I getVertexId(Et entry,
 I representativeVertexId)
```
 Get vertexId for a given key
 
 Parameters:
 
 entry - for vertexId key
 
 representativeVertexId - representativeVertexId
 
 Returns:
 
 vertex Id
 - createVertexId
```
protected abstract I createVertexId(Et entry)
```
 Create vertexId from a given key
 
 Parameters:
 
 entry - for vertexId key
 
 Returns:
 
 new vertexId
 - getPartitionEdges
```
protected abstract Map<K,OutEdges<I,E>> getPartitionEdges(int partitionId)
```
 Get OutEdges for a given partition
 
 Parameters:
 
 partitionId - id of partition
 
 Returns:
 
 OutEdges for the partition
 - getPartitionEdges
```
protected abstract OutEdges<I,E> getPartitionEdges(Et entry)
```
 Return the OutEdges for a given partition
 
 Parameters:
 
 entry - for vertexId key
 
 Returns:
 
 out edges
 - writeVertexKey
```
protected abstract void writeVertexKey(K key,
 DataOutput output)
 throws IOException
```
 Writes the given key to the output
 
 Parameters:
 
 key - input key to be written
 
 output - output to write the key to
 
 Throws:
 
 IOException
 - readVertexKey
```
protected abstract K readVertexKey(DataInput input)
 throws IOException
```
 Reads the given key from the input
 
 Parameters:
 
 input - input to read the key from
 
 Returns:
 
 Key read from the input
 
 Throws:
 
 IOException
 - getPartitionEdgesIterator
```
protected abstract Iterator<Et> getPartitionEdgesIterator(Map<K,OutEdges<I,E>> partitionEdges)
```
 Get iterator for partition edges
 
 Parameters:
 
 partitionEdges - map of out-edges for vertices in a partition
 
 Returns:
 
 iterator
 - hasEdgesForPartition
```
public boolean hasEdgesForPartition(int partitionId)
```
 Description copied from interface: EdgeStore
 
 Check if edge store has edge for a given partition
 
 Specified by:
 
 hasEdgesForPartition in interface EdgeStore
 
 Parameters:
 
 partitionId - Id of partition
 
 Returns:
 
 True iff edge store have messages for the given partition
 - writePartitionEdgeStore
```
public void writePartitionEdgeStore(int partitionId,
 DataOutput output)
 throws IOException
```
 Description copied from interface: EdgeStore
 
 Deserialize the edges of a given partition, and removes the associated data from the store. Note: This method is not thread-safe (i.e. should not be called for the same partition at the same time).
 
 Specified by:
 
 writePartitionEdgeStore in interface EdgeStore
 
 Parameters:
 
 partitionId - Id of partition to deserialize
 
 output - Output to write the edge store to
 
 Throws:
 
 IOException
 - readPartitionEdgeStore
```
public void readPartitionEdgeStore(int partitionId,
 DataInput input)
 throws IOException
```
 Description copied from interface: EdgeStore
 
 Serialize the edges of a given partition, and adds it to the partition store (assumes that the edge store does not have any edge from the partition already). Note: This method is not thread-safe (i.e. should not be called for the same partition at the same time).
 
 Specified by:
 
 readPartitionEdgeStore in interface EdgeStore
 
 Parameters:
 
 partitionId - Id of partition to serialize
 
 input - Input to read the partition from
 
 Throws:
 
 IOException
 - getVertexOutEdges
```
protected abstract OutEdges<I,E> getVertexOutEdges(VertexIdEdgeIterator<I,E> vertexIdEdgeIterator,
 Map<K,OutEdges<I,E>> partitionEdgesIn)
```
 Get out-edges for a given vertex
 
 Parameters:
 
 vertexIdEdgeIterator - vertex Id Edge iterator
 
 partitionEdgesIn - map of out-edges for vertices in a partition
 
 Returns:
 
 out-edges for the vertex
 - addPartitionEdges
```
public void addPartitionEdges(int partitionId,
 VertexIdEdges<I,E> edges)
```
 Description copied from interface: EdgeStore
 
 Add edges belonging to a given partition on this worker. Note: This method is thread-safe.
 
 Specified by:
 
 addPartitionEdges in interface EdgeStore
 
 Parameters:
 
 partitionId - Partition id for the incoming edges.
 
 edges - Incoming edges
 - moveEdgesToVertices
```
public void moveEdgesToVertices()
```
 Description copied from interface: EdgeStore
 
 Move all edges from temporary storage to their source vertices. Note: this method is not thread-safe and is called once all vertices and edges are read in INPUT_SUPERSTEP.
 
 Specified by:
 
 moveEdgesToVertices in interface EdgeStore

Class AbstractEdgeStore<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,K,Et>

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

Methods inherited from class java.lang.Object

Field Detail

PROGRESS_COUNTER

service

configuration

progressable

transientEdges

reuseEdgeObjects

useInputOutEdges

Constructor Detail

AbstractEdgeStore

Method Detail

getVertexId

createVertexId

getPartitionEdges

getPartitionEdges

writeVertexKey

readVertexKey

getPartitionEdgesIterator

hasEdgesForPartition

writePartitionEdgeStore

readPartitionEdgeStore

getVertexOutEdges

addPartitionEdges

moveEdgesToVertices