org.apache.giraph.partition

Class WorkerGraphPartitionerImpl<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.partition.WorkerGraphPartitionerImpl<I,V,E>

Type Parameters:

I - Vertex id type

V - Vertex value type

E - Edge value type

All Implemented Interfaces:

WorkerGraphPartitioner<I,V,E>

public abstract class WorkerGraphPartitionerImpl<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends Object
implements WorkerGraphPartitioner<I,V,E>

Abstracts and implements all WorkerGraphPartitioner logic on top of a single user function - getPartitionIndex.

Constructor Summary

Constructors

Constructor and Description
WorkerGraphPartitionerImpl()

Method Summary

Modifier and Type	Method and Description
PartitionOwner	createPartitionOwner() Instantiate the PartitionOwner implementation used to read the master assignments.
void	extractAvailableWorkers() Update availableWorkers
Collection<PartitionStats>	finalizePartitionStats(Collection<PartitionStats> workerPartitionStats, PartitionStore<I,V,E> partitionStore) At the end of a superstep, workers have PartitionStats generated for each of their partitions.
protected abstract int	getPartitionIndex(I id, int partitionCount, int workerCount) Calculates in which partition current vertex belongs to, from interval [0, partitionCount).
PartitionOwner	getPartitionOwner(I vertexId) Figure out the owner of a vertex
Collection<? extends PartitionOwner>	getPartitionOwners() Get a collection of the PartitionOwner objects.
PartitionExchange	updatePartitionOwners(WorkerInfo myWorkerInfo, Collection<? extends PartitionOwner> masterSetPartitionOwners) Get the partitions owners and update locally.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WorkerGraphPartitionerImpl

public WorkerGraphPartitionerImpl()

Method Detail

createPartitionOwner

public PartitionOwner createPartitionOwner()

Description copied from interface: WorkerGraphPartitioner

Instantiate the PartitionOwner implementation used to read the master assignments.

Specified by:

createPartitionOwner in interface WorkerGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Instantiated PartitionOwner object

getPartitionOwner

public PartitionOwner getPartitionOwner(I vertexId)

Description copied from interface: WorkerGraphPartitioner

Figure out the owner of a vertex

Specified by:

getPartitionOwner in interface WorkerGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertexId - Vertex id to get the partition for

Returns:

Correct partition owner

finalizePartitionStats

public Collection<PartitionStats> finalizePartitionStats(Collection<PartitionStats> workerPartitionStats,
                                                         PartitionStore<I,V,E> partitionStore)

Description copied from interface: WorkerGraphPartitioner

At the end of a superstep, workers have PartitionStats generated for each of their partitions. This method will allow the user to modify or create their own PartitionStats interfaces to send to the master.

Specified by:

finalizePartitionStats in interface WorkerGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

workerPartitionStats - Stats generated by the infrastructure during the superstep

partitionStore - Partition store for this worker (could be used to provide more useful stat information)

Returns:

Final partition stats

updatePartitionOwners

public PartitionExchange updatePartitionOwners(WorkerInfo myWorkerInfo,
                                               Collection<? extends PartitionOwner> masterSetPartitionOwners)

Description copied from interface: WorkerGraphPartitioner

Get the partitions owners and update locally. Returns the partitions to send to other workers and other dependencies.

Specified by:

updatePartitionOwners in interface WorkerGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

myWorkerInfo - Worker info.

masterSetPartitionOwners - Master set partition owners, received prior to beginning the superstep

Returns:

Information for the partition exchange.

getPartitionOwners

public Collection<? extends PartitionOwner> getPartitionOwners()

Description copied from interface: WorkerGraphPartitioner

Get a collection of the PartitionOwner objects.

Specified by:

getPartitionOwners in interface WorkerGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Collection of owners for every partition.

extractAvailableWorkers

public void extractAvailableWorkers()

Update availableWorkers

getPartitionIndex

protected abstract int getPartitionIndex(I id,
                                         int partitionCount,
                                         int workerCount)

Calculates in which partition current vertex belongs to, from interval [0, partitionCount).

Parameters:

id - Vertex id

partitionCount - Number of partitions

workerCount - Number of active workers

Returns:

partition