org.apache.giraph.partition

Class MasterGraphPartitionerImpl<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.partition.MasterGraphPartitionerImpl<I,V,E>

Type Parameters:

I - Vertex id type

V - Vertex value type

E - Edge value type

All Implemented Interfaces:

MasterGraphPartitioner<I,V,E>

public abstract class MasterGraphPartitionerImpl<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends Object
implements MasterGraphPartitioner<I,V,E>

Abstracts and implements all MasterGraphPartitioner logic on top of a single user function - getWorkerIndex.

Constructor Summary

Constructors

Constructor and Description
MasterGraphPartitionerImpl(ImmutableClassesGiraphConfiguration<I,V,E> conf) Constructor.

Method Summary

Modifier and Type	Method and Description
Collection<PartitionOwner>	createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos, int maxWorkers) Set some initial partition owners for the graph.
PartitionStats	createPartitionStats() Instantiate the PartitionStats implementation used to read the worker stats
Collection<PartitionOwner>	generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList, Collection<WorkerInfo> availableWorkers, int maxWorkers, long superstep) After the worker stats have been merged to a single list, the master can use this information to send commands to the workers for any Partition changes.
Collection<PartitionOwner>	getCurrentPartitionOwners() Get current partition owners at this time.
protected abstract int	getWorkerIndex(int partition, int partitionCount, int workerCount) Calculates worker that should be responsible for passed partition.
void	setPartitionOwners(Collection<PartitionOwner> partitionOwners) Sets partition owners for the graph.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MasterGraphPartitionerImpl

public MasterGraphPartitionerImpl(ImmutableClassesGiraphConfiguration<I,V,E> conf)

Constructor.

Parameters:

conf - Configuration used.

Method Detail

createInitialPartitionOwners

public Collection<PartitionOwner> createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
                                                               int maxWorkers)

Description copied from interface: MasterGraphPartitioner

Set some initial partition owners for the graph. Guaranteed to be called prior to the graph being loaded (initial or restart).

Specified by:

createInitialPartitionOwners in interface MasterGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

availableWorkerInfos - Workers available for partition assignment

maxWorkers - Maximum number of workers

Returns:

Collection of generated partition owners.

setPartitionOwners

public void setPartitionOwners(Collection<PartitionOwner> partitionOwners)

Description copied from interface: MasterGraphPartitioner

Sets partition owners for the graph. Used then loading from checkpoint.

Specified by:

setPartitionOwners in interface MasterGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

partitionOwners - assigned partition owners.

generateChangedPartitionOwners

public Collection<PartitionOwner> generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList,
                                                                 Collection<WorkerInfo> availableWorkers,
                                                                 int maxWorkers,
                                                                 long superstep)

Description copied from interface: MasterGraphPartitioner

After the worker stats have been merged to a single list, the master can use this information to send commands to the workers for any Partition changes. This protocol is specific to the MasterGraphPartitioner implementation.

Specified by:

generateChangedPartitionOwners in interface MasterGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

allPartitionStatsList - All partition stats from all workers.

availableWorkers - Workers available for partition assignment

maxWorkers - Maximum number of workers

superstep - Partition owners will be set for this superstep

Returns:

Collection of PartitionOwner objects that changed from the previous superstep, empty list if no change.

getCurrentPartitionOwners

public Collection<PartitionOwner> getCurrentPartitionOwners()

Description copied from interface: MasterGraphPartitioner

Get current partition owners at this time.

Specified by:

getCurrentPartitionOwners in interface MasterGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Collection of current PartitionOwner objects

createPartitionStats

public PartitionStats createPartitionStats()

Description copied from interface: MasterGraphPartitioner

Instantiate the PartitionStats implementation used to read the worker stats

Specified by:

createPartitionStats in interface MasterGraphPartitioner<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Instantiated PartitionStats object

getWorkerIndex

protected abstract int getWorkerIndex(int partition,
                                      int partitionCount,
                                      int workerCount)

Calculates worker that should be responsible for passed partition.

Parameters:

partition - Current partition

partitionCount - Number of partitions

workerCount - Number of workers

Returns:

index of worker responsible for current partition