org.apache.giraph.partition

Class GraphPartitionerFactory<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.partition.GraphPartitionerFactory<I,V,E>

Type Parameters:

I - Vertex id value

V - Vertex value

E - Edge value

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>, GraphPartitionerFactoryInterface<I,V,E>

Direct Known Subclasses:

HashPartitionerFactory, HashRangePartitionerFactory, LongMappingStorePartitionerFactory, SimpleIntRangePartitionerFactory, SimpleLongRangePartitionerFactory

public abstract class GraphPartitionerFactory<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends DefaultImmutableClassesGiraphConfigurable<I,V,E>
implements GraphPartitionerFactoryInterface<I,V,E>

Defines the partitioning framework for this application. Abstracts and implements all GraphPartitionerFactoryInterface logic on top of two functions which define partitioning scheme: - which partition vertex should be in, and - which partition should belong to which worker

Constructor Summary

Constructors

Constructor and Description
GraphPartitionerFactory()

Method Summary

Modifier and Type	Method and Description
MasterGraphPartitioner<I,V,E>	createMasterGraphPartitioner() Create the MasterGraphPartitioner used by the master.
WorkerGraphPartitioner<I,V,E>	createWorkerGraphPartitioner() Create the WorkerGraphPartitioner used by the worker.
abstract int	getPartition(I id, int partitionCount, int workerCount) Calculates in which partition current vertex belongs to, from interval [0, partitionCount).
static int	getPartitionInRange(int value, int max, int partitions) Utility function for calculating in which partition value from interval [0, max) should belong to.
static int	getPartitionInRange(long value, long max, int partitions) Utility function for calculating in which partition value from interval [0, max) should belong to.
abstract int	getWorker(int partition, int partitionCount, int workerCount) Calculates worker that should be responsible for passed partition.
void	initialize(LocalData<I,V,E,? extends org.apache.hadoop.io.Writable> localData) Use some local data present in the worker

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.giraph.conf.ImmutableClassesGiraphConfigurable

getConf

Methods inherited from interface org.apache.giraph.conf.GiraphConfigurationSettable

setConf

Constructor Detail

GraphPartitionerFactory

public GraphPartitionerFactory()

Method Detail

initialize

public void initialize(LocalData<I,V,E,? extends org.apache.hadoop.io.Writable> localData)

Description copied from interface: GraphPartitionerFactoryInterface

Use some local data present in the worker

Specified by:

initialize in interface GraphPartitionerFactoryInterface<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

localData - localData present in the worker

createMasterGraphPartitioner

public final MasterGraphPartitioner<I,V,E> createMasterGraphPartitioner()

Description copied from interface: GraphPartitionerFactoryInterface

Create the MasterGraphPartitioner used by the master. Instantiated once by the master and reused.

Specified by:

createMasterGraphPartitioner in interface GraphPartitionerFactoryInterface<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Instantiated master graph partitioner

createWorkerGraphPartitioner

public final WorkerGraphPartitioner<I,V,E> createWorkerGraphPartitioner()

Description copied from interface: GraphPartitionerFactoryInterface

Create the WorkerGraphPartitioner used by the worker. Instantiated once by every worker and reused.

Specified by:

createWorkerGraphPartitioner in interface GraphPartitionerFactoryInterface<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Instantiated worker graph partitioner

getPartition

public abstract int getPartition(I id,
                                 int partitionCount,
                                 int workerCount)

Calculates in which partition current vertex belongs to, from interval [0, partitionCount).

Parameters:

id - Vertex id

partitionCount - Number of partitions

workerCount - Number of workers

Returns:

partition

getWorker

public abstract int getWorker(int partition,
                              int partitionCount,
                              int workerCount)

Calculates worker that should be responsible for passed partition.

Parameters:

partition - Current partition

partitionCount - Number of partitions

workerCount - Number of workers

Returns:

index of worker responsible for current partition

getPartitionInRange

public static int getPartitionInRange(int value,
                                      int max,
                                      int partitions)

Utility function for calculating in which partition value from interval [0, max) should belong to.

Parameters:

value - Value for which partition is requested

max - Maximum possible value

partitions - Number of partitions, equally sized.

Returns:

Index of partition where value belongs to.

getPartitionInRange

public static int getPartitionInRange(long value,
                                      long max,
                                      int partitions)

Utility function for calculating in which partition value from interval [0, max) should belong to.

Parameters:

value - Value for which partition is requested

max - Maximum possible value

partitions - Number of partitions, equally sized.

Returns:

Index of partition where value belongs to.