org.apache.giraph.comm.aggregators

Class OwnerAggregatorServerData

java.lang.Object

org.apache.giraph.comm.aggregators.OwnerAggregatorServerData

public class OwnerAggregatorServerData
extends Object

Class for holding aggregators which current worker owns, and aggregating partial aggregator values from workers. Protocol: 1. Before the beginning of superstep, worker receives its aggregators from master, and these aggregators will be registered to this class. Multiple registrations can be called concurrently. 2. During the superstep, whenever a worker finishes computation, it will send partial aggregated values to worker owner. This class is used to help deserialize the arriving aggregator values, and aggregate the values at the destination owner worker; these can happen concurrently. (we know step 1. is finished before anything from step 2. happens because other workers can't start computation before they receive aggregators which this worker owns) 3. This class also tracks the number of partial aggregator requests which worker received. In the end of superstep, getMyAggregatorValuesWhenReady will be called to ensure everything was received and get the values which need to be sent to master. Because of this counting, in step 2. even if worker owns no aggregators, it will still send a message without aggregator data. 4. In the end we reset to prepare for the next superstep.

Constructor Summary

Constructors

Constructor and Description
OwnerAggregatorServerData(org.apache.hadoop.util.Progressable progressable) Constructor

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.io.Writable	createInitialValue(String name) Create initial value for a reducer.
Iterable<Map.Entry<String,org.apache.hadoop.io.Writable>>	getMyReducedValuesWhenReady(Set<Integer> workerIds) This function will wait until all partial aggregated values from all workers are ready and aggregated, and return final aggregated values afterwards.
void	receivedRequestCountFromWorker(long requestCount, int taskId) Notify this object about the total number of requests which should arrive from one of the workers.
void	receivedRequestFromWorker() Notify this object that a partial aggregated values request from some worker have been received.
void	reduce(String name, org.apache.hadoop.io.Writable value) Reduce partial value of one of current worker's reducers.
void	registerReducer(String name, ReduceOperation<Object,org.apache.hadoop.io.Writable> reduceOp) Register a reducer which current worker owns.
void	reset() Prepare for next superstep

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OwnerAggregatorServerData

public OwnerAggregatorServerData(org.apache.hadoop.util.Progressable progressable)

Constructor

Parameters:

progressable - Progressable used to report progress

Method Detail

registerReducer

public void registerReducer(String name,
                            ReduceOperation<Object,org.apache.hadoop.io.Writable> reduceOp)

Register a reducer which current worker owns. Thread-safe.

Parameters:

name - Name of aggregator

reduceOp - Reduce operation

reduce

public void reduce(String name,
                   org.apache.hadoop.io.Writable value)

Reduce partial value of one of current worker's reducers. Thread-safe. Call only after reducers have been registered.

Parameters:

name - Name of the reducer

value - Value to reduce to it

createInitialValue

public org.apache.hadoop.io.Writable createInitialValue(String name)

Create initial value for a reducer. Used so requests would be able to deserialize data. Thread-safe. Call only after reducer has been registered.

Parameters:

name - Name of the reducer

Returns:

Empty value

receivedRequestFromWorker

public void receivedRequestFromWorker()

Notify this object that a partial aggregated values request from some worker have been received. Thread-safe.

receivedRequestCountFromWorker

public void receivedRequestCountFromWorker(long requestCount,
                                           int taskId)

Notify this object about the total number of requests which should arrive from one of the workers. Thread-safe.

Parameters:

requestCount - Number of requests which should arrive

taskId - Task id of that worker

getMyReducedValuesWhenReady

public Iterable<Map.Entry<String,org.apache.hadoop.io.Writable>> getMyReducedValuesWhenReady(Set<Integer> workerIds)

This function will wait until all partial aggregated values from all workers are ready and aggregated, and return final aggregated values afterwards.

Parameters:

workerIds - All workers in the job apart from the current one

Returns:

Iterable through final aggregated values which this worker owns

reset

public void reset()

Prepare for next superstep