org.apache.giraph.comm.aggregators

Class AllAggregatorServerData

java.lang.Object

org.apache.giraph.comm.aggregators.AllAggregatorServerData

public class AllAggregatorServerData
extends Object

Accepts aggregators and their values from previous superstep from master and workers which own aggregators. Keeps data received from master so it could be distributed later. Also counts the requests so we would know when we are done receiving requests. Only restriction is that we need to call registerAggregatorClass before calling createAggregatorInitialValue, other than that methods of this class are thread-safe.

Constructor Summary

Constructors

Constructor and Description
AllAggregatorServerData(org.apache.hadoop.util.Progressable progressable, ImmutableClassesGiraphConfiguration conf) Constructor

Method Summary

Modifier and Type	Method and Description
void	fillNextSuperstepMapsWhenReady(Set<Integer> workerIds, Map<String,org.apache.hadoop.io.Writable> broadcastedMapToFill, Map<String,Reducer<Object,org.apache.hadoop.io.Writable>> reducerMapToFill) This function will wait until all aggregator requests from workers have arrived, and fill the maps for next superstep when ready.
Iterable<byte[]>	getDataFromMasterWhenReady(MasterInfo masterInfo) This function will wait until all aggregator requests from master have arrived, and return that data afterwards.
void	receivedRequestCountFromMaster(long requestCount, int taskId) Notify this object about the total number of requests which should arrive from master.
void	receivedRequestCountFromWorker(long requestCount, int taskId) Notify this object about the total number of requests which should arrive from one of the workers.
void	receivedRequestFromMaster(byte[] data) Notify this object that an aggregator request from master has been received.
void	receivedRequestFromWorker() Notify this object that an aggregator request from some worker has been received.
void	receiveValueFromMaster(String name, GlobalCommType type, org.apache.hadoop.io.Writable value) Received value through global communication from master.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AllAggregatorServerData

public AllAggregatorServerData(org.apache.hadoop.util.Progressable progressable,
                               ImmutableClassesGiraphConfiguration conf)

Constructor

Parameters:

progressable - Progressable used to report progress

conf - Configuration

Method Detail

receiveValueFromMaster

public void receiveValueFromMaster(String name,
                                   GlobalCommType type,
                                   org.apache.hadoop.io.Writable value)

Received value through global communication from master.

Parameters:

name - Name

type - Global communication type

value - Object value

receivedRequestFromMaster

public void receivedRequestFromMaster(byte[] data)

Notify this object that an aggregator request from master has been received.

Parameters:

data - Byte request with data received from master

receivedRequestCountFromMaster

public void receivedRequestCountFromMaster(long requestCount,
                                           int taskId)

Notify this object about the total number of requests which should arrive from master.

Parameters:

requestCount - Number of requests which should arrive

taskId - Task id of master

receivedRequestFromWorker

public void receivedRequestFromWorker()

Notify this object that an aggregator request from some worker has been received.

receivedRequestCountFromWorker

public void receivedRequestCountFromWorker(long requestCount,
                                           int taskId)

Notify this object about the total number of requests which should arrive from one of the workers.

Parameters:

requestCount - Number of requests which should arrive

taskId - Task id of that worker

getDataFromMasterWhenReady

public Iterable<byte[]> getDataFromMasterWhenReady(MasterInfo masterInfo)

This function will wait until all aggregator requests from master have arrived, and return that data afterwards.

Parameters:

masterInfo - Master info

Returns:

Iterable through data received from master

fillNextSuperstepMapsWhenReady

public void fillNextSuperstepMapsWhenReady(Set<Integer> workerIds,
                                           Map<String,org.apache.hadoop.io.Writable> broadcastedMapToFill,
                                           Map<String,Reducer<Object,org.apache.hadoop.io.Writable>> reducerMapToFill)

This function will wait until all aggregator requests from workers have arrived, and fill the maps for next superstep when ready.

Parameters:

workerIds - All workers in the job apart from the current one

broadcastedMapToFill - Broadcast map to fill out

reducerMapToFill - Registered reducer map to fill out.