org.apache.giraph.master

Class MasterAggregatorHandler

java.lang.Object

org.apache.giraph.master.MasterAggregatorHandler

All Implemented Interfaces:

MasterGlobalCommUsageAggregators, org.apache.hadoop.io.Writable

public class MasterAggregatorHandler
extends Object
implements MasterGlobalCommUsageAggregators, org.apache.hadoop.io.Writable

Handler for reduce/broadcast on the master

Constructor Summary

Constructors

Constructor and Description
MasterAggregatorHandler(ImmutableClassesGiraphConfiguration<?,?,?> conf, org.apache.hadoop.util.Progressable progressable) Constructor

Method Summary

Modifier and Type	Method and Description
void	acceptReducedValues(DataInput reducedValuesInput) Accept reduced values sent by worker.
void	broadcast(String name, org.apache.hadoop.io.Writable object) Broadcast given value to all workers for next computation.
void	close() Close AggregatorWriter
void	finishSuperstep() Finalize aggregators for current superstep
<T extends org.apache.hadoop.io.Writable> T	getReduced(String name) Get reduced value from previous worker computation.
void	initialize(BspService service) Initialize AggregatorWriter
void	prepareSuperstep() Prepare reduced values for current superstep's master compute
void	readFields(DataInput in)
<S,R extends org.apache.hadoop.io.Writable> void	registerReducer(String name, ReduceOperation<S,R> reduceOp) Register reducer to be reduced in the next worker computation, using given name and operations.
<S,R extends org.apache.hadoop.io.Writable> void	registerReducer(String name, ReduceOperation<S,R> reduceOp, R globalInitialValue) Register reducer to be reduced in the next worker computation, using given name and operations, starting globally from globalInitialValue.
void	sendDataToOwners(MasterClient masterClient) Send data to workers (through owner workers)
void	write(DataOutput out)
void	writeAggregators(long superstep, SuperstepState superstepState) Write aggregators to AggregatorWriter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MasterAggregatorHandler

public MasterAggregatorHandler(ImmutableClassesGiraphConfiguration<?,?,?> conf,
                               org.apache.hadoop.util.Progressable progressable)

Constructor

Parameters:

conf - Configuration

progressable - Progress reporter

Method Detail

registerReducer

public final <S,R extends org.apache.hadoop.io.Writable> void registerReducer(String name,
                                                                              ReduceOperation<S,R> reduceOp)

Description copied from interface: MasterGlobalCommUsageAggregators

Register reducer to be reduced in the next worker computation, using given name and operations.

Specified by:

registerReducer in interface MasterGlobalCommUsageAggregators

Type Parameters:

S - Single value type

R - Reduced value type

Parameters:

name - Name of the reducer

reduceOp - Reduce operations

registerReducer

public <S,R extends org.apache.hadoop.io.Writable> void registerReducer(String name,
                                                                        ReduceOperation<S,R> reduceOp,
                                                                        R globalInitialValue)

Description copied from interface: MasterGlobalCommUsageAggregators

Register reducer to be reduced in the next worker computation, using given name and operations, starting globally from globalInitialValue. (globalInitialValue is reduced only once, each worker will still start from neutral initial value)

Specified by:

registerReducer in interface MasterGlobalCommUsageAggregators

Type Parameters:

S - Single value type

R - Reduced value type

Parameters:

name - Name of the reducer

reduceOp - Reduce operations

globalInitialValue - Global initial value

getReduced

public <T extends org.apache.hadoop.io.Writable> T getReduced(String name)

Description copied from interface: MasterGlobalCommUsageAggregators

Get reduced value from previous worker computation.

Specified by:

getReduced in interface MasterGlobalCommUsageAggregators

Type Parameters:

T - Reduced value type

Parameters:

name - Name of the reducer

Returns:

Reduced value

broadcast

public void broadcast(String name,
                      org.apache.hadoop.io.Writable object)

Description copied from interface: MasterGlobalCommUsageAggregators

Broadcast given value to all workers for next computation.

Specified by:

broadcast in interface MasterGlobalCommUsageAggregators

Parameters:

name - Name of the broadcast object

object - Value

prepareSuperstep

public void prepareSuperstep()

Prepare reduced values for current superstep's master compute

finishSuperstep

public void finishSuperstep()

Finalize aggregators for current superstep

sendDataToOwners

public void sendDataToOwners(MasterClient masterClient)

Send data to workers (through owner workers)

Parameters:

masterClient - IPC client on master

acceptReducedValues

public void acceptReducedValues(DataInput reducedValuesInput)
                         throws IOException

Accept reduced values sent by worker. Every value will be sent only once, by its owner. We don't need to count the number of these requests because global superstep barrier will happen after workers ensure all requests of this type have been received and processed by master.

Parameters:

reducedValuesInput - Input in which aggregated values are written in the following format: numReducers name_1 REDUCED_VALUE value_1 name_2 REDUCED_VALUE value_2 ...

Throws:

IOException

writeAggregators

public void writeAggregators(long superstep,
                             SuperstepState superstepState)

Write aggregators to AggregatorWriter

Parameters:

superstep - Superstep which just finished

superstepState - State of the superstep which just finished

initialize

public void initialize(BspService service)

Initialize AggregatorWriter

Parameters:

service - BspService

close

public void close()
           throws IOException

Close AggregatorWriter

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException