org.apache.giraph.master

Class MasterCompute

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

org.apache.giraph.master.MasterCompute

All Implemented Interfaces:

AggregatorUsage, GiraphConfigurationSettable, ImmutableClassesGiraphConfigurable, MasterAggregatorUsage, MasterGlobalCommUsage, MasterGlobalCommUsageAggregators, org.apache.hadoop.io.Writable

Direct Known Subclasses:

BlockMasterCompute, DefaultMasterCompute

public abstract class MasterCompute
extends DefaultImmutableClassesGiraphConfigurable
implements MasterAggregatorUsage, MasterGlobalCommUsage, org.apache.hadoop.io.Writable

Interface for defining a master vertex that can perform centralized computation between supersteps. This class will be instantiated on the master node and will run every superstep before the workers do. Communication with the workers should be performed via aggregators. The values of the aggregators are broadcast to the workers before vertex.compute() is called and collected by the master before master.compute() is called. This means aggregator values used by the workers are consistent with aggregator values from the master from the same superstep and aggregator used by the master are consistent with aggregator values from the workers from the previous superstep.

Constructor Summary

Constructors

Constructor and Description
MasterCompute()

Method Summary

Modifier and Type	Method and Description
void	broadcast(String name, org.apache.hadoop.io.Writable object) Broadcast given value to all workers for next computation.
abstract void	compute() Must be defined by user to specify what the master has to do.
<A extends org.apache.hadoop.io.Writable> A	getAggregatedValue(String name) Get value of an aggregator.
Class<? extends Computation>	getComputation() Get Computation class to be used
org.apache.hadoop.mapreduce.Mapper.Context	getContext() Get the mapper context
Class<? extends MessageCombiner>	getMessageCombiner() Get MessageCombiner class to be used
<T extends org.apache.hadoop.io.Writable> T	getReduced(String name) Get reduced value from previous worker computation.
long	getSuperstep() Retrieves the current superstep.
long	getTotalNumEdges() Get the total (all workers) number of edges that existed in the previous superstep.
long	getTotalNumVertices() Get the total (all workers) number of vertices that existed in the previous superstep.
List<WorkerInfo>	getWorkerInfoList() Get list of workers
void	haltComputation() After this is called, the computation will stop, even if there are still messages in the system or vertices that have not voted to halt.
abstract void	initialize() Initialize the MasterCompute class, this is the place to register aggregators.
boolean	isHalted() Has the master halted?
void	logToCommandLine(String line) Call this to log a line to command line of the job.
<A extends org.apache.hadoop.io.Writable> boolean	registerAggregator(String name, Class<? extends Aggregator<A>> aggregatorClass) Register an aggregator in preSuperstep() and/or preApplication().
<A extends org.apache.hadoop.io.Writable> boolean	registerPersistentAggregator(String name, Class<? extends Aggregator<A>> aggregatorClass) Register persistent aggregator in preSuperstep() and/or preApplication().
<S,R extends org.apache.hadoop.io.Writable> void	registerReducer(String name, ReduceOperation<S,R> reduceOp) Register reducer to be reduced in the next worker computation, using given name and operations.
<S,R extends org.apache.hadoop.io.Writable> void	registerReducer(String name, ReduceOperation<S,R> reduceOp, R globalInitialValue) Register reducer to be reduced in the next worker computation, using given name and operations, starting globally from globalInitialValue.
<A extends org.apache.hadoop.io.Writable> void	setAggregatedValue(String name, A value) Sets value of an aggregator.
void	setComputation(Class<? extends Computation> computationClass) Set Computation class to be used
void	setGraphState(GraphState graphState)
void	setIncomingMessage(Class<? extends org.apache.hadoop.io.Writable> incomingMessageClass) Deprecated.
void	setMasterService(CentralizedServiceMaster serviceMaster)
void	setMessageCombiner(Class<? extends MessageCombiner> combinerClass) Set MessageCombiner class to be used
void	setOutgoingMessage(Class<? extends org.apache.hadoop.io.Writable> outgoingMessageClass) Set outgoing message class to be used
void	setOutgoingMessageClasses(MessageClasses<? extends org.apache.hadoop.io.WritableComparable,? extends org.apache.hadoop.io.Writable> outgoingMessageClasses) Set outgoing message classes to be used
void	setSuperstepClasses(SuperstepClasses superstepClasses)

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.io.Writable

readFields, write

Constructor Detail

MasterCompute

public MasterCompute()

Method Detail

compute

public abstract void compute()

Must be defined by user to specify what the master has to do.

initialize

public abstract void initialize()
                         throws InstantiationException,
                                IllegalAccessException

Initialize the MasterCompute class, this is the place to register aggregators.

Throws:

InstantiationException

IllegalAccessException

getSuperstep

public final long getSuperstep()

Retrieves the current superstep.

Returns:

Current superstep

getTotalNumVertices

public final long getTotalNumVertices()

Get the total (all workers) number of vertices that existed in the previous superstep.

Returns:

Total number of vertices (-1 if first superstep)

getTotalNumEdges

public final long getTotalNumEdges()

Get the total (all workers) number of edges that existed in the previous superstep.

Returns:

Total number of edges (-1 if first superstep)

haltComputation

public final void haltComputation()

After this is called, the computation will stop, even if there are still messages in the system or vertices that have not voted to halt.

isHalted

public final boolean isHalted()

Has the master halted?

Returns:

True if halted, false otherwise.

getContext

public final org.apache.hadoop.mapreduce.Mapper.Context getContext()

Get the mapper context

Returns:

Mapper context

getWorkerInfoList

public final List<WorkerInfo> getWorkerInfoList()

Get list of workers

Returns:

List of workers

setComputation

public final void setComputation(Class<? extends Computation> computationClass)

Set Computation class to be used

Parameters:

computationClass - Computation class

getComputation

public final Class<? extends Computation> getComputation()

Get Computation class to be used

Returns:

Computation class

setMessageCombiner

public final void setMessageCombiner(Class<? extends MessageCombiner> combinerClass)

Set MessageCombiner class to be used

Parameters:

combinerClass - MessageCombiner class

getMessageCombiner

public final Class<? extends MessageCombiner> getMessageCombiner()

Get MessageCombiner class to be used

Returns:

MessageCombiner class

setIncomingMessage

@Deprecated
public final void setIncomingMessage(Class<? extends org.apache.hadoop.io.Writable> incomingMessageClass)

Deprecated.

Set incoming message class to be used

Parameters:

incomingMessageClass - incoming message class

setOutgoingMessage

public final void setOutgoingMessage(Class<? extends org.apache.hadoop.io.Writable> outgoingMessageClass)

Set outgoing message class to be used

Parameters:

outgoingMessageClass - outgoing message class

setOutgoingMessageClasses

public void setOutgoingMessageClasses(MessageClasses<? extends org.apache.hadoop.io.WritableComparable,? extends org.apache.hadoop.io.Writable> outgoingMessageClasses)

Set outgoing message classes to be used

Parameters:

outgoingMessageClasses - outgoing message classes

registerReducer

public final <S,R extends org.apache.hadoop.io.Writable> void registerReducer(String name,
                                                                              ReduceOperation<S,R> reduceOp)

Description copied from interface: MasterGlobalCommUsageAggregators

Register reducer to be reduced in the next worker computation, using given name and operations.

Specified by:

registerReducer in interface MasterGlobalCommUsageAggregators

Type Parameters:

S - Single value type

R - Reduced value type

Parameters:

name - Name of the reducer

reduceOp - Reduce operations

registerReducer

public final <S,R extends org.apache.hadoop.io.Writable> void registerReducer(String name,
                                                                              ReduceOperation<S,R> reduceOp,
                                                                              R globalInitialValue)

Description copied from interface: MasterGlobalCommUsageAggregators

Register reducer to be reduced in the next worker computation, using given name and operations, starting globally from globalInitialValue. (globalInitialValue is reduced only once, each worker will still start from neutral initial value)

Specified by:

registerReducer in interface MasterGlobalCommUsageAggregators

Type Parameters:

S - Single value type

R - Reduced value type

Parameters:

name - Name of the reducer

reduceOp - Reduce operations

globalInitialValue - Global initial value

getReduced

public final <T extends org.apache.hadoop.io.Writable> T getReduced(String name)

Description copied from interface: MasterGlobalCommUsageAggregators

Get reduced value from previous worker computation.

Specified by:

getReduced in interface MasterGlobalCommUsageAggregators

Type Parameters:

T - Reduced value type

Parameters:

name - Name of the reducer

Returns:

Reduced value

broadcast

public final void broadcast(String name,
                            org.apache.hadoop.io.Writable object)

Description copied from interface: MasterGlobalCommUsageAggregators

Broadcast given value to all workers for next computation.

Specified by:

broadcast in interface MasterGlobalCommUsageAggregators

Parameters:

name - Name of the broadcast object

object - Value

registerAggregator

public final <A extends org.apache.hadoop.io.Writable> boolean registerAggregator(String name,
                                                                                  Class<? extends Aggregator<A>> aggregatorClass)
                                                                           throws InstantiationException,
                                                                                  IllegalAccessException

Description copied from interface: MasterAggregatorUsage

Register an aggregator in preSuperstep() and/or preApplication(). This aggregator will have its value reset at the end of each super step.

Specified by:

registerAggregator in interface MasterAggregatorUsage

Type Parameters:

A - Aggregator type

Parameters:

name - of aggregator

aggregatorClass - Class type of the aggregator

Returns:

True iff aggregator wasn't already registered

Throws:

InstantiationException

IllegalAccessException

registerPersistentAggregator

public final <A extends org.apache.hadoop.io.Writable> boolean registerPersistentAggregator(String name,
                                                                                            Class<? extends Aggregator<A>> aggregatorClass)
                                                                                     throws InstantiationException,
                                                                                            IllegalAccessException

Description copied from interface: MasterAggregatorUsage

Register persistent aggregator in preSuperstep() and/or preApplication(). This aggregator will not reset value at the end of super step.

Specified by:

registerPersistentAggregator in interface MasterAggregatorUsage

Type Parameters:

A - Aggregator type

Parameters:

name - of aggregator

aggregatorClass - Class type of the aggregator

Returns:

True iff aggregator wasn't already registered

Throws:

InstantiationException

IllegalAccessException

getAggregatedValue

public final <A extends org.apache.hadoop.io.Writable> A getAggregatedValue(String name)

Description copied from interface: AggregatorUsage

Get value of an aggregator.

Specified by:

getAggregatedValue in interface AggregatorUsage

Type Parameters:

A - Aggregated value

Parameters:

name - Name of aggregator

Returns:

Value of the aggregator

setAggregatedValue

public final <A extends org.apache.hadoop.io.Writable> void setAggregatedValue(String name,
                                                                               A value)

Description copied from interface: MasterAggregatorUsage

Sets value of an aggregator.

Specified by:

setAggregatedValue in interface MasterAggregatorUsage

Type Parameters:

A - Aggregated value

Parameters:

name - Name of aggregator

value - Value to set

logToCommandLine

public void logToCommandLine(String line)

Call this to log a line to command line of the job. Use in moderation - it's a synchronous call to Job client

Parameters:

line - Line to print

setGraphState

public final void setGraphState(GraphState graphState)

setMasterService

public final void setMasterService(CentralizedServiceMaster serviceMaster)

setSuperstepClasses

public final void setSuperstepClasses(SuperstepClasses superstepClasses)