org.apache.giraph.block_app.framework.piece.delegate

Class DelegatePiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

java.lang.Object

org.apache.giraph.block_app.framework.piece.AbstractPiece<I,V,E,M,WV,WM,S>

org.apache.giraph.block_app.framework.piece.delegate.DelegatePiece<I,V,E,M,WV,WM,S>

Type Parameters:

I - Vertex id type

V - Vertex value type

E - Edge value type

M - Message type

WV - Worker value type

WM - Worker message type

S - Execution stage type

All Implemented Interfaces:

Iterable<AbstractPiece>, Block

Direct Known Subclasses:

FilteringPiece

public class DelegatePiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
extends AbstractPiece<I,V,E,M,WV,WM,S>

Delegate Piece which allows combining multiple pieces in same iteration: new DelegatePiece(new LogicPiece(), new StatsPiece()) You should be careful when doing so, since those pieces must not interact, and only one can send messages. Execution of any of the Piece methods by the framework is going to trigger sequential execution of that method on all of the pieces this DelegatePiece wraps. That means for example, getVertexSender is going to be called on all pieces before masterCompute is called on all pieces, which is called before getVertexReceiver on all pieces. Also, via overriding, it provides an abstract class for filtering. I.e. if you want piece that filters out calls to masterCompute, you can have: new FilterMasterPiece(new LogicPiece()), with FilterMasterPiece extends DelegatePiece, and only overrides getMaster function and DelegateMasterPiece class.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	DelegatePiece.DelegateWorkerReceiveFunctions Delegating WorkerReceivePiece
protected class	DelegatePiece.DelegateWorkerSendFunctions Delegating WorkerSendPiece

Nested classes/interfaces inherited from class org.apache.giraph.block_app.framework.piece.AbstractPiece

AbstractPiece.InnerVertexReceiver, AbstractPiece.InnerVertexSender

Constructor Summary

Constructors

Constructor and Description
DelegatePiece(AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S>... innerPieces)
DelegatePiece(AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)

Method Summary

Modifier and Type	Method and Description
protected DelegatePiece.DelegateWorkerReceiveFunctions	delegateWorkerReceiveFunctions(ArrayList<VertexReceiver<I,V,E,M>> workerReceiveFunctions, BlockWorkerReceiveApi<I> workerApi, S executionStage)
protected DelegatePiece.DelegateWorkerSendFunctions	delegateWorkerSendFunctions(ArrayList<AbstractPiece.InnerVertexSender> workerSendFunctions, BlockWorkerSendApi<I,V,E,M> workerApi, S executionStage)
protected String	delegationName()
void	forAllPossiblePieces(Consumer<AbstractPiece> consumer) Calls consumer for each Piece: - in no particular order - potentially calling multiple times on same Piece - even if Piece might never be returned in the iterator - it will be called at least once for every piece that is going to be returned by iterator Can be used for static analysis/introspection of the block, without actually executing them.
MessageClasses<I,M>	getMessageClasses(ImmutableClassesGiraphConfiguration conf) Returns MessageClasses definition for messages being sent by this Piece.
PieceCount	getPieceCount() How many pieces are in this block.
AbstractPiece.InnerVertexReceiver	getVertexReceiver(BlockWorkerReceiveApi<I> workerApi, S executionStage) Override to do vertex receive processing.
AbstractPiece.InnerVertexSender	getWrappedVertexSender(BlockWorkerSendApi<I,V,E,M> workerApi, S executionStage) Add automatic handling of reducers to getVertexSender.
void	masterCompute(BlockMasterApi api, S executionStage) Function that is called on master, after send phase, before receive phase.
S	nextExecutionStage(S executionStage) Override to provide different next execution stage for Pieces that come after it.
void	registerAggregators(BlockMasterApi master) Override to register any potential aggregators used by this piece.
String	toString()
void	workerContextReceive(BlockWorkerContextReceiveApi workerContextApi, S executionStage, WV workerValue, List<WM> workerMessages) Override to have worker context receive computation.
void	workerContextSend(BlockWorkerContextSendApi<I,WM> workerContextApi, S executionStage, WV workerValue) Override to have worker context send computation.
void	wrappedRegisterReducers(BlockMasterApi masterApi, S executionStage) Add automatic handling of reducers to registerReducers.

Methods inherited from class org.apache.giraph.block_app.framework.piece.AbstractPiece

equals, hashCode, iterator

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

DelegatePiece

@SafeVarargs
public DelegatePiece(AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S>... innerPieces)

DelegatePiece

public DelegatePiece(AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)

Method Detail

delegateWorkerSendFunctions

protected DelegatePiece.DelegateWorkerSendFunctions delegateWorkerSendFunctions(ArrayList<AbstractPiece.InnerVertexSender> workerSendFunctions,
                                                                                BlockWorkerSendApi<I,V,E,M> workerApi,
                                                                                S executionStage)

delegateWorkerReceiveFunctions

protected DelegatePiece.DelegateWorkerReceiveFunctions delegateWorkerReceiveFunctions(ArrayList<VertexReceiver<I,V,E,M>> workerReceiveFunctions,
                                                                                      BlockWorkerReceiveApi<I> workerApi,
                                                                                      S executionStage)

getWrappedVertexSender

public AbstractPiece.InnerVertexSender getWrappedVertexSender(BlockWorkerSendApi<I,V,E,M> workerApi,
                                                              S executionStage)

Description copied from class: AbstractPiece

Add automatic handling of reducers to getVertexSender. Only for Framework internal use.

Specified by:

getWrappedVertexSender in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

getVertexReceiver

public AbstractPiece.InnerVertexReceiver getVertexReceiver(BlockWorkerReceiveApi<I> workerApi,
                                                           S executionStage)

Description copied from class: AbstractPiece

Override to do vertex receive processing. Creates handler that defines what should be executed on worker for each vertex during receive phase. This logic executed last. This function is called once on each worker on each thread, in parallel, on their copy of Piece object to create functions handler. If returned object implements Postprocessor interface, then corresponding postprocess() function is going to be called once, after all vertices corresponding thread needed to process are done.

Overrides:

getVertexReceiver in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

masterCompute

public void masterCompute(BlockMasterApi api,
                          S executionStage)

Description copied from class: AbstractPiece

Function that is called on master, after send phase, before receive phase. It can: - read aggregators sent from worker - do global processing - send data to workers through aggregators

Overrides:

masterCompute in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

workerContextSend

public void workerContextSend(BlockWorkerContextSendApi<I,WM> workerContextApi,
                              S executionStage,
                              WV workerValue)

Description copied from class: AbstractPiece

Override to have worker context send computation. Called once per worker, after all vertices have been processed with getVertexSender.

Overrides:

workerContextSend in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

workerContextReceive

public void workerContextReceive(BlockWorkerContextReceiveApi workerContextApi,
                                 S executionStage,
                                 WV workerValue,
                                 List<WM> workerMessages)

Description copied from class: AbstractPiece

Override to have worker context receive computation. Called once per worker, before all vertices are going to be processed with getVertexReceiver.

Overrides:

workerContextReceive in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

nextExecutionStage

public S nextExecutionStage(S executionStage)

Description copied from class: AbstractPiece

Override to provide different next execution stage for Pieces that come after it. Execution stage should be immutable, and this function should be returning a new object, if it needs to return different value. It affects pieces that come after this piece, and isn't applied to execution stage this piece sees.

Overrides:

nextExecutionStage in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

getMessageClasses

public MessageClasses<I,M> getMessageClasses(ImmutableClassesGiraphConfiguration conf)

Description copied from class: AbstractPiece

Returns MessageClasses definition for messages being sent by this Piece.

Specified by:

getMessageClasses in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

forAllPossiblePieces

public void forAllPossiblePieces(Consumer<AbstractPiece> consumer)

Description copied from interface: Block

Calls consumer for each Piece: - in no particular order - potentially calling multiple times on same Piece - even if Piece might never be returned in the iterator - it will be called at least once for every piece that is going to be returned by iterator Can be used for static analysis/introspection of the block, without actually executing them.

Specified by:

forAllPossiblePieces in interface Block

Overrides:

forAllPossiblePieces in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

getPieceCount

public PieceCount getPieceCount()

Description copied from interface: Block

How many pieces are in this block. Sometimes we don't know (eg RepeatBlock).

Specified by:

getPieceCount in interface Block

Overrides:

getPieceCount in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

Returns:

How many pieces are in this block.

registerAggregators

public void registerAggregators(BlockMasterApi master)
                         throws InstantiationException,
                                IllegalAccessException

Description copied from class: AbstractPiece

Override to register any potential aggregators used by this piece.

Overrides:

registerAggregators in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

Throws:

InstantiationException

IllegalAccessException

wrappedRegisterReducers

public void wrappedRegisterReducers(BlockMasterApi masterApi,
                                    S executionStage)

Description copied from class: AbstractPiece

Add automatic handling of reducers to registerReducers. Only for internal use.

Specified by:

wrappedRegisterReducers in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

delegationName

protected String delegationName()

toString

public String toString()

Overrides:

toString in class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>