AbstractPiece (Apache Giraph Parent 1.4.0-SNAPSHOT API)

java.lang.Object
- org.apache.giraph.block_app.framework.piece.AbstractPiece<I,V,E,M,WV,WM,S>

Type Parameters:

I - Vertex id type

V - Vertex value type

E - Edge value type

M - Message type

WV - Worker value type

WM - Worker message type

S - Execution stage type

All Implemented Interfaces:

Iterable<AbstractPiece>, Block

Direct Known Subclasses:

DefaultParentPiece, DelegatePiece
```
public abstract class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
extends Object
implements Block
```
Parent of all Pieces, contains comprehensive list of methods Piece can support. Specific subclasses should be extended directly, to simplify usage - most frequently for example Piece class. Single unit of execution, capturing: - sending and then receiving messages from vertices - sending data to be aggregated from workers to master - sending values from master, via aggregators, to workers - sending and receiving worker messages Order of execution is: - On master, once at the start of the application -- registerAggregators (deprecated, use registerReducers instead) - After masterCompute of previous piece, on master: -- registerReducers - Send logic on workers: -- getVertexSender per each worker thread, and on object returned by it: --- vertexSend on each vertex --- postprocess on each worker thread -- workerContextSend per worker - Logic on master: -- masterCompute - Receive logic on workers: -- workerContextReceive per worker -- getVertexReceiver per each worker thread, and on object returned by it: --- vertexReceive on each vertex --- postprocess on each worker thread And before everything, during initialization, registerAggregators. Only masterCompute and registerReducers/registerAggregators should modify the Piece, all of the worker methods should treat Piece as read-only. Each piece should be encapsulated unit of execution. Vertex value should be used as a single implicit "communication" channel between different pieces, all other dependencies should be explicitly defined and passed through constructor, via interfaces (as explained below). I.e. state of the vertex value is invariant that Pieces act upon. Best is not to depend on explicit vertex value class, but on interface that provides all needed functions, so that pieces can be freely combined, as long as vertex value implements appropriate ones. Similarly, use most abstract class you need - if Piece doesn't depend on edge value, don't use NullWritable, but Writable. Or if it doesn't depend on ExecutionStage, use Object for it. All other external dependencies should be explicitly passed through constructor, through interfaces. All Pieces will be created within one context - on the master. They are then going to be replicated across all workers, and across all threads within each worker, and will see everything that happens in global context (masterCompute) before them, including any state master has. Through ObjectHolder/ObjectTransfer, you can pass data between Pieces in global context, and from global context to worker functions of a Piece that happens in the future. VertexReceiver of previous Piece and VertexSender of next Piece live in the same context, and vertexReceive of the next Piece is executed immediately after vertexSend of the previous piece, before vertexSend is called on the next vertex. This detail allows you to have external dependency on each other through memory only mediator objects - like ObjectTransfer. All other logic going to live in different contexts, specifically VertexSender and VertexReceiver of the same Piece, or workerContextSend and VertexSender of the same Piece, and cannot interact with each other outside of changing the state of the graph or using global communication api. All methods on this class (or objects it returns) will be called serially, so there is no need for any Thread synchronization. Each Thread will have a complete deep copy of the Piece, to achieve that, so all static fields must be written to be Thread safe! (i.e. either immutable, or have synchronized/locked access to them)

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`class`	`AbstractPiece.InnerVertexReceiver` Inner class to provide clean use without specifying types
`class`	`AbstractPiece.InnerVertexSender` Inner class to provide clean use without specifying types

Constructor Summary

Constructors
Constructor and Description

AbstractPiece()

Constructors
Constructor and Description
`AbstractPiece()`

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`boolean`	`equals(Object obj)`
`void`	`forAllPossiblePieces(Consumer<AbstractPiece> consumer)` Calls consumer for each Piece: - in no particular order - potentially calling multiple times on same Piece - even if Piece might never be returned in the iterator - it will be called at least once for every piece that is going to be returned by iterator Can be used for static analysis/introspection of the block, without actually executing them.
`abstract MessageClasses<I,M>`	`getMessageClasses(ImmutableClassesGiraphConfiguration conf)` Returns MessageClasses definition for messages being sent by this Piece.
`PieceCount`	`getPieceCount()` How many pieces are in this block.
`VertexReceiver<I,V,E,M>`	`getVertexReceiver(BlockWorkerReceiveApi<I> workerApi, S executionStage)` Override to do vertex receive processing.
`abstract AbstractPiece.InnerVertexSender`	`getWrappedVertexSender(BlockWorkerSendApi<I,V,E,M> workerApi, S executionStage)` Add automatic handling of reducers to getVertexSender.
`int`	`hashCode()`
`Iterator<AbstractPiece>`	`iterator()` Create iterator representing all pieces needed to be executed in this block.
`void`	`masterCompute(BlockMasterApi masterApi, S executionStage)` Function that is called on master, after send phase, before receive phase.
`S`	`nextExecutionStage(S executionStage)` Override to provide different next execution stage for Pieces that come after it.
`void`	`registerAggregators(BlockMasterApi masterApi)` Deprecated. Use registerReducers instead.
`String`	`toString()`
`void`	`workerContextReceive(BlockWorkerContextReceiveApi workerContextApi, S executionStage, WV workerValue, List<WM> workerMessages)` Override to have worker context receive computation.
`void`	`workerContextSend(BlockWorkerContextSendApi<I,WM> workerContextApi, S executionStage, WV workerValue)` Override to have worker context send computation.
`abstract void`	`wrappedRegisterReducers(BlockMasterApi masterApi, S executionStage)` Add automatic handling of reducers to registerReducers.

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable
forEach, spliterator

- Constructor Detail
  - AbstractPiece
```
public AbstractPiece()
```
- Method Detail
  - wrappedRegisterReducers
```
public abstract void wrappedRegisterReducers(BlockMasterApi masterApi,
                                             S executionStage)
```
    Add automatic handling of reducers to registerReducers. Only for internal use.
  - getWrappedVertexSender
```
public abstract AbstractPiece.InnerVertexSender getWrappedVertexSender(BlockWorkerSendApi<I,V,E,M> workerApi,
                                                                       S executionStage)
```
    Add automatic handling of reducers to getVertexSender. Only for Framework internal use.
  - workerContextSend
```
public void workerContextSend(BlockWorkerContextSendApi<I,WM> workerContextApi,
                              S executionStage,
                              WV workerValue)
```
    Override to have worker context send computation. Called once per worker, after all vertices have been processed with getVertexSender.
  - masterCompute
```
public void masterCompute(BlockMasterApi masterApi,
                          S executionStage)
```
    Function that is called on master, after send phase, before receive phase. It can: - read aggregators sent from worker - do global processing - send data to workers through aggregators
  - workerContextReceive
```
public void workerContextReceive(BlockWorkerContextReceiveApi workerContextApi,
                                 S executionStage,
                                 WV workerValue,
                                 List<WM> workerMessages)
```
    Override to have worker context receive computation. Called once per worker, before all vertices are going to be processed with getVertexReceiver.
  - getVertexReceiver
```
public VertexReceiver<I,V,E,M> getVertexReceiver(BlockWorkerReceiveApi<I> workerApi,
                                                 S executionStage)
```
    Override to do vertex receive processing. Creates handler that defines what should be executed on worker for each vertex during receive phase. This logic executed last. This function is called once on each worker on each thread, in parallel, on their copy of Piece object to create functions handler. If returned object implements Postprocessor interface, then corresponding postprocess() function is going to be called once, after all vertices corresponding thread needed to process are done.
  - getMessageClasses
```
public abstract MessageClasses<I,M> getMessageClasses(ImmutableClassesGiraphConfiguration conf)
```
    Returns MessageClasses definition for messages being sent by this Piece.
  - nextExecutionStage
```
public S nextExecutionStage(S executionStage)
```
    Override to provide different next execution stage for Pieces that come after it. Execution stage should be immutable, and this function should be returning a new object, if it needs to return different value. It affects pieces that come after this piece, and isn't applied to execution stage this piece sees.
  - registerAggregators
```
@Deprecated
public void registerAggregators(BlockMasterApi masterApi)
                                     throws InstantiationException,
                                            IllegalAccessException
```
    Deprecated. Use registerReducers instead.
    
    Override to register any potential aggregators used by this piece.
    
    Throws:
    
    InstantiationException
    
    IllegalAccessException
  - iterator
```
public final Iterator<AbstractPiece> iterator()
```
    Description copied from interface: Block
    
    Create iterator representing all pieces needed to be executed in this block. After Iterator.next call returns, master compute of returned Piece is guaranteed to be called before calling hasNext/next on the iterator. (allows for iterators logic to depend on the execution dynamically, and not be only static)
    
    Specified by:
    
    iterator in interface Iterable<AbstractPiece>
    
    Specified by:
    
    iterator in interface Block
  - forAllPossiblePieces
```
public void forAllPossiblePieces(Consumer<AbstractPiece> consumer)
```
    Description copied from interface: Block
    
    Calls consumer for each Piece: - in no particular order - potentially calling multiple times on same Piece - even if Piece might never be returned in the iterator - it will be called at least once for every piece that is going to be returned by iterator Can be used for static analysis/introspection of the block, without actually executing them.
    
    Specified by:
    
    forAllPossiblePieces in interface Block
  - getPieceCount
```
public PieceCount getPieceCount()
```
    Description copied from interface: Block
    
    How many pieces are in this block. Sometimes we don't know (eg RepeatBlock).
    
    Specified by:
    
    getPieceCount in interface Block
    
    Returns:
    
    How many pieces are in this block.
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - hashCode
```
public final int hashCode()
```
    Overrides:
    
    hashCode in class Object
  - equals
```
public final boolean equals(Object obj)
```
    Overrides:
    
    equals in class Object

Class AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface java.lang.Iterable

Constructor Detail

AbstractPiece

Method Detail

wrappedRegisterReducers

getWrappedVertexSender

workerContextSend

masterCompute

workerContextReceive

getVertexReceiver

getMessageClasses

nextExecutionStage

registerAggregators

iterator

forAllPossiblePieces

getPieceCount

toString

hashCode

equals