Package | Description |
---|---|
org.apache.giraph.block_app.framework.block |
Block interface as a composable unit of execution, and its common
implementations.
|
org.apache.giraph.block_app.framework.piece.delegate |
Pieces that delegate their work to a set of one or multiple other Pieces.
|
org.apache.giraph.block_app.library |
Core library of Pieces and Suppliers, providing most common usages.
|
org.apache.giraph.block_app.library.internal |
Internal implementation of Pieces needed for Pieces utility class.
|
org.apache.giraph.function |
Package for interfaces representing different functions,
that all extends Serializable in order for Kryo to be able
to serialize them.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createReceiveFiltering(SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block innerBlock)
Creates filtering block, that filters only vertexReceive function,
and always calls vertexSend function.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createSendFiltering(SupplierFromVertex<I,V,E,Boolean> toCallSend,
Block innerBlock)
Creates filtering block, that filters only vertexSend function,
and always calls vertexReceive function.
|
Constructor and Description |
---|
FilteringBlock(SupplierFromVertex<I,V,E,Boolean> toCallSendAndReceive,
Block block)
Creates filtering block, where both vertexSend and vertexReceive is
filtered based on same supplier.
|
FilteringBlock(SupplierFromVertex<I,V,E,Boolean> toCallSend,
SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block block)
Creates filtering block which uses passed
toCallSend to filter
calls to vertexSend , and passed toCallReceive to filter
calls to vertexReceive , on all pieces within passed block . |
FilteringBlock(SupplierFromVertex<I,V,E,Boolean> toCallSend,
SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block block)
Creates filtering block which uses passed
toCallSend to filter
calls to vertexSend , and passed toCallReceive to filter
calls to vertexReceive , on all pieces within passed block . |
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S> |
FilteringPiece.createReceiveFiltering(SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallReceive,
AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)
Creates filtering piece, that filters only vertexReceive function,
and always calls vertexSend function.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S> |
FilteringPiece.createSendFiltering(SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallSend,
AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)
Creates filtering block, that filters only vertexSend function,
and always calls vertexReceive function.
|
Constructor and Description |
---|
FilteringPiece(SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallSendAndReceive,
AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)
Creates filtering piece, where both vertexSend and vertexReceive is
filtered based on same supplier.
|
FilteringPiece(SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallSend,
SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallReceive,
AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)
Creates filtering piece which uses passed
toCallSend to filter
calls to vertexSend , and passed toCallReceive to filter
calls to vertexReceive , on passed innerPiece . |
FilteringPiece(SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallSend,
SupplierFromVertex<? super I,? super V,? super E,Boolean> toCallReceive,
AbstractPiece<? super I,? super V,? super E,? super M,? super WV,? super WM,? super S> innerPiece)
Creates filtering piece which uses passed
toCallSend to filter
calls to vertexSend , and passed toCallReceive to filter
calls to vertexReceive , on passed innerPiece . |
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexEdgesSupplier()
Supplier which extracts and returns edges object.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexIdSupplier()
Supplier which extracts and returns vertex ID.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexNeighborsSupplier()
Supplier which extracts and returns Iterator over all neighbor IDs.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexNeighborsSupplier(Predicate<I> toSupply)
Supplier which extracts and returns Iterator over neighbor IDs
that return true for given predicate.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexNeighborsSupplierWithIndex(PairPredicate<PrimitiveRefs.IntRef,I> toSupply)
Supplier which gives Iterator over neighbor IDs that return true for given
predicate over (index, target)
Note - iterator returns reused object, so you need to "use" them,
before calling next() again.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
VertexSuppliers.vertexValueSupplier()
Supplier which extracts and returns vertex value.
|
Modifier and Type | Method and Description |
---|---|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduce(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
Consumer<R> reducedValueConsumer)
Creates single reducer piece - given reduce class, supplier of values on
worker, reduces and passes the result to given consumer on master.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceAndBroadcast(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
ConsumerWithVertex<I,V,E,R> reducedValueConsumer)
Creates single reducer and broadcast piece - given reduce class, supplier
of values on worker, reduces and broadcasts the value, passing it to the
consumer on worker for each vertex.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceAndBroadcastWithArrayOfHandles(String name,
int numHandles,
Supplier<ReduceOperation<S,R>> reduceOp,
SupplierFromVertex<I,V,E,Long> handleHashSupplier,
SupplierFromVertex<I,V,E,S> valueSupplier,
ConsumerWithVertex<I,V,E,R> reducedValueConsumer)
Like reduceAndBroadcast, but uses array of handles for reducers and
broadcasts, to make it feasible and performant when values are large.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceAndBroadcastWithArrayOfHandles(String name,
int numHandles,
Supplier<ReduceOperation<S,R>> reduceOp,
SupplierFromVertex<I,V,E,Long> handleHashSupplier,
SupplierFromVertex<I,V,E,S> valueSupplier,
ConsumerWithVertex<I,V,E,R> reducedValueConsumer)
Like reduceAndBroadcast, but uses array of handles for reducers and
broadcasts, to make it feasible and performant when values are large.
|
static <S,R extends org.apache.hadoop.io.Writable,I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.reduceWithMaster(String name,
ReduceOperation<S,R> reduceOp,
SupplierFromVertex<I,V,E,S> valueSupplier,
PairConsumer<R,BlockMasterApi> reducedValueConsumer)
Creates single reducer piece - given reduce class, supplier of values on
worker, reduces and passes the result to given consumer on master.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
Pieces.removeVertices(String pieceName,
SupplierFromVertex<I,V,E,Boolean> shouldRemoveVertex)
Creates Piece which removes vertices for which supplier returns true.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessage(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier,
ConsumerWithVertex<I,V,E,Iterable<M>> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all targets provided by targetsSupplier.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessage(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier,
ConsumerWithVertex<I,V,E,Iterable<M>> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all targets provided by targetsSupplier.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessage(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier,
ConsumerWithVertex<I,V,E,M> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all targets provided by targetsSupplier,
and uses given messageCombiner to combine messages together.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessage(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier,
ConsumerWithVertex<I,V,E,M> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all targets provided by targetsSupplier,
and uses given messageCombiner to combine messages together.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessageToNeighbors(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier,
ConsumerWithVertex<I,V,E,Iterable<M>> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all neighbors of current vertex.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
Pieces.sendMessageToNeighbors(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier,
ConsumerWithVertex<I,V,E,M> messagesConsumer)
Creates Piece that for each vertex, sends message provided by
messageSupplier to all neighbors of current vertex,
and uses given messageCombiner to combine messages together.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSend(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Start chain with sending message provided by messageSupplier to all
targets provided by targetsSupplier.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSend(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Start chain with sending message provided by messageSupplier to all
targets provided by targetsSupplier.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSend(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Start chain with sending message provided by messageSupplier to all
targets provided by targetsSupplier, and use given messageCombiner to
combine messages together.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSend(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Start chain with sending message provided by messageSupplier to all
targets provided by targetsSupplier, and use given messageCombiner to
combine messages together.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSendToNeighbors(String name,
Class<M> messageClass,
SupplierFromVertex<I,V,E,M> messageSupplier)
Start chain with sending message provided by messageSupplier to all
neighbors of a current vertex.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable> |
SendMessageChain.startSendToNeighbors(String name,
MessageCombiner<? super I,M> messageCombiner,
SupplierFromVertex<I,V,E,M> messageSupplier)
Start chain with sending message provided by messageSupplier to all
neighbors of a current vertex, and use given messageCombiner to
combine messages together.
|
<M extends org.apache.hadoop.io.Writable> |
SendMessageChain.thenSend(String name,
Class<M> messageClass,
FunctionWithVertex<I,V,E,P,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Give previously received message(s) to messageSupplier, and send message
it returns to all targets provided by targetsSupplier.
|
<M extends org.apache.hadoop.io.Writable> |
SendMessageChain.thenSend(String name,
MessageCombiner<? super I,M> messageCombiner,
FunctionWithVertex<I,V,E,P,M> messageSupplier,
SupplierFromVertex<I,V,E,Iterator<I>> targetsSupplier)
Give previously received message(s) to messageSupplier, and send message
it returns to all targets provided by targetsSupplier, and use given
messageCombiner to combine messages together.
|
Modifier and Type | Method and Description |
---|---|
Block |
SendMessageChain.endCustom(Function<SupplierFromVertex<I,V,E,P>,Block> createBlockToAttach)
End chain by providing a function that will produce Block to be attached
to the end of current chain, given a supplier of messages received
within the last link in the chain.
|
Modifier and Type | Method and Description |
---|---|
<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
ObjectTransfer.castToSupplier()
To be called when needing to pass it as a Supplier - making it
obvious that K, V and E on supplier side can be any types,
and to make code work without compile errors/warnings.
|
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.