Package | Description |
---|---|
org.apache.giraph.block_app.framework |
Block application abstraction package.
|
org.apache.giraph.block_app.framework.api.local |
Local (single machine) implementation of graph processing system API used by
Blocks Framework.
|
org.apache.giraph.block_app.framework.block |
Block interface as a composable unit of execution, and its common
implementations.
|
org.apache.giraph.block_app.framework.internal |
Implementation of execution logic, guiding internal execution of
Block Application.
|
org.apache.giraph.block_app.framework.piece |
Single execution object - Piece, and related classes.
|
org.apache.giraph.block_app.framework.piece.delegate |
Pieces that delegate their work to a set of one or multiple other Pieces.
|
org.apache.giraph.block_app.library |
Core library of Pieces and Suppliers, providing most common usages.
|
org.apache.giraph.block_app.library.gc |
GC utility pieces.
|
org.apache.giraph.block_app.library.internal |
Internal implementation of Pieces needed for Pieces utility class.
|
org.apache.giraph.block_app.library.iteration |
Utilities for having iteration within execution stage object.
|
org.apache.giraph.block_app.library.striping |
Utilities for striping.
|
org.apache.giraph.block_app.migration |
Migration utility for transforming standard Giraph applications, into
Block Applications.
|
org.apache.giraph.examples.block_app |
Package of Giraph examples for Block Applications.
|
Modifier and Type | Method and Description |
---|---|
Block |
BlockFactory.createBlock(GiraphConfiguration conf)
Create a block (representing a full Giraph job), based on the given
configuration.
|
Modifier and Type | Method and Description |
---|---|
static void |
BlockUtils.checkBlockTypes(Block executionBlock,
Object executionStage,
ImmutableClassesGiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
LocalBlockRunner.runBlock(TestGraph<I,V,E> graph,
Block block,
Object executionStage)
Run Block from a specified execution stage on a given graph,
locally, in-memory.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
LocalBlockRunner.runBlockWithVertexOutput(Block block,
Object executionStage,
TestGraph<I,V,E> graph,
SimpleVertexWriter<I,V,E> vertexSaver)
Run Block from a specified execution stage on a given graph,
locally, in-memory, with a given vertexSaver.
|
Modifier and Type | Interface and Description |
---|---|
interface |
BlockWithApiHandle
Applications that need access to a
BlockApiHandle should return a
Block of this type. |
Modifier and Type | Class and Description |
---|---|
class |
EmptyBlock
Block without any pieces
|
class |
FilteringBlock<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
Block which filters out calls to vertexSend/vertexReceive functions
of all pieces in a given block.
|
class |
IfBlock
Block that executes one of two branches based on a condition
|
class |
RepeatBlock
Block that repeats another block given number of times.
|
class |
RepeatUntilBlock
Block that repeats another block until toQuit supplier returns true,
but at most given number of times.
|
class |
SequenceBlock
Block that executes provided blocks sequentially.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createReceiveFiltering(SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block innerBlock)
Creates filtering block, that filters only vertexReceive function,
and always calls vertexSend function.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createSendFiltering(SupplierFromVertex<I,V,E,Boolean> toCallSend,
Block innerBlock)
Creates filtering block, that filters only vertexSend function,
and always calls vertexReceive function.
|
static Block |
RepeatBlock.unlimited(Block block)
Create a repeat block that executes unlimited number of times.
|
static Block |
RepeatUntilBlock.unlimited(Block block,
Supplier<Boolean> toQuit)
Repeat unlimited number of times, until toQuit supplier returns true.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createReceiveFiltering(SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block innerBlock)
Creates filtering block, that filters only vertexReceive function,
and always calls vertexSend function.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable> |
FilteringBlock.createSendFiltering(SupplierFromVertex<I,V,E,Boolean> toCallSend,
Block innerBlock)
Creates filtering block, that filters only vertexSend function,
and always calls vertexReceive function.
|
static Block |
RepeatBlock.unlimited(Block block)
Create a repeat block that executes unlimited number of times.
|
static Block |
RepeatUntilBlock.unlimited(Block block,
Supplier<Boolean> toQuit)
Repeat unlimited number of times, until toQuit supplier returns true.
|
Constructor and Description |
---|
FilteringBlock(SupplierFromVertex<I,V,E,Boolean> toCallSendAndReceive,
Block block)
Creates filtering block, where both vertexSend and vertexReceive is
filtered based on same supplier.
|
FilteringBlock(SupplierFromVertex<I,V,E,Boolean> toCallSend,
SupplierFromVertex<I,V,E,Boolean> toCallReceive,
Block block)
Creates filtering block which uses passed
toCallSend to filter
calls to vertexSend , and passed toCallReceive to filter
calls to vertexReceive , on all pieces within passed block . |
IfBlock(Supplier<Boolean> condition,
Block thenBlock) |
IfBlock(Supplier<Boolean> condition,
Block thenBlock,
Block elseBlock) |
RepeatBlock(int repeatTimes,
Block block) |
RepeatBlock(IntSupplier repeatTimes,
Block block)
Creates a repeat block, that before starting execution takes number of
iterations from the given supplier.
|
RepeatUntilBlock(int repeatTimes,
Block block,
Supplier<Boolean> toQuit) |
SequenceBlock(Block... blocks) |
Constructor and Description |
---|
SequenceBlock(List<? extends Block> blocks) |
Modifier and Type | Method and Description |
---|---|
void |
BlockMasterLogic.initialize(Block executionBlock,
S executionStage,
BlockMasterApi masterApi)
Initialize Master Logic to execute given block, starting
with given executionStage.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
Parent of all Pieces, contains comprehensive list of methods Piece
can support.
|
class |
DefaultParentPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
Additional abstract implementations for all pieces to be used.
|
class |
Piece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,S>
Piece that should be extended in common usecases, when we want to be:
- sending and then receiving messages from vertices
- sending data to be aggregated from workers to master
- sending values from master, via aggregators, to workers
(basically - we don't want to use WorkerContext)
|
class |
PieceWithWorkerContext<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
Piece that should be extended when WorkerContext is used.
|
Modifier and Type | Class and Description |
---|---|
class |
DelegatePiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
Delegate Piece which allows combining multiple pieces in same iteration:
new DelegatePiece(new LogicPiece(), new StatsPiece())
You should be careful when doing so, since those pieces must not interact,
and only one can send messages.
|
class |
FilteringPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable,WV,WM extends org.apache.hadoop.io.Writable,S>
Piece which uses a provided suppliers to decide whether or not to run
receive/send piece part on a certain vertex.
|
Modifier and Type | Method and Description |
---|---|
Block |
SendMessageChain.endConsume(ConsumerWithVertex<I,V,E,P> messagesConsumer)
End chain by processing messages received within the last link
in the chain.
|
Block |
SendMessageChain.endCustom(Function<SupplierFromVertex<I,V,E,P>,Block> createBlockToAttach)
End chain by providing a function that will produce Block to be attached
to the end of current chain, given a supplier of messages received
within the last link in the chain.
|
<S,R extends org.apache.hadoop.io.Writable> |
SendMessageChain.endReduce(String name,
ReduceOperation<S,R> reduceOp,
FunctionWithVertex<I,V,E,P,S> valueSupplier,
Consumer<R> reducedValueConsumer)
End chain by giving received messages to valueSupplier,
to produce value that should be reduced, and consumed on master
by reducedValueConsumer.
|
<S,R extends org.apache.hadoop.io.Writable> |
SendMessageChain.endReduceWithMaster(String name,
ReduceOperation<S,R> reduceOp,
FunctionWithVertex<I,V,E,P,S> valueSupplier,
PairConsumer<R,BlockMasterApi> reducedValueConsumer)
End chain by giving received messages to valueSupplier,
to produce value that should be reduced, and consumed on master
by reducedValueConsumer.
|
Modifier and Type | Method and Description |
---|---|
Block |
SendMessageChain.endCustom(Function<SupplierFromVertex<I,V,E,P>,Block> createBlockToAttach)
End chain by providing a function that will produce Block to be attached
to the end of current chain, given a supplier of messages received
within the last link in the chain.
|
static <I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,P extends org.apache.hadoop.io.Writable> |
SendMessageChain.startCustom(Function<ConsumerWithVertex<I,V,E,P>,Block> createStartingBlock)
Start chain by providing a function that will produce Block representing
beginning of the chain, given a consumer of messages send
by the last link in the created block.
|
Modifier and Type | Class and Description |
---|---|
class |
WorkerGCPiece
Dummy piece to hint System.gc()
|
Modifier and Type | Class and Description |
---|---|
class |
SendMessagePiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
Piece that sends a message provided through messageProducer to given set of
neighbors, and passes them to messagesConsumer.
|
class |
SendMessageWithCombinerPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
Piece that sends a message provided through messageProducer to given set of
neighbors, uses a message combiner and passes them to messagesConsumer.
|
Modifier and Type | Method and Description |
---|---|
Block |
SendMessageWithCombinerPiece.stripeByReceiver(int stripes,
Int2ObjFunction<Int2ObjFunction<Predicate<I>>> stripeSupplier)
Stripe message sending computation across multiple stripes, in
each stripe only part of the vertices will receive messages.
|
Block |
SendMessagePiece.stripeByReceiver(int stripes,
Int2ObjFunction<Int2ObjFunction<Predicate<I>>> stripeSupplier)
Stripe message sending computation across multiple stripes, in
each stripe only part of the vertices will receive messages.
|
Modifier and Type | Class and Description |
---|---|
class |
IterationCounterPiece
Piece that increments execution stage iteration.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable> |
StripingUtils.generateStripedBlock(int stripes,
Function<Predicate<I>,Block> blockGenerator,
Int2ObjFunction<Int2ObjFunction<Predicate<I>>> stripeSupplier)
Generate striped block, with given number of
stripes ,
using given blockGenerator to generate block for each stripe,
and using striping based on given stripeSupplier . |
static Block |
StripingUtils.generateStripedBlock(int stripes,
Function<Predicate<org.apache.hadoop.io.LongWritable>,Block> blockGenerator)
Generate striped block, with given number of
stripes ,
using given blockGenerator to generate block for each stripe. |
static Block |
StripingUtils.stripeBlockBySenders(int stripes,
Block block)
Stripe given block, by calling vertexSend only in it's corresponding
stripe.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable> |
StripingUtils.createSingleStripeBySendersFunction(Block block)
Given a block, creates a function that will given a predicate filter
calls to vertexSend function based on that predicate.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable> |
StripingUtils.createSingleStripeBySendersFunction(Block block)
Given a block, creates a function that will given a predicate filter
calls to vertexSend function based on that predicate.
|
static Block |
StripingUtils.stripeBlockBySenders(int stripes,
Block block)
Stripe given block, by calling vertexSend only in it's corresponding
stripe.
|
Modifier and Type | Method and Description |
---|---|
static <I extends org.apache.hadoop.io.WritableComparable> |
StripingUtils.generateStripedBlock(int stripes,
Function<Predicate<I>,Block> blockGenerator,
Int2ObjFunction<Int2ObjFunction<Predicate<I>>> stripeSupplier)
Generate striped block, with given number of
stripes ,
using given blockGenerator to generate block for each stripe,
and using striping based on given stripeSupplier . |
static Block |
StripingUtils.generateStripedBlock(int stripes,
Function<Predicate<org.apache.hadoop.io.LongWritable>,Block> blockGenerator)
Generate striped block, with given number of
stripes ,
using given blockGenerator to generate block for each stripe. |
Modifier and Type | Class and Description |
---|---|
class |
MigrationPiece<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,MPrev extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
Piece used when migrating applications to Blocks Framework.
|
Modifier and Type | Method and Description |
---|---|
<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,MR extends org.apache.hadoop.io.Writable,MS extends org.apache.hadoop.io.Writable> |
MigrationFullBlockFactory.createMigrationAppBlock(Class<? extends MigrationAbstractComputation.MigrationFullAbstractComputation<I,V,E,MR,MS>> computationClass,
MigrationMasterCompute.MigrationFullMasterCompute masterCompute,
Class<MS> messageClass,
Class<? extends MessageCombiner<? super I,MS>> messageCombinerClass,
GiraphConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
Block |
SimpleMigrationMasterBlockFactory.createBlock(GiraphConfiguration conf) |
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.