org.apache.giraph.bsp
Interface CentralizedServiceWorker<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>

Type Parameters:

I - Vertex id

V - Vertex value

E - Edge value

M - Message data

All Superinterfaces:

CentralizedService<I,V,E,M>

All Known Implementing Classes:

BspServiceWorker

public interface CentralizedServiceWorker<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,M extends org.apache.hadoop.io.Writable>
extends CentralizedService<I,V,E,M>

All workers should have access to this centralized service to execute the following methods.

Method Summary

void	cleanup(FinishedSuperstepStats finishedSuperstepStats) Clean up the service (no calls may be issued after this)
void	exchangeVertexPartitions(Collection<? extends PartitionOwner> masterSetPartitionOwners) If desired by the user, vertex partitions are redistributed among workers according to the chosen WorkerGraphPartitioner.
void	failureCleanup() Operations that will be called if there is a failure by a worker.
FinishedSuperstepStats	finishSuperstep(GraphState<I,V,E,M> graphState, List<PartitionStats> partitionStatsList) Worker is done with its portion of the superstep.
WorkerAggregatorHandler	getAggregatorHandler() Get worker aggregator handler
GraphTaskManager<I,V,E,M>	getGraphTaskManager() Get the GraphTaskManager that this service is using.
MasterInfo	getMasterInfo() Get master info
Integer	getPartitionId(I vertexId) Get the partition id that a vertex id would belong to.
Iterable<? extends PartitionOwner>	getPartitionOwners() Get all partition owners.
PartitionStore<I,V,E,M>	getPartitionStore() Get the partition store for this worker.
ServerData<I,V,E,M>	getServerData() Get server data
SuperstepOutput<I,V,E>	getSuperstepOutput() Get the superstep output class
PartitionOwner	getVertexPartitionOwner(I vertexId) Every client will need to get a partition owner from a vertex id so that they know which worker to sent the request to.
WorkerClient<I,V,E,M>	getWorkerClient() Get the worker client (for instantiating WorkerClientRequestProcessor instances.
WorkerContext	getWorkerContext() Get the worker context.
WorkerInfo	getWorkerInfo() Get the worker information
WorkerObserver[]	getWorkerObservers() Get the observers for this Worker.
boolean	hasPartition(Integer partitionId) Whether a partition with given id exists on this worker.
VertexEdgeCount	loadCheckpoint(long superstep) Load the vertices, edges, messages from the beginning of a superstep.
void	prepareSuperstep() Final preparation for superstep, called after startSuperstep and potential loading from checkpoint, right before the computation started TODO how to avoid this additional function
FinishedSuperstepStats	setup() Setup (must be called prior to any other function)
Collection<? extends PartitionOwner>	startSuperstep(GraphState<I,V,E,M> graphState) Take all steps prior to actually beginning the computation of a superstep.
void	storeCheckpoint() Both the vertices and the messages need to be checkpointed in order for them to be used.

Methods inherited from interface org.apache.giraph.bsp.CentralizedService

checkpointFrequencyMet, getRestartedSuperstep, getSuperstep, getWorkerInfoList

Method Detail

setup

FinishedSuperstepStats setup()

Setup (must be called prior to any other function)

Returns:

Finished superstep stats for the input superstep

getWorkerInfo

WorkerInfo getWorkerInfo()

Get the worker information

Returns:

Worker information

getWorkerClient

WorkerClient<I,V,E,M> getWorkerClient()

Get the worker client (for instantiating WorkerClientRequestProcessor instances.

Returns:

Worker client

getWorkerContext

WorkerContext getWorkerContext()

Get the worker context.

Returns:

worker's WorkerContext

getWorkerObservers

WorkerObserver[] getWorkerObservers()

Get the observers for this Worker.

Returns:

array of WorkerObservers.

getPartitionStore

PartitionStore<I,V,E,M> getPartitionStore()

Get the partition store for this worker. The partitions contain the vertices for this worker and can be used to run compute() for the vertices or do checkpointing.

Returns:

The partition store for this worker.

storeCheckpoint

void storeCheckpoint()
                     throws IOException

Both the vertices and the messages need to be checkpointed in order for them to be used. This is done after all messages have been delivered, but prior to a superstep starting.

Throws:

IOException

loadCheckpoint

VertexEdgeCount loadCheckpoint(long superstep)
                               throws IOException

Load the vertices, edges, messages from the beginning of a superstep. Will load the vertex partitions as designated by the master and set the appropriate superstep.

Parameters:

superstep - which checkpoint to use

Returns:

Graph-wide vertex and edge counts

Throws:

IOException

startSuperstep

Collection<? extends PartitionOwner> startSuperstep(GraphState<I,V,E,M> graphState)

Take all steps prior to actually beginning the computation of a superstep.

Parameters:

graphState - Current graph state

Returns:

Collection of all the partition owners from the master for this superstep.

finishSuperstep

FinishedSuperstepStats finishSuperstep(GraphState<I,V,E,M> graphState,
                                       List<PartitionStats> partitionStatsList)

Worker is done with its portion of the superstep. Report the worker level statistics after the computation.

Parameters:

graphState - Current graph state

partitionStatsList - All the partition stats for this worker

Returns:

Stats of the superstep completion

getPartitionId

Integer getPartitionId(I vertexId)

Get the partition id that a vertex id would belong to.

Parameters:

vertexId - Vertex id

Returns:

Partition id

hasPartition

boolean hasPartition(Integer partitionId)

Whether a partition with given id exists on this worker.

Parameters:

partitionId - Partition id

Returns:

True iff this worker has the specified partition

getVertexPartitionOwner

PartitionOwner getVertexPartitionOwner(I vertexId)

Every client will need to get a partition owner from a vertex id so that they know which worker to sent the request to.

Parameters:

vertexId - Vertex index to look for

Returns:

PartitionOnwer that should contain this vertex if it exists

getPartitionOwners

Iterable<? extends PartitionOwner> getPartitionOwners()

Get all partition owners.

Returns:

Iterable through partition owners

exchangeVertexPartitions

void exchangeVertexPartitions(Collection<? extends PartitionOwner> masterSetPartitionOwners)

If desired by the user, vertex partitions are redistributed among workers according to the chosen WorkerGraphPartitioner.

Parameters:

masterSetPartitionOwners - Partition owner info passed from the master.

getMasterInfo

MasterInfo getMasterInfo()

Get master info

Returns:

Master info

getGraphTaskManager

GraphTaskManager<I,V,E,M> getGraphTaskManager()

Get the GraphTaskManager that this service is using. Vertices need to know this.

Returns:

the GraphTaskManager instance for this compute node

failureCleanup

void failureCleanup()

Operations that will be called if there is a failure by a worker.

getServerData

ServerData<I,V,E,M> getServerData()

Get server data

Returns:

Server data

getAggregatorHandler

WorkerAggregatorHandler getAggregatorHandler()

Get worker aggregator handler

Returns:

Worker aggregator handler

prepareSuperstep

void prepareSuperstep()

Final preparation for superstep, called after startSuperstep and potential loading from checkpoint, right before the computation started TODO how to avoid this additional function

getSuperstepOutput

SuperstepOutput<I,V,E> getSuperstepOutput()

Get the superstep output class

Returns:

SuperstepOutput

cleanup

void cleanup(FinishedSuperstepStats finishedSuperstepStats)
             throws IOException,
                    InterruptedException

Clean up the service (no calls may be issued after this)

Parameters:

finishedSuperstepStats - Finished supestep stats

Throws:

IOException

InterruptedException