org.apache.giraph.bsp

Interface CentralizedServiceMaster<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Type Parameters:

I - Vertex id

V - Vertex value

E - Edge value

All Superinterfaces:

CentralizedService<I,V,E>

All Known Implementing Classes:

BspServiceMaster

public interface CentralizedServiceMaster<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends CentralizedService<I,V,E>

At most, there will be one active master at a time, but many threads can be trying to be the active master.

Method Summary

Modifier and Type	Method and Description
void	addGiraphTimersAndSendCounters(long superstep) Add the Giraph Timers to thirft counter struct, and send to the job client Counters include the Giraph Timers for setup, initialise, shutdown, total, and time for the given superstep
boolean	becomeMaster() Become the master.
List<WorkerInfo>	checkWorkers() Check all the WorkerInfo objects to ensure that a minimum number of good workers exists out of the total that have reported.
void	cleanup(SuperstepState superstepState) Clean up the service (no calls may be issued after this)
SuperstepState	coordinateSuperstep() Master coordinates the superstep
int	createEdgeInputSplits() Create the BspInputSplit objects from the index range based on the user-defined EdgeInputFormat.
int	createMappingInputSplits() Create the BspInputSplit objects from the index range based on the user-defined MappingInputFormat.
int	createVertexInputSplits() Create the BspInputSplit objects from the index range based on the user-defined VertexInputFormat.
void	failureCleanup(Exception e) Called when the job fails in order to let the Master do any cleanup.
AggregatorToGlobalCommTranslation	getAggregatorTranslationHandler() Handler for aggregators to reduce/broadcast translation
MasterGlobalCommHandler	getGlobalCommHandler() Get handler for global communication
long	getLastGoodCheckpoint() Get the last known good checkpoint
MasterCompute	getMasterCompute() Get MasterCompute object
void	postApplication() Application has finished.
void	postSuperstep() Superstep has finished.
void	restartFromCheckpoint(long checkpoint) Master can decide to restart from the last good checkpoint if a worker fails during a superstep.
void	setJobState(ApplicationState state, long applicationAttempt, long desiredSuperstep) If the master decides that this job doesn't have the resources to continue, it can fail the job.
void	setup() Setup (must be called prior to any other function)

Methods inherited from interface org.apache.giraph.bsp.CentralizedService

getConfiguration, getJobProgressTracker, getMasterInfo, getRestartedSuperstep, getSuperstep, getWorkerInfoList

Method Detail

setup

void setup()

Setup (must be called prior to any other function)

becomeMaster

boolean becomeMaster()

Become the master.

Returns:

true if became the master, false if the application is done.

checkWorkers

List<WorkerInfo> checkWorkers()

Check all the WorkerInfo objects to ensure that a minimum number of good workers exists out of the total that have reported.

Returns:

List of of healthy workers such that the minimum has been met, otherwise null

createMappingInputSplits

int createMappingInputSplits()

Create the BspInputSplit objects from the index range based on the user-defined MappingInputFormat. The BspInputSplit objects will processed by the workers later on during the INPUT_SUPERSTEP.

Returns:

Number of splits. Returns -1 on failure to create valid input splits.

createVertexInputSplits

int createVertexInputSplits()

Create the BspInputSplit objects from the index range based on the user-defined VertexInputFormat. The BspInputSplit objects will processed by the workers later on during the INPUT_SUPERSTEP.

Returns:

Number of splits. Returns -1 on failure to create valid input splits.

createEdgeInputSplits

int createEdgeInputSplits()

Create the BspInputSplit objects from the index range based on the user-defined EdgeInputFormat. The BspInputSplit objects will processed by the workers later on during the INPUT_SUPERSTEP.

Returns:

Number of splits. Returns -1 on failure to create valid input splits.

coordinateSuperstep

SuperstepState coordinateSuperstep()
                            throws org.apache.zookeeper.KeeperException,
                                   InterruptedException

Master coordinates the superstep

Returns:

State of the application as a result of this superstep

Throws:

InterruptedException

org.apache.zookeeper.KeeperException

restartFromCheckpoint

void restartFromCheckpoint(long checkpoint)

Master can decide to restart from the last good checkpoint if a worker fails during a superstep.

Parameters:

checkpoint - Checkpoint to restart from

getLastGoodCheckpoint

long getLastGoodCheckpoint()
                    throws IOException

Get the last known good checkpoint

Returns:

Last good superstep number

Throws:

IOException

setJobState

void setJobState(ApplicationState state,
                 long applicationAttempt,
                 long desiredSuperstep)

If the master decides that this job doesn't have the resources to continue, it can fail the job. It can also designate what to do next. Typically this is mainly informative.

Parameters:

state - State of the application.

applicationAttempt - Attempt to start on

desiredSuperstep - Superstep to restart from (if applicable)

getGlobalCommHandler

MasterGlobalCommHandler getGlobalCommHandler()

Get handler for global communication

Returns:

Global communication handler

getAggregatorTranslationHandler

AggregatorToGlobalCommTranslation getAggregatorTranslationHandler()

Handler for aggregators to reduce/broadcast translation

Returns:

aggregator translation handler

getMasterCompute

MasterCompute getMasterCompute()

Get MasterCompute object

Returns:

MasterCompute object

postSuperstep

void postSuperstep()

Superstep has finished.

postApplication

void postApplication()

Application has finished.

failureCleanup

void failureCleanup(Exception e)

Called when the job fails in order to let the Master do any cleanup.

Parameters:

e - Exception job failed from. May be null.

cleanup

void cleanup(SuperstepState superstepState)
      throws IOException,
             InterruptedException

Clean up the service (no calls may be issued after this)

Parameters:

superstepState - what was the state of the last complete superstep?

Throws:

IOException

InterruptedException

addGiraphTimersAndSendCounters

void addGiraphTimersAndSendCounters(long superstep)

Add the Giraph Timers to thirft counter struct, and send to the job client Counters include the Giraph Timers for setup, initialise, shutdown, total, and time for the given superstep

Parameters:

superstep - superstep for which the GiraphTimer will be sent