org.apache.giraph.job

Class DefaultJobProgressTrackerService

java.lang.Object

org.apache.giraph.job.DefaultJobProgressTrackerService

All Implemented Interfaces:

JobProgressTracker, JobProgressTrackerService

public class DefaultJobProgressTrackerService
extends Object
implements JobProgressTrackerService

Default implementation of JobProgressTrackerService

Field Summary

Fields

Modifier and Type	Field and Description
static IntConfOption	MAX_ALLOWED_TIME_WITHOUT_PROGRESS_MS Max time job is allowed to not make progress before getting killed

Constructor Summary

Constructors

Constructor and Description
DefaultJobProgressTrackerService()

Method Summary

Modifier and Type	Method and Description
protected boolean	canFinishInTime(GiraphConfiguration conf, org.apache.hadoop.mapreduce.Job job, CombinedWorkerProgress progress) Determine if the job will finish in allotted time
static JobProgressTrackerService	createJobProgressTrackerService(GiraphConfiguration conf, GiraphJobObserver jobObserver) Create job progress server on job client if enabled in configuration.
void	init(GiraphConfiguration conf, GiraphJobObserver jobObserver) Initialize the service
protected void	jobGotAllMappers() Called when job got all mappers, used to check MAX_ALLOWED_JOB_TIME_MS and potentially start a thread which will kill the job after this time
protected boolean	killJobWithMessage(String message) Kill job with message describing why it's being killed
void	logError(String logLine, byte[] exByteArray) Call this when you want to log an error line and exception object from any mapper to command line KryoWritableWrapper.convertFromByteArray can be used to get exception object back
void	logFailure(String reason) Notify that job is failing
void	logInfo(String logLine) Call this when you want to log an info line from any mapper to command line
void	mapperStarted() Notify JobProgressTracker that mapper started
void	sendMasterCounters(GiraphCountersThriftStruct giraphCounters) Master should call this method after all supersteps are finished, and send the aggregated counters to the job client
void	setJob(org.apache.hadoop.mapreduce.Job job) Set job
void	stop(boolean succeeded) Stop the thread which logs application progress and server
void	updateMasterProgress(MasterProgress masterProgress) Master should call this method to update its progress
void	updateProgress(WorkerProgress workerProgress) Workers should call this method to update their progress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_ALLOWED_TIME_WITHOUT_PROGRESS_MS

public static final IntConfOption MAX_ALLOWED_TIME_WITHOUT_PROGRESS_MS

Max time job is allowed to not make progress before getting killed

Constructor Detail

DefaultJobProgressTrackerService

public DefaultJobProgressTrackerService()

Method Detail

init

public void init(GiraphConfiguration conf,
                 GiraphJobObserver jobObserver)

Description copied from interface: JobProgressTrackerService

Initialize the service

Specified by:

init in interface JobProgressTrackerService

Parameters:

conf - Configuration

jobObserver - Giraph job callbacks

canFinishInTime

protected boolean canFinishInTime(GiraphConfiguration conf,
                                  org.apache.hadoop.mapreduce.Job job,
                                  CombinedWorkerProgress progress)

Determine if the job will finish in allotted time

Parameters:

conf - Giraph configuration

job - Job

progress - Combined worker progress

Returns:

true it the job can finish in allotted time, false otherwise

killJobWithMessage

protected boolean killJobWithMessage(String message)

Kill job with message describing why it's being killed

Parameters:

message - Message describing why job is being killed

Returns:

True iff job was killed successfully, false if job was already done or kill failed

setJob

public void setJob(org.apache.hadoop.mapreduce.Job job)

Description copied from interface: JobProgressTrackerService

Set job

Specified by:

setJob in interface JobProgressTrackerService

Parameters:

job - Job

jobGotAllMappers

protected void jobGotAllMappers()

Called when job got all mappers, used to check MAX_ALLOWED_JOB_TIME_MS and potentially start a thread which will kill the job after this time

mapperStarted

public void mapperStarted()

Description copied from interface: JobProgressTracker

Notify JobProgressTracker that mapper started

Specified by:

mapperStarted in interface JobProgressTracker

logInfo

public void logInfo(String logLine)

Description copied from interface: JobProgressTracker

Call this when you want to log an info line from any mapper to command line

Specified by:

logInfo in interface JobProgressTracker

Parameters:

logLine - Line to log

logError

public void logError(String logLine,
                     byte[] exByteArray)

Description copied from interface: JobProgressTracker

Call this when you want to log an error line and exception object from any mapper to command line KryoWritableWrapper.convertFromByteArray can be used to get exception object back

Specified by:

logError in interface JobProgressTracker

Parameters:

logLine - Line to log

exByteArray - Exception byte array

logFailure

public void logFailure(String reason)

Description copied from interface: JobProgressTracker

Notify that job is failing

Specified by:

logFailure in interface JobProgressTracker

Parameters:

reason - Reason for failure

updateProgress

public void updateProgress(WorkerProgress workerProgress)

Description copied from interface: JobProgressTracker

Workers should call this method to update their progress

Specified by:

updateProgress in interface JobProgressTracker

Parameters:

workerProgress - Progress of the worker

updateMasterProgress

public void updateMasterProgress(MasterProgress masterProgress)

Description copied from interface: JobProgressTracker

Master should call this method to update its progress

Specified by:

updateMasterProgress in interface JobProgressTracker

Parameters:

masterProgress - Progress of the master

sendMasterCounters

public void sendMasterCounters(GiraphCountersThriftStruct giraphCounters)

Description copied from interface: JobProgressTracker

Master should call this method after all supersteps are finished, and send the aggregated counters to the job client

Specified by:

sendMasterCounters in interface JobProgressTracker

Parameters:

giraphCounters - Giraph-based counters

stop

public void stop(boolean succeeded)

Description copied from interface: JobProgressTrackerService

Stop the thread which logs application progress and server

Specified by:

stop in interface JobProgressTrackerService

Parameters:

succeeded - Whether job succeeded or not

createJobProgressTrackerService

public static JobProgressTrackerService createJobProgressTrackerService(GiraphConfiguration conf,
                                                                        GiraphJobObserver jobObserver)

Create job progress server on job client if enabled in configuration.

Parameters:

conf - Configuration

jobObserver - Giraph job callbacks

Returns:

JobProgressTrackerService