org.apache.giraph.zk

Class GiraphZooKeeperAdmin

java.lang.Object

org.apache.giraph.zk.GiraphZooKeeperAdmin

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool, org.apache.zookeeper.Watcher

public class GiraphZooKeeperAdmin
extends Object
implements org.apache.zookeeper.Watcher, org.apache.hadoop.util.Tool

A Utility class to be used by Giraph admins to occasionally clean up the ZK remnants of jobs that have failed or were killed before finishing. Usage (note that defaults are used if giraph.XYZ args are missing): bin/giraph-admin -Dgiraph.zkBaseNode=... -Dgiraph.zkList=... -Dgiraph.zkServerPort=... -cleanZk alterantely, the Configuration file will populate these fields as it would in a bin/giraph run. WARNING: Obviously, running this while actual Giraph jobs using your cluster are in progress is not recommended.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher

org.apache.zookeeper.Watcher.Event

Constructor Summary

Constructors

Constructor and Description
GiraphZooKeeperAdmin()

Method Summary

Modifier and Type	Method and Description
void	doZooKeeperCleanup(ZooKeeperExt zooKeeper, String zkBasePath) Cleans the ZooKeeper quorum of in-memory failed/killed job fragments.
org.apache.hadoop.conf.Configuration	getConf()
static void	main(String[] args) Entry point from shell script
void	process(org.apache.zookeeper.WatchedEvent event) Implement watcher to receive event at the end of the cleaner run
int	run(String[] args) Clean the ZooKeeper of all failed and cancelled in-memory job remnants that pile up on the ZK quorum over time.
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GiraphZooKeeperAdmin

public GiraphZooKeeperAdmin()

Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

run

public int run(String[] args)

Clean the ZooKeeper of all failed and cancelled in-memory job remnants that pile up on the ZK quorum over time.

Specified by:

run in interface org.apache.hadoop.util.Tool

Parameters:

args - the input command line arguments, if any.

Returns:

the System.exit value to return to the console.

process

public final void process(org.apache.zookeeper.WatchedEvent event)

Implement watcher to receive event at the end of the cleaner run

Specified by:

process in interface org.apache.zookeeper.Watcher

Parameters:

event - the WatchedEvent returned by ZK after the cleaning job.

doZooKeeperCleanup

public void doZooKeeperCleanup(ZooKeeperExt zooKeeper,
                               String zkBasePath)
                        throws org.apache.zookeeper.KeeperException,
                               InterruptedException

Cleans the ZooKeeper quorum of in-memory failed/killed job fragments.

Parameters:

zooKeeper - the connected ZK instance (session) to delete from.

zkBasePath - the base node to begin erasing from.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

main

public static void main(String[] args)
                 throws Exception

Entry point from shell script

Parameters:

args - the command line arguments

Throws:

Exception