org.apache.giraph.utils

Class DistributedCacheUtils

java.lang.Object

org.apache.giraph.utils.DistributedCacheUtils

public class DistributedCacheUtils
extends Object

Helpers for dealing with DistributedCache

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	copyAndAdd(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Copy a file to HDFS if it is local, and adds it to the distributed cache.
static org.apache.hadoop.fs.Path	copyToHdfs(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Copy a file to HDFS if it is local.
static com.google.common.base.Optional<org.apache.hadoop.fs.Path>	getLocalCacheFile(org.apache.hadoop.conf.Configuration conf, String pathToMatch) Get local path to file from a DistributedCache.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getLocalCacheFile

public static com.google.common.base.Optional<org.apache.hadoop.fs.Path> getLocalCacheFile(org.apache.hadoop.conf.Configuration conf,
                                                                                           String pathToMatch)

Get local path to file from a DistributedCache.

Parameters:

conf - Configuration

pathToMatch - Path that was used to insert into DistributedCache

Returns:

Path matched, or Optional.absent()

copyToHdfs

public static org.apache.hadoop.fs.Path copyToHdfs(org.apache.hadoop.fs.Path path,
                                                   org.apache.hadoop.conf.Configuration conf)

Copy a file to HDFS if it is local. If the path is already in HDFS, this call does nothing.

Parameters:

path - path to file

conf - Configuration

Returns:

path to file on HDFS.

copyAndAdd

public static org.apache.hadoop.fs.Path copyAndAdd(org.apache.hadoop.fs.Path path,
                                                   org.apache.hadoop.conf.Configuration conf)

Copy a file to HDFS if it is local, and adds it to the distributed cache.

Parameters:

path - path to file

conf - Configuration

Returns:

remote path to file