org.apache.giraph.io.formats

Class PseudoRandomVertexInputFormat

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.io.GiraphInputFormat<I,V,E>

org.apache.giraph.io.VertexInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

org.apache.giraph.io.formats.PseudoRandomVertexInputFormat

All Implemented Interfaces:

GiraphConfigurationSettable<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>, ImmutableClassesGiraphConfigurable<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

public class PseudoRandomVertexInputFormat
extends VertexInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

This VertexInputFormat is meant for large scale testing. It allows the user to create an input data source that a variable number of aggregate vertices and edges per vertex that is repeatable for the exact same parameter (pseudo-random).

Constructor Summary

Constructors

Constructor and Description
PseudoRandomVertexInputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkInputSpecs(org.apache.hadoop.conf.Configuration conf) Check that input is valid.
VertexReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>	createVertexReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) Create a vertex reader for a given split.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context, int minSplitCountHint) Get the list of input splits for the format.

Methods inherited from class org.apache.giraph.io.GiraphInputFormat

readInputSplit, writeInputSplit

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PseudoRandomVertexInputFormat

public PseudoRandomVertexInputFormat()

Method Detail

checkInputSpecs

public void checkInputSpecs(org.apache.hadoop.conf.Configuration conf)

Description copied from class: GiraphInputFormat

Check that input is valid.

Specified by:

checkInputSpecs in class GiraphInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

Parameters:

conf - Configuration

getSplits

public final List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context,
                                                                    int minSplitCountHint)
                                                             throws IOException,
                                                                    InterruptedException

Description copied from class: GiraphInputFormat

Get the list of input splits for the format.

Specified by:

getSplits in class GiraphInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

Parameters:

context - The job context

minSplitCountHint - Minimum number of splits to create (hint)

Returns:

The list of input splits

Throws:

IOException

InterruptedException

createVertexReader

public VertexReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable> createVertexReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                                                  org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                           throws IOException

Description copied from class: VertexInputFormat

Create a vertex reader for a given split. Guaranteed to have been configured with setConf() prior to use. The framework will also call VertexReader.initialize(InputSplit, TaskAttemptContext) before the split is used.

Specified by:

createVertexReader in class VertexInputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.DoubleWritable,org.apache.hadoop.io.DoubleWritable>

Parameters:

split - the split to be read

context - the information about the task

Returns:

a new record reader

Throws:

IOException