org.apache.giraph.io.formats

Class SequenceFileVertexOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,OK extends org.apache.hadoop.io.Writable,OV extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.io.OutputFormat<I,V,E>

org.apache.giraph.io.VertexOutputFormat<I,V,E>

org.apache.giraph.io.formats.SequenceFileVertexOutputFormat<I,V,E,OK,OV>

Type Parameters:

I - Vertex id type

V - Vertex value type

E - Edge value type

OK - Output key data type for a sequence file

OV - Output value data type for a sequence file

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>

public abstract class SequenceFileVertexOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,OK extends org.apache.hadoop.io.Writable,OV extends org.apache.hadoop.io.Writable>
extends VertexOutputFormat<I,V,E>

Sequence file vertex output format. It allows to convert a vertex into a key and value pair of desired types, and output the pair into a sequence file. A subclass has to provide two conversion methods convertToSequenceFileKey() and convertToSequenceFileValue().

Constructor Summary

Constructors

Constructor and Description
SequenceFileVertexOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Check for validity of the output-specification for the job.
protected abstract OK	convertToSequenceFileKey(I vertexId) Converts a vertex identifier into a sequence file key.
protected abstract OV	convertToSequenceFileValue(V vertexValue) Converts a vertex value into a sequence file value.
VertexWriter	createVertexWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Create a vertex writer for a given split.
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the output committer for this output format.

Methods inherited from class org.apache.giraph.io.OutputFormat

postWriting, preWriting

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequenceFileVertexOutputFormat

public SequenceFileVertexOutputFormat()

Method Detail

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Description copied from class: OutputFormat

Check for validity of the output-specification for the job. (Copied from Hadoop OutputFormat)

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Specified by:

checkOutputSpecs in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - information about the job

Throws:

IOException - when output should not be attempted

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Description copied from class: OutputFormat

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly. (Copied from Hadoop OutputFormat)

Specified by:

getOutputCommitter in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the task context

Returns:

an output committer

Throws:

IOException

InterruptedException

createVertexWriter

public VertexWriter createVertexWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                throws IOException,
                                       InterruptedException

Description copied from class: VertexOutputFormat

Create a vertex writer for a given split. The framework will call VertexWriter.initialize(TaskAttemptContext) before the split is used.

Specified by:

createVertexWriter in class VertexOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the information about the task

Returns:

a new vertex writer

Throws:

IOException

InterruptedException

convertToSequenceFileKey

protected abstract OK convertToSequenceFileKey(I vertexId)

Converts a vertex identifier into a sequence file key.

Parameters:

vertexId - Vertex identifier.

Returns:

Sequence file key.

convertToSequenceFileValue

protected abstract OV convertToSequenceFileValue(V vertexValue)

Converts a vertex value into a sequence file value.

Parameters:

vertexValue - Vertex value.

Returns:

Sequence file value.