org.apache.giraph.io.internal

Class WrappedEdgeOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.io.OutputFormat<I,V,E>

org.apache.giraph.io.EdgeOutputFormat<I,V,E>

org.apache.giraph.io.internal.WrappedEdgeOutputFormat<I,V,E>

Type Parameters:

I - Vertex id

V - Vertex data

E - Edge data

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>

public class WrappedEdgeOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends EdgeOutputFormat<I,V,E>

For internal use only. Wraps user set EdgeOutputFormat to make sure proper configuration parameters are passed around, that user can set parameters in configuration and they will be available in other methods related to this format.

Constructor Summary

Constructors

Constructor and Description
WrappedEdgeOutputFormat(EdgeOutputFormat<I,V,E> edgeOutputFormat) Constructor

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Check for validity of the output-specification for the job.
EdgeWriter<I,V,E>	createEdgeWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Create an edge writer for a given split.
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get the output committer for this output format.
void	postWriting(org.apache.hadoop.mapreduce.TaskAttemptContext context) This method will be called after all writers are closed
void	preWriting(org.apache.hadoop.mapreduce.TaskAttemptContext context) This method will be called before creating any writers

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WrappedEdgeOutputFormat

public WrappedEdgeOutputFormat(EdgeOutputFormat<I,V,E> edgeOutputFormat)

Constructor

Parameters:

edgeOutputFormat - Edge output format to wrap

Method Detail

createEdgeWriter

public EdgeWriter<I,V,E> createEdgeWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                   throws IOException,
                                          InterruptedException

Description copied from class: EdgeOutputFormat

Create an edge writer for a given split. The framework will call EdgeWriter.initialize(TaskAttemptContext) before the split is used.

Specified by:

createEdgeWriter in class EdgeOutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the information about the task

Returns:

a new vertex writer

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Description copied from class: OutputFormat

Check for validity of the output-specification for the job. (Copied from Hadoop OutputFormat)

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Specified by:

checkOutputSpecs in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - information about the job

Throws:

IOException - when output should not be attempted

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Description copied from class: OutputFormat

Get the output committer for this output format. This is responsible for ensuring the output is committed correctly. (Copied from Hadoop OutputFormat)

Specified by:

getOutputCommitter in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the task context

Returns:

an output committer

Throws:

IOException

InterruptedException

preWriting

public void preWriting(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Description copied from class: OutputFormat

This method will be called before creating any writers

Overrides:

preWriting in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the task context

postWriting

public void postWriting(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Description copied from class: OutputFormat

This method will be called after all writers are closed

Overrides:

postWriting in class OutputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - the task context