org.apache.giraph.io.formats

Class TextVertexInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.io.GiraphInputFormat<I,V,E>

org.apache.giraph.io.VertexInputFormat<I,V,E>

org.apache.giraph.io.formats.TextVertexInputFormat<I,V,E>

Type Parameters:

I - Vertex index value

V - Vertex value

E - Edge value

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>

Direct Known Subclasses:

AdjacencyListTextVertexInputFormat, BrachaTouegDeadlockInputFormat, IntIntNullTextInputFormat, IntIntNullTextVertexInputFormat, IntNullNullTextInputFormat, JsonBase64VertexInputFormat, JsonLongDoubleFloatDoubleVertexInputFormat, LongDoubleDoubleTextInputFormat, LongDoubleFloatTextInputFormat, LongDoubleNullTextInputFormat, LongLongNullTextInputFormat, NormalizingLongDoubleDoubleTextInputFormat, SccLongLongNullTextInputFormat

public abstract class TextVertexInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends VertexInputFormat<I,V,E>

Abstract class that users should subclass to use their own text based vertex input format.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	TextVertexInputFormat.TextVertexReader Abstract class to be implemented by the user based on their specific vertex input.
protected class	TextVertexInputFormat.TextVertexReaderFromEachLine Abstract class to be implemented by the user to read a vertex from each text line.
protected class	TextVertexInputFormat.TextVertexReaderFromEachLineProcessed<T> Abstract class to be implemented by the user to read a vertex from each text line after preprocessing it.
protected class	TextVertexInputFormat.TextVertexReaderFromEachLineProcessedHandlingExceptions<T,X extends Throwable> Abstract class to be implemented by the user to read a vertex from each text line after preprocessing it with exception handling.

Field Summary

Fields

Modifier and Type	Field and Description
protected GiraphTextInputFormat	textInputFormat Uses the GiraphTextInputFormat to do everything

Constructor Summary

Constructors

Constructor and Description
TextVertexInputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkInputSpecs(org.apache.hadoop.conf.Configuration conf) Check that input is valid.
abstract TextVertexInputFormat.TextVertexReader	createVertexReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) The factory method which produces the TextVertexInputFormat.TextVertexReader used by this input format.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context, int minSplitCountHint) Get the list of input splits for the format.

Methods inherited from class org.apache.giraph.io.GiraphInputFormat

readInputSplit, writeInputSplit

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

textInputFormat

protected GiraphTextInputFormat textInputFormat

Uses the GiraphTextInputFormat to do everything

Constructor Detail

TextVertexInputFormat

public TextVertexInputFormat()

Method Detail

checkInputSpecs

public void checkInputSpecs(org.apache.hadoop.conf.Configuration conf)

Description copied from class: GiraphInputFormat

Check that input is valid.

Specified by:

checkInputSpecs in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

conf - Configuration

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context,
                                                              int minSplitCountHint)
                                                       throws IOException,
                                                              InterruptedException

Description copied from class: GiraphInputFormat

Get the list of input splits for the format.

Specified by:

getSplits in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - The job context

minSplitCountHint - Minimum number of splits to create (hint)

Returns:

The list of input splits

Throws:

IOException

InterruptedException

createVertexReader

public abstract TextVertexInputFormat.TextVertexReader createVertexReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                          org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                   throws IOException

The factory method which produces the TextVertexInputFormat.TextVertexReader used by this input format.

Specified by:

createVertexReader in class VertexInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

split - the split to be read

context - the information about the task

Returns:

the text vertex reader to be used

Throws:

IOException