org.apache.giraph.io.formats

Class TextVertexInputFormat.TextVertexReader

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.worker.WorkerAggregatorDelegator<I,V,E>

org.apache.giraph.io.VertexReader<I,V,E>

org.apache.giraph.io.formats.TextVertexInputFormat.TextVertexReader

All Implemented Interfaces:

AggregatorUsage, GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>, WorkerAggregatorUsage, WorkerBroadcastUsage, WorkerGlobalCommUsage, WorkerReduceUsage

Direct Known Subclasses:

LongDoubleDoubleTextInputFormat.LongDoubleDoubleDoubleVertexReader, LongDoubleFloatTextInputFormat.LongDoubleFloatVertexReader, LongDoubleNullTextInputFormat.LongDoubleNullDoubleVertexReader, NormalizingLongDoubleDoubleTextInputFormat.NormalizingLongDoubleDoubleDoubleVertexReader, TextVertexInputFormat.TextVertexReaderFromEachLine, TextVertexInputFormat.TextVertexReaderFromEachLineProcessed, TextVertexInputFormat.TextVertexReaderFromEachLineProcessedHandlingExceptions

Enclosing class:

TextVertexInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

protected abstract class TextVertexInputFormat.TextVertexReader
extends VertexReader<I,V,E>

Abstract class to be implemented by the user based on their specific vertex input. Easiest to ignore the key value separator and only use key instead. When reading a vertex from each line, extend TextVertexInputFormat.TextVertexReaderFromEachLine. If you need to preprocess each line first, then extend TextVertexInputFormat.TextVertexReaderFromEachLineProcessed. If you need common exception handling while preprocessing, then extend TextVertexInputFormat.TextVertexReaderFromEachLineProcessedHandlingExceptions.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	TextVertexReader()

Method Summary

Modifier and Type	Method and Description
void	close() Close this VertexReader to future operations.
protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>	createLineRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context) Create the line record reader.
protected org.apache.hadoop.mapreduce.TaskAttemptContext	getContext() Get the context.
float	getProgress() How much of the input has the VertexReader consumed i.e.
protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>	getRecordReader() Get the line record reader.
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context) Use the input split and context to setup reading the vertices.

Methods inherited from class org.apache.giraph.io.VertexReader

getCurrentVertex, nextVertex

Methods inherited from class org.apache.giraph.worker.WorkerAggregatorDelegator

aggregate, getAggregatedValue, getBroadcast, reduce, reduceMerge, setWorkerGlobalCommUsage

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextVertexReader

protected TextVertexReader()

Method Detail

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws IOException,
                       InterruptedException

Description copied from class: VertexReader

Use the input split and context to setup reading the vertices. Guaranteed to be called prior to any other function.

Specified by:

initialize in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

inputSplit - Input split to be used for reading vertices.

context - Context from the task.

Throws:

IOException

InterruptedException

createLineRecordReader

protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> createLineRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                                                                                                                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                throws IOException,
                                                                                                                                       InterruptedException

Create the line record reader. Override this to use a different underlying record reader (useful for testing).

Parameters:

inputSplit - the split to read

context - the context passed to initialize

Returns:

the record reader to be used

Throws:

IOException - exception that can be thrown during creation

InterruptedException - exception that can be thrown during creation

close

public void close()
           throws IOException

Description copied from class: VertexReader

Close this VertexReader to future operations.

Specified by:

close in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Throws:

IOException

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException

Description copied from class: VertexReader

How much of the input has the VertexReader consumed i.e. has been processed by?

Specified by:

getProgress in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Progress from 0.0 to 1.0.

Throws:

IOException

InterruptedException

getRecordReader

protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> getRecordReader()

Get the line record reader.

Returns:

Record reader to be used for reading.

getContext

protected org.apache.hadoop.mapreduce.TaskAttemptContext getContext()

Get the context.

Returns:

Context passed to initialize.