org.apache.giraph.io.hcatalog

Class HCatalogVertexInputFormat.HCatalogVertexReader

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.worker.WorkerAggregatorDelegator<I,V,E>

org.apache.giraph.io.VertexReader<I,V,E>

org.apache.giraph.io.hcatalog.HCatalogVertexInputFormat.HCatalogVertexReader

All Implemented Interfaces:

AggregatorUsage, GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>, WorkerAggregatorUsage, WorkerBroadcastUsage, WorkerGlobalCommUsage, WorkerReduceUsage

Direct Known Subclasses:

HCatalogVertexInputFormat.MultiRowHCatalogVertexReader, HCatalogVertexInputFormat.SingleRowHCatalogVertexReader

Enclosing class:

HCatalogVertexInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

protected abstract class HCatalogVertexInputFormat.HCatalogVertexReader
extends VertexReader<I,V,E>

Abstract class that users should subclass based on their specific vertex input. HCatRecord can be parsed to get the required data for implementing getCurrentVertex(). If the vertex spans more than one HCatRecord, nextVertex() should be overwritten to handle that logic as well.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HCatalogVertexReader()

Method Summary

Modifier and Type	Method and Description
void	close() Close this VertexReader to future operations.
protected org.apache.hadoop.mapreduce.TaskAttemptContext	getContext() Get the context.
float	getProgress() How much of the input has the VertexReader consumed i.e.
protected org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.WritableComparable,org.apache.hcatalog.data.HCatRecord>	getRecordReader() Get the record reader.
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext ctxt) Use the input split and context to setup reading the vertices.
boolean	nextVertex()

Methods inherited from class org.apache.giraph.io.VertexReader

getCurrentVertex

Methods inherited from class org.apache.giraph.worker.WorkerAggregatorDelegator

aggregate, getAggregatedValue, getBroadcast, reduce, reduceMerge, setWorkerGlobalCommUsage

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HCatalogVertexReader

protected HCatalogVertexReader()

Method Detail

initialize

public final void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                             org.apache.hadoop.mapreduce.TaskAttemptContext ctxt)
                      throws IOException,
                             InterruptedException

Description copied from class: VertexReader

Use the input split and context to setup reading the vertices. Guaranteed to be called prior to any other function.

Specified by:

initialize in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

inputSplit - Input split to be used for reading vertices.

ctxt - Context from the task.

Throws:

IOException

InterruptedException

nextVertex

public boolean nextVertex()
                   throws IOException,
                          InterruptedException

Specified by:

nextVertex in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

false iff there are no more vertices

Throws:

IOException

InterruptedException

close

public final void close()
                 throws IOException

Description copied from class: VertexReader

Close this VertexReader to future operations.

Specified by:

close in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Throws:

IOException

getProgress

public final float getProgress()
                        throws IOException,
                               InterruptedException

Description copied from class: VertexReader

How much of the input has the VertexReader consumed i.e. has been processed by?

Specified by:

getProgress in class VertexReader<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Progress from 0.0 to 1.0.

Throws:

IOException

InterruptedException

getRecordReader

protected final org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.WritableComparable,org.apache.hcatalog.data.HCatRecord> getRecordReader()

Get the record reader.

Returns:

Record reader to be used for reading.

getContext

protected final org.apache.hadoop.mapreduce.TaskAttemptContext getContext()

Get the context.

Returns:

Context passed to initialize.