org.apache.giraph.io.formats.multi

Class MultiMappingInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,B extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable<I,V,E>

org.apache.giraph.io.GiraphInputFormat<I,V,E>

org.apache.giraph.io.MappingInputFormat<I,V,E,B>

org.apache.giraph.io.formats.multi.MultiMappingInputFormat<I,V,E,B>

Type Parameters:

I - Vertex id

V - Vertex value

E - Edge data

B - Mapping target

All Implemented Interfaces:

GiraphConfigurationSettable<I,V,E>, ImmutableClassesGiraphConfigurable<I,V,E>

public class MultiMappingInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,B extends org.apache.hadoop.io.Writable>
extends MappingInputFormat<I,V,E,B>

Mapping input format which wraps several mapping input formats. Provides the way to read data from multiple sources, using several different input formats.

Constructor Summary

Constructors

Constructor and Description
MultiMappingInputFormat() Default constructor.

Method Summary

Modifier and Type	Method and Description
void	checkInputSpecs(org.apache.hadoop.conf.Configuration conf) Check that input is valid.
MappingReader	createMappingReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context) Create a vertex reader for a given split.
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context, int minSplitCountHint) Get the list of input splits for the format.
org.apache.hadoop.mapreduce.InputSplit	readInputSplit(DataInput dataInput) Read input split info from DataInput.
void	setConf(ImmutableClassesGiraphConfiguration<I,V,E> conf) Set the configuration to be used by this object.
void	writeInputSplit(org.apache.hadoop.mapreduce.InputSplit inputSplit, DataOutput dataOutput) Write input split info to DataOutput.

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiMappingInputFormat

public MultiMappingInputFormat()

Default constructor.

Method Detail

setConf

public void setConf(ImmutableClassesGiraphConfiguration<I,V,E> conf)

Description copied from interface: GiraphConfigurationSettable

Set the configuration to be used by this object.

Specified by:

setConf in interface GiraphConfigurationSettable<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Overrides:

setConf in class DefaultImmutableClassesGiraphConfigurable<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

conf - Set configuration

createMappingReader

public MappingReader createMappingReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                         org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                  throws IOException

Description copied from class: MappingInputFormat

Create a vertex reader for a given split. Guaranteed to have been configured with setConf() prior to use. The framework will also call VertexReader.initialize(InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext) before the split is used.

Specified by:

createMappingReader in class MappingInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable,B extends org.apache.hadoop.io.Writable>

Parameters:

inputSplit - the split to be read

context - the information about the task

Returns:

a new record reader

Throws:

IOException

checkInputSpecs

public void checkInputSpecs(org.apache.hadoop.conf.Configuration conf)

Description copied from class: GiraphInputFormat

Check that input is valid.

Specified by:

checkInputSpecs in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

conf - Configuration

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context,
                                                              int minSplitCountHint)
                                                       throws IOException,
                                                              InterruptedException

Description copied from class: GiraphInputFormat

Get the list of input splits for the format.

Specified by:

getSplits in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

context - The job context

minSplitCountHint - Minimum number of splits to create (hint)

Returns:

The list of input splits

Throws:

IOException

InterruptedException

writeInputSplit

public void writeInputSplit(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                            DataOutput dataOutput)
                     throws IOException

Description copied from class: GiraphInputFormat

Write input split info to DataOutput.

Overrides:

writeInputSplit in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

inputSplit - InputSplit

dataOutput - DataOutput

Throws:

IOException

readInputSplit

public org.apache.hadoop.mapreduce.InputSplit readInputSplit(DataInput dataInput)
                                                      throws IOException,
                                                             ClassNotFoundException

Description copied from class: GiraphInputFormat

Read input split info from DataInput.

Overrides:

readInputSplit in class GiraphInputFormat<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

dataInput - DataInput

Returns:

InputSplit

Throws:

IOException

ClassNotFoundException