org.apache.giraph.bsp

Class BspInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

org.apache.giraph.bsp.BspInputFormat

public class BspInputFormat
extends org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

This InputFormat supports the BSP model by ensuring that the user specifies how many splits (number of mappers) should be started simultaneously. The number of splits depends on whether the master and worker processes are separate. It is not meant to do any meaningful split of user-data.

Constructor Summary

Constructors

Constructor and Description
BspInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
static int	getMaxTasks(org.apache.hadoop.conf.Configuration conf) Get the correct number of mappers based on the configuration
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BspInputFormat

public BspInputFormat()

Method Detail

getMaxTasks

public static int getMaxTasks(org.apache.hadoop.conf.Configuration conf)

Get the correct number of mappers based on the configuration

Parameters:

conf - Configuration to determine the number of mappers

Returns:

Maximum number of tasks

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                       throws IOException,
                                                              InterruptedException

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                        org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                 throws IOException,
                                                                                                                        InterruptedException

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException