org.apache.giraph.aggregators

Class TextAggregatorWriter

java.lang.Object

org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

org.apache.giraph.aggregators.TextAggregatorWriter

All Implemented Interfaces:

AggregatorWriter, GiraphConfigurationSettable, ImmutableClassesGiraphConfigurable

public class TextAggregatorWriter
extends DefaultImmutableClassesGiraphConfigurable
implements AggregatorWriter

Default implementation of AggregatorWriter. Each line consists of text and contains the aggregator name, the aggregator value and the aggregator class.

Field Summary

Fields

Modifier and Type	Field and Description
static int	ALWAYS Signal for "write values in every superstep" frequency
static int	AT_THE_END Signal for "write only the final values" frequency
static String	FILENAME The filename of the outputfile
static String	FREQUENCY The frequency of writing: - NEVER: never write, files aren't created at all - AT_THE_END: aggregators are written only when the computation is over - int: i.e.
static int	NEVER Signal for "never write" frequency
protected org.apache.hadoop.fs.FSDataOutputStream	output Handle to the outputfile

Fields inherited from interface org.apache.giraph.aggregators.AggregatorWriter

LAST_SUPERSTEP

Constructor Summary

Constructors

Constructor and Description
TextAggregatorWriter()

Method Summary

Modifier and Type	Method and Description
protected String	aggregatorToString(String aggregatorName, org.apache.hadoop.io.Writable value, long superstep) Implements the way an aggregator is converted into a String.
void	close() The method is called at the end of a successful computation.
void	initialize(org.apache.hadoop.mapreduce.Mapper.Context context, long attempt) The method is called at the initialization of the AggregatorWriter.
void	writeAggregator(Iterable<Map.Entry<String,org.apache.hadoop.io.Writable>> aggregatorMap, long superstep) The method is called at the end of each superstep.

Methods inherited from class org.apache.giraph.conf.DefaultImmutableClassesGiraphConfigurable

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.giraph.conf.ImmutableClassesGiraphConfigurable

getConf

Methods inherited from interface org.apache.giraph.conf.GiraphConfigurationSettable

setConf

Field Detail

FILENAME

public static final String FILENAME

The filename of the outputfile

See Also:

Constant Field Values

NEVER

public static final int NEVER

Signal for "never write" frequency

See Also:

Constant Field Values

AT_THE_END

public static final int AT_THE_END

Signal for "write only the final values" frequency

See Also:

Constant Field Values

ALWAYS

public static final int ALWAYS

Signal for "write values in every superstep" frequency

See Also:

Constant Field Values

FREQUENCY

public static final String FREQUENCY

The frequency of writing: - NEVER: never write, files aren't created at all - AT_THE_END: aggregators are written only when the computation is over - int: i.e. 1 is every superstep, 2 every two supersteps and so on

See Also:

Constant Field Values

output

protected org.apache.hadoop.fs.FSDataOutputStream output

Handle to the outputfile

Constructor Detail

TextAggregatorWriter

public TextAggregatorWriter()

Method Detail

initialize

public void initialize(org.apache.hadoop.mapreduce.Mapper.Context context,
                       long attempt)
                throws IOException

Description copied from interface: AggregatorWriter

The method is called at the initialization of the AggregatorWriter. More precisely, the aggregatorWriter is initialized each time a new master is elected.

Specified by:

initialize in interface AggregatorWriter

Parameters:

context - Mapper Context where the master is running on

attempt - ID of the applicationAttempt, used to disambiguate aggregator writes for different attempts

Throws:

IOException

writeAggregator

public void writeAggregator(Iterable<Map.Entry<String,org.apache.hadoop.io.Writable>> aggregatorMap,
                            long superstep)
                     throws IOException

Description copied from interface: AggregatorWriter

The method is called at the end of each superstep. The user might decide whether to write the aggregators values for the current superstep. For the last superstep, AggregatorWriter.LAST_SUPERSTEP is passed.

Specified by:

writeAggregator in interface AggregatorWriter

Parameters:

aggregatorMap - Map from aggregator name to aggregator value

superstep - Current superstep

Throws:

IOException

aggregatorToString

protected String aggregatorToString(String aggregatorName,
                                    org.apache.hadoop.io.Writable value,
                                    long superstep)

Implements the way an aggregator is converted into a String. Override this if you want to implement your own text format.

Parameters:

aggregatorName - Name of the aggregator

value - Value of aggregator

superstep - Current superstep

Returns:

The String representation for the aggregator

close

public void close()
           throws IOException

Description copied from interface: AggregatorWriter

The method is called at the end of a successful computation. The method is not called when the job fails and a new master is elected. For this reason it's advised to flush data at the end of AggregatorWriter.writeAggregator(Iterable, long).

Specified by:

close in interface AggregatorWriter

Throws:

IOException