org.apache.giraph.utils.io

Class BigDataOutput

java.lang.Object

org.apache.giraph.utils.io.BigDataOutput

All Implemented Interfaces:

DataOutput, org.apache.hadoop.io.Writable

public class BigDataOutput
extends Object
implements DataOutput, org.apache.hadoop.io.Writable

Implementations of ExtendedDataOutput are limited because they can only handle up to 1GB of data. This DataOutput overcomes that limitation, with almost no additional cost when data is not huge. Goes in pair with BigDataInput

Field Summary

Fields

Modifier and Type	Field and Description
protected ImmutableClassesGiraphConfiguration	conf Configuration
protected ExtendedDataOutput	currentDataOutput Data output which we are currently writing to
protected List<ExtendedDataOutput>	dataOutputs List of filled outputs, will be null until we get a lot of data

Constructor Summary

Constructors

Constructor and Description
BigDataOutput(ImmutableClassesGiraphConfiguration conf) Constructor
BigDataOutput(int initialSize, ImmutableClassesGiraphConfiguration conf) Constructor

Method Summary

Modifier and Type	Method and Description
protected ExtendedDataOutput	createOutput(int size) Create next data output
ImmutableClassesGiraphConfiguration	getConf()
Iterable<ExtendedDataOutput>	getDataOutputs() Get DataOutputs which contain written data.
protected int	getMaxSize() Get max size for single data output
int	getNumberOfDataOutputs() Get number of DataOutputs which contain written data.
long	getSize() Get number of bytes written to this data output
void	readFields(DataInput in)
void	write(byte[] b)
void	write(byte[] b, int off, int len)
void	write(DataOutput out)
void	write(int b)
void	writeBoolean(boolean v)
void	writeByte(int v)
void	writeBytes(String s)
void	writeChar(int v)
void	writeChars(String s)
void	writeDouble(double v)
void	writeFloat(float v)
void	writeInt(int v)
void	writeLong(long v)
void	writeShort(int v)
void	writeUTF(String s)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

currentDataOutput

protected ExtendedDataOutput currentDataOutput

Data output which we are currently writing to

dataOutputs

protected List<ExtendedDataOutput> dataOutputs

List of filled outputs, will be null until we get a lot of data

conf

protected final ImmutableClassesGiraphConfiguration conf

Configuration

Constructor Detail

BigDataOutput

public BigDataOutput(ImmutableClassesGiraphConfiguration conf)

Constructor

Parameters:

conf - Configuration

BigDataOutput

public BigDataOutput(int initialSize,
                     ImmutableClassesGiraphConfiguration conf)

Constructor

Parameters:

initialSize - Initial size of data output

conf - Configuration

Method Detail

getMaxSize

protected int getMaxSize()

Get max size for single data output

Returns:

Max size for single data output

createOutput

protected ExtendedDataOutput createOutput(int size)

Create next data output

Parameters:

size - Size of data output to create

Returns:

Created data output

getNumberOfDataOutputs

public int getNumberOfDataOutputs()

Get number of DataOutputs which contain written data.

Returns:

Number of DataOutputs which contain written data

getDataOutputs

public Iterable<ExtendedDataOutput> getDataOutputs()

Get DataOutputs which contain written data.

Returns:

DataOutputs which contain written data

getConf

public ImmutableClassesGiraphConfiguration getConf()

getSize

public long getSize()

Get number of bytes written to this data output

Returns:

Size in bytes

write

public void write(int b)
           throws IOException

Specified by:

write in interface DataOutput

Throws:

IOException

write

public void write(byte[] b)
           throws IOException

Specified by:

write in interface DataOutput

Throws:

IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws IOException

Specified by:

write in interface DataOutput

Throws:

IOException

writeBoolean

public void writeBoolean(boolean v)
                  throws IOException

Specified by:

writeBoolean in interface DataOutput

Throws:

IOException

writeByte

public void writeByte(int v)
               throws IOException

Specified by:

writeByte in interface DataOutput

Throws:

IOException

writeShort

public void writeShort(int v)
                throws IOException

Specified by:

writeShort in interface DataOutput

Throws:

IOException

writeChar

public void writeChar(int v)
               throws IOException

Specified by:

writeChar in interface DataOutput

Throws:

IOException

writeInt

public void writeInt(int v)
              throws IOException

Specified by:

writeInt in interface DataOutput

Throws:

IOException

writeLong

public void writeLong(long v)
               throws IOException

Specified by:

writeLong in interface DataOutput

Throws:

IOException

writeFloat

public void writeFloat(float v)
                throws IOException

Specified by:

writeFloat in interface DataOutput

Throws:

IOException

writeDouble

public void writeDouble(double v)
                 throws IOException

Specified by:

writeDouble in interface DataOutput

Throws:

IOException

writeBytes

public void writeBytes(String s)
                throws IOException

Specified by:

writeBytes in interface DataOutput

Throws:

IOException

writeChars

public void writeChars(String s)
                throws IOException

Specified by:

writeChars in interface DataOutput

Throws:

IOException

writeUTF

public void writeUTF(String s)
              throws IOException

Specified by:

writeUTF in interface DataOutput

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException