org.apache.giraph.ooc.persistence

Class LocalDiskDataAccessor

java.lang.Object

org.apache.giraph.ooc.persistence.LocalDiskDataAccessor

All Implemented Interfaces:

OutOfCoreDataAccessor

public class LocalDiskDataAccessor
extends Object
implements OutOfCoreDataAccessor

Data accessor object to read/write data in local disk. Note: This class assumes that the data are partitioned across IO threads, i.e. each part of data can be accessed by one and only one IO thread throughout the execution. Also, each IO thread reads a particular type of data completely and, only then, it can read other type of data; i.e. an IO thread cannot be used to read two different files at the same time. These assumptions are based on the assumptions that the current out-of-core mechanism is designed for.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.giraph.ooc.persistence.OutOfCoreDataAccessor

OutOfCoreDataAccessor.DataInputWrapper, OutOfCoreDataAccessor.DataOutputWrapper

Field Summary

Fields

Modifier and Type	Field and Description
static IntConfOption	OOC_DISK_BUFFER_SIZE Size of the buffer used for (de)serializing data when reading/writing from/to disk

Constructor Summary

Constructors

Constructor and Description
LocalDiskDataAccessor(ImmutableClassesGiraphConfiguration<?,?,?> conf) Constructor

Method Summary

Modifier and Type	Method and Description
boolean	dataExist(int threadId, DataIndex index) Whether the data for the given thread and index chain exists?
int	getNumAccessorThreads()
void	initialize() Initialize the DAO
OutOfCoreDataAccessor.DataInputWrapper	prepareInput(int threadId, DataIndex index) Prepare a wrapper containing DataInput representation for a given thread involved in persistence for a given index chain for data.
OutOfCoreDataAccessor.DataOutputWrapper	prepareOutput(int threadId, DataIndex index, boolean shouldAppend) Prepare a wrapper containing DataOutput representation for a given thread involved in persistence for a given index chain for data.
void	shutdown() Shut down the DAO

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OOC_DISK_BUFFER_SIZE

public static final IntConfOption OOC_DISK_BUFFER_SIZE

Size of the buffer used for (de)serializing data when reading/writing from/to disk

Constructor Detail

LocalDiskDataAccessor

public LocalDiskDataAccessor(ImmutableClassesGiraphConfiguration<?,?,?> conf)

Constructor

Parameters:

conf - Configuration

Method Detail

initialize

public void initialize()

Description copied from interface: OutOfCoreDataAccessor

Initialize the DAO

Specified by:

initialize in interface OutOfCoreDataAccessor

shutdown

public void shutdown()

Description copied from interface: OutOfCoreDataAccessor

Shut down the DAO

Specified by:

shutdown in interface OutOfCoreDataAccessor

getNumAccessorThreads

public int getNumAccessorThreads()

Specified by:

getNumAccessorThreads in interface OutOfCoreDataAccessor

Returns:

the number of threads involved in data persistence

prepareInput

public OutOfCoreDataAccessor.DataInputWrapper prepareInput(int threadId,
                                                           DataIndex index)
                                                    throws IOException

Description copied from interface: OutOfCoreDataAccessor

Prepare a wrapper containing DataInput representation for a given thread involved in persistence for a given index chain for data.

Specified by:

prepareInput in interface OutOfCoreDataAccessor

Parameters:

threadId - id of the thread involved in persistence

index - index chain of the data to access the serialized data form

Returns:

the wrapper for DataInput representation of data

Throws:

IOException

prepareOutput

public OutOfCoreDataAccessor.DataOutputWrapper prepareOutput(int threadId,
                                                             DataIndex index,
                                                             boolean shouldAppend)
                                                      throws IOException

Description copied from interface: OutOfCoreDataAccessor

Prepare a wrapper containing DataOutput representation for a given thread involved in persistence for a given index chain for data.

Specified by:

prepareOutput in interface OutOfCoreDataAccessor

Parameters:

threadId - id of the thread involved in persistence

index - index chain of the data to access the serialized data form

shouldAppend - whether the DataOutput should be used for appending to already existing data for the given index or the DataOutput should create new instance to store serialized data

Returns:

the wrapper for DataOutput representation of data

Throws:

IOException

dataExist

public boolean dataExist(int threadId,
                         DataIndex index)

Description copied from interface: OutOfCoreDataAccessor

Whether the data for the given thread and index chain exists?

Specified by:

dataExist in interface OutOfCoreDataAccessor

Parameters:

threadId - id of the thread involved in persistence

index - index chain used to access the data

Returns:

True if the data exists for the given index chain for the given thread, False otherwise