org.apache.giraph.conf

Class FacebookConfiguration

java.lang.Object

org.apache.giraph.conf.FacebookConfiguration

All Implemented Interfaces:

BulkConfigurator

public class FacebookConfiguration
extends Object
implements BulkConfigurator

Default configuration used in Facebook

Field Summary

Fields

Modifier and Type	Field and Description
static BooleanConfOption	CONFIGURE_JAVA_OPTS Whether to configure java opts.
static FloatConfOption	CORES_FRACTION_DURING_COMMUNICATION Which fraction of cores to use for threads when computation and communication overlap
static IntConfOption	MAPPER_CORES How many cores per mapper should we use
static IntConfOption	MAPPER_MEMORY How much memory per mapper should we use
static StrConfOption	MAPRED_JAVA_JOB_OPTIONS Java options passed to mappers.
static FloatConfOption	NEW_GEN_MEMORY_FRACTION Fraction of MAPPER_MEMORY to use for new generation
static BooleanConfOption	USE_G1_COLLECTOR Note: using G1 is often faster, but we've seen it add off heap memory overhead which can cause issues.

Constructor Summary

Constructors

Constructor and Description
FacebookConfiguration()

Method Summary

Modifier and Type	Method and Description
void	configure(GiraphConfiguration conf) Expand GiraphConfiguration with default Facebook settings.
static List<String>	getGcJavaOpts(org.apache.hadoop.conf.Configuration conf) Get garbage collection java opts to use
static List<String>	getMemoryJavaOpts(org.apache.hadoop.conf.Configuration conf) Get memory java opts to use

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAPPER_MEMORY

public static final IntConfOption MAPPER_MEMORY

How much memory per mapper should we use

MAPPER_CORES

public static final IntConfOption MAPPER_CORES

How many cores per mapper should we use

NEW_GEN_MEMORY_FRACTION

public static final FloatConfOption NEW_GEN_MEMORY_FRACTION

Fraction of MAPPER_MEMORY to use for new generation

USE_G1_COLLECTOR

public static final BooleanConfOption USE_G1_COLLECTOR

Note: using G1 is often faster, but we've seen it add off heap memory overhead which can cause issues.

CORES_FRACTION_DURING_COMMUNICATION

public static final FloatConfOption CORES_FRACTION_DURING_COMMUNICATION

Which fraction of cores to use for threads when computation and communication overlap

CONFIGURE_JAVA_OPTS

public static final BooleanConfOption CONFIGURE_JAVA_OPTS

Whether to configure java opts.

MAPRED_JAVA_JOB_OPTIONS

public static final StrConfOption MAPRED_JAVA_JOB_OPTIONS

Java options passed to mappers.

Constructor Detail

FacebookConfiguration

public FacebookConfiguration()

Method Detail

configure

public void configure(GiraphConfiguration conf)

Expand GiraphConfiguration with default Facebook settings. Assumes MAPPER_CORES and number of workers to use are already set correctly in Configuration. For all conf options it changed it will only do so if they are not set, so it won't override any of your custom settings. The only exception is mapred.child.java.opts, this one will be overwritten depending on the CONFIGURE_JAVA_OPTS setting

Specified by:

configure in interface BulkConfigurator

Parameters:

conf - Configuration

getMemoryJavaOpts

public static List<String> getMemoryJavaOpts(org.apache.hadoop.conf.Configuration conf)

Get memory java opts to use

Parameters:

conf - Configuration

Returns:

Java opts

getGcJavaOpts

public static List<String> getGcJavaOpts(org.apache.hadoop.conf.Configuration conf)

Get garbage collection java opts to use

Parameters:

conf - Configuration

Returns:

Java opts