Package | Description |
---|---|
org.apache.giraph.bsp |
Package of generic bulk synchronous processing objects.
|
org.apache.giraph.comm |
Package of communication related objects, IPC service.
|
org.apache.giraph.comm.aggregators |
Package for classes which are used to handle aggregators.
|
org.apache.giraph.comm.netty |
Package for netty implementations.
|
org.apache.giraph.graph |
Package of all the graph related objects, built on the
org.apache.bsp package.
|
org.apache.giraph.master |
Package of all the master related things.
|
org.apache.giraph.master.input |
Input related master classes
|
org.apache.giraph.partition |
Package of partitioning related objects.
|
org.apache.giraph.worker |
Package of all the worker related things
|
Modifier and Type | Method and Description |
---|---|
WorkerInfo |
CentralizedServiceWorker.getWorkerInfo()
Get the worker information
|
protected WorkerInfo |
BspService.getWorkerInfoById(int id)
Returns worker info corresponding to specified worker id.
|
Modifier and Type | Method and Description |
---|---|
List<WorkerInfo> |
CentralizedServiceMaster.checkWorkers()
Check all the
WorkerInfo objects to ensure
that a minimum number of good workers exists out of the total that have
reported. |
List<WorkerInfo> |
CentralizedService.getWorkerInfoList()
Get list of workers
|
Modifier and Type | Method and Description |
---|---|
String |
BspService.getPartitionExchangeWorkerPath(long attempt,
long superstep,
WorkerInfo workerInfo)
Based on the superstep, worker info, and attempt, get the appropriate
worker path for the exchange.
|
protected int |
BspService.getWorkerId(WorkerInfo workerInfo)
For every worker this method returns unique number
between 0 and N, where N is the total number of workers.
|
Modifier and Type | Method and Description |
---|---|
protected Map<WorkerInfo,List<Integer>> |
SendDataCache.getWorkerPartitions() |
PairList<WorkerInfo,PairList<Integer,D>> |
SendDataCache.removeAllData()
Gets all the data and removes it from the cache.
|
PairList<WorkerInfo,PairList<Integer,VertexIdEdges<I,E>>> |
SendEdgeCache.removeAllEdges()
Gets all the edges and removes them from the cache.
|
Modifier and Type | Method and Description |
---|---|
int |
SendVertexIdDataCache.addData(WorkerInfo workerInfo,
int partitionId,
byte[] serializedId,
int idPos,
T data)
This method is similar to the method above,
but use a serialized id to replace original I type
destVertexId.
|
int |
SendVertexIdDataCache.addData(WorkerInfo workerInfo,
int partitionId,
I destVertexId,
T data)
Add data to the cache.
|
int |
SendEdgeCache.addEdge(WorkerInfo workerInfo,
int partitionId,
I destVertexId,
Edge<I,E> edge)
Add an edge to the cache.
|
protected int |
SendMessageCache.addMessage(WorkerInfo workerInfo,
int partitionId,
byte[] serializedId,
int idSerializerPos,
M message)
Add a message to the cache with serialized ids.
|
int |
SendMessageCache.addMessage(WorkerInfo workerInfo,
int partitionId,
I destVertexId,
M message)
Add a message to the cache.
|
PairList<Integer,D> |
SendDataCache.removeWorkerData(WorkerInfo workerInfo)
Gets the data for a worker and removes it from the cache.
|
PairList<Integer,VertexIdEdges<I,E>> |
SendEdgeCache.removeWorkerEdges(WorkerInfo workerInfo)
Gets the edges for a worker and removes it from the cache.
|
protected PairList<Integer,VertexIdMessages<I,M>> |
SendMessageCache.removeWorkerMessages(WorkerInfo workerInfo)
Gets the messages for a worker and removes it from the cache.
|
void |
WorkerClientRequestProcessor.sendPartitionRequest(WorkerInfo workerInfo,
Partition<I,V,E> partition)
Send a partition request (no batching).
|
Modifier and Type | Method and Description |
---|---|
static WorkerInfo |
AggregatorUtils.getOwner(String aggregatorName,
List<WorkerInfo> workers)
Get owner of aggregator with selected name from the list of workers
|
Modifier and Type | Method and Description |
---|---|
static WorkerInfo |
AggregatorUtils.getOwner(String aggregatorName,
List<WorkerInfo> workers)
Get owner of aggregator with selected name from the list of workers
|
Modifier and Type | Method and Description |
---|---|
void |
NettyWorkerClientRequestProcessor.doRequest(WorkerInfo workerInfo,
WritableRequest writableRequest)
When doing the request, short circuit if it is local
|
void |
NettyWorkerClientRequestProcessor.sendPartitionRequest(WorkerInfo workerInfo,
Partition<I,V,E> partition) |
Modifier and Type | Method and Description |
---|---|
List<WorkerInfo> |
AddressesAndPartitionsWritable.getWorkerInfos()
Get all workers
|
Constructor and Description |
---|
AddressesAndPartitionsWritable(MasterInfo masterInfo,
List<WorkerInfo> workerInfos,
Collection<PartitionOwner> partitionOwners)
Constructor when we want to serialize object
|
Modifier and Type | Method and Description |
---|---|
List<WorkerInfo> |
BspServiceMaster.checkWorkers() |
List<WorkerInfo> |
BspServiceMaster.getWorkerInfoList() |
List<WorkerInfo> |
MasterCompute.getWorkerInfoList()
Get list of workers
|
Modifier and Type | Method and Description |
---|---|
void |
MasterInputSplitsHandler.initialize(MasterClient masterClient,
List<WorkerInfo> workers)
Initialize
|
Constructor and Description |
---|
LocalityAwareInputSplitsMasterOrganizer(List<byte[]> serializedSplits,
List<org.apache.hadoop.mapreduce.InputSplit> splits,
List<WorkerInfo> workers)
Constructor
|
MappingInputSplitsMasterOrganizer(List<byte[]> serializedSplits,
List<WorkerInfo> workers)
Constructor
|
Modifier and Type | Method and Description |
---|---|
WorkerInfo |
BasicPartitionOwner.getPreviousWorkerInfo() |
WorkerInfo |
PartitionOwner.getPreviousWorkerInfo()
Get the worker information that was previously responsible for the
partition id.
|
WorkerInfo |
BasicPartitionOwner.getWorkerInfo() |
WorkerInfo |
PartitionOwner.getWorkerInfo()
Get the worker information that is currently responsible for
the partition id.
|
Modifier and Type | Method and Description |
---|---|
Set<WorkerInfo> |
PartitionExchange.getMyDependencyWorkerSet()
Get the workers that I must wait for
|
Map<WorkerInfo,List<Integer>> |
PartitionExchange.getSendWorkerPartitionMap()
Get a mapping of worker to list of partition ids I need to send to.
|
Modifier and Type | Method and Description |
---|---|
void |
BasicPartitionOwner.setPreviousWorkerInfo(WorkerInfo workerInfo) |
void |
PartitionOwner.setPreviousWorkerInfo(WorkerInfo workerInfo)
Set the previous worker info.
|
void |
BasicPartitionOwner.setWorkerInfo(WorkerInfo workerInfo) |
void |
PartitionOwner.setWorkerInfo(WorkerInfo workerInfo)
Set the current worker info.
|
static PartitionExchange |
PartitionBalancer.updatePartitionOwners(List<PartitionOwner> partitionOwnerList,
WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners)
Helper function to update partition owners and determine which
partitions need to be sent from a specific worker.
|
PartitionExchange |
WorkerGraphPartitionerImpl.updatePartitionOwners(WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners) |
PartitionExchange |
WorkerGraphPartitioner.updatePartitionOwners(WorkerInfo myWorkerInfo,
Collection<? extends PartitionOwner> masterSetPartitionOwners)
Get the partitions owners and update locally.
|
Modifier and Type | Method and Description |
---|---|
static Collection<PartitionOwner> |
PartitionBalancer.balancePartitionsAcrossWorkers(org.apache.hadoop.conf.Configuration conf,
Collection<PartitionOwner> partitionOwners,
Collection<PartitionStats> allPartitionStats,
Collection<WorkerInfo> availableWorkerInfos)
Balance the partitions with an algorithm based on a value.
|
Collection<PartitionOwner> |
MasterGraphPartitionerImpl.createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
int maxWorkers) |
Collection<PartitionOwner> |
MasterGraphPartitioner.createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
int maxWorkers)
Set some initial partition owners for the graph.
|
Collection<PartitionOwner> |
MasterGraphPartitionerImpl.generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList,
Collection<WorkerInfo> availableWorkers,
int maxWorkers,
long superstep) |
Collection<PartitionOwner> |
MasterGraphPartitioner.generateChangedPartitionOwners(Collection<PartitionStats> allPartitionStatsList,
Collection<WorkerInfo> availableWorkers,
int maxWorkers,
long superstep)
After the worker stats have been merged to a single list, the master can
use this information to send commands to the workers for any
Partition changes. |
void |
BasicPartitionOwner.readFieldsWithWorkerIds(DataInput input,
Map<Integer,WorkerInfo> workerInfoMap) |
void |
PartitionOwner.readFieldsWithWorkerIds(DataInput input,
Map<Integer,WorkerInfo> workerInfoMap)
A match for writeWithWorkerIds method - for WorkerInfos it will read
just task ids from input and then find the matching WorkerInfo in the
provided map and set it
|
Constructor and Description |
---|
BasicPartitionOwner(int partitionId,
WorkerInfo workerInfo)
Constructor with partition id and worker info.
|
BasicPartitionOwner(int partitionId,
WorkerInfo workerInfo,
WorkerInfo previousWorkerInfo,
String checkpointFilesPrefix)
Constructor with partition id and worker info.
|
Constructor and Description |
---|
PartitionExchange(Set<WorkerInfo> myDependencyWorkerSet,
Map<WorkerInfo,List<Integer>> sendWorkerPartitionMap)
Only constructor.
|
PartitionExchange(Set<WorkerInfo> myDependencyWorkerSet,
Map<WorkerInfo,List<Integer>> sendWorkerPartitionMap)
Only constructor.
|
Modifier and Type | Method and Description |
---|---|
WorkerInfo |
BspServiceWorker.getWorkerInfo() |
Modifier and Type | Method and Description |
---|---|
List<WorkerInfo> |
BspServiceWorker.getWorkerInfoList() |
List<WorkerInfo> |
AllWorkersInfo.getWorkerList()
List of WorkerInfos
|
Modifier and Type | Method and Description |
---|---|
int |
AllWorkersInfo.getWorkerIndex(WorkerInfo workerInfo)
For every worker this method returns unique number
between 0 and N, where N is the total number of workers.
|
Constructor and Description |
---|
AllWorkersInfo(List<WorkerInfo> workers,
WorkerInfo myWorker)
Constructor
|
WorkerInputSplitsHandler(WorkerInfo workerInfo,
int masterTaskId,
WorkerClient workerClient)
Constructor
|
Constructor and Description |
---|
AllWorkersInfo(List<WorkerInfo> workers,
WorkerInfo myWorker)
Constructor
|
Copyright © 2011-2020 The Apache Software Foundation. All Rights Reserved.