org.apache.giraph.comm.netty

Class NettyWorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor<I,V,E>

Type Parameters:

I - Vertex id

V - Vertex data

E - Edge data

All Implemented Interfaces:

WorkerClientRequestProcessor<I,V,E>

public class NettyWorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>
extends Object
implements WorkerClientRequestProcessor<I,V,E>

Aggregate requests and sends them to the thread-safe NettyClient. This class is not thread-safe and expected to be used and then thrown away after a phase of communication has completed.

Constructor Summary

Constructors

Constructor and Description
NettyWorkerClientRequestProcessor(org.apache.hadoop.mapreduce.Mapper.Context context, ImmutableClassesGiraphConfiguration<I,V,E> conf, CentralizedServiceWorker<I,V,E> serviceWorker, boolean useOneMessageToManyIdsEncoding) Constructor.

Method Summary

Modifier and Type	Method and Description
void	addEdgeRequest(I vertexIndex, Edge<I,E> edge) Sends a request to the appropriate vertex range owner to add an edge
void	addVertexRequest(Vertex<I,V,E> vertex) Sends a request to the appropriate vertex range owner to add a vertex
void	doRequest(WorkerInfo workerInfo, WritableRequest writableRequest) When doing the request, short circuit if it is local
void	flush() Flush all outgoing messages.
void	removeEdgesRequest(I vertexIndex, I destinationVertexIndex) Sends a request to the appropriate vertex range owner to remove all edges pointing to a given vertex.
void	removeVertexRequest(I vertexIndex) Sends a request to the appropriate vertex range owner to remove a vertex
long	resetMessageBytesCount() Get the message bytes sent during this superstep and clear them.
long	resetMessageCount() Get the messages sent during this superstep and clear them.
boolean	sendEdgeRequest(I sourceVertexId, Edge<I,E> edge) Sends a request to the source vertex owner to add an edge.
void	sendMessageRequest(I destVertexId, org.apache.hadoop.io.Writable message) Sends a message to destination vertex.
void	sendMessageToAllRequest(Iterator<I> vertexIdIterator, org.apache.hadoop.io.Writable message) Sends a message to the targets in the iterator.
void	sendMessageToAllRequest(Vertex<I,V,E> vertex, org.apache.hadoop.io.Writable message) Sends a message through all edges to all destinations.
void	sendPartitionRequest(WorkerInfo workerInfo, Partition<I,V,E> partition) Send a partition request (no batching).
boolean	sendVertexRequest(PartitionOwner partitionOwner, Vertex<I,V,E> vertex) Sends a vertex to the appropriate partition owner

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NettyWorkerClientRequestProcessor

public NettyWorkerClientRequestProcessor(org.apache.hadoop.mapreduce.Mapper.Context context,
                                         ImmutableClassesGiraphConfiguration<I,V,E> conf,
                                         CentralizedServiceWorker<I,V,E> serviceWorker,
                                         boolean useOneMessageToManyIdsEncoding)

Constructor.

Parameters:

context - Context

conf - Configuration

serviceWorker - Service worker

useOneMessageToManyIdsEncoding - should use one message to many

Method Detail

sendMessageRequest

public void sendMessageRequest(I destVertexId,
                               org.apache.hadoop.io.Writable message)

Description copied from interface: WorkerClientRequestProcessor

Sends a message to destination vertex.

Specified by:

sendMessageRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

destVertexId - Destination vertex id.

message - Message to send.

sendMessageToAllRequest

public void sendMessageToAllRequest(Vertex<I,V,E> vertex,
                                    org.apache.hadoop.io.Writable message)

Description copied from interface: WorkerClientRequestProcessor

Sends a message through all edges to all destinations.

Specified by:

sendMessageToAllRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertex - The source vertex.

message - Message to send.

sendMessageToAllRequest

public void sendMessageToAllRequest(Iterator<I> vertexIdIterator,
                                    org.apache.hadoop.io.Writable message)

Description copied from interface: WorkerClientRequestProcessor

Sends a message to the targets in the iterator.

Specified by:

sendMessageToAllRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertexIdIterator - The iterator of target vertex ids.

message - Message to send.

sendPartitionRequest

public void sendPartitionRequest(WorkerInfo workerInfo,
                                 Partition<I,V,E> partition)

Description copied from interface: WorkerClientRequestProcessor

Send a partition request (no batching).

Specified by:

sendPartitionRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

workerInfo - Worker to send the partition to

partition - Partition to send

sendVertexRequest

public boolean sendVertexRequest(PartitionOwner partitionOwner,
                                 Vertex<I,V,E> vertex)

Description copied from interface: WorkerClientRequestProcessor

Sends a vertex to the appropriate partition owner

Specified by:

sendVertexRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

partitionOwner - Owner of the vertex

vertex - Vertex to send

Returns:

Returns true iff any network I/O occurred.

addEdgeRequest

public void addEdgeRequest(I vertexIndex,
                           Edge<I,E> edge)
                    throws IOException

Description copied from interface: WorkerClientRequestProcessor

Sends a request to the appropriate vertex range owner to add an edge

Specified by:

addEdgeRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertexIndex - Index of the vertex to get the request

edge - Edge to be added

Throws:

IOException

sendEdgeRequest

public boolean sendEdgeRequest(I sourceVertexId,
                               Edge<I,E> edge)
                        throws IOException

Description copied from interface: WorkerClientRequestProcessor

Sends a request to the source vertex owner to add an edge. Note: this request follows an optimized code path used by edge-based input, and doesn't coordinate with mutations.

Specified by:

sendEdgeRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

sourceVertexId - Source vertex id.

edge - Edge to be added.

Returns:

Returns true iff any network I/O occurred.

Throws:

IOException

removeEdgesRequest

public void removeEdgesRequest(I vertexIndex,
                               I destinationVertexIndex)
                        throws IOException

Description copied from interface: WorkerClientRequestProcessor

Sends a request to the appropriate vertex range owner to remove all edges pointing to a given vertex.

Specified by:

removeEdgesRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertexIndex - Index of the vertex to get the request

destinationVertexIndex - Index of the edge to be removed

Throws:

IOException

addVertexRequest

public void addVertexRequest(Vertex<I,V,E> vertex)
                      throws IOException

Description copied from interface: WorkerClientRequestProcessor

Sends a request to the appropriate vertex range owner to add a vertex

Specified by:

addVertexRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertex - Vertex to be added

Throws:

IOException

removeVertexRequest

public void removeVertexRequest(I vertexIndex)
                         throws IOException

Description copied from interface: WorkerClientRequestProcessor

Sends a request to the appropriate vertex range owner to remove a vertex

Specified by:

removeVertexRequest in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Parameters:

vertexIndex - Index of the vertex to be removed

Throws:

IOException

flush

public void flush()
           throws IOException

Description copied from interface: WorkerClientRequestProcessor

Flush all outgoing messages. This ensures that all the messages have been sent, but not guaranteed to have been delivered yet.

Specified by:

flush in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Throws:

IOException

resetMessageCount

public long resetMessageCount()

Description copied from interface: WorkerClientRequestProcessor

Get the messages sent during this superstep and clear them.

Specified by:

resetMessageCount in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Number of messages sent before the reset.

resetMessageBytesCount

public long resetMessageBytesCount()

Description copied from interface: WorkerClientRequestProcessor

Get the message bytes sent during this superstep and clear them.

Specified by:

resetMessageBytesCount in interface WorkerClientRequestProcessor<I extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable,E extends org.apache.hadoop.io.Writable>

Returns:

Bytes of messages sent before the reset.

doRequest

public void doRequest(WorkerInfo workerInfo,
                      WritableRequest writableRequest)

When doing the request, short circuit if it is local

Parameters:

workerInfo - Worker info

writableRequest - Request to either submit or run locally