1. 程式人生 > >Giraph原始碼分析(二)—啟動Master/Worker服務

Giraph原始碼分析(二)—啟動Master/Worker服務

作者 | 白松

注:本文為原創,引用轉載需與數瀾聯絡。

1、org.apache.giraph.bsp.CentralizedService 介面

功能:Basic service interface shared by both CentralizedServiceMaster and CentralizedServiceWorker.

2、org.apache.giraph.bsp.CentralizedServiceMaster介面

功能:At most, there will be one active master at a time, but many threads can be trying to be the active master.

3、org.apache.giraph.bsp.CentralizedServiceWorker介面

功能:All workers should have access to this centralized service to execute the following methods.

4、org.apache.giraph.bsp.BspService抽象類

**功能:**Zookeeper-based implementation of CentralizedService.

5、org.apache.giraph.master.BspServiceMaster類

功能:ZooKeeper-based implementation of CentralizedServiceMaster.

6、org.apache.giraph.worker.BspServiceWorker類

功能:ZooKeeper-based implementation of CentralizedServiceWorker.

BspServiceWorker類有WorkerClient和WorkerServer例項,分別作為IPC通訊的客戶端和伺服器端,通過Netty來發送資料。WorkerClient例項實際為NettyWorkerClient物件,WorkerServert例項實際為NettyWorkerServer物件。

NettyWorkerClient implements WorkerClient介面,NettyWorkerServer implements WorkerServer介面。

NettyWorkerServer類的構造方法中建立一個NettyServer物件,用於底層的IPC的通訊,還有一個ServerData物件,作為資料實體。ServerData中包含該Worker的partitionStore、edgeStore、incomingMessageStore、currentMessageStore、聚集值等。

NettyWorkerClient類的構造方法中建立一個NettyClient物件,用於底層的IPC的通訊,作為客戶端。

7、org.apache.giraph.worker.InputSplitsCallable 抽象類,繼承 Callable介面。

功能:用於載入頂點或邊 輸入splits,每個執行緒都有一個WorkerClientRequestProcessor例項(實為 NettyWorkerClientRequestProcessor物件),負責向遠端的worker傳送資料。

NettyWorkerClientRequestProcessor物件用於傳送的WorkerClient物件就是BspServiceWorker裡面的WorkerClient物件。

VertexInputSplitsCallable類中的readInputSplit()方法用來從split中讀取頂點的資訊,然後呼叫NettyWorkerClientRequestProcessor物件的sendVertexRequest()方法把頂點發送到它所屬的Partition上。

8、org.apache.giraph.graph.ComputeCallable 類,繼承Callable介面。

在該物件中完成“計算-通訊-同步”的過程。每個執行緒都有一個WorkerClientRequestProcessor例項(實為 NettyWorkerClientRequestProcessor物件),負責向遠端的