【kubernetes/k8s原始碼分析】CNI flannel原始碼分析
阿新 • • 發佈:2019-01-04
原始碼路徑: https://github.com/containernetworking/plugins
版本: v.0.10.0
flannel cni路徑: plugins/plugins/meta/flannel/flannel.go
subnet.env檔案
# cat /run/flannel/subnet.env
FLANNEL_NETWORK=172.30.0.0/16
FLANNEL_SUBNET=172.30.45.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false
cni 配置檔案
# cat /etc/cni/net.d/10-flannel.conf
{"name":"cbr0","type":"flannel","delegate": {"bridge": "docker0", "isDefaultGateway": true, "ipMasq": false}}
NetConf結構體
subnetFile檔案預設為:/run/flannel/subnet.env
dataDIr預設為:/var/lib/cni/flannel
type NetConf struct { types.NetConf SubnetFile string `json:"subnetFile"` DataDir string `json:"dataDir"` Delegate map[string]interface{} `json:"delegate"` }
1. cmdAdd函式
1.1 loadFlannelNetConf函式
將傳入的引數解析到NetConf結構體,這裡設定了預設的subnetFile,與dataDir
n, err := loadFlannelNetConf(args.StdinData)
if err != nil {
return err
}
1.2 loadFlannelSubnetEnv函式
從/run/flannel/subnet.env讀取配置,包括:
- FLANNEL_NETWORK=172.30.0.0/16
- FLANNEL_SUBNET=172.30.46.1/24
- FLANNEL_MTU=1500
- FLANNEL_IPMASQ=false
fenv, err := loadFlannelSubnetEnv(n.SubnetFile)
if err != nil {
return err
}
1.3 subnet.env驗證引數合法性
if n.Delegate == nil {
n.Delegate = make(map[string]interface{})
} else {
if hasKey(n.Delegate, "type") && !isString(n.Delegate["type"]) {
return fmt.Errorf("'delegate' dictionary, if present, must have (string) 'type' field")
}
if hasKey(n.Delegate, "name") {
return fmt.Errorf("'delegate' dictionary must not have 'name' field, it'll be set by flannel")
}
if hasKey(n.Delegate, "ipam") {
return fmt.Errorf("'delegate' dictionary must not have 'ipam' field, it'll be set by flannel")
}
}
1.4 doCmdAdd函式
對於未設定的引數初始化,如果未設定ipam則使用預設host-local
func doCmdAdd(args *skel.CmdArgs, n *NetConf, fenv *subnetEnv) error {
n.Delegate["name"] = n.Name
if !hasKey(n.Delegate, "type") {
n.Delegate["type"] = "bridge"
}
if !hasKey(n.Delegate, "ipMasq") {
// if flannel is not doing ipmasq, we should
ipmasq := !*fenv.ipmasq
n.Delegate["ipMasq"] = ipmasq
}
if !hasKey(n.Delegate, "mtu") {
mtu := fenv.mtu
n.Delegate["mtu"] = mtu
}
if n.Delegate["type"].(string) == "bridge" {
if !hasKey(n.Delegate, "isGateway") {
n.Delegate["isGateway"] = true
}
}
if n.CNIVersion != "" {
n.Delegate["cniVersion"] = n.CNIVersion
}
n.Delegate["ipam"] = map[string]interface{}{
"type": "host-local",
"subnet": fenv.sn.String(),
"routes": []types.Route{
{
Dst: *fenv.nw,
},
},
}
return delegateAdd(args.ContainerID, n.DataDir, n.Delegate)
}
2. delegateAdd函式
saveScratchNetConf函式建立/var/lib/cni/flannel目錄
呼叫DelegateAdd函式
func delegateAdd(cid, dataDir string, netconf map[string]interface{}) error {
netconfBytes, err := json.Marshal(netconf)
if err != nil {
return fmt.Errorf("error serializing delegate netconf: %v", err)
}
// save the rendered netconf for cmdDel
if err = saveScratchNetConf(cid, dataDir, netconfBytes); err != nil {
return err
}
result, err := invoke.DelegateAdd(netconf["type"].(string), netconfBytes, nil)
if err != nil {
return err
}
return result.Print()
}
3. 根據設定的bridge,呼叫bridge cni
參照https://blog.csdn.net/zhonglinzhang/article/details/82733201
3.1 setupBridge函式
- 通過netlink.LinkAdd(br)建立網橋,相當於ip link add br-test type bridge
- 然後通過 netlink.LinkSetUp(br)啟動網橋,相當於ip link set dev br-test up
3.2 setupVeth函式
- 呼叫netlink.LinkAdd(veth)建立veth,這個是一個管道,Linux的網絡卡對,在容器對應的namespace下建立好虛擬網路介面,相當於ip link add test-veth0 type veth peer name test-veth1
- 呼叫netlink.LinkSetUp(contVeth)啟動容器端網絡卡,相當於ip link set dev test-veth0 up
- 呼叫netlink.LinkSetNsFd(hostVeth, int(hostNS.Fd()))將host端加入namespace中,相當於ip link set $link netns $ns
- 呼叫netlink.LinkSetMaster(hostVeth, br)綁到bridge,相當於ip link set dev test-veth0 master br-test
與calico不同點:
calico 綁到eth0
flannel 綁到docker0 bridge上,在呼叫bridage cni建立一大堆網路相關