[系统运维]Calico网络策略原理

注：本文基于Calico v3.20.1版本编写

1 calico支持网络策略的基础

calico网络插件中，所有pod的网络设备并没有像flannel一样连接到网桥docker0上，这样每个pod的网络都有独立的链路，而这就是支持网络策略的基础。如果都连接到网桥，那所有pod都互通，并且没有办法做隔离。

2 实例分析

我们以上一篇——Calico网络策略为背景，看下我们添加的策略是如何生效的。

kind: NetworkPolicy
apiVersion: projectcalico.org/v3
metadata:
  name: allow-same-namespace-green
  namespace: default
spec:
  selector: color == 'green'

  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: color == 'yellow'

  egress:
  - action: Allow
    protocol: TCP
    source:
      selector: color == 'green'
    destination:
      ports:
        - 9999

网络策略的实现仍然是基于iptables，因此还是raw->mangle->nat->filter这一套，我们就不一一展开，只对filter这个进行说明。

测试所在pod的网卡为，cali93097fbc44d

[root@node1 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.244.166.129  0.0.0.0         255.255.255.255 UH    0      0        0 cali93097fbc44d
...

当报文从测试pod发出，经过raw->mangle->nat的各个表链，经过路由决策，确定报文不是发完当前host，就会进入forward流程，来到filter表中，进入cali-FORWARD chain中，

[root@node1 ~]# iptables -vn -t filter -L FORWARD
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
41360 4224K cali-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:wUHhoiAYhphO9Mso */
...

在cali-FORWARD中，由于报文是从cali93097fbc44d进入，因此会匹配到第二条cali-from-wl-dispatch

[root@node1 ~]# iptables -vn -t filter -L cali-FORWARD
Chain cali-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination         
41471 4240K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:vjrMJCRpqwy5oRoX */ MARK and 0xfff1ffff
41471 4240K cali-from-hep-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:A_sPAO0mcxbT9mOV */ mark match 0x0/0x10000
 2192  246K cali-from-wl-dispatch  all  --  cali+  *       0.0.0.0/0            0.0.0.0/0            /* cali:8ZoYfO5HKXWbB3pk */
39279 3994K cali-to-wl-dispatch  all  --  *      cali+   0.0.0.0/0            0.0.0.0/0            /* cali:jdEuaPBe14V2hutn */
    7   420 cali-to-hep-forward  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:12bc6HljsMKsmfr- */
    7   420 cali-cidr-block  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:NOSxoaGx8OIstr1z */

而cali-from-wl-dispatch里就是各个pod网络设备的匹配规则了，该node上有几个pod，就会有几条匹配规则，这里肯定就匹配到cali-fw-cali93097fbc44d

root@node1 ~]# iptables -vn -t filter -L cali-from-wl-dispatch
Chain cali-from-wl-dispatch (2 references)
 pkts bytes target     prot opt in     out     source               destination         
35037 3502K cali-fw-cali4a245372073  all  --  cali4a245372073 *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:3ZusskG_3Ca3VOIL */
    0     0 cali-fw-cali707398f917c  all  --  cali707398f917c *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:M-QKg0w_9mBlCw0H */
 7272  607K cali-fw-cali8515f2aefbe  all  --  cali8515f2aefbe *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:Gzwe6u1_zA2pfkwA */
    2   168 cali-fw-cali93097fbc44d  all  --  cali93097fbc44d *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:1p5khMZK7EFGJyut */
    0     0 cali-fw-calic3248e80700  all  --  calic3248e80700 *       0.0.0.0/0            0.0.0.0/0           [goto]  /* cali:GCPyCoAiS9kNxEWA */
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:K3T6ZG38-FX9omYZ */ /* Unknown interface */

cali-fw-cali93097fbc44d chain就是关于这个pod的所有出口规则，也就是egress，首先会匹配到的是cali-po-_sLkPxPTmSi3LEyBEL_F

[root@node1 ~]# iptables -vn -t filter -L cali-fw-cali93097fbc44d
Chain cali-fw-cali93097fbc44d (2 references)
 pkts bytes target     prot opt in     out     source               destination         
...
    3   252 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:gPKxhI74-DaPbbKA */ /* Start of policies */ MARK and 0xfffdffff
    3   252 cali-po-_sLkPxPTmSi3LEyBEL_F  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:90eQeqZMb-_qic6G */ mark match 0x0/0x20000
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:GlgnD49_7My26f-d */ /* Return if policy accepted */ mark match 0x10000/0x10000
    1    84 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:fDkgHxOLOpTv9xD6 */ /* Drop if no policies passed packet */ mark match 0x0/0x20000
...

cali-po-_sLkPxPTmSi3LEyBEL_F其实对应的就是我们测试的策略规则里的egress，源ip使用ipset保存，即测试pod ip，

[root@node1 ~]# iptables -vn -t filter -L cali-po-_sLkPxPTmSi3LEyBEL_F
Chain cali-po-_sLkPxPTmSi3LEyBEL_F (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:LcJtzkK0Ol0FbU8Z */ match-set cali40s:Dg3pLGe1BF-EMx2RxAeEJIv src multiport dports 9999 MARK or 0x10000
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:SFUTPPobAHfmUfeo */ mark match 0x10000/0x10000
[root@node1 ~]# ipset list cali40s:Dg3pLGe1BF-EMx2RxAeEJIv
Name: cali40s:Dg3pLGe1BF-EMx2RxAeEJIv
Type: hash:net
Revision: 3
Header: family inet hashsize 1024 maxelem 1048576
Size in memory: 16816
References: 1
Members:
10.244.166.129

匹配到这条规则后，报文会被标记0x10000，返回后就会被cali-fw-cali93097fbc44d里的下一条规则return，然后继续后面的postrouting和发送报文。

如果没匹配到，就会到再下一条规则，而这就是默认的drop，报文也就被丢弃了。

    1    84 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:fDkgHxOLOpTv9xD6 */ /* Drop if no policies passed packet */ mark match 0x0/0x20000