Cloudpods的服务运行在一个Kubernetes集群之上，该Kubernets集群的网络方案采用了Calico。因此运行Cloudpods服务的节点的iptables规则被Calico接管。这就导致我们在Cloudpods服务节点上配置的防火墙规则会被Calico配置的iptables规则覆盖，导致防火墙规则不生效。本文介绍如何使用Calico的HostEndpoint和GlobalNetworkPolicy来设置主机节点的防火墙规则。

1. 准备calicoctl工具​

下载二进制

curl -O -L https://github.com/projectcalico/calicoctl/releases/download/v3.12.1/calicoctl
chmod +x calicoctl

设置环境变量

export DATASTORE_TYPE=kubernetes
export KUBECONFIG=/etc/kubernetes/admin.conf

2. 配置HostEndpoint规则​

对每一台主机的每个需要控制防火墙规则接口，定义对应的HostEndpoint规则

- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: <node_name>-<interface_name>
    labels:
      role: master
      env: production
  spec:
    interfaceName: <interface_name>
    node: <node_name>
    expectedIPs: ["<interface_ip>"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: <node_name>-<interface_name>
    labels:
      role: master
      env: production
  spec:
    interfaceName: <interface_name>
    node: <node_name>
    expectedIPs: ["<interface_ip>"]

应用该规则：

./calicoctl apply -f hep.yaml

3. 定义网络规则​

定义好HostEndpoint之后，采用Calico的GlobalNetworkPolicy定义防火墙规则。

- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: <whitelist_gnp_name>
  spec:
    order: 10
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Allow
        protocol: TCP
        source:
          nets: [<src_net_block1>, <src_net_block2>]
        destination:
          ports: [<dst_port1>, <dst_port2>]
    selector: "role==\"master\""
- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: drop-other-ingress
  spec:
    order: 20
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Deny
    selector: "role==\"master\""

应用规则

./calicoctl apply -f gnp.yaml

4. failSafe机制​

为防止用户错误配置导致node无法网络访问的风险，calico设计了failSafe机制，即在用户编写规则有误的情况下，部分端口也不会被封禁，导致节点功能失效。这里是FailSafe端口的信息：https://docs.projectcalico.org/reference/host-endpoints/failsafe

5. 配置举例​

举例：master节点的外网端口只允许80和443端口，其他都禁止：

HostEndpoint定义​

- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master1-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master1
    expectedIPs: ["120.133.60.219"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master2-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master2
    expectedIPs: ["120.133.60.220"]
- apiVersion: projectcalico.org/v3
  kind: HostEndpoint
  metadata:
    name: master3-em4
    labels:
      role: master
      type: external
  spec:
    interfaceName: em4
    node: master3
    expectedIPs: ["120.133.60.221"]

GlobalNetworkPolicy定义​

- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: allow-http-https-traffic-only
  spec:
    order: 10
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Allow
        protocol: TCP
        destination:
          ports: [80,443]
    selector: "role==\"master\" && type==\"external\""
- apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: drop-other-ingress
  spec:
    order: 20
    preDNAT: true
    applyOnForward: true
    ingress:
      - action: Deny