Fedora 31 k8s kubernetes kubeasz 防火墙 firewalld 导致 harbor pod 容器 实例 网络不通 connect: connection refused
防火墙开启状态，harbor pod出错Events:TypeReasonAgeFromMessage-------------------------WarningUnhealthy19m (x319 over 7h28m)kubelet, 10.51.72.167
·
防火墙开启状态，harbor pod出错
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 19m (x319 over 7h28m) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 4m4s (x1728 over 7h27m) kubelet, 192.168.1.167 Back-off restarting failed container
关闭防火墙
sudo systemctl stop firewalld等待一段时间，harbor恢复正常
$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 51 59d 172.20.0.170 192.168.1.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 2/2 Running 1731 58d 172.20.0.178 192.168.1.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 1/1 Running 1452 59d 172.20.0.175 192.168.1.167 <none> <none>
harbor-harbor-database-0 1/1 Running 51 59d 172.20.0.172 192.168.1.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 1/1 Running 1196 59d 172.20.0.174 192.168.1.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1601 126d 172.20.0.177 192.168.1.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1600 126d 172.20.0.167 192.168.1.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 51 59d 172.20.0.181 192.168.1.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 51 59d 172.20.0.182 192.168.1.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 102 59d 172.20.0.176 192.168.1.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 51 59d 172.20.0.169 192.168.1.167 <none> <none>
分析：
禁用防火墙，重启
sudo systemctl disable firewalld
rebootharbor故障
[yeqiang@harbor ~]$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 52 59d 172.20.0.201 192.168.1.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 CrashLoopBackOff 1738 58d 172.20.0.187 192.168.1.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 0/1 CrashLoopBackOff 1457 59d 172.20.0.188 192.168.1.167 <none> <none>
harbor-harbor-database-0 1/1 Running 52 59d 172.20.0.196 192.168.1.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 0/1 Running 1200 59d 172.20.0.191 192.168.1.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 0/1 CrashLoopBackOff 1607 126d 172.20.0.186 192.168.1.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 0/1 CrashLoopBackOff 1606 126d 172.20.0.192 192.168.1.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 52 59d 172.20.0.190 192.168.1.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 52 59d 172.20.0.185 192.168.1.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 104 59d 172.20.0.202 192.168.1.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 52 59d 172.20.0.197 192.168.1.167 <none> <none>
查看故障
[yeqiang@harbor ~]$ kubectl describe pod -n harbor harbor-harbor-core-7d5d7588bb-f6vh5
Name: harbor-harbor-core-7d5d7588bb-f6vh5
Namespace: harbor
Priority: 0
Node: 192.168.1.167/192.168.1.167
Start Time: Tue, 27 Oct 2020 08:53:03 +0800
Labels: app=harbor
component=core
pod-template-hash=7d5d7588bb
release=harbor
Annotations: checksum/configmap: 3e352575d9b0f3eafcd9910f11194507b7605186b387f6a586fe0378e357e944
checksum/secret: 1d810e908cbf53045d899e09f3610777d0875f4cbf737c8457a4c426bc4c96a4
checksum/secret-jobservice: a9746a4cd6c2a3e3fb4a4d4c19beaf93a2652c07692641e8a5a4d90bfdf1ace0
Status: Running
IP: 172.20.0.188
IPs:
IP: 172.20.0.188
Controlled By: ReplicaSet/harbor-harbor-core-7d5d7588bb
Containers:
core:
Container ID: docker://d015706c507ac823bb936bdcd58bf2529c1f015c200d78e087f48d6570322a89
Image: r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0
Image ID: docker-pullable://r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core@sha256:75b5900d2335c87ca34ee80e64e2a6b56cbe218aceaef8614efc31594ad84d38
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 25 Dec 2020 15:57:32 +0800
Finished: Fri, 25 Dec 2020 15:58:34 +0800
Ready: False
Restart Count: 1457
Liveness: http-get http://:8080/api/v2.0/ping delay=300s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/api/v2.0/ping delay=20s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
harbor-harbor-core ConfigMap Optional: false
harbor-harbor-core Secret Optional: false
Environment:
CORE_SECRET: <set to the key 'secret' in secret 'harbor-harbor-core'> Optional: false
JOBSERVICE_SECRET: <set to the key 'JOBSERVICE_SECRET' in secret 'harbor-harbor-jobservice'> Optional: false
Mounts:
/etc/core/app.conf from config (rw,path="app.conf")
/etc/core/ca from ca-download (rw)
/etc/core/key from secret-key (rw,path="key")
/etc/core/private_key.pem from token-service-private-key (rw,path="tls.key")
/etc/core/token from psc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lqv24 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: harbor-harbor-core
Optional: false
secret-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
token-service-private-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
ca-download:
Type: Secret (a volume populated by a Secret)
SecretName: hknaruto.com
Optional: false
psc:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
default-token-lqv24:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-lqv24
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 35m (x319 over 7h44m) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 20m (x1728 over 7h44m) kubelet, 192.168.1.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 9m43s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "8d4a1398862a5a2af909879f58e14fcd96702a53e032b79cf0bc37561285511e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 9m43s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f1d1b91d8b9142b7b3da3a7406b07c716f0ef75902243951f518a892ee26a803" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 9m41s (x4 over 9m44s) kubelet, 192.168.1.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 9m41s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3933a5273489a38d8832a9813972088ae18f570a68a5a4cfd106c844314252e0" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning BackOff 8m31s (x2 over 8m37s) kubelet, 192.168.1.167 Back-off restarting failed container
Normal Pulled 8m19s (x2 over 9m40s) kubelet, 192.168.1.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 8m19s (x2 over 9m40s) kubelet, 192.168.1.167 Created container core
Normal Started 8m19s (x2 over 9m40s) kubelet, 192.168.1.167 Started container core
Warning Unhealthy 4m37s (x16 over 9m17s) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.188:8080/api/v2.0/ping: dial tcp 172.20.0.188:8080: connect: connection refused
当前iptables状态
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination [yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination [yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination 启用防火墙，重启
[yeqiang@harbor startup_firewalld_off]$ sudo systemctl enable firewalld
Created symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service → /usr/lib/systemd/system/firewalld.service.
Created symlink /etc/systemd/system/multi-user.target.wants/firewalld.service → /usr/lib/systemd/system/firewalld.service.
[yeqiang@harbor startup_firewalld_off]$ reboot
检查harbor状态
[yeqiang@harbor ~]$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 53 59d 172.20.0.222 192.168.1.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 CrashLoopBackOff 1742 58d 172.20.0.221 192.168.1.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 0/1 Error 1459 59d 172.20.0.208 192.168.1.167 <none> <none>
harbor-harbor-database-0 1/1 Running 53 59d 172.20.0.205 192.168.1.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 0/1 Running 1202 59d 172.20.0.210 192.168.1.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1609 126d 172.20.0.214 192.168.1.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1608 126d 172.20.0.212 192.168.1.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 53 59d 172.20.0.215 192.168.1.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 53 59d 172.20.0.219 192.168.1.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 106 59d 172.20.0.218 192.168.1.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 53 59d 172.20.0.211 192.168.1.167 <none> <none>
harbor-core错误
[yeqiang@harbor ~]$ kubectl describe pod -n harbor harbor-harbor-core-7d5d7588bb-f6vh5
Name: harbor-harbor-core-7d5d7588bb-f6vh5
Namespace: harbor
Priority: 0
Node: 192.168.1.167/192.168.1.167
Start Time: Tue, 27 Oct 2020 08:53:03 +0800
Labels: app=harbor
component=core
pod-template-hash=7d5d7588bb
release=harbor
Annotations: checksum/configmap: 3e352575d9b0f3eafcd9910f11194507b7605186b387f6a586fe0378e357e944
checksum/secret: 1d810e908cbf53045d899e09f3610777d0875f4cbf737c8457a4c426bc4c96a4
checksum/secret-jobservice: a9746a4cd6c2a3e3fb4a4d4c19beaf93a2652c07692641e8a5a4d90bfdf1ace0
Status: Running
IP: 172.20.0.208
IPs:
IP: 172.20.0.208
Controlled By: ReplicaSet/harbor-harbor-core-7d5d7588bb
Containers:
core:
Container ID: docker://8057d34c9f708d2000e4cfc4449e4be49a7a73417aba76750145d6cc24149260
Image: r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0
Image ID: docker-pullable://r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core@sha256:75b5900d2335c87ca34ee80e64e2a6b56cbe218aceaef8614efc31594ad84d38
Port: 8080/TCP
Host Port: 0/TCP
State: Running
Started: Fri, 25 Dec 2020 16:06:15 +0800
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 25 Dec 2020 16:04:52 +0800
Finished: Fri, 25 Dec 2020 16:05:54 +0800
Ready: False
Restart Count: 1460
Liveness: http-get http://:8080/api/v2.0/ping delay=300s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/api/v2.0/ping delay=20s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
harbor-harbor-core ConfigMap Optional: false
harbor-harbor-core Secret Optional: false
Environment:
CORE_SECRET: <set to the key 'secret' in secret 'harbor-harbor-core'> Optional: false
JOBSERVICE_SECRET: <set to the key 'JOBSERVICE_SECRET' in secret 'harbor-harbor-jobservice'> Optional: false
Mounts:
/etc/core/app.conf from config (rw,path="app.conf")
/etc/core/ca from ca-download (rw)
/etc/core/key from secret-key (rw,path="key")
/etc/core/private_key.pem from token-service-private-key (rw,path="tls.key")
/etc/core/token from psc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lqv24 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: harbor-harbor-core
Optional: false
secret-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
token-service-private-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
ca-download:
Type: Secret (a volume populated by a Secret)
SecretName: hknaruto.com
Optional: false
psc:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
default-token-lqv24:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-lqv24
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 41m (x319 over 7h50m) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 26m (x1728 over 7h50m) kubelet, 192.168.1.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 15m kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "8d4a1398862a5a2af909879f58e14fcd96702a53e032b79cf0bc37561285511e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 15m kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f1d1b91d8b9142b7b3da3a7406b07c716f0ef75902243951f518a892ee26a803" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 15m (x4 over 15m) kubelet, 192.168.1.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 15m kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3933a5273489a38d8832a9813972088ae18f570a68a5a4cfd106c844314252e0" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal Pulled 14m (x2 over 15m) kubelet, 192.168.1.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 14m (x2 over 15m) kubelet, 192.168.1.167 Created container core
Normal Started 14m (x2 over 15m) kubelet, 192.168.1.167 Started container core
Warning Unhealthy 10m (x16 over 15m) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.188:8080/api/v2.0/ping: dial tcp 172.20.0.188:8080: connect: connection refused
Warning BackOff 5m33s (x27 over 14m) kubelet, 192.168.1.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 90s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1729d36f31a0d7a96f92e457e34ec2da90162d3db0835c5b0719f06867da7a9e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 89s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "bb5669a2e842d59dccb9e71647a37365a09459980834a9f356dad0c1561608ea" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 88s (x4 over 91s) kubelet, 192.168.1.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 88s kubelet, 192.168.1.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "df5c38f5ba29f3601cc5fec1df73803ebbbbe53905bed96d75119dd2828af45a" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning Unhealthy 27s (x4 over 57s) kubelet, 192.168.1.167 Readiness probe failed: Get http://172.20.0.208:8080/api/v2.0/ping: dial tcp 172.20.0.208:8080: connect: connection refused
Warning BackOff 15s (x2 over 23s) kubelet, 192.168.1.167 Back-off restarting failed container
Normal Pulled 3s (x2 over 87s) kubelet, 192.168.1.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 3s (x2 over 86s) kubelet, 192.168.1.167 Created container core
Normal Started 3s (x2 over 86s) kubelet, 192.168.1.167 Started container core
iptables状态
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED,DNAT
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
INPUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED,DNAT
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_IN_ZONES (1 references)
target prot opt source destination
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_OUT_ZONES (1 references)
target prot opt source destination
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation (2 references)
target prot opt source destination
FWDI_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
Chain FWDI_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation (2 references)
target prot opt source destination
FWDO_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain FWDO_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain INPUT_ZONES (1 references)
target prot opt source destination
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain INPUT_direct (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation (2 references)
target prot opt source destination
IN_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
Chain IN_FedoraWorkstation_allow (1 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:111 ctstate NEW,UNTRACKED
Chain IN_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain POSTROUTING_ZONES (1 references)
target prot opt source destination
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain POSTROUTING_direct (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation (2 references)
target prot opt source destination
POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain POST_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
Chain INPUT (policy ACCEPT)
target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain POSTROUTING_direct (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
此时，关闭防火墙
[yeqiang@harbor startup_firewalld_on]$ sudo systemctl stop firewalld等待一段时间，故障小时
[yeqiang@harbor ~]$ kubectl get pod -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 53 59d 172.20.0.222 192.168.1.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 Running 1747 58d 172.20.0.221 192.168.1.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 1/1 Running 1464 59d 172.20.0.208 192.168.1.167 <none> <none>
harbor-harbor-database-0 1/1 Running 53 59d 172.20.0.205 192.168.1.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 1/1 Running 1206 59d 172.20.0.210 192.168.1.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1613 126d 172.20.0.214 192.168.1.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1612 126d 172.20.0.212 192.168.1.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 53 59d 172.20.0.215 192.168.1.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 53 59d 172.20.0.219 192.168.1.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 106 59d 172.20.0.218 192.168.1.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 53 59d 172.20.0.211 192.168.1.167 <none> <none>
此时iptables状态
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n
[sudo] password for yeqiang:
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination [yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination [yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination 对比防火墙开启到关闭差异
[yeqiang@harbor iptables]$ diff -y -r startup_firewalld_on startup_firewalld_on2off/
diff -y -r startup_firewalld_on/filter.txt startup_firewalld_on2off/filter.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
INPUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0 ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16 ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER (1 references) <
target prot opt source destination <
<
Chain DOCKER-ISOLATION-STAGE-1 (1 references) <
target prot opt source destination <
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-ISOLATION-STAGE-2 (1 references) <
target prot opt source destination <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-USER (1 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_IN_ZONES (1 references) <
target prot opt source destination <
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain FORWARD_OUT_ZONES (1 references) <
target prot opt source destination <
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation (2 references) <
target prot opt source destination <
FWDI_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
FWDI_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
FWDI_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
FWDI_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
FWDI_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FWDI_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation (2 references) <
target prot opt source destination <
FWDO_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
FWDO_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
FWDO_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
FWDO_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
FWDO_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
<
Chain FWDO_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain INPUT_ZONES (1 references) <
target prot opt source destination <
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 <
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation (2 references) <
target prot opt source destination <
IN_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0 <
IN_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0 <
IN_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0. <
IN_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0 <
IN_FedoraWorkstation_post all -- 0.0.0.0/0 0.0. <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain IN_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain IN_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
Chain KUBE-FIREWALL (2 references) Chain KUBE-FIREWALL (2 references)
target prot opt source destination target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FORWARD (1 references) Chain KUBE-FORWARD (1 references)
target prot opt source destination target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/mangle.txt startup_firewalld_on2off/mangle.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain POSTROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/nat.txt startup_firewalld_on2off/nat.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 <
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 172.20.0.0/16 172.20.0.0/16 RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4
RETURN all -- !172.20.0.0/16 172.20.0.0/24 RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16
Chain DOCKER (2 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain KUBE-FIREWALL (0 references) Chain KUBE-FIREWALL (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references) Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references) Chain KUBE-MARK-DROP (1 references)
target prot opt source destination target prot opt source destination
Chain KUBE-MARK-MASQ (3 references) Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-NODE-PORT (1 references) Chain KUBE-NODE-PORT (1 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-POSTROUTING (1 references) Chain KUBE-POSTROUTING (1 references)
target prot opt source destination target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-SERVICES (2 references) Chain KUBE-SERVICES (2 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain POSTROUTING_ZONES (1 references) <
target prot opt source destination <
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain POSTROUTING_direct (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation (2 references) <
target prot opt source destination <
POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
POST_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
<
Chain POST_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
diff -y -r startup_firewalld_on/raw.txt startup_firewalld_on2off/raw.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/security.txt startup_firewalld_on2off/security.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination target prot opt source destination
对比关闭防火墙状态与开启防火墙后再关闭
[yeqiang@harbor iptables]$ diff -y -r startup_firewalld_off startup_firewalld_on2off/
diff -y -r startup_firewalld_off/filter.txt startup_firewalld_on2off/filter.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0 ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16 ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
<
Chain DOCKER (1 references) <
target prot opt source destination <
<
Chain DOCKER-ISOLATION-STAGE-1 (1 references) <
target prot opt source destination <
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-ISOLATION-STAGE-2 (1 references) <
target prot opt source destination <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-USER (1 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
Chain KUBE-FIREWALL (2 references) Chain KUBE-FIREWALL (2 references)
target prot opt source destination target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FORWARD (1 references) Chain KUBE-FORWARD (1 references)
target prot opt source destination target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/mangle.txt startup_firewalld_on2off/mangle.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/nat.txt startup_firewalld_on2off/nat.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 <
RETURN all -- 172.20.0.0/16 172.20.0.0/16 RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4
RETURN all -- !172.20.0.0/16 172.20.0.0/24 RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16
<
Chain DOCKER (2 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
Chain KUBE-FIREWALL (0 references) Chain KUBE-FIREWALL (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references) Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references) Chain KUBE-MARK-DROP (1 references)
target prot opt source destination target prot opt source destination
Chain KUBE-MARK-MASQ (3 references) Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-NODE-PORT (1 references) Chain KUBE-NODE-PORT (1 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-POSTROUTING (1 references) Chain KUBE-POSTROUTING (1 references)
target prot opt source destination target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-SERVICES (2 references) Chain KUBE-SERVICES (2 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
diff -y -r startup_firewalld_off/raw.txt startup_firewalld_on2off/raw.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/security.txt startup_firewalld_on2off/security.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
可以看到，开机，防火墙关闭状态下，nat Chain PREROUTING, Chain OUTPUT 多了一行DOCKER链
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
打印出规则编号
[yeqiang@harbor iptables]$ sudo iptables -L -n -t nat --line-number
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
2 PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
3 PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
4 DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
num target prot opt source destination
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
2 OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
3 DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination
1 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
2 MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
3 POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
4 POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
5 RETURN all -- 172.20.0.0/16 172.20.0.0/16
6 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
7 RETURN all -- !172.20.0.0/16 172.20.0.0/24
8 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
num target prot opt source destination
1 RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
num target prot opt source destination
1 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
num target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
num target prot opt source destination
1 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
num target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
num target prot opt source destination
1 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
num target prot opt source destination
1 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
num target prot opt source destination
1 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
2 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
num target prot opt source destination
1 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
2 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
Chain OUTPUT_direct (1 references)
num target prot opt source destination
Chain POSTROUTING_ZONES (1 references)
num target prot opt source destination
1 POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain POSTROUTING_direct (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation (2 references)
num target prot opt source destination
1 POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
2 POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
3 POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
4 POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
5 POST_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain POST_FedoraWorkstation_allow (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_deny (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_log (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_post (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_pre (1 references)
num target prot opt source destination
Chain PREROUTING_ZONES (1 references)
num target prot opt source destination
1 PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
num target prot opt source destination
1 PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
2 PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
3 PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
4 PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
5 PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
num target prot opt source destination
手动删除掉这两条规则
[yeqiang@harbor iptables]$ sudo iptables -t nat -D PREROUTING 4
[yeqiang@harbor iptables]$ sudo iptables -t nat -D OUTPUT 3
还是无效，只能手动停止防火墙服务。没有头绪。。。
开源、云原生的融合云平台
更多推荐
0
0- 0
已为社区贡献5条内容
所有评论(0)