[root@node1 ~]# kubectl get pod -A | grep cil
kube-system cilium-mbjx2 1/1 Running 3 (39m ago) 29h
kube-system cilium-operator-5547b984f4-5d9c8 1/1 Running 3 (39m ago) 29h
kube-system cilium-operator-5547b984f4-z9kgk 1/1 Running 3 (39m ago) 29h
kube-system cilium-pc8hh 1/1 Running 3 (39m ago) 29h
[root@node1 ~]# kubectl get svc -n nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service LoadBalancer 10.233.50.245 172.27.0.7 80:32180/TCP 95m
[root@node1 ~]#
[root@node1 ~]# curl -I 172.27.0.7
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 08 May 2024 07:42:02 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/8.2.7
[root@node1 ~]# ping 172.27.0.7
PING 172.27.0.7 (172.27.0.7) 56(84) bytes of data.
From 172.27.0.7 icmp_seq=1 Destination Port Unreachable
From 172.27.0.7 icmp_seq=2 Destination Port Unreachable
From 172.27.0.7 icmp_seq=3 Destination Port Unreachable
^C
--- 172.27.0.7 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3115ms
1
Garrettx 11 天前 via iPhone
跟网络插件没关系,跟 vip 宣告模式有关系。要用 arping 去 ping ,试试看。
|
2
ZeroAsh 11 天前
nft list ruleset 看看,我印象中规则都是 l4proto tcp/l4proto udp (看你暴露的 service port 是 udp 还是 tcp )
|
3
ukec OP @ZeroAsh [root@node1 ~]# kubectl get svc -n nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-service LoadBalancer 10.233.49.91 172.27.0.7 80:32109/TCP 20h [root@node1 ~]# arping 172.27.0.7 Interface "lo" is not ARPable |
4
ukec OP @ZeroAsh
``` [root@node1 ~]# kubectl get svc -n nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-service LoadBalancer 10.233.49.91 172.27.0.7 80:32109/TCP 20h [root@node1 ~]# nft list ruleset | grep 172.27.0.7 [root@node1 ~]# [root@node1 ~]# nft list ruleset | grep 10.233.49.91 [root@node1 ~]# [root@node1 ~]# [root@node1 ~]# nft list ruleset | grep 80 counter packets 73636 bytes 25380793 jump CILIUM_OUTPUT meta mark & 0x00000e00 == 0x00000800 counter packets 0 bytes 0 accept meta mark & 0x00000f00 != 0x00000e00 meta mark & 0x00000f00 != 0x00000d00 meta mark & 0x00000f00 != 0x00000400 meta mark & 0x00000e00 != 0x00000a00 meta mark & 0x00000e00 != 0x00000800 meta mark & 0x00000f00 != 0x00000f00 counter packets 73464 bytes 25359005 meta mark set mark and 0xfffff0ff xor 0xc00 ip saddr 127.0.0.0/8 counter packets 3 bytes 180 return meta mark & 0x00004000 != 0x00004000 counter packets 53 bytes 3180 return oifname "cilium_host" ip saddr != 10.233.65.0/24 ip daddr != 10.233.65.0/24 counter packets 18 bytes 1080 snat to 10.233.65.116 oifname "lxc*" meta mark & 0x00000e00 == 0x00000800 counter packets 0 bytes 0 notrack oifname "cilium_host" meta mark & 0x00000e00 == 0x00000800 counter packets 0 bytes 0 notrack ip saddr 169.254.25.10 tcp sport 8080 counter packets 0 bytes 0 notrack ip daddr 169.254.25.10 tcp dport 8080 counter packets 0 bytes 0 notrack [root@node1 ~]# nft list ruleset | grep 32109 [root@node1 ~]# ``` |
5
guanzhangzhang 11 天前
你可以去看看源码,cilium 是不是类似 kube-proxy 的 ipvs 一个安全更新
有人 https://github.com/kubernetes/kubernetes/issues/72236 发现 ipvs 下,访问 svcIP+宿主机的端口,例如 22 也能访问,这不安全,然后就有大佬 2022/09/02 [合入的 pr]( https://github.com/kubernetes/kubernetes/pull/108460) 加了个链 `KUBE-IPVS-FILTER` 让 svcIP:非 svcPort 无法访问,ping 也 ping 不通了 |
7
ukec OP @guanzhangzhang 我这是 ping 不通,但不影响访问,所以想看看为什么 ping 不通 Load Balancer
|
8
ukec OP @ZeroAsh
``` # MetalLB deployment metallb_enabled: true metallb_speaker_enabled: "{{ metallb_enabled }}" metallb_namespace: metallb-system metallb_protocol: "layer2" metallb_config: address_pools: primary: ip_range: - 172.27.0.7-172.27.0.9 auto_assign: true layer2: - primary ``` |
9
guanzhangzhang 10 天前
@ukec #7 我知道,可能也是类似原因,如果之前版本可以,去 github compare 下两个版本
|
10
ukec OP |
11
guanzhangzhang 6 天前
@ukec #10 他说他也发现是这样
|
12
wencaiwulue 3 天前
用 kubevpn 可能可以解决问题?不过我没试过 cilium 的 cni... https://github.com/kubenetworks/kubevpn
|
13
wencaiwulue 3 天前
看看在集群中的 pod 是不是可以 ping 通尼?
|