proxy - iptables模式下的kube-proxy在路由中不起作用

Question

我有的是

Kubernetes:v.1.1.2

iptables v1.4.21

内核:3.10.0-327.3.1.el7.x86_64 Centos

通过绒布udp

进行联网

没有云提供商

我所做的

我已使用 --proxy_mode = iptables 参数启用了它。我检查了iptables

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
KUBE-SERVICES  all  --  anywhere             anywhere             /* kubernetes service portals */
DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
KUBE-SERVICES  all  --  anywhere             anywhere             /* kubernetes service portals */
DOCKER     all  --  anywhere            !loopback/8           ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  SIDR26KUBEAPMORANGE-005/26  anywhere
MASQUERADE  all  --  172.17.0.0/16        anywhere
MASQUERADE  all  --  anywhere             anywhere             /* kubernetes service traffic requiring SNAT */ mark match 0x4d415351

Chain DOCKER (2 references)
target     prot opt source               destination

Chain KUBE-NODEPORTS (1 references)
target     prot opt source               destination

Chain KUBE-SEP-3SX6E5663KCZDTLC (1 references)
target     prot opt source               destination
MARK       all  --  172.20.10.130        anywhere             /* default/nc-service: */ MARK set 0x4d415351
DNAT       tcp  --  anywhere             anywhere             /* default/nc-service: */ tcp to:172.20.10.130:9000

Chain KUBE-SEP-Q4LJF4YJE6VUB3Y2 (1 references)
target     prot opt source               destination
MARK       all  --  SIDR26KUBEAPMORANGE-001.serviceengage.com  anywhere             /* default/kubernetes: */ MARK set 0x4d415351
DNAT       tcp  --  anywhere             anywhere             /* default/kubernetes: */ tcp to:10.62.66.254:9443

Chain KUBE-SERVICES (2 references)
target     prot opt source               destination
KUBE-SVC-6N4SJQIF3IX3FORG  tcp  --  anywhere             172.21.0.1           /* default/kubernetes: cluster IP */ tcp dpt:https
KUBE-SVC-362XK5X6TGXLXGID  tcp  --  anywhere             172.21.145.28        /* default/nc-service: cluster IP */ tcp dpt:commplex-main
KUBE-NODEPORTS  all  --  anywhere             anywhere             /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL

Chain KUBE-SVC-362XK5X6TGXLXGID (1 references)
target     prot opt source               destination
KUBE-SEP-3SX6E5663KCZDTLC  all  --  anywhere             anywhere             /* default/nc-service: */

Chain KUBE-SVC-6N4SJQIF3IX3FORG (1 references)
target     prot opt source               destination
KUBE-SEP-Q4LJF4YJE6VUB3Y2  all  --  anywhere             anywhere             /* default/kubernetes: */

当我从另一台机器向服务ip发出 nc 请求时，在我的情况下是10.116.0.2，我得到了如下错误
nc -v 172.21.145.28 5000
Ncat:版本6.40( http://nmap.org/ncat)
你好
Ncat:连接超时。

而当我向172.20.10.130:9000服务器请求时，它工作正常。

nc -v 172.20.10.130 9000
Ncat:版本6.40( http://nmap.org/ncat)
Ncat:已连接到172.20.10.130:9000。
你好
是

从dmesg日志中，我可以看到

[10153.318195] DBG@OUTPUT: IN= OUT=eth0 SRC=10.62.66.223 DST=172.21.145.28 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=62466 DF PROTO=TCP SPT=59075 DPT=5000 WINDOW=29200 RES=0x00 SYN URGP=0
[10153.318282] DBG@OUTPUT: IN= OUT=eth0 SRC=10.62.66.223 DST=172.21.145.28 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=62466 DF PROTO=TCP SPT=59075 DPT=5000 WINDOW=29200 RES=0x00 SYN URGP=0
[10153.318374] DBG@POSTROUTING: IN= OUT=flannel0 SRC=10.62.66.223 DST=172.20.10.130 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=62466 DF PROTO=TCP SPT=59075 DPT=9000 WINDOW=29200 RES=0x00 SYN URGP=0

我发现我是否在Pod正在运行的计算机上。我可以通过服务ip成功连接。

nc -v 172.21.145.28 5000
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 172.21.145.28:5000.
hello
yes

我想知道为什么以及如何解决它。

Answer 1

我在Kubernetes 1.1.7和1.2.0上遇到了同样的问题。我开始没有--ip-masq的法兰绒，并为kube-proxy添加参数--masquerade-all = true，这很有帮助。