networking - 具有 ipv6 地址的容器在 k8s/calico 环境中无法连接到外部

Question

我正在尝试在 k8s 环境中测试 ipv6 连接，并安装了 calico 网络插件；问题是容器无法ping到ipv6网关或集群节点的其他地址，k8s和calico的版本是v1.18.2和calico v1.12(也试过v1.13)；配置如下:

centos7，内核为4.4(升级)
开通ipv6转发
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1

印花布配置:

[root@k8s-master-01 ~]# calicoctl get ipp -owide
NAME                  CIDR            NAT    IPIPMODE   VXLANMODE   DISABLED   SELECTOR   
default-ipv4-ippool   10.244.0.0/16   true   Never      Never       false      all()      
default-ipv6-ippool   fc00:f00::/24   true   Never      Never       false      all()      

within the pod, can see ipv6 address is allocated
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1440
        inet 10.244.36.196  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::a8c6:c1ff:fe61:258c  prefixlen 64  scopeid 0x20<link>
        inet6 fc00:fd8:4bce:9a48:4ab7:a333:5ec8:c684  prefixlen 128  scopeid 0x0<global>
        ether aa:c6:c1:61:25:8c  txqueuelen 0  (Ethernet)
        RX packets 23026  bytes 3522721 (3.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 24249  bytes 3598501 (3.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@k8s-worker-01 ~]# ip -6 route show
fc00:fd8:4bce:9a48:4ab7:a333:5ec8:c684 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::ecee:eeff:feee:eeee dev eth0 metric 1024 pref medium

实际上，我从主机用 tcpdump 捕获了消息，并且可以看到一些 icmp 请求进入了 like cali66e9f9aafee 接口(interface)，但看起来没有进一步处理，我检查了 ip6tables 并看到没有任何包进入 masqurade 的右链

[root@k8s-worker-01 ~]# ip6tables -t nat -vnL
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1    80 cali-PREROUTING  all      *      *       ::/0                 ::/0                 /* cali:6gwbT8clXdHdC1b1 */

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 791 packets, 63280 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  796 63680 cali-OUTPUT  all      *      *       ::/0                 ::/0                 /* cali:tVnHkvAo15HuiPy0 */

Chain POSTROUTING (policy ACCEPT 791 packets, 63280 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  796 63680 cali-POSTROUTING  all      *      *       ::/0                 ::/0                 /* cali:O3lYWMrLQYEMJtB5 */

Chain cali-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  796 63680 cali-fip-dnat  all      *      *       ::/0                 ::/0                 /* cali:GBTAv2p5CwevEyJm */

Chain cali-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
  796 63680 cali-fip-snat  all      *      *       ::/0                 ::/0                 /* cali:Z-c7XtVd2Bq7s_hA */
  796 63680 cali-nat-outgoing  all      *      *       ::/0                 ::/0                 /* cali:nYKhEzDlr11Jccal */

Chain cali-PREROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    80 cali-fip-dnat  all      *      *       ::/0                 ::/0                 /* cali:r6XmIziWUJsdOK6Z */

Chain cali-fip-dnat (2 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain cali-fip-snat (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain cali-nat-outgoing (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all      *      *       ::/0                 ::/0                 /* cali:Ir_z2t1P6-CxTDof */ match-set cali60masq-ipam-pools src ! match-set cali60all-ipam-pools dst

尝试了很多次，但都失败了，我错过了什么吗？

问候

Answer 1

在您的集群上启用 ipv6 并不像您做的那么简单。仅仅在你的网络中配置 ipv6 并不适用于 Kubernetes。

本类(class)的第一个也是最重要的主题是 IPv4/IPv6 dual-stack是一个阿尔法特征。作为任何 alpha 功能，它可能无法按预期工作。

请转this文档以更好地了解使其在您的集群中工作的必要条件(您必须添加一个功能门)。

还有一些工作正在做以使引导 Kubernetes cluster with Dual Stack using Kubeadm 成为可能。 ，但它还不能使用，并且没有 ETA。

this repository 中有一些 IPv6 和双栈设置与其他网络插件的示例.

This project serves two primary purposes: (i) study and validate ipv6 support in kubernetes and associated plugins (ii) provide a dev environment for implementing and testing additional functionality (e.g.dual-stack)