docker - kubernetes 安装和 kube-dns : open/run/flannel/subnet. env: no such file or directory

Question

概述

kubeadm init 和网络设置后，kube-dns 无法启动(SetupNetworkError):

Error syncing pod, skipping: failed to "SetupNetwork" for 
"kube-dns-654381707-w4mpg_kube-system" with SetupNetworkError: 
"Failed to setup network for pod 
\"kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\" 
using network plugins \"cni\": open /run/flannel/subnet.env: 
no such file or directory; Skipping pod"

Kubernetes 版本

Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:42:39Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

环境

适用于 Mac 的 VMWare Fusion

操作系统

NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

内核(例如 uname -a)

Linux ubuntu-master 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

问题是什么

kube-system   kube-dns-654381707-w4mpg                0/3       ContainerCreating   0          2m
FirstSeen     LastSeen        Count   From                    SubobjectPath   Type            Reason          Message
  ---------     --------        -----   ----                    -------------   --------        ------          -------
  3m            3m              1       {default-scheduler }                    Normal          Scheduled       Successfully assigned kube-dns-654381707-w4mpg to ubuntu-master
  2m            1s              177     {kubelet ubuntu-master}                 Warning         FailedSync      Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-654381707-w4mpg_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod"

我期望发生的事情

kube-dns 正在运行

如何重现它

root@ubuntu-master:~# kubeadm init
Running pre-flight checks
<master/tokens> generated token: "247a8e.b7c8c1a7685bf204"
<master/pki> generated Certificate Authority key and certificate:
Issuer: CN=kubernetes | Subject: CN=kubernetes | CA: true
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2026-11-08 11:40:21 +0000 UTC
Public: /etc/kubernetes/pki/ca-pub.pem
Private: /etc/kubernetes/pki/ca-key.pem
Cert: /etc/kubernetes/pki/ca.pem
<master/pki> generated API Server key and certificate:
Issuer: CN=kubernetes | Subject: CN=kube-apiserver | CA: false
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2017-11-10 11:40:21 +0000 UTC
Alternate Names: [172.20.10.4 10.96.0.1 kubernetes kubernetes.default     kubernetes.default.svc kubernetes.default.svc.cluster.local]
Public: /etc/kubernetes/pki/apiserver-pub.pem
Private: /etc/kubernetes/pki/apiserver-key.pem
Cert: /etc/kubernetes/pki/apiserver.pem
<master/pki> generated Service Account Signing keys:
Public: /etc/kubernetes/pki/sa-pub.pem
Private: /etc/kubernetes/pki/sa-key.pem
<master/pki> created keys and certificates in "/etc/kubernetes/pki"
<util/kubeconfig> created "/etc/kubernetes/kubelet.conf"
<util/kubeconfig> created "/etc/kubernetes/admin.conf"
<master/apiclient> created API client configuration
<master/apiclient> created API client, waiting for the control plane to become ready
<master/apiclient> all control plane components are healthy after 14.053453 seconds
<master/apiclient> waiting for at least one node to register and become ready
<master/apiclient> first node is ready after 0.508561 seconds
<master/apiclient> attempting a test deployment
<master/apiclient> test deployment succeeded
<master/discovery> created essential addon: kube-discovery, waiting for it to become ready
<master/discovery> kube-discovery is ready after 1.503838 seconds
<master/addons> created essential addon: kube-proxy
<master/addons> created essential addon: kube-dns

Kubernetes master initialised successfully!

You can now join any number of machines by running the following on each node:

kubeadm join --token=247a8e.b7c8c1a7685bf204 172.20.10.4
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl get pods --all-namespaces
NAMESPACE     NAME                                    READY     STATUS              RESTARTS   AGE
kube-system   dummy-2088944543-eo1ua                  1/1       Running             0          47s
kube-system   etcd-ubuntu-master                      1/1       Running             3          51s
kube-system   kube-apiserver-ubuntu-master            1/1       Running             0          49s
kube-system   kube-controller-manager-ubuntu-master   1/1       Running             3          51s
kube-system   kube-discovery-1150918428-qmu0b         1/1       Running             0          46s
kube-system   kube-dns-654381707-mv47d                0/3       ContainerCreating   0          44s
kube-system   kube-proxy-k0k9q                        1/1       Running             0          44s
kube-system   kube-scheduler-ubuntu-master            1/1       Running             3          51s
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl apply -f https://git.io/weave-kube
daemonset "weave-net" created
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl get pods --all-namespaces
NAMESPACE     NAME                                    READY     STATUS              RESTARTS   AGE
kube-system   dummy-2088944543-eo1ua                  1/1       Running             0          47s
kube-system   etcd-ubuntu-master                      1/1       Running             3          51s
kube-system   kube-apiserver-ubuntu-master            1/1       Running             0          49s
kube-system   kube-controller-manager-ubuntu-master   1/1       Running             3          51s
kube-system   kube-discovery-1150918428-qmu0b         1/1       Running             0          46s
kube-system   kube-dns-654381707-mv47d                0/3       ContainerCreating   0          44s
kube-system   kube-proxy-k0k9q                        1/1       Running             0          44s
kube-system   kube-scheduler-ubuntu-master            1/1       Running             3          51s
kube-system   weave-net-ja736                         2/2       Running             0          1h

Answer 1

看起来你在运行 kubeadm init 之前已经配置了 flannel .您可以尝试通过删除法兰绒来解决此问题(删除配置文件 rm -f /etc/cni/net.d/*flannel* 可能就足够了)，但最好重新开始。