kubernetes - 卷 "nfs": mount failed: exit status 32 的 MountVolume.SetUp 失败

Question

这是第一个问题之后的第二个问题
PersistentVolumeClaim is not bound: "nfs-pv-provisioning-demo"

我正在建立一个仅使用一个节点的 kubernetes 实验室并学习设置 kubernetes nfs。我正在从以下链接逐步关注 kubernetes nfs 示例:https://github.com/kubernetes/examples/tree/master/staging/volumes/nfs

根据'helmbert'提供的反馈，我修改了内容
https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/provisioner/nfs-server-gce-pv.yaml

它有效，我再也看不到事件“PersistentVolumeClaim is not bound: “nfs-pv-provisioning-demo””。

$ cat nfs-server-local-pv01.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01
  labels:
    type: local
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/data01"


$ cat nfs-server-local-pvc01.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-pv-provisioning-demo
  labels:
    demo: nfs-pv-provisioning
spec:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 5Gi


$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM     STORAGECLASS   REASON    AGE
pv01      10Gi       RWO            Retain           Available                                      4s


$ kubectl get pvc
NAME                       STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-pv-provisioning-demo   Bound     pv01      10Gi       RWO                           2m



$ kubectl get pod
NAME                READY     STATUS              RESTARTS   AGE
nfs-server-nlzlv    1/1       Running             0          1h


$ kubectl describe pods nfs-server-nlzlv
Name:           nfs-server-nlzlv
Namespace:      default
Node:           lab-kube-06/10.0.0.6
Start Time:     Tue, 21 Nov 2017 19:32:21 +0000
Labels:         role=nfs-server
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"default","name":"nfs-server","uid":"b1b00292-cef2-11e7-8ed3-000d3a04eb...
Status:         Running
IP:             10.32.0.3
Created By:     ReplicationController/nfs-server
Controlled By:  ReplicationController/nfs-server
Containers:
  nfs-server:
    Container ID:   docker://1ea76052920d4560557cfb5e5bfc9f8efc3af5f13c086530bd4e0aded201955a
    Image:          gcr.io/google_containers/volume-nfs:0.8
    Image ID:       docker-pullable://gcr.io/google_containers/volume-nfs@sha256:83ba87be13a6f74361601c8614527e186ca67f49091e2d0d4ae8a8da67c403ee
    Ports:          2049/TCP, 20048/TCP, 111/TCP
    State:          Running
      Started:      Tue, 21 Nov 2017 19:32:43 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /exports from mypvc (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-grgdz (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  mypvc:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  nfs-pv-provisioning-demo
    ReadOnly:   false
  default-token-grgdz:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-grgdz
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                 node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

我继续其余的步骤并到达“设置假后端”部分并运行以下命令:

$ kubectl create -f examples/volumes/nfs/nfs-busybox-rc.yaml

对于两个 nfs-busybox pod，我看到状态“ContainerCreating”并且永远不会更改为“Running”。这是因为容器镜像适用于 yaml 中所示的 Google Cloud 吗？

https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/nfs-server-rc.yaml

  containers:
  - name: nfs-server
    image: gcr.io/google_containers/volume-nfs:0.8
    ports:
      - name: nfs
        containerPort: 2049
      - name: mountd
        containerPort: 20048
      - name: rpcbind
        containerPort: 111
    securityContext:
      privileged: true
    volumeMounts:
      - mountPath: /exports
        name: mypvc

我是否必须将该“图像”行替换为其他内容，因为我在此实验室中不使用 Google Cloud？我的实验室中只有一个节点。我是否必须重写上面“容器”的定义？我应该用什么替换“图像”行？我需要从某个地方下载 dockerized 'nfs image' 吗？

$ kubectl describe pvc nfs
Name:          nfs
Namespace:     default
StorageClass:
Status:        Bound
Volume:        nfs
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
Capacity:      1Mi
Access Modes:  RWX
Events:        <none>


$ kubectl describe pv nfs
Name:            nfs
Labels:          <none>
Annotations:     pv.kubernetes.io/bound-by-controller=yes
StorageClass:
Status:          Bound
Claim:           default/nfs
Reclaim Policy:  Retain
Access Modes:    RWX
Capacity:        1Mi
Message:
Source:
    Type:      NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    10.111.29.157
    Path:      /
    ReadOnly:  false
Events:        <none>


$ kubectl get rc
NAME          DESIRED   CURRENT   READY     AGE
nfs-busybox   2         2         0         25s
nfs-server    1         1         1         1h


$ kubectl get pod
NAME                READY     STATUS              RESTARTS   AGE
nfs-busybox-lmgtx   0/1       ContainerCreating   0          3m
nfs-busybox-xn9vz   0/1       ContainerCreating   0          3m
nfs-server-nlzlv    1/1       Running             0          1h


$ kubectl get service
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP                      20m
nfs-server   ClusterIP   10.111.29.157   <none>        2049/TCP,20048/TCP,111/TCP   9s


$ kubectl describe services nfs-server
Name:              nfs-server
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          role=nfs-server
Type:              ClusterIP
IP:                10.111.29.157
Port:              nfs  2049/TCP
TargetPort:        2049/TCP
Endpoints:         10.32.0.3:2049
Port:              mountd  20048/TCP
TargetPort:        20048/TCP
Endpoints:         10.32.0.3:20048
Port:              rpcbind  111/TCP
TargetPort:        111/TCP
Endpoints:         10.32.0.3:111
Session Affinity:  None
Events:            <none>

$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                              STORAGECLASS   REASON    AGE
nfs       1Mi        RWX            Retain           Bound     default/nfs                                                 38m
pv01      10Gi       RWO            Retain           Bound     default/nfs-pv-provisioning-demo                            1h

我看到重复事件 - 卷“nfs”的 MountVolume.SetUp 失败:安装失败:退出状态 32

    $ kubectl describe pod nfs-busybox-lmgtx
Name:           nfs-busybox-lmgtx
Namespace:      default
Node:           lab-kube-06/10.0.0.6
Start Time:     Tue, 21 Nov 2017 20:39:35 +0000
Labels:         name=nfs-busybox
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"default","name":"nfs-busybox","uid":"15d683c2-cefc-11e7-8ed3-000d3a04e...
Status:         Pending
IP:
Created By:     ReplicationController/nfs-busybox
Controlled By:  ReplicationController/nfs-busybox
Containers:
  busybox:
    Container ID:
    Image:         busybox
    Image ID:
    Port:          <none>
    Command:
      sh
      -c
      while true; do date > /mnt/index.html; hostname >> /mnt/index.html; sleep $(($RANDOM % 5 + 5)); done
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /mnt from nfs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-grgdz (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  nfs:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  nfs
    ReadOnly:   false
  default-token-grgdz:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-grgdz
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                 node.alpha.kubernetes.io/unreachable:NoExecute for 300s

Events:
  Type     Reason                 Age   From                  Message
  ----     ------                 ----  ----                  -------
  Normal   Scheduled              17m   default-scheduler     Successfully assigned nfs-busybox-lmgtx to lab-kube-06
  Normal   SuccessfulMountVolume  17m   kubelet, lab-kube-06  MountVolume.SetUp succeeded for volume "default-token-grgdz"
  Warning  FailedMount            17m   kubelet, lab-kube-06  MountVolume.SetUp failed for volume "nfs" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/15d8d6d6-cefc-11e7-8ed3-000d3a04ebcd/volumes/kubernetes.io~nfs/nfs --scope -- mount -t nfs 10.111.29.157:/ /var/lib/kubelet/pods/15d8d6d6-cefc-11e7-8ed3-000d3a04ebcd/volumes/kubernetes.io~nfs/nfs
Output: Running scope as unit run-43641.scope.
mount: wrong fs type, bad option, bad superblock on 10.111.29.157:/,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

  Warning  FailedMount  9m (x4 over 15m)  kubelet, lab-kube-06  Unable to mount volumes for pod "nfs-busybox-lmgtx_default(15d8d6d6-cefc-11e7-8ed3-000d3a04ebcd)": timeout expired waiting for volumes to attach/mount for pod "default"/"nfs-busybox-lmgtx". list of unattached/unmounted volumes=[nfs]
  Warning  FailedMount  4m (x8 over 15m)  kubelet, lab-kube-06  (combined from similar events): Unable to mount volumes for pod "nfs-busybox-lmgtx_default(15d8d6d6-cefc-11e7-8ed3-000d3a04ebcd)": timeout expired waiting for volumes to attach/mount for pod "default"/"nfs-busybox-lmgtx". list of unattached/unmounted volumes=[nfs]
  Warning  FailedSync   2m (x7 over 15m)  kubelet, lab-kube-06  Error syncing pod



$ kubectl describe pod nfs-busybox-xn9vz
Name:           nfs-busybox-xn9vz
Namespace:      default
Node:           lab-kube-06/10.0.0.6
Start Time:     Tue, 21 Nov 2017 20:39:35 +0000
Labels:         name=nfs-busybox
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"default","name":"nfs-busybox","uid":"15d683c2-cefc-11e7-8ed3-000d3a04e...
Status:         Pending
IP:
Created By:     ReplicationController/nfs-busybox
Controlled By:  ReplicationController/nfs-busybox
Containers:
  busybox:
    Container ID:
    Image:         busybox
    Image ID:
    Port:          <none>
    Command:
      sh
      -c
      while true; do date > /mnt/index.html; hostname >> /mnt/index.html; sleep $(($RANDOM % 5 + 5)); done
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /mnt from nfs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-grgdz (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  nfs:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  nfs
    ReadOnly:   false
  default-token-grgdz:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-grgdz
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                 node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age               From                  Message
  ----     ------       ----              ----                  -------
  Warning  FailedMount  59m (x6 over 1h)  kubelet, lab-kube-06  Unable to mount volumes for pod "nfs-busybox-xn9vz_default(15d7fb5e-cefc-11e7-8ed3-000d3a04ebcd)": timeout expired waiting for volumes to attach/mount for pod "default"/"nfs-busybox-xn9vz". list of unattached/unmounted volumes=[nfs]
  Warning  FailedMount  7m (x32 over 1h)  kubelet, lab-kube-06  (combined from similar events): MountVolume.SetUp failed for volume "nfs" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/15d7fb5e-cefc-11e7-8ed3-000d3a04ebcd/volumes/kubernetes.io~nfs/nfs --scope -- mount -t nfs 10.111.29.157:/ /var/lib/kubelet/pods/15d7fb5e-cefc-11e7-8ed3-000d3a04ebcd/volumes/kubernetes.io~nfs/nfs
Output: Running scope as unit run-59365.scope.
mount: wrong fs type, bad option, bad superblock on 10.111.29.157:/,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
  Warning  FailedSync  2m (x31 over 1h)  kubelet, lab-kube-06  Error syncing pod

Answer 1

遇到了同样的问题，

sudo apt install nfs-kernel-server

直接在节点上为 ubuntu 18.04 服务器修复了它。