linux - 使用 xfs 的 Kubernetes PVC 挂载失败并显示代码 32

Question

我有一个 Pod 从 PersistentVolumeClaim 安装卷。 PVC 使用 StorageClass 通过 xfs 文件系统配置 EBS 卷。设置如下:

        volumeMounts:
        - mountPath: "/opt/st1"
          name: opt-st1
      volumes:
      - name: opt-st1
        persistentVolumeClaim:
          claimName: st1-xfs-pvc

kind: PersistentVolumeClaim
metadata:
  name: st1-xfs-pvc
  labels:
    app: st1-xfs-pvc
spec:
  storageClassName: st1-xfs-sc
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: st1-xfs-sc
provisioner: kubernetes.io/aws-ebs
parameters:
  type: st1
  fsType: xfs
reclaimPolicy: Retain
mountOptions:
  - debug
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
allowedTopologies:
- matchLabelExpressions:
  - key: failure-domain.beta.kubernetes.io/zone
    values:
    - us-east-1a

当我在基于 EKS 的集群(版本 1.13)上运行此设置时，出现以下错误:

Events:
  Type     Reason                  Age                From                                    Message
  ----     ------                  ----               ----                                    -------
  Normal   Scheduled               45s                default-scheduler                       Successfully assigned jira-node-deployment-5f4f59c44d-jbc4c to ip-10-237-86-124.ec2.internal
  Warning  FailedAttachVolume      40s (x4 over 44s)  attachdetach-controller                 AttachVolume.Attach failed for volume "pvc-50996814-bf53-11e9-848f-0ec61103f6e0" : "Error attaching EBS volume \"vol-077709885f54252c7\"" to instance "i-0fe9867c4129f058e" since volume is in "creating" state
  Normal   SuccessfulAttachVolume  33s                attachdetach-controller                 AttachVolume.Attach succeeded for volume "pvc-50996814-bf53-11e9-848f-0ec61103f6e0"
  Warning  FailedMount             24s                kubelet, ip-10-237-86-124.ec2.internal  MountVolume.MountDevice failed for volume "pvc-50996814-bf53-11e9-848f-0ec61103f6e0" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7 --scope -- mount -t xfs -o debug,defaults /dev/xvdbp /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7
Output: Running scope as unit run-979548.scope.
mount: /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7: wrong fs type, bad option, bad superblock on /dev/xvdbp, missing codepage or helper program, or other error.
  Warning  FailedMount  22s  kubelet, ip-10-237-86-124.ec2.internal  MountVolume.MountDevice failed for volume "pvc-50996814-bf53-11e9-848f-0ec61103f6e0" : mount failed: exit status 32

如果我连接到 Kubernetes Worker 并手动运行相同的命令，我就能够重现该错误:

$ systemd-run --description='Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-068d85e415249b896' --scope -- mount -t xfs -o debug,defaults /dev/xvdcg /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-068d85e415249b896
Running scope as unit run-982245.scope.
mount: /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7: mount point does not exist.

$ mkdir /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7

$ systemd-run --description='Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-068d85e415249b896' --scope -- mount -t xfs -o debug,defaults /dev/xvdcg /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-068d85e415249b896
Running scope as unit run-982245.scope.
mount: /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7: wrong fs type, bad option, bad superblock on /dev/xvdbp, missing codepage or helper program, or other error.

$ echo $?
32

我注意到，通过从命令中删除 debug 选项并再次运行它，然后卷就可以正常安装...

$ systemd-run --description='Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7' --scope -- mount -t xfs -o defaults /dev/xvdbp /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1a/vol-077709885f54252c7
Running scope as unit run-986177.scope.

...并且 Pod 几秒钟后运行良好:

  Normal   Pulled       50s                  kubelet, ip-10-237-86-124.ec2.internal  Container image "nginx:alpine" already present on machine
  Normal   Created      49s                  kubelet, ip-10-237-86-124.ec2.internal  Created container
  Normal   Started      46s                  kubelet, ip-10-237-86-124.ec2.internal  Started container

我还注意到，如果我使用 ext4 而不是 xfs，上述设置可以正常工作。

Answer 1

过了一会儿，我意识到 debug 操作是我自己在 StorageClass 配置中添加的:

mountOptions:
  - debug

删除这两行后，一切都按预期工作。