docker - 谷歌 Kubernetes 引擎 : Not seeing mount persistent volume in the instance

Question

我使用命令创建了一个 200G 的磁盘 gcloud compute disks create --size 200GB my-disk

然后创建一个 PersistentVolume

    apiVersion: v1    kind: PersistentVolume    metadata:      name: my-volume    spec:      capacity:        storage: 200Gi      accessModes:        - ReadWriteOnce      gcePersistentDisk:        pdName: my-disk        fsType: ext4

然后创建了一个 PersistentVolumeClaim

    apiVersion: v1    kind: PersistentVolumeClaim    metadata:      name: my-claim    spec:      accessModes:        - ReadWriteOnce      resources:        requests:          storage: 200Gi

然后创建一个 StatefulSet 并将卷挂载到/mnt/disks，这是一个现有目录。 statefulset.yaml:

    apiVersion: apps/v1beta2    kind: StatefulSet    metadata:      name: ...    spec:        ...        spec:          containers:          - name: ...            ...            volumeMounts:            - name: my-volume              mountPath: /mnt/disks          volumes:          - name: my-volume            emptyDir: {}      volumeClaimTemplates:      - metadata:          name: my-claim        spec:          accessModes: [ "ReadWriteOnce" ]          resources:            requests:              storage: 200Gi

我运行命令 kubectl get pv 并看到磁盘已成功挂载到每个实例

    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                    STORAGECLASS   REASON    AGE    my-volume                                  200Gi      RWO            Retain           Available                                                                     19m    pvc-17c60f45-2e4f-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claim-xxx_1   standard                 13m    pvc-5972c804-2e4e-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claim                         standard                 18m    pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e   200Gi      RWO            Delete           Bound       default/my-claimxxx_0   standard                 18m

但是当我通过 ssh 进入一个实例并运行 df -hT 时，我没有看到已安装的卷。下面是输出:

    Filesystem     Type      Size  Used Avail Use% Mounted on    /dev/root      ext2      1.2G  447M  774M  37% /    devtmpfs       devtmpfs  1.9G     0  1.9G   0% /dev    tmpfs          tmpfs     1.9G     0  1.9G   0% /dev/shm    tmpfs          tmpfs     1.9G  744K  1.9G   1% /run    tmpfs          tmpfs     1.9G     0  1.9G   0% /sys/fs/cgroup    tmpfs          tmpfs     1.9G     0  1.9G   0% /tmp    tmpfs          tmpfs     256K     0  256K   0% /mnt/disks    /dev/sda8      ext4       12M   28K   12M   1% /usr/share/oem    /dev/sda1      ext4       95G  3.5G   91G   4% /mnt/stateful_partition    tmpfs          tmpfs     1.0M  128K  896K  13% /var/lib/cloud    overlayfs      overlay   1.0M  148K  876K  15% /etc

有人知道吗？

另外值得一提的是，我正在尝试将磁盘挂载到在 kubernete 引擎中运行的 docker 镜像。 Pod 是使用以下命令创建的:

    docker build -t gcr.io/xxx .    gcloud docker -- push gcr.io/xxx    kubectl create -f statefulset.yaml

我进入的实例是运行 docker 镜像的实例。我在实例和 docker 容器中都没有看到卷

更新我找到了卷，我在实例中运行了 df -ahT，并看到了相关条目

    /dev/sdb       -               -     -     -    - /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/gke-xxx-cluster-c-pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e    /dev/sdb       -               -     -     -    - /home/kubernetes/containerized_mounter/rootfs/var/lib/kubelet/pods/61bb679b-2e4e-11e8-9b77-42010af0000e/volumes/kubernetes.io~gce-pd/pvc-61b9daf9-2e4e-11e8-9b77-42010af0000e

然后我进入 docker 容器并运行 df -ahT，我得到了

    Filesystem     Type     Size  Used Avail Use% Mounted on    /dev/sda1      ext4      95G  3.5G   91G   4% /mnt/disks

为什么我看到的总大小是 95G，而不是我的卷大小 200G？

更多信息:kubectl describe pod

    Name:           xxx-replicaset-0    Namespace:      default    Node:           gke-xxx-cluster-default-pool-5e49501c-nrzt/10.128.0.17    Start Time:     Fri, 23 Mar 2018 11:40:57 -0400    Labels:         app=xxx-replicaset                    controller-revision-hash=xxx-replicaset-755c4f7cff    Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"default","name":"xxx-replicaset","uid":"d6c3511f-2eaf-11e8-b14e-42010af0000...                    kubernetes.io/limit-ranger=LimitRanger plugin set: cpu request for container xxx-deployment    Status:         Running    IP:             10.52.4.5    Created By:     StatefulSet/xxx-replicaset    Controlled By:  StatefulSet/xxx-replicaset    Containers:      xxx-deployment:        Container ID:   docker://137b3966a14538233ed394a3d0d1501027966b972d8ad821951f53d9eb908615        Image:          gcr.io/sampeproject/xxxstaging:v1        Image ID:       docker-pullable://gcr.io/sampeproject/xxxstaging@sha256:a96835c2597cfae3670a609a69196c6cd3d9cc9f2f0edf5b67d0a4afdd772e0b        Port:           8080/TCP        State:          Running          Started:      Fri, 23 Mar 2018 11:42:17 -0400        Ready:          True        Restart Count:  0        Requests:          cpu:        100m        Environment:          Mounts:          /mnt/disks from my-volume (rw)          /var/run/secrets/kubernetes.io/serviceaccount from default-token-hj65g (ro)    Conditions:      Type           Status      Initialized    True      Ready          True      PodScheduled   True    Volumes:      my-claim:        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)        ClaimName:  my-claim-xxx-replicaset-0        ReadOnly:   false      my-volume:        Type:    EmptyDir (a temporary directory that shares a pod's lifetime)        Medium:      default-token-hj65g:        Type:        Secret (a volume populated by a Secret)        SecretName:  default-token-hj65g        Optional:    false    QoS Class:       Burstable    Node-Selectors:      Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s                     node.alpha.kubernetes.io/unreachable:NoExecute for 300s    Events:      Type     Reason                 Age                From                                                      Message      ----     ------                 ----               ----                                                      -------      Warning  FailedScheduling       10m (x4 over 10m)  default-scheduler                                         PersistentVolumeClaim is not bound: "my-claim-xxx-replicaset-0" (repeated 5 times)      Normal   Scheduled              9m                 default-scheduler                                         Successfully assigned xxx-replicaset-0 to gke-xxx-cluster-default-pool-5e49501c-nrzt      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "my-volume"      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "default-token-hj65g"      Normal   SuccessfulMountVolume  9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  MountVolume.SetUp succeeded for volume "pvc-902c57c5-2eb0-11e8-b14e-42010af0000e"      Normal   Pulling                9m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  pulling image "gcr.io/sampeproject/xxxstaging:v1"      Normal   Pulled                 8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Successfully pulled image "gcr.io/sampeproject/xxxstaging:v1"      Normal   Created                8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Created container      Normal   Started                8m                 kubelet, gke-xxx-cluster-default-pool-5e49501c-nrzt  Started container

似乎没有安装正确的卷。我在 docker 容器中运行了 lsblk

NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT    sda       8:0    0  100G  0 disk     ├─sda1    8:1    0 95.9G  0 part /mnt/disks    ├─sda2    8:2    0   16M  0 part     ├─sda3    8:3    0    2G  0 part     ├─sda4    8:4    0   16M  0 part     ├─sda5    8:5    0    2G  0 part     ├─sda6    8:6    0  512B  0 part     ├─sda7    8:7    0  512B  0 part     ├─sda8    8:8    0   16M  0 part     ├─sda9    8:9    0  512B  0 part     ├─sda10   8:10   0  512B  0 part     ├─sda11   8:11   0    8M  0 part     └─sda12   8:12   0   32M  0 part     sdb       8:16   0  200G  0 disk 

为什么会这样？

Answer 1

当您使用 PVC 时，K8s 会为您管理永久性磁盘。

存储类中供应商定义 PV 的确切方式。由于您使用 GKE，因此您的默认 SC 使用 kubernetes.io/gce-pd provisioner ( https://kubernetes.io/docs/concepts/storage/storage-classes/#gce )。

换句话说，为每个 pod 创建新的 PV。

如果您想使用现有磁盘，您可以使用卷而不是 PVC (https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk)