amazon-web-services - 无法获取 AWS 云提供商。 GetCloudProvider 返回了

Question

我有一个手工构建的 Kubernets 集群 1.11.4，使用 CentOS 作为 AWS ec2 实例运行，1 个主节点和 1 个从节点。集群非常稳定。我想将 JupyterHub 部署到集群中。医生 here和 here调出配置 EFS 的一些细节。我选择使用 EBS。

pvc 失败:

Failed to get AWS Cloud Provider. GetCloudProvider returned <nil> instead
Mounted By:  hub-76ffd7d94b-dmj8l

下面是 StorageClass 定义:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: gp2
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
  fsType: ext4

pv yaml:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: jupyterhub-pv
  labels:
    type: amazonEBS
spec:
  capacity:
    storage: 30Gi
  accessModes:
    - ReadWriteMany
  awsElasticBlockStore:
    volumeID: vol-0ddb700735db435c7
    fsType: ext4

pvc yaml:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: jupyterhub-pvc
  labels:
    type: amazonEBS
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

$ kubectl -n jhub describe pvc hub-db-dir

返回:

Name:          hub-db-dir
Namespace:     jhub
StorageClass:  standard  <========from an earlier try
Status:        Pending
Volume:
Labels:        app=jupyterhub
               chart=jupyterhub-0.8.2
               component=hub
               heritage=Tiller
               release=jhub
Annotations:   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Events:
  Type       Reason              Age                     From                         Message
  ----       ------              ----                    ----                         -------
  Warning    ProvisioningFailed  110s (x106 over 3h43m)  persistentvolume-controller  Failed to provision volume with StorageClass "standard": Failed to get AWS Cloud Provider. GetCloudProvider returned <nil> instead
Mounted By:  hub-76ffd7d94b-dmj8l

对我来说，这看起来像是 pod 尝试挂载存储，但失败了。隔离此错误一直是一个挑战。我尝试修补 pvc 以将存储类更新为 gp2，它现在被标记为默认值，但在我部署 pvc 策略时还没有。修补失败:

$ kubectl -n jhub patch pvc hub-db-dir -p '{"spec":{"StorageClass":"gp2"}}'
persistentvolumeclaim/hub-db-dir patched (no change)

$ kubectl -n jhub describe pvc hub-db-dir
Name:          hub-db-dir
Namespace:     jhub
StorageClass:  standard  <====== Not changed
Status:        Pending
Volume:
Labels:        app=jupyterhub
               chart=jupyterhub-0.8.2
               component=hub
               heritage=Tiller
               release=jhub
Annotations:   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Events:
  Type       Reason              Age                      From                         Message
  ----       ------              ----                     ----                         -------
  Warning    ProvisioningFailed  2m26s (x108 over 3h48m)  persistentvolume-controller  Failed to provision volume with StorageClass "standard": Failed to get AWS Cloud Provider. GetCloudProvider returned <nil> instead
Mounted By:  hub-76ffd7d94b-dmj8l

JupyterHub 部署由 Helm/tiller 管理，因此当进行任何更改时，我使用以下内容来更新 pod:

$ helm upgrade jhub jupyterhub/jupyterhub --version=0.8.2 -f config.yaml

config.yaml 文件中分配用户存储的相关部分是:

proxy:
  secretToken: "<random value>"
singleuser:
  cloudMetadata:
    enabled: true
singleuser:
  storage:
    dynamic:
      storageClass: gp2
singleuser:
  storage:
    extraVolumes:
      - name: jupyterhub-pv
        persistentVolumeClaim:
          claimName: jupyterhub-pvc
    extraVolumeMounts:
      - name: jupyterhub-pv
        mountPath: /home/shared

部分故障排除还侧重于让集群知道其资源由 AWS 提供。为此，我在 kubernets 配置文件中有:

/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

线路:

Environment="KUBELET_EXTRA_ARGS=--cloud-provider=aws --cloud-config=/etc/kubernetes/cloud-config.conf

其中:/etc/kubernetes/cloud-config.conf 包含:

[Global]
KubernetesClusterTag=kubernetes
KubernetesClusterID=kubernetes

在文件 kube-controller-manager.yaml 和 kube-apiserver.yaml 我添加了一行:

- --cloud-provider=aws

我还没有标记任何 AWS 资源，但会根据 this 开始标记.

我接下来的故障排除步骤是什么？

谢谢!

Answer 1

可能是this链接有帮助吗？

You must have the --cloud-provider=aws flag added to the Kubelet before adding the node to the cluster. Key to the AWS integration is a particular field on the Node object—the .spec.providerID field - and that field will only get populated if the flag is present when the node is added to the cluster. If you add a node to the cluster and then add the command-line flag afterward, this field/value won’t get populated and the integration won’t work as expected. No error is surfaced in this situation (at least, not that I’ve been able to find).

If you do find yourself with a missing .spec.providerID field on the Node object, you can add it with a kubectl edit node command. The format of the value for this field is aws:///<az-of-instance>/<instance-id>.