azure - ImagePullBackOff 未经授权 : authentication required

Question

我已经完成了所有的 Action ，但我遇到了一个似乎是常见的问题。不幸的是，我从 github 和 SO 尝试过的所有解决方案都还没有工作。错误如下:

Warning  Failed     4m (x4 over 5m)    kubelet, aks-agentpool-97052351-0  Failed to pull image "ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi": [rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required]

-- 创建服务主体

az ad sp create-for-rbac 
  --scopes /subscriptions/11870e73-bdb2-47b0-bf27-25d24c41ae24/resourcegroups/USS-MicroService-Test/providers/Microsoft.ContainerRegistry/registries/UssMicroServiceRegistry
  --role Reader 
  --name kimage-reader

--为 Kube 创建 secret

kubectl create secret docker-registry kimagereadersecret --docker-server ussmicroserviceregistry.azurecr.io --docker-email <a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="5f3c302d3a262f1f2a31362b3a3b722c262c2b3a322c713c3032" rel="noreferrer noopener nofollow">[email protected]</a> --docker-username=kimage-reader --docker-password 4b37b896-a04e-48b4-a950-5f1abdd3e7aa

-- kubectl.exe 描述 pod simpledotnetapi-deployment-6fbf97df55-2hg2m

Name:               simpledotnetapi-deployment-6fbf97df55-2hg2m
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               aks-agentpool-97052351-0/10.240.0.4
Start Time:         Mon, 17 Jun 2019 15:22:30 -0500
Labels:             app=simpledotnetapi-pod
                    pod-template-hash=6fbf97df55
Annotations:        <none>
Status:             Pending
IP:                 10.240.0.26
Controlled By:      ReplicaSet/simpledotnetapi-deployment-6fbf97df55
Containers:
  simpledotnetapi-simpledotnetapi:
    Container ID:
    Image:          ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi
    Image ID:
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hj9b5 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-hj9b5:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hj9b5
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                               Message
  ----     ------     ----               ----                               -------
  Normal   Scheduled  5m                 default-scheduler                  Successfully assigned default/simpledotnetapi-deployment-6fbf97df55-2hg2m to aks-agentpool-97052351-0
  Normal   BackOff    4m (x6 over 5m)    kubelet, aks-agentpool-97052351-0  Back-off pulling image "ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi"
  Normal   Pulling    4m (x4 over 5m)    kubelet, aks-agentpool-97052351-0  pulling image "ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi"
  Warning  Failed     4m (x4 over 5m)    kubelet, aks-agentpool-97052351-0  Failed to pull image "ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi": [rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required]
  Warning  Failed     4m (x4 over 5m)    kubelet, aks-agentpool-97052351-0  Error: ErrImagePull
  Warning  Failed     24s (x22 over 5m)  kubelet, aks-agentpool-97052351-0  Error: ImagePullBackOff

-- kubectl.exe 获取 pod simpledotnetapi-deployment-6fbf97df55-2hg2m -o yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: 2019-06-17T20:22:30Z
  generateName: simpledotnetapi-deployment-6fbf97df55-
  labels:
    app: simpledotnetapi-pod
    pod-template-hash: 6fbf97df55
  name: simpledotnetapi-deployment-6fbf97df55-2hg2m
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: simpledotnetapi-deployment-6fbf97df55
    uid: a99e4ac8-8ec3-11e9-9bf8-86d46846735e
  resourceVersion: "813190"
  selfLink: /api/v1/namespaces/default/pods/simpledotnetapi-deployment-6fbf97df55-2hg2m
  uid: a1c220a2-913d-11e9-801a-c6aef815c06a
spec:
  containers:
  - image: ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi
    imagePullPolicy: Always
    name: simpledotnetapi-simpledotnetapi
    ports:
    - containerPort: 5000
      protocol: TCP
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-hj9b5
      readOnly: true
  dnsPolicy: ClusterFirst
  imagePullSecrets:
  - name: kimagereadersecret
  nodeName: aks-agentpool-97052351-0
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: default-token-hj9b5
    secret:
      defaultMode: 420
      secretName: default-token-hj9b5
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: 2019-06-17T20:22:30Z
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: 2019-06-17T20:22:30Z
    message: 'containers with unready status: [simpledotnetapi_simpledotnetapi]'
    reason: ContainersNotReady
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: 2019-06-17T20:22:30Z
    message: 'containers with unready status: [simpledotnetapi_simpledotnetapi]'
    reason: ContainersNotReady
    status: "False"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: 2019-06-17T20:22:30Z
    status: "True"
    type: PodScheduled
  containerStatuses:
  - image: ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi
    imageID: ""
    lastState: {}
    name: simpledotnetapi-simpledotnetapi
    ready: false
    restartCount: 0
    state:
      waiting:
        message: Back-off pulling image "ussmicroserviceregistry.azurecr.io/simpledotnetapi-simpledotnetapi"
        reason: ImagePullBackOff
  hostIP: 10.240.0.4
  phase: Pending
  podIP: 10.240.0.26
  qosClass: BestEffort
  startTime: 2019-06-17T20:22:30Z

--yaml配置文件

apiVersion: apps/v1
kind: Deployment
metadata:
    name: simpledotnetapi-deployment
spec:
    replicas: 3
    selector:
        matchLabels:
            app: simpledotnetapi-pod
    template:
        metadata:
            labels:
                app: simpledotnetapi-pod
        spec:
            imagePullSecrets:
              - name: kimagereadersecret
            containers:
              - name: simpledotnetapi_simpledotnetapi
                image: ussmicroserviceregistry.azurecr.io/simpledotnetapi-simpledotnetapi
                ports:
                  - containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
    name: simpledotnetapi-service
spec:
    type: LoadBalancer
    ports:
    - port: 80
    selector:
       app: simpledotnetapi
       type: front-end

-- kubectl get Secret kimagereadersecret 的输出

NAME                 TYPE                             DATA      AGE
kimagereadersecret   kubernetes.io/dockerconfigjson   1         1h

--来自 Kube 仪表板的凭据/ secret

{
  "kind": "Secret",
  "apiVersion": "v1",
  "metadata": {
    "name": "kimagereadersecret",
    "namespace": "default",
    "selfLink": "/api/v1/namespaces/default/secrets/kimagereadersecret",
    "uid": "86006aff-9156-11e9-801a-c6aef815c06a",
    "resourceVersion": "830006",
    "creationTimestamp": "2019-06-17T23:20:41Z"
  },
  "data": {
    ".dockerconfigjson": "eyJhdXRocyI6eyJ1c3NtaWNyb3NlcnZpY2VyZWdpc3RyeS5henVyZWNyLmlvIjp7InVzZXJuYW1lIjoiMzNjYjBjZTQtOTVmMC00NGJkLWJiYmYtNTZkNTA2ZmY0ZWIzIiwicGFzc3dvcmQiOiI0YjM3Yjg5Ni1hMDRlLTQ4YjQtYTk1MC01ZjFhYmRkM2U3YWEiLCJlbWFpbCI6ImNvcmV5cEB1bml0ZWQtc3lzdGVtcy5jb20iLCJhdXRoIjoiTXpOallqQmpaVFF0T1RWbU1DMDBOR0prTFdKaVltWXROVFprTlRBMlptWTBaV0l6T2pSaU16ZGlPRGsyTFdFd05HVXRORGhpTkMxaE9UVXdMVFZtTVdGaVpHUXpaVGRoWVE9PSJ9fX0="
  },
  "type": "kubernetes.io/dockerconfigjson"
}

-- Kube 仪表板的完整转储

Failed to pull image "ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi": [rpc error: code = Unknown desc = Error response from daemon: manifest for ussmicroserviceregistry.azurecr.io/simpledotnetapi_simpledotnetapi:latest not found: manifest unknown: manifest unknown, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required, rpc error: code = Unknown desc = Error response from daemon: Get https://ussmicroserviceregistry.azurecr.io/v2/simpledotnetapi_simpledotnetapi/manifests/latest: unauthorized: authentication required]

整个项目都在 GitHub @ https://github.com/coreyperkins/KubeSimpleDotNetApi

--ACR 截图

-- Kube 中的 Pod 故障

Answer 1

我相当确定您没有授予它足够的权限:

az ad sp create-for-rbac 
  --scopes /subscriptions/11870e73-bdb2-47b0-bf27-25d24c41ae24/resourcegroups/USS-MicroService-Test/providers/Microsoft.ContainerRegistry/registries/UssMicroServiceRegistry
  --role Reader 
  --name kimage-reader

角色应该是acrpull，而不是读者。只需删除此 secret :`kimagereadersecret 1 并在 pod 中引用它即可。 kubernetes 将为您处理这个问题。