kubernetes - 在 Kubernetes 上运行 Gitlab CI 运行程序时 pod 的挂起状态

Question

我目前正在尝试为 Gitlab CI 使用 Kubernetes 集群。
虽然遵循不太好的文档(https://docs.gitlab.com/runner/install/kubernetes.html)，但我所做的是使用 Gitlab CI 部分的 token 手动注册一个运行器，这样我就可以获得另一个 token 并在我用于部署的 ConfigMap 中使用它。

-ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-runner
  namespace: gitlab
data:
  config.toml: |
    concurrent = 4
    [[runners]]
        name = "Kubernetes Runner"
        url = "https://url/ci"
        token = "TOKEN"
        executor = "kubernetes"
        [runners.kubernetes]
            namespace = "gitlab"

-部署

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab
spec:
  replicas: 4
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:latest
        imagePullPolicy: Always
        name: gitlab-runner
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
      restartPolicy: Always
      volumes:
      - configMap:
          name: gitlab-runner
        name: config

有了这两个，我可以在 Gitlab Runner 部分看到 runner，但是每当我开始工作时，新创建的 pod 都会保持挂起状态。

我想修复它，但我只知道节点和 pod 会收到这些事件:

- pod :

Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                    -------------   --------        ------                  -------
  35s           4s              7       {default-scheduler }                    Warning         FailedScheduling        No nodes are available that match all of the following predicates:: MatchNodeSelector (2).

-节点:

Events:
  FirstSeen     LastSeen        Count   From                                            SubObjectPath   Type            Reason                                  Message
  ---------     --------        -----   ----                                            -------------   --------        ------                                  -------
  4d            31s             6887    {kubelet gitlab-ci-hc6k3ffax54o-master-0}                       Warning         FailedNodeAllocatableEnforcement        Failed to update Node Allocatable Limits "": failed to set supported cgroup subsystems for cgroup : Failed to set config for supported subsystems : failed to write 3783761920 to memory.limit_in_bytes: write /rootfs/sys/fs/cgroup/memory/memory.limit_in_bytes: invalid argument 

知道为什么会这样吗？

编辑:kubectl 描述添加:

Name:           runner-45384765-project-1570-concurrent-00mb7r
Namespace:      gitlab
Node:           /
Labels:         <none>
Status:         Pending
IP:
Controllers:    <none>
Containers:
  build:
    Image:      blablabla:latest
    Port:
    Command:
      sh
      -c
      if [ -x /usr/local/bin/bash ]; then
        exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
        exec /usr/bin/bash
elif [ -x /bin/bash ]; then
        exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
        exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
        exec /usr/bin/sh
elif [ -x /bin/sh ]; then
        exec /bin/sh
else
        echo shell not found
        exit 1
fi


    Volume Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-1qm5n (ro)
      /vcs from repo (rw)
    Environment Variables:
      CI_PROJECT_DIR:           blablabla
      CI_SERVER:                yes
      CI_SERVER_TLS_CA_FILE:    -----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla 
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----

      CI:                       true
      GITLAB_CI:                true
      CI_SERVER_NAME:           GitLab
      CI_SERVER_VERSION:        9.5.5-ee
      CI_SERVER_REVISION:       cfe2d5c
      CI_JOB_ID:                5625
      CI_JOB_NAME:              pylint
      CI_JOB_STAGE:             build
      CI_COMMIT_SHA:            ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_COMMIT_REF_NAME:       master
      CI_COMMIT_REF_SLUG:       master
      CI_REGISTRY_USER:         gitlab-ci-token
      CI_BUILD_ID:              5625
      CI_BUILD_REF:             ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_BUILD_BEFORE_SHA:      ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_BUILD_REF_NAME:        master
      CI_BUILD_REF_SLUG:        master
      CI_BUILD_NAME:            pylint
      CI_BUILD_STAGE:           build
      CI_PROJECT_ID:            1570
      CI_PROJECT_NAME:          blablabla
      CI_PROJECT_PATH:          blablabla
      CI_PROJECT_PATH_SLUG:     blablabla
      CI_PROJECT_NAMESPACE:     vcs
      CI_PROJECT_URL:           https://blablabla
      CI_PIPELINE_ID:           2574
      CI_CONFIG_PATH:           .gitlab-ci.yml
      CI_PIPELINE_SOURCE:       push
      CI_RUNNER_ID:             111
      CI_RUNNER_DESCRIPTION:    testing on kubernetes
      CI_RUNNER_TAGS:           docker-image-build
      CI_REGISTRY:              blablabla
      CI_REGISTRY_IMAGE:        blablabla
      PYLINTHOME:               ./pylint-home
      GITLAB_USER_ID:           2277
      GITLAB_USER_EMAIL:        blablabla
  helper:
    Image:      gitlab/gitlab-runner-helper:x86_64-a9a76a50
    Port:
    Command:
      sh
      -c
      if [ -x /usr/local/bin/bash ]; then
        exec /usr/local/bin/bash
elif [ -x /usr/bin/bash ]; then
        exec /usr/bin/bash
elif [ -x /bin/bash ]; then
        exec /bin/bash
elif [ -x /usr/local/bin/sh ]; then
        exec /usr/local/bin/sh
elif [ -x /usr/bin/sh ]; then
        exec /usr/bin/sh
elif [ -x /bin/sh ]; then
        exec /bin/sh
else
        echo shell not found
        exit 1
fi


    Volume Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-1qm5n (ro)
      /vcs from repo (rw)
    Environment Variables:
      CI_PROJECT_DIR:           blablabla
      CI_SERVER:                yes
      CI_SERVER_TLS_CA_FILE:    -----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
blablabla
-----END CERTIFICATE-----

      CI:                       true
      GITLAB_CI:                true
      CI_SERVER_NAME:           GitLab
      CI_SERVER_VERSION:        9.5.5-ee
      CI_SERVER_REVISION:       cfe2d5c
      CI_JOB_ID:                5625
      CI_JOB_NAME:              pylint
      CI_JOB_STAGE:             build
      CI_COMMIT_SHA:            ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_COMMIT_REF_NAME:       master
      CI_COMMIT_REF_SLUG:       master
      CI_REGISTRY_USER:         gitlab-ci-token
      CI_BUILD_ID:              5625
      CI_BUILD_REF:             ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_BUILD_BEFORE_SHA:      ece31293f8eeb3a36a8585b79d4d21e0ebe8008f
      CI_BUILD_REF_NAME:        master
      CI_BUILD_REF_SLUG:        master
      CI_BUILD_NAME:            pylint
      CI_BUILD_STAGE:           build
      CI_PROJECT_ID:            1570
      CI_PROJECT_NAME:          blablabla
      CI_PROJECT_PATH:          blablabla
      CI_PROJECT_PATH_SLUG:     blablabla
      CI_PROJECT_NAMESPACE:     vcs
      CI_PROJECT_URL:           blablabla
      CI_PIPELINE_ID:           2574
      CI_CONFIG_PATH:           .gitlab-ci.yml
      CI_PIPELINE_SOURCE:       push
      CI_RUNNER_ID:             111
      CI_RUNNER_DESCRIPTION:    testing on kubernetes
      CI_RUNNER_TAGS:           docker-image-build
      CI_REGISTRY:              blablabla
      CI_REGISTRY_IMAGE:        blablabla
      PYLINTHOME:               ./pylint-home
      GITLAB_USER_ID:           2277
      GITLAB_USER_EMAIL:        blablabla
Conditions:
  Type          Status
  PodScheduled  False
Volumes:
  repo:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
  default-token-1qm5n:
    Type:       Secret (a volume populated by a Secret)
    SecretName: default-token-1qm5n
QoS Class:      BestEffort
Tolerations:    <none>
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                    -------------   --------        ------                  -------
  39s           8s              7       {default-scheduler }                    Warning         FailedScheduling        No nodes are available that match all of the following predicates:: MatchNodeSelector (2).

Answer 1

@djuarez 只要部署选择器与模板部分中的 pods 标签匹配，在这种情况下，我可以看到就是这种情况:

selector:
  matchLabels:
    name: gitlab-runner
template:
  metadata:
    labels:
      name: gitlab-runner

这应该不是问题；前提是使用了正确的 API，在这种情况下为 apiVersion: extensions/v1beta1也是正确的。 describe输出显示 MatchNodeSelector这与部署选择器无关。我的猜测是这里没有显示完整的部署配置，还有其他一些问题，比如尝试通过 nodeSeletor 将 Pod 部署到特定节点。在 nodeSelector 条件中没有请求的标签。