kubernetes - K8s Pod Topology Spread 在 rollout 后没有得到尊重？

Question

我正在尝试传播我的 ingress-nginx-controller pod，这样:

• 每个可用区都有相同数量的 pod (+- 1)。
• Pod 更喜欢当前运行最少 Pod 的节点。

按照此处的其他问题，我在我的 pod 部署中设置了 Pod Topology Spread Constraints:

      replicas: 4
      topologySpreadConstraints:
      - labelSelector:
          matchLabels:
            app.kubernetes.io/name: ingress-nginx
        maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
      - labelSelector:
          matchLabels:
            app.kubernetes.io/name: ingress-nginx
        maxSkew: 1
        topologyKey: kubernetes.io/hostname
        whenUnsatisfiable: DoNotSchedule

我目前有 2 个节点，每个节点位于不同的可用区:

$ kubectl get nodes --label-columns=topology.kubernetes.io/zone,kubernetes.io/hostname
NAME                            STATUS   ROLES                  AGE    VERSION   ZONE         HOSTNAME
ip-{{node1}}.compute.internal   Ready    node                   136m   v1.20.2   us-west-2a   ip-{{node1}}.compute.internal
ip-{{node2}}.compute.internal   Ready    node                   20h    v1.20.2   us-west-2b   ip-{{node2}}.compute.internal

为该部署运行 kubectl rollout restart 后，我在一个节点中获得 3 个 pod，在另一个节点中获得 1 个 pod，其偏差为 2 > 1:

$ kubectl describe pod ingress-nginx-controller -n ingress-nginx | grep 'Node:'
Node:         ip-{{node1}}.compute.internal/{{node1}}
Node:         ip-{{node2}}.compute.internal/{{node2}}
Node:         ip-{{node1}}.compute.internal/{{node1}}
Node:         ip-{{node1}}.compute.internal/{{node1}}

为什么我的约束没有得到遵守？如何调试 Pod 调度程序？

我的 kubectl 版本:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.0-beta.0.607+269d62d895c297", GitCommit:"269d62d895c29743931bfaaec6e8d37ced43c35f", GitTreeState:"clean", BuildDate:"2021-03-05T22:28:02Z", GoVersion:"go1.16", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

Answer 1

提高评论的可见度:

Daemonset worked and was easy enough. It won't work for our deployments with several pods per node, but there are mitigations there (descheduler) and it should self resolve as the cluster grows.

请将此答案视为解决方法:

A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.

-- Kubernetes.io: Docs: Concepts: Workloads: Controllers: Daemonset

它的一个例子如下:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      name: nginx
  template:
    metadata:
      labels:
        name: nginx 
    spec:
      #nodeSelector: 
          #schedule: here 
      tolerations:
      # this toleration is to have the daemonset runnable on master nodes
      # remove it if your masters can't run pods
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: nginx
        image: nginx

此定义将在集群中的每个 Node 上生成一个 Pod。您可以通过指定 nodeSelector 进一步限制 Pod 调度。

假设您有一些 Controller /逻辑负责使用特定标签标记节点，您可以在特定节点上安排 Pod。负责它的部分在上面的 list 中被注释掉了:

      nodeSelector: 
          schedule: here 

节点(raven-sgdm 和 raven-xvvw 被标记)::

NAME         STATUS   ROLES    AGE    VERSION
raven-6k6m   Ready    <none>   159m   v1.20
raven-sgdm   Ready    <none>   159m   v1.20
raven-xvvw   Ready    <none>   159m   v1.20

Daemonset:

NAME    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
nginx   2         2         2       2            2           schedule=here   99m

---

其他资源:

• Kubernetes.io: Docs: Concepts: Workloads: Pods: Pod topology spread constraints
• Github.com: Kubernetes: Issues: 98215