kubernetes - 无法覆盖部署 k8s 的 `replicas`

Question

我遇到过这样的问题:

首先，我使用 helm 创建一个版本 nginx:

helm upgrade --install --namespace test nginx bitnami/nginx --debug

LAST DEPLOYED: Wed Jul 22 15:17:50 2020
NAMESPACE: test
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                DATA  AGE
nginx-server-block  1     2s

==> v1/Deployment
NAME   READY  UP-TO-DATE  AVAILABLE  AGE
nginx  0/1    1           0          2s

==> v1/Pod(related)
NAME                    READY  STATUS             RESTARTS  AGE
nginx-6bcbfcd548-kdf4x  0/1    ContainerCreating  0         1s

==> v1/Service
NAME   TYPE          CLUSTER-IP    EXTERNAL-IP  PORT(S)                     AGE
nginx  LoadBalancer  10.219.6.148  <pending>    80:30811/TCP,443:31260/TCP  2s


NOTES:
Get the NGINX URL:

  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        Watch the status with: 'kubectl get svc --namespace test -w nginx'

  export SERVICE_IP=$(kubectl get svc --namespace test nginx --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}")
  echo "NGINX URL: http://$SERVICE_IP/"

K8s 仅创建具有 1 个 pod 的部署:

# Source: nginx/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app.kubernetes.io/name: nginx
    helm.sh/chart: nginx-6.0.2
    app.kubernetes.io/instance: nginx
    app.kubernetes.io/managed-by: Tiller
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: nginx
  replicas: 1
  ...

其次，我使用 kubectl 命令编辑部署以扩展到 2 个 pod

kubectl -n test  edit deployment nginx

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-07-22T08:17:51Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: nginx
    app.kubernetes.io/managed-by: Tiller
    app.kubernetes.io/name: nginx
    helm.sh/chart: nginx-6.0.2
  name: nginx
  namespace: test
  resourceVersion: "128636260"
  selfLink: /apis/extensions/v1beta1/namespaces/test/deployments/nginx
  uid: d63b0f05-cbf3-11ea-99d5-42010a8a00f1
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  ...

然后我保存它，检查状态以查看部署是否已扩展到 2 个 pod:

kubectl -n test get deployment

NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   2/2     2            2           7m50s

最后，我使用 helm 升级版本，正如预期的那样，helm 将像第一步一样覆盖部署到 1 个 pod，但现在，部署将保留值 replicas: 2 即使你设置值(在 helm 的 values.yaml 文件中)为任意数字。我使用了 helm 命令的选项 --recreate-pods:

helm upgrade --install --namespace test nginx bitnami/nginx --debug --recreate-pods
Release "nginx" has been upgraded. Happy Helming!
LAST DEPLOYED: Wed Jul 22 15:31:24 2020
NAMESPACE: test
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                DATA  AGE
nginx-server-block  1     13m

==> v1/Deployment
NAME   READY  UP-TO-DATE  AVAILABLE  AGE
nginx  0/2    2           0          13m

==> v1/Pod(related)
NAME                    READY  STATUS             RESTARTS  AGE
nginx-6bcbfcd548-b4bfs  0/1    ContainerCreating  0         1s
nginx-6bcbfcd548-bzhf2  0/1    ContainerCreating  0         1s
nginx-6bcbfcd548-kdf4x  0/1    Terminating        0         13m
nginx-6bcbfcd548-xfxbv  1/1    Terminating        0         6m16s

==> v1/Service
NAME   TYPE          CLUSTER-IP    EXTERNAL-IP    PORT(S)                     AGE
nginx  LoadBalancer  10.219.6.148  34.82.120.134  80:30811/TCP,443:31260/TCP  13m


NOTES:
Get the NGINX URL:

  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        Watch the status with: 'kubectl get svc --namespace test -w nginx'

  export SERVICE_IP=$(kubectl get svc --namespace test nginx --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}")
  echo "NGINX URL: http://$SERVICE_IP/"

结果:在我手动编辑部署中的 replicas 后，我无法使用 helm 覆盖此值 replicas，但我仍然可以更改图像等，。 ..只有副本不会改变我已经运行 --debug 并且 helm 仍然使用 replicas: 1

创建部署

# Source: nginx/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app.kubernetes.io/name: nginx
    helm.sh/chart: nginx-6.0.2
    app.kubernetes.io/instance: nginx
    app.kubernetes.io/managed-by: Tiller
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        helm.sh/chart: nginx-6.0.2
        app.kubernetes.io/instance: nginx
        app.kubernetes.io/managed-by: Tiller
    spec:      
      containers:
        - name: nginx
          image: docker.io/bitnami/nginx:1.19.1-debian-10-r0
          imagePullPolicy: "IfNotPresent"
          ports:
            - name: http
              containerPort: 8080
            
          livenessProbe:
            failureThreshold: 6
            initialDelaySeconds: 30
            tcpSocket:
              port: http
            timeoutSeconds: 5
            
          readinessProbe:
            initialDelaySeconds: 5
            periodSeconds: 5
            tcpSocket:
              port: http
            timeoutSeconds: 3
            
          resources:
            limits: {}
            requests: {}
            
          volumeMounts:
            - name: nginx-server-block-paths
              mountPath: /opt/bitnami/nginx/conf/server_blocks
      volumes:
        - name: nginx-server-block-paths
          configMap:
            name: nginx-server-block
            items:
              - key: server-blocks-paths.conf
                path: server-blocks-paths.conf

但是k8s部署会保持值replicas和编辑手册一样一次replicas: 2

据我所知，helm 命令的输出是创建 k8s yaml 文件，为什么我不能使用 helm 来覆盖特定值 replicas 在这种情况下？

提前致谢!!!

P/S: 我只想知道这里的行为是什么，Tks

Helm 版本

Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.0", GitCommit:"79d07943b03aea2b76c12644b4b54733bc5958d6", GitTreeState:"clean"}

Answer 1

遵循来自 Helm 的官方文档:Helm | Docs

Helm 2 used a two-way strategic merge patch. During an upgrade, it compared the most recent chart's manifest against the proposed chart's manifest (the one supplied during helm upgrade). It compared the differences between these two charts to determine what changes needed to be applied to the resources in Kubernetes. If changes were applied to the cluster out-of-band (such as during a kubectl edit), those changes were not considered. This resulted in resources being unable to roll back to its previous state: because Helm only considered the last applied chart's manifest as its current state, if there were no changes in the chart's state, the live state was left unchanged.

而这件事将在 Helm v3 中得到改进，因为 Helm v3 已经删除了 Tiller，您的值将完全应用于 Kubernetes资源，Helm和Kubernetes的值会保持一致。

==> 结果是如果你使用 Helm version 3 就不会再遇到这个问题了