kubernetes - K8s http Liveness探针仅在1个工作节点上失败

Question

我在K8S场上安装了.netcore 2.2 API Pod，我实现了运行状况检查api，以便进行k8s liveness 检查。
这是我的设置。

     livenessProbe:
        httpGet:
          path: /api/Authentication/CheckLiveness
          port: 80
          scheme: HTTP
        initialDelaySeconds: 100
        timeoutSeconds: 60
        periodSeconds: 30
        successThreshold: 1
        failureThreshold: 1
      readinessProbe:
        httpGet:
          path: /api/Authentication/CheckReadiness
          port: 80
          scheme: HTTP
        initialDelaySeconds: 50
        timeoutSeconds: 30
        periodSeconds: 15
        successThreshold: 1
        failureThreshold: 1

问题是其他工作节点似乎正常工作，但工作节点1上的pod除外。



这是错误事件。

Liveness probe failed: Get 
http://10.244.3.218:80/api/Authentication/CheckLiveness:net/http: 
request canceled (Client.Timeout exceeded while awaiting headers)

Readiness probe failed: Get 
http://10.244.3.218:80/api/Authentication/CheckReadiness: net/http: 
request canceled (Client.Timeout exceeded while awaiting headers)

我可以通过执行将其 curl 到我的 pods 中。但是pod不断重启。

我认为这是一个节点问题。该K8s Farm在带有centos7 os的Vmware上运行。
我在具有相同基础结构的开发环境上尝试了此配置。而且全部绿色无问题。
需要任何建议以调试或解决此问题。
谢谢。

@mWatney编辑

这就是你的结果

Name:           authenservice-dpm-7d468bfcc4-px44m
Namespace:      pbtsapi
Priority:       0
Node:           ptseclsbtwkn01/192.168.10.136
Start Time:     Fri, 12 Jun 2020 11:23:07 +0700
Labels:         app=authenservice-api
                pod-template-hash=7d468bfcc4
Annotations:    <none>
Status:         Running
IP:             10.244.3.218
Controlled By:  ReplicaSet/authenservice-dpm-7d468bfcc4
Containers:
  authenservice:
    Container ID:   docker://1b1acffeae54421201d1bbc54b8020a75db660e1ae1a0f0d18a56930bbca0d12
    Image:          10.99.21.89:5000/authenservice:v1.0.4
    Image ID:       docker-pullable://10.99.21.89:5000/authenservice@sha256:b9244059195edff3cc3592d3e19a94ac00e481e9936413a4315a3cf41b0023ea
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 12 Jun 2020 14:46:22 +0700
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 12 Jun 2020 14:37:52 +0700
      Finished:     Fri, 12 Jun 2020 14:46:21 +0700
    Ready:          False
    Restart Count:  28
    Limits:
      cpu:     500m
      memory:  400Mi
    Requests:
      cpu:      250m
      memory:   200Mi
    Liveness:   http-get http://:80/api/Authentication/CheckLiveness delay=300s timeout=60s period=30s #success=1 #failure=1
    Readiness:  http-get http://:80/api/Authentication/CheckReadiness delay=300s timeout=60s period=30s #success=1 #failure=1
    Environment:
      MSSQL_PORT:  1433
    Mounts:
      /app/appsettings.json from authen-v (rw,path="appsettings.json")
      /etc/localtime from tz-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-h8x2b (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  authen-v:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      authenservice-config
    Optional:  false
  tz-config:
    Type:          HostPath (bare host directory volume)
    Path:          /usr/share/zoneinfo/Asia/Bangkok
    HostPathType:  
  default-token-h8x2b:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-h8x2b
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                     Message
  ----     ------     ----                    ----                     -------
  Normal   Killing    22m (x26 over 3h23m)    kubelet, ptseclsbtwkn01  Container authenservice failed liveness probe, will be restarted
  Warning  Unhealthy  6m42s (x28 over 3h23m)  kubelet, ptseclsbtwkn01  Liveness probe failed: Get http://10.244.3.218:80/api/Authentication/CheckLiveness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
  Warning  Unhealthy  10s (x22 over 3h8m)     kubelet, ptseclsbtwkn01  Readiness probe failed: Get http://10.244.3.218:80/api/Authentication/CheckReadiness: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Answer 1

虚拟机永远不会相同。您可以设置具有相同规格的虚拟机，这样做可以确保它们具有相似性，但这并不意味着它们将具有完全相同的性能。

服务器也发生同样的事情。您可以购买两个相同的物理服务器，它们的性能相似，但永远不会相同。

日志清楚地表明，在测试该应用程序时，它尚未准备就绪。

正如zerkms所说，您不能具有完全相同的“红色”和“活力”探针。您一定要对此进行审查。您还必须查看您的准备情况调查initialDelaySeconds。尝试增加它，以便有更多时间启动应用程序。

要解决suren建议并增加您的超时时间，我将改为增加initialDelaySeconds。