kubernetes pod 的探测失败 - Client.Timeout exceeded while waiting headers

Question

我进行了搜索，但没有任何帮助。

我的套装。
k8s - v1.20.2.
印花布 - 3.16.6.
pod-cidr = 10.214.0.0/16.
服务 cidr = 10.215.0.1/16。

由 kubespray 与此一起安装 https://kubernetes.io/ko/docs/setup/production-environment/tools/kubespray

pod restarts again and again .
ingress-nginx-controller pod describe

[dns-autoscaler pod 日志]

github.com/kubernetes-incubator/cluster-proportional-autoscaler/pkg/autoscaler/k8sclient/k8sclient.go:96: Failed to list *v1.Node: Get https://10.215.0.1:443/api/v1/nodes: dial tcp 10.215.0.1:443: i/o timeout

[dns-autoscaler pod 描述]

kubelet  Readiness probe failed: Get "http://10.214.116.129:8080/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

[coredns pod 日志]

pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.215.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.215.0.1:443: i/o timeout

[coredns pod 描述]

Get "http://10.214.122.1:8080/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

我尝试安装 ingress-nginx-controller，它获取了我的日志和描述。
[入口 Controller 日志]

W0106 04:17:16.715661       6 flags.go:243] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0106 04:17:16.715911       6 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0106 04:17:16.716200       6 main.go:182] Creating API client for https://10.215.0.1:

[入口 Controller 描述]

Liveness probe failed: Get "https://10.214.233.2:8443/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

所有这些 pod 都在与就绪/事件探测失败作斗争:获取“http://10.214.116.155:10254/healthz”:超出上下文截止日期(在等待 header 时超出了 Client.Timeout)。

Calico 正在运行。我检查了 pod 到 pod 的通信(OK)。
calico is Running

[kubectl 获取组件状态]

controller-manager   Unhealthy   Get "http://127.0.0.1:10252/healthz": dial tcp 127.0.0.1:10252: connect: connection refused
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
etcd-1               Healthy     {"health":"true"}
etcd-2               Healthy     {"health":"true"}
etcd-0               Healthy     {"health":"true"}

kubectl get componentstatuses我关注了How to resolve scheduler and controller-manager unhealthy state in Kubernetes现在调度程序和 Controller 管理器是健康的。

[kubectl 获取节点]

Nodes are ready.

我做错了什么？ T.T.
提前致谢

Answer 1

在将应用程序部署到 Kubernetes 时遇到此问题。

Warning Unhealthy 10m (x3206 over 3d16h) kubelet Liveness probe failed: Get "http://10.2.0.97:80/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

我在 pod 中执行了一个 exec:

kubectl exec <pod-name> -it --namespace default /bin/bash

然后我对 pod 的 IP 和端口运行了一个 curl 请求:

curl 10.2.0.97:80

它返回了一个成功的响应。但 liveness 探测仍然未能成功执行。

我是这样解决的:

我所要做的就是将 timeoutSeconds 增加到 10:

livenessProbe:
  httpGet:
    path: /
    port: http
  initialDelaySeconds: 300       
  periodSeconds: 20
  timeoutSeconds: 10

之后 liveness probe 开始执行成功

可以对就绪探测进行同样的操作:

ReadinessProbe:
  httpGet:
    path: /
    port: http
  initialDelaySeconds: 30
  periodSeconds: 20
  timeoutSeconds: 10

引用:Sometime Liveness/Readiness Probes fail because of net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting head