kubernetes pods http : TLS handshake error from x. x.x.x:38676: EOF

Question

启动 cert-manager 时，我收到以下消息

TLS handshake error from 10.42.152.128:38676: EOF

$ kubectl -n cert-manager logs cert-manager-webhook-8575f88c85-l4tlw
I0214 19:41:28.147106       1 main.go:64]  "msg"="enabling TLS as certificate file flags specified"  
I0214 19:41:28.147365       1 server.go:126]  "msg"="listening for insecure healthz connections"  "address"=":6080"
I0214 19:41:28.147418       1 server.go:138]  "msg"="listening for secure connections"  "address"=":10250"
I0214 19:41:28.147437       1 server.go:155]  "msg"="registered pprof handlers"  
I0214 19:41:28.147570       1 tls_file_source.go:144]  "msg"="detected private key or certificate data on disk has changed. reloading certificate"  
2020/02/14 19:43:32 http: TLS handshake error from 10.42.152.128:38676: EOF

有趣的是没有那个IP的pod

$ kubectl get pod -o wide --all-namespaces | grep 128
cert-manager    cert-manager-webhook-8575f88c85-l4tlw             1/1     Running     0          4m56s   10.42.112.128   node002   <none>           <none>

cert-manager pod 上的类似错误

E0214 19:38:22.540589       1 controller.go:131] cert-manager/controller/ingress-shim "msg"="re-queuing item  due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: net/http: TLS handshake timeout" "key"="kube-system/dashboard-kubernetes-dashboard"

我有两个ClusterIssuer

kubectl get ClusterIssuer --namespace cert-manager
NAME              READY   AGE
letsencrypt-prd   True    42d
letsencrypt-stg   True    42d

但还没有证书:

kubectl get certificate --all-namespaces
No resources found

当我尝试申请证书时，我遇到了同样的错误

kubectl apply -f mycert.yml                                                                                                                                                                  
Error from server (InternalError): error when creating "cert-wyssmann-dev.yml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: net/http: TLS handshake timeout

我不确定如何才能找到问题的根源。我跑了 sonobouy看看这是否对我有帮助，但是我的 3 个节点中有 2 个测试失败。

Plugin: e2e
Status: failed
Total: 1
Passed: 0
Failed: 1
Skipped: 0

Failed tests:
Container e2e is in a terminated state (exit code 1) due to reason: Error: 

Plugin: systemd-logs
Status: failed
Total: 3
Passed: 1
Failed: 2
Skipped: 0

Failed tests:
timeout waiting for results

对于失败的节点，我可以在 sonobouy 日志中看到这一点

E0214 19:38:22.540589       1 controller.go:131] cert-manager/controller/ingress-shim "msg"="re-queuing item  due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=30s: net/http: TLS handshake timeout" "key"="kube-system/dashboard-kubernetes-dashboard"

Answer 1

如果您真的不需要 webhook，那么解决此问题的一种快速方法是根据 documentation 禁用 webhook