kubernetes - 为什么我的 Nginx Ingress Controller 给出 503s？

Question

在给 nginx-controller 一个 externalIPs 列表后，我可以访问 nginx 负载均衡器，但它给了我 503s。我检查了生成的 nginx 文件，我看不到上游服务器的任何提及，这部分没有填充到 nginx 配置文件中，我得到上游部分的占位符。
我的部署:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

我的入口 Controller YAML:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: nginx-service
          servicePort: 80

我的服务:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  type: NodePort
  ports:
    - port: 80
      name: http
  selector:
    name: nginx

我来自 nginx 的日志:

-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v0.34.1
  Build:         v20200715-ingress-nginx-2.11.0-8-gda5fa45e2
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.1

-------------------------------------------------------------------------------

I0808 10:03:07.530453       6 flags.go:205] Watching for Ingress class: nginx
W0808 10:03:07.530755       6 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0808 10:03:07.530805       6 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0808 10:03:07.531018       6 main.go:231] Creating API client for https://10.96.0.1:443
I0808 10:03:07.535965       6 main.go:275] Running in Kubernetes cluster version v1.18 (v1.18.6) - git (clean) commit dff82dc0de47299ab66c83c626e08b245ab19037 - platform linux/amd64
I0808 10:03:07.661731       6 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0808 10:03:07.662262       6 main.go:113] Enabling new Ingress features available since Kubernetes v1.18
W0808 10:03:07.663809       6 main.go:125] No IngressClass resource with name nginx found. Only annotation will be used.
I0808 10:03:07.667982       6 ssl.go:528] loading tls certificate from certificate path /usr/local/certificates/cert and key path /usr/local/certificates/key
I0808 10:03:07.695581       6 nginx.go:263] Starting NGINX Ingress controller
I0808 10:03:07.699045       6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"7f8efb11-d326-4a79-96e1-87e9e610d189", APIVersion:"v1", ResourceVersion:"11100", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0808 10:03:08.799119       6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"test-ingress", UID:"5b3b5c37-d81e-4144-9c8e-d4ff424401ea", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"31926", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/test-ingress
I0808 10:03:08.895939       6 nginx.go:307] Starting NGINX process
I0808 10:03:08.896060       6 leaderelection.go:242] attempting to acquire leader lease  ingress-nginx/ingress-controller-leader-nginx...
I0808 10:03:08.896177       6 nginx.go:327] Starting validation webhook on :8443 with keys /usr/local/certificates/cert /usr/local/certificates/key
I0808 10:03:08.896317       6 controller.go:141] Configuration changes detected, backend reload required.
I0808 10:03:08.897732       6 status.go:86] new leader elected: ingress-nginx-controller-7fd7d8df56-7kls4
I0808 10:03:08.943105       6 controller.go:157] Backend successfully reloaded.
I0808 10:03:08.943137       6 controller.go:166] Initial sync, sleeping for 1 second.
I0808 10:03:48.343214       6 leaderelection.go:252] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
I0808 10:03:48.343262       6 status.go:86] new leader elected: ingress-nginx-controller-7fd7d8df56-54v6b
I0808 10:03:48.353140       6 status.go:275] updating Ingress default/test-ingress status from [] to [{10.0.2.15 } {49.12.86.195 }]
I0808 10:03:48.355684       6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"test-ingress", UID:"5b3b5c37-d81e-4144-9c8e-d4ff424401ea", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"33002", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/test-ingress
10.32.0.1 - - [08/Aug/2020:18:37:39 +0000] "GET / HTTP/1.1" 503 197 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" 424 0.000 [default-nginx-1337] [] - - - - 68e58595b011156ca288edba56a53a99
10.32.0.1 - - [08/Aug/2020:18:37:42 +0000] "GET / HTTP/1.1" 503 197 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" 367 0.000 [default-nginx-1337] [] - - - - 77659ecd8f16cbc8a1d9f9e603dcba90
W0808 18:38:37.982021       6 controller.go:916] Service "default/nginx" does not have any active Endpoint.
I0808 18:38:38.010466       6 main.go:115] successfully validated configuration, accepting ingress test-ingress in namespace default
I0808 18:38:38.012769       6 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"test-ingress", UID:"5b3b5c37-d81e-4144-9c8e-d4ff424401ea", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"106029", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/test-ingress
W0808 18:38:38.012885       6 controller.go:916] Service "default/nginx" does not have any active Endpoint.
I0808 18:38:38.012926       6 controller.go:141] Configuration changes detected, backend reload required.
I0808 18:38:38.060293       6 controller.go:157] Backend successfully reloaded.
10.32.0.1 - - [08/Aug/2020:18:38:39 +0000] "GET / HTTP/1.1" 503 197 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" 367 0.000 [default-nginx-80] [] - - - - 49267e557ee39cfed6c41b1592d2ceaf

我不确定 nginx 服务是什么类型的服务，我怀疑是 NodePort。
对我来说可疑的部分是

Service "default/nginx" does not have any active Endpoint.

但我无法弄清楚这意味着什么。
编辑: kubectl describe svc ngin 的输出

Name:                     nginx
Namespace:                default
Labels:                   name=nginx
Annotations:              Selector:  name=nginx
Type:                     NodePort
IP:                       10.105.68.13
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  32250/TCP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

kubectl 描述 pod nginx 部署

Name:                      nginx-deployment-68bd55b8f6-5gcpn
Namespace:                 default
Priority:                  0
Node:                      ubuntu/10.0.2.15
Start Time:                Sat, 08 Aug 2020 08:33:11 +0100
Labels:                    app=nginx
                           pod-template-hash=68bd55b8f6
Annotations:               <none>
Status:                    Terminating (lasts 3d5h)
Termination Grace Period:  30s
IP:                        10.44.0.2
IPs:
  IP:           10.44.0.2
Controlled By:  ReplicaSet/nginx-deployment-68bd55b8f6
Containers:
  nginx:
    Container ID:   docker://d6553f9fe848c4b535cbc05a3009accb06f7825f742f39db9e15d6511b0dba00
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           1337/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 08 Aug 2020 08:33:22 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Name:                      nginx-deployment-68bd55b8f6-zpzj5
Namespace:                 default
Priority:                  0
Node:                      ubuntu/10.0.2.15
Start Time:                Sat, 08 Aug 2020 08:33:11 +0100
Labels:                    app=nginx
                           pod-template-hash=68bd55b8f6
Annotations:               <none>
Status:                    Terminating (lasts 3d5h)
Termination Grace Period:  30s
IP:                        10.44.0.1
IPs:
  IP:           10.44.0.1
Controlled By:  ReplicaSet/nginx-deployment-68bd55b8f6
Containers:
  nginx:
    Container ID:   docker://4ed8b3e6896cc3923a4e446c102a4822e118eb66cb531df1e865ee4ad78e3fdb
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           1337/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 08 Aug 2020 08:33:21 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Name:         nginx-deployment-6b474476c4-9zsbx
Namespace:    default
Priority:     0
Node:         ubuntu-2004-focal-64-minimal/49.12.86.195
Start Time:   Tue, 11 Aug 2020 16:14:55 +0100
Labels:       app=nginx
              pod-template-hash=6b474476c4
Annotations:  <none>
Status:       Running
IP:           10.32.0.7
IPs:
  IP:           10.32.0.7
Controlled By:  ReplicaSet/nginx-deployment-6b474476c4
Containers:
  nginx:
    Container ID:   docker://c1610bd533ea78339aca54a252f4058e08805940dc2e55ab803728d00bda9274
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 11 Aug 2020 16:14:56 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From                                   Message
  ----    ------     ----   ----                                   -------
  Normal  Scheduled  9m51s  default-scheduler                      Successfully assigned default/nginx-deployment-6b474476c4-9zsbx to ubuntu-2004-focal-64-minimal
  Normal  Pulled     9m51s  kubelet, ubuntu-2004-focal-64-minimal  Container image "nginx:1.14.2" already present on machine
  Normal  Created    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Created container nginx
  Normal  Started    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Started container nginx

Name:         nginx-deployment-6b474476c4-xlj4m
Namespace:    default
Priority:     0
Node:         ubuntu-2004-focal-64-minimal/49.12.86.195
Start Time:   Tue, 11 Aug 2020 16:14:56 +0100
Labels:       app=nginx
              pod-template-hash=6b474476c4
Annotations:  <none>
Status:       Running
IP:           10.32.0.4
IPs:
  IP:           10.32.0.4
Controlled By:  ReplicaSet/nginx-deployment-6b474476c4
Containers:
  nginx:
    Container ID:   docker://05d30b9bcbb92bccfaec96dd8298a2ed1690394910db4703a3b7bb6624920071
    Image:          nginx:1.14.2
    Image ID:       docker-pullable://nginx@sha256:f7988fb6c02e0ce69257d9bd9cf37ae20a60f1df7563c3a2a6abe24160306b8d
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 11 Aug 2020 16:14:57 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8hfzj (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-8hfzj:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-8hfzj
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age    From                                   Message
  ----    ------     ----   ----                                   -------
  Normal  Scheduled  9m50s  default-scheduler                      Successfully assigned default/nginx-deployment-6b474476c4-xlj4m to ubuntu-2004-focal-64-minimal
  Normal  Pulled     9m50s  kubelet, ubuntu-2004-focal-64-minimal  Container image "nginx:1.14.2" already present on machine
  Normal  Created    9m50s  kubelet, ubuntu-2004-focal-64-minimal  Created container nginx
  Normal  Started    9m49s  kubelet, ubuntu-2004-focal-64-minimal  Started container nginx

Answer 1

在这种情况下，您想使用 service with selectors .还可以选择创建 service without selectors .
在您的 Deployment你用过app: nginx , 但是在 service你用过name: nginx .更多详情请查看 Service Documentation .
第二件事是在 Ingress你指向 serviceName调用nginx-service您的service名字是nginx .
另一件事是您只指定了 port在您的 service显现。在这种情况下，无论 Nginx 是否监听端口 80，但是如果您不指定它，它将分配 targetPort与 port 具有相同的值.

A Service can map any incoming port to a targetPort. By default and for convenience, the targetPort is set to the same value as the port field.

此 YAML 应该适用于您的情况:

apiVersion: apps/v1
kind:  Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
          - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels: 
    app: nginx
spec:
  type: NodePort 
  selector:
    app: nginx
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          serviceName: nginx-service
          servicePort: 80
      

最后，请记住 pathType: Prefix介绍于 Kubernetes 1.18 ，所以如果你在旧版本中使用它会出错。