ssl - HTTPS 重定向不适用于 nginx-ingress-controller 的默认后端

Question

对于 kubernetes 的 NGINX 入口 Controller 的默认后端，我无法从 HTTP -> HTTPS 进行自动重定向，其中 Controller 位于 AWS Classic ELB 后面；可能吗？

According to the guide似乎默认情况下启用 HSTS

HTTP Strict Transport Security
HTTP Strict Transport Security (HSTS) is an opt-in security enhancement specified through the use of a special response header. Once a supported browser receives this header that browser will prevent any communications from being sent over HTTP to the specified domain and will instead send all communications over HTTPS.

HSTS is enabled by default.

并重定向 HTTP -> 启用 HTTPS

Server-side HTTPS enforcement through redirect
By default the controller redirects HTTP clients to the HTTPS port 443 using a 308 Permanent Redirect response if TLS is enabled for that Ingress.

但是，当我按如下配置部署 Controller 并导航到 http://<ELB>.elb.amazonaws.com 时我无法得到任何回应(curl 报告 Empty reply from server)。相反，我希望发生的是我应该看到 308 重定向到 https，然后是 404。

这个问题类似:Redirection from http to https not working for custom backend service in Kubernetes Nginx Ingress Controller但他们通过部署自定义后端并在入口资源上指定使用 TLS 来解决它。我试图避免部署自定义后端，只想使用默认后端，因此此解决方案不适用于我的情况。

I've shared my deployment files on gist并将它们也复制到这里:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx-sit
  labels:
    app.kubernetes.io/name: ingress-nginx-sit
    app.kubernetes.io/part-of: ingress-nginx-sit
spec:
  minReadySeconds: 2
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: '50%'
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx-sit
      app.kubernetes.io/part-of: ingress-nginx-sit
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx-sit
        app.kubernetes.io/part-of: ingress-nginx-sit
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.23.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --ingress-class=$(POD_NAMESPACE)
            - --election-id=leader
            - --watch-namespace=$(POD_NAMESPACE)
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx-sit
  labels:
    app.kubernetes.io/name: ingress-nginx-sit
    app.kubernetes.io/part-of: ingress-nginx-sit
data:
  hsts: "true"
  ssl-redirect: "true"
  use-proxy-protocol: "false"
  use-forwarded-headers: "true"
  enable-access-log-for-default-backend: "true"
  enable-owasp-modsecurity-crs: "true"
  proxy-real-ip-cidr: "10.0.0.0/24,10.0.1.0/24" # restrict this to the IP addresses of ELB

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx-sit
  labels:
    app.kubernetes.io/name: ingress-nginx-sit
    app.kubernetes.io/part-of: ingress-nginx-sit
  annotations:
    # replace with the correct value of the generated certificate in the AWS console
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:<region>:<account>:certificate/<id>"
    # Specify the ssl policy to apply to the ELB
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-2017-01"
    # the backend instances are HTTP
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    # Terminate ssl on https port
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "*"
    # Ensure the ELB idle timeout is less than nginx keep-alive timeout. By default,
    # NGINX keep-alive is set to 75s. If using WebSockets, the value will need to be
    # increased to '3600' to avoid any potential issues.
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
    # Security group used for the load balancer.
    service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: "sg-xxxxx"
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx-sit
    app.kubernetes.io/part-of: ingress-nginx-sit
  loadBalancerSourceRanges:
    # Restrict allowed source IP ranges
    - "192.168.1.1/16"
  ports:
    - name: http
      port: 80
      targetPort: http
      # The range of valid ports is 30000-32767
      nodePort: 30080
    - name: https
      port: 443
      targetPort: http
      # The range of valid ports is 30000-32767
      nodePort: 30443

Answer 1

我想我找到了问题。

由于某些原因，默认服务器在确定是否应将传入请求重定向到 HTTPS 时将 force_ssl_redirect 设置为 false:

cat/etc/nginx/nginx.conf 注意 rewrite_by_lua_block 发送 force_ssl_redirect = false

...
    ## start server _
    server {
        server_name _ ;

        listen 80 default_server reuseport backlog=511;

        set $proxy_upstream_name "-";
        set $pass_access_scheme $scheme;
        set $pass_server_port $server_port;
        set $best_http_host $http_host;
        set $pass_port $pass_server_port;

        listen 443  default_server reuseport backlog=511 ssl http2;

        # PEM sha: 601213c2dd57a30b689e1ccdfaa291bf9cc264c3
        ssl_certificate                         /etc/ingress-controller/ssl/default-fake-certificate.pem;
        ssl_certificate_key                     /etc/ingress-controller/ssl/default-fake-certificate.pem;

        ssl_certificate_by_lua_block {
            certificate.call()
        }

        location / {

            set $namespace      "";
            set $ingress_name   "";
            set $service_name   "";
            set $service_port   "0";
            set $location_path  "/";

            rewrite_by_lua_block {
                lua_ingress.rewrite({
                    force_ssl_redirect = false,
                    use_port_in_redirects = false,
                })
                balancer.rewrite()
                plugins.run()
            }
...

然后，LUA代码需要force_ssl_redirect 和 redirect_to_https()

cat/etc/nginx/lua/lua_ingress.lua

...
  if location_config.force_ssl_redirect and redirect_to_https() then
    local uri = string_format("https://%s%s", redirect_host(), ngx.var.request_uri)

    if location_config.use_port_in_redirects then
      uri = string_format("https://%s:%s%s", redirect_host(), config.listen_ports.https, ngx.var.request_uri)
    end

    ngx_redirect(uri, config.http_redirect_code)
  end
...

据我所知，force_ssl_redirect 设置仅为 controlled at the Ingress resource通过注解 nginx.ingress.kubernetes.io/force-ssl-redirect: "true" 级别。因为我没有设置入口规则(这是不匹配任何入口的请求的默认服务器)，所以我无法更改此设置。

因此，我确定我必须做的是在将 force_ssl_redirect 设置为 true 的不同端口上定义我自己的自定义服务器片段，然后将服务负载均衡器指向该自定义服务器而不是默认服务器.具体来说:

添加到ConfigMap:

...
  http-snippet: |
    server {
      server_name _ ;
      listen 8080 default_server reuseport backlog=511;

      set $proxy_upstream_name "-";
      set $pass_access_scheme $scheme;
      set $pass_server_port $server_port;
      set $best_http_host $http_host;
      set $pass_port $pass_server_port;

      server_tokens off;
      location / {
        rewrite_by_lua_block {
            lua_ingress.rewrite({
                force_ssl_redirect = true,
                use_port_in_redirects = false,
            })
            balancer.rewrite()
            plugins.run()
        }
      }
      location /healthz {
        access_log off;
        return 200;
      }
    }
  server-snippet: |
    more_set_headers "Strict-Transport-Security: max-age=31536000; includeSubDomains; preload";

注意 我还添加了 server-snippet 以正确启用 HSTS。我认为因为从 ELB 到 NGINX 的流量是 HTTP 而不是 HTTPS，所以默认情况下没有正确添加 HSTS header 。

添加到 DaemonSet:

...
        ports:
          - name: http
            containerPort: 80
          - name: http-redirect
            containerPort: 8080
...

修改了服务:

...
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
...
  ports:
    - name: http
      port: 80
      targetPort: http-redirect
      # The range of valid ports is 30000-32767
      nodePort: 30080
    - name: https
      port: 443
      targetPort: http
      # The range of valid ports is 30000-32767
      nodePort: 30443
...

现在一切似乎都奏效了。我更新了the Gist所以它包括我正在使用的完整配置。