traefik - 无法从 X-Forwarded-For 获取真实用户的 IP

Question

我在单节点 Kubernetes 集群上运行 Traefik 1.7.3，我试图从 X-Forwarded-For 获取真实用户 IP标题，但我得到的是 X-Forwarded-For: 10.244.0.1这是我的 k8s 集群中的一个 IP。

这是我的 Traefik 部署和服务:

    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: traefik-conf
    data:
      traefik.toml: |
        # traefik.toml
        debug = true
        logLevel = "DEBUG"
        defaultEntryPoints = ["http","https"]
        [entryPoints]
          [entryPoints.http]
          address = ":80"
          compress = true
          [entryPoints.http.forwardedHeaders]
          trustedIPs = [ "0.0.0.0/0" ]
          entryPoint = "https"
          [entryPoints.https]
          address = ":443"
          compress = true
          [entryPoints.https.forwardedHeaders]
          trustedIPs = [ "0.0.0.0/0" ]
          [entryPoints.https.tls]

        [acme]
        email = "xxxx"
        storage = "/acme/acme.json"
        entryPoint = "https"
        onHostRule = true
        #caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
        acmeLogging = true
        [[acme.domains]]
        main = "xxxx"
        [acme.dnsChallenge]
        provider = "route53"
        delayBeforeCheck = 0

        [persistence]
        enabled = true
        existingClaim = "pvc0"
        annotations = {}
        accessMode = "ReadWriteOnce"
        size = "1Gi"

        [kubernetes]
        namespaces = ["default"]
        [accessLog]
        filePath = "/acme/access.log"
        [accessLog.fields]
        defaultMode = "keep"
    ---
    kind: Deployment
    apiVersion: extensions/v1beta1
    metadata:
      name: traefik-ingress-controller
      namespace: default
      labels:
        k8s-app: traefik-ingress-lb
    spec:
      replicas: 1
      selector:
        matchLabels:
          k8s-app: traefik-ingress-lb
      template:
        metadata:
          labels:
            k8s-app: traefik-ingress-lb
            name: traefik-ingress-lb
        spec:
          serviceAccountName: traefik-ingress-controller
          terminationGracePeriodSeconds: 60
          containers:
          - image: traefik
            name: traefik-ingress-lb
            env:
              - name: AWS_ACCESS_KEY_ID
                value: xxxx
              - name: AWS_SECRET_ACCESS_KEY
                value: xxxx
              - name: AWS_REGION
                value: us-west-2
              - name: AWS_HOSTED_ZONE_ID
                value: xxxx
            ports:
            - name: http
              containerPort: 80
            - name: admin
              containerPort: 8080
            args:
            - --api
            - --kubernetes
            - --configfile=/config/traefik.toml
            volumeMounts:
            - mountPath: /config
              name: config
            - mountPath: /acme
              name: acme
          volumes:
          - name: config
            configMap:
              name: traefik-conf
          - name: acme
            persistentVolumeClaim:
              claimName: "pvc0"
    ---
    kind: Service
    apiVersion: v1
    metadata:
      name: traefik-ingress-service
      namespace: default
    spec:
      externalIPs:
      - x.x.x.x
      externalTrafficPolicy: Local
      selector:
        k8s-app: traefik-ingress-lb
      ports:
        - protocol: TCP
          port: 80
          name: web
        - protocol: TCP
          port: 443
          name: https
        - protocol: TCP
          port: 8080
          name: admin
      type: NodePort

这是我的入口:

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: headers-test
      namespace: default
      annotations:
        ingress.kubernetes.io/proxy-body-size: 500m
        kubernetes.io/ingress.class: traefik
    spec:
      rules:
      - host: xxxx
        http:
          paths:
          - path: /
            backend:
              serviceName: headers-test
              servicePort: 8080

我读到我只需要添加 [entryPoints.http.forwardedHeaders]以及 trustedIPs 的列表但这似乎不起作用。我错过了什么吗？

Answer 1

如果您将 NodePort 用于 Traefik Ingress 服务，则必须将 service.spec.externalTrafficPolicy 设置为“本地”。否则，当您的连接进入 K8s 集群时，您将有一个 SNAT。如果 Pod 不在同一节点上运行，则此 SNAT 是将传入连接转发到您的 Pod 所必需的。

但请注意，将 service.spec.externalTrafficPolicy 设置为“Local”后，只有执行 Traefik pod 的节点才会接受 80、443、8080 上的请求。不再从其他节点转发到 pod。这可能会导致连接到您的服务时出现奇怪的延迟。为避免您的 Traefik 需要在 HA 设置(DaemonSet)中运行。请记住，您需要一个 K/V-Store 用于分布式 Traefik 设置，以使 Letsencrypt 运行良好。

如果 service.spec.externalTrafficPolicy 设置还没有解决您的问题，您可能还需要配置 kubernetes 覆盖网络不执行任何 SNAT。

service.spec.externalTrafficPolicy 在这里得到了很好的解释:
https://kubernetes.io/docs/tutorials/services/source-ip/