kubernetes - 如何使用 Terraform 在 AWS EKS 上为 kubernetes 设置 letencrypt 证书颁发者

Question

我正在尝试在 kubernetes 集群上设置 letencrypt cert-issuer。我的地形看起来像这样:

resource "helm_release" "cert_manager" {
  keyring = ""
  name = "cert-manager"
  chart = "stable/cert-manager"
  namespace = "kube-system"
  depends_on = ["helm_release.ingress"]
  set {
    name  = "webhook.enabled"
    value = "false"
  }

  provisioner "local-exec" {
    command = "kubectl --server=${aws_eks_cluster.demo.endpoint} --insecure-skip-tls-verify=true --token=${data.aws_eks_cluster_auth.demo.token} apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.6/deploy/manifests/00-crds.yaml"
  }

  provisioner "local-exec" {
    command = "kubectl --server=${aws_eks_cluster.demo.endpoint} --insecure-skip-tls-verify=true --token=${data.aws_eks_cluster_auth.demo.token} label namespace kube-system certmanager.k8s.io/disable-validation=\"true\" --overwrite"
  }

  provisioner "local-exec" {
    command = <<EOT
cat <<EOF | kubectl --server=${aws_eks_cluster.demo.endpoint} --insecure-skip-tls-verify=true --token=${data.aws_eks_cluster_auth.demo.token} create -f -
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: mymail@gmail.com
    privateKeySecretRef:
      name: letsencrypt
    http01: {}
EOF
EOT
  }
}

我部署了简单的测试 pod 和服务。当我去 http://<cluster-address>/apple它响应 apple .所以我尝试为它创建入口:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt
    kubernetes.io/ingress.class: nginx
  labels:
    app: apple
    heritage: Tiller
    release: apple
spec:
  rules:
    - http:
        paths:
          - path: /apple
            backend:
              serviceName: apple-service
              servicePort: 5678
  tls:
    - hosts:
      - my.domain.alias.to.cluster.address.io
      secretName: my.domain.alias.to.cluster.address.io

但是，当我去 https://my.domain.alias.to.cluster.address.io/apple 时我的浏览器警告我，我可以看到证书是 Kubernetes Ingress Controller Fake Certificate。

我错过了什么？我应该怎么做才能让 letencrypt 在那里创建证书？

更新:

来自我的 cert-manager pod 的日志:

I0220 16:34:49.071883       1 sync.go:180] Certificate "my.domain.alias.to.cluster.address.io" for ingress "example-ingress" is up to date
I0220 16:34:49.072121       1 controller.go:179] ingress-shim controller: Finished processing work item "default/example-ingress"
I0220 16:34:49.071454       1 controller.go:145] certificates controller: syncing item 'default/my.domain.alias.to.cluster.address.io'
I0220 16:34:49.073892       1 helpers.go:183] Setting lastTransitionTime for Certificate "my.domain.alias.to.cluster.address.io" condition "Ready" to 2019-02-20 16:34:49.073885527 +0000 UTC m=+889.175312552
I0220 16:34:49.074450       1 sync.go:263] Certificate default/my.domain.alias.to.cluster.address.io scheduled for renewal in 1438h47m42.92555861s
I0220 16:34:49.081224       1 controller.go:151] certificates controller: Finished processing work item "default/my.domain.alias.to.cluster.address.io"
I0220 16:34:49.081479       1 controller.go:173] ingress-shim controller: syncing item 'default/example-ingress'
I0220 16:34:49.081567       1 sync.go:177] Certificate "my.domain.alias.to.cluster.address.io" for ingress "example-ingress" already exists
I0220 16:34:49.081631       1 sync.go:180] Certificate "my.domain.alias.to.cluster.address.io" for ingress "example-ingress" is up to date
I0220 16:34:49.081672       1 controller.go:179] ingress-shim controller: Finished processing work item "default/example-ingress"
I0220 16:34:49.081743       1 controller.go:145] certificates controller: syncing item 'default/my.domain.alias.to.cluster.address.io'
I0220 16:34:49.082384       1 sync.go:263] Certificate default/my.domain.alias.to.cluster.address.io scheduled for renewal in 1438h47m42.917624001s
I0220 16:34:49.087552       1 controller.go:151] certificates controller: Finished processing work item "default/my.domain.alias.to.cluster.address.io"
I0220 16:35:04.571789       1 controller.go:173] ingress-shim controller: syncing item 'default/example-ingress'

这就是 kubectl describe certificate my.domain.alias.to.cluster.address.io返回:

Name:         my.domain.alias.to.cluster.address.io
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2019-02-20T16:34:49Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  example-ingress
    UID:                   709a55df-352d-11e9-bf9d-06ede39599be
  Resource Version:        278211
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/default/certificates/my.domain.alias.to.cluster.address.io
  UID:                     709bf1bd-352d-11e9-b941-026486635030
Spec:
  Acme:
    Config:
      Domains:
        my.domain.alias.to.cluster.address.io
      Http 01:
        Ingress:
        Ingress Class:  nginx
  Dns Names:
    my.domain.alias.to.cluster.address.io
  Issuer Ref:
    Kind:       ClusterIssuer
    Name:       letsencrypt
  Secret Name:  my.domain.alias.to.cluster.address.io
Status:
  Conditions:
    Last Transition Time:  2019-02-20T16:34:49Z
    Message:               Certificate is up to date and has not expired
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2019-05-21T15:22:32Z
Events:                    <none>

在入口 Controller 的日志中，我可以找到:

I0220 16:22:34.428736       8 store.go:446] secret default/my.domain.alias.to.cluster.address.io was updated and it is used in ingress annotations. Parsing...
I0220 16:22:34.429898       8 backend_ssl.go:68] Adding Secret "default/my.domain.alias.to.cluster.address.io" to the local store
I0220 16:22:35.410950       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:22:35.522502       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:22:35 +0000]TCP200000.000
I0220 16:27:39.225810       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:27:39.226685       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"f2f0c9bd-345d-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"277488", FieldPath:""}): type: 'Normal' reason: 'DELETE' Ingress default/example-ingress
I0220 16:27:39.336879       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:27:39 +0000]TCP200000.001
I0220 16:27:53.090686       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"78ab0815-352c-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"277520", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/example-ingress
I0220 16:27:53.091216       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:27:53.212854       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:27:53 +0000]TCP200000.000
I0220 16:28:04.566342       8 status.go:388] updating Ingress default/example-ingress status from [] to [{34.245.112.11 }]
I0220 16:28:04.576525       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"78ab0815-352c-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"277542", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/example-ingress
I0220 16:28:05.676217       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"78ab0815-352c-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"277546", FieldPath:""}): type: 'Normal' reason: 'DELETE' Ingress default/example-ingress
I0220 16:28:07.909830       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:28:08.019070       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:28:08 +0000]TCP200000.000
I0220 16:28:22.557334       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:28:22.557490       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"cm-acme-http-solver-dmnqh", UID:"7f8f4be4-3461-11e9-b941-026486635030", APIVersion:"extensions/v1beta1", ResourceVersion:"277576", FieldPath:""}): type: 'Normal' reason: 'DELETE' Ingress default/cm-acme-http-solver-dmnqh
I0220 16:28:22.662971       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:28:22 +0000]TCP200000.000
I0220 16:34:49.057385       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"709a55df-352d-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"278207", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/example-ingress
I0220 16:34:49.057688       8 controller.go:172] Configuration changes detected, backend reload required.
I0220 16:34:49.175039       8 controller.go:190] Backend successfully reloaded.
[20/Feb/2019:16:34:49 +0000]TCP200000.000
I0220 16:35:04.565324       8 status.go:388] updating Ingress default/example-ingress status from [] to [{34.245.112.11 }]
I0220 16:35:04.572954       8 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"example-ingress", UID:"709a55df-352d-11e9-bf9d-06ede39599be", APIVersion:"extensions/v1beta1", ResourceVersion:"278236", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/example-ingress
10.0.1.114 - [10.0.1.114] - - [20/Feb/2019:18:38:33 +0000] "\x05\x01\x00" 400 157 "-" "-" 0 0.751 [] - - - - e0aec2a9e3e71e136a1c62939e341b49
10.0.1.114 - [10.0.1.114] - - [20/Feb/2019:18:39:50 +0000] "\x04\x01\x00P\x05\xBC\xD2\x0C\x00" 400 157 "-" "-" 0 0.579 [] - - - - 7f825a3ef2f94e200b14fe3691e4fdde
10.0.1.114 - [10.0.1.114] - - [20/Feb/2019:18:41:30 +0000] "GET http://5.188.210.12/echo.php HTTP/1.1" 400 657 "https://www.google.com/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36" 359 0.000 [] - - - - 1167890a763ddc360051046c84a47d21
10.0.1.114 - [10.0.1.114] - - [20/Feb/2019:19:46:35 +0000] "GET /apple HTTP/1.1" 308 171 "-" "Mozilla/5.0 (Linux; Android 8.0.0; ONEPLUS A3003) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.105 Mobile Safari/537.36" 555 0.000 [default-apple-service-5678] - - - - b1f1bb0da3e465c3a54e963663dffb61
10.0.1.114 - [10.0.1.114] - - [20/Feb/2019:20:38:39 +0000] "\x03\x00\x00/*\xE0\x00\x00\x00\x00\x00Cookie: mstshash=Administr" 400 157 "-" "-" 0 0.065 [] - - - - cd420e70b3f78bee069f8bac97918e36

Answer 1

基本上，letsencrypt 不会为您颁发证书，因此它默认为假证书。您需要确保 my.domain.alias.to.cluster.address.io可公开解析，例如通过 DNS 服务器，例如 8.8.8.8然后它需要解析为可公开访问的 IP 地址。您可以通过查看 certmanager pod 日志来调试正在发生的事情。

$ kubectl logs <certmanagerpod>

您还可以查看有关证书的详细信息(并且您可能会看到它没有被颁发的原因)。

$ kubectl get certificates
$ kubectl describe <certificate-name>

另一方面是您可能会受到 https://acme-v02.api.letsencrypt.org/directory 的限速这是他们的生产环境。你也可以试试: https://acme-staging-v02.api.letsencrypt.org/directory这是他们的登台环境。