ssl - terraform kubernetes 提供程序 - tls secret 未正确创建

Question

1. 我通过 terraform 创建了一个自签名的 tls 证书和私钥。这些文件称为 server.key 和 server.crt

2. 我使用此命令使用此证书和私钥创建一个 kubernetes tls secret:
   kubectl create secret tls dpaas-secret -n dpaas-prod --key server.key --cert server.crt

3. 这工作正常，nginx ingress ssl 终止工作，以及以下 kubectl 命令:kubectl get secret test-secret -o yaml -n dpaas-prod
   使用 tls.crt pem 数据和 tls.key pem 数据返回正确的输出(请参阅下面第 6 步中的正确输出)

4. 由于我们使用 terraform，我尝试通过 terraform kubernetes 提供程序使用相同的 server.key 和 servert.crt 文件创建相同的 secret 。然而这一次，命令:
   kubectl get secret test-secret -o yaml -n dpaas-prod
   为 crt pem 和 key pem 返回了奇怪的输出(请参阅下面第 5 步中的输出)并且我的 nginx ingress 上的 ssl 终止不起作用。
   这就是我通过 terraform 创建 kubernetes secret 的方式:

      resource "kubernetes_secret" "this" {
          metadata {
            name = "dpaas-secret"
            namespace = "dpaas-prod"
          }

          data = {
            "tls.crt" = "${path.module}/certs/server.crt"
            "tls.key" = "${path.module}/certs/server.key"
          }

          type = "kubernetes.io/tls"
       }  

1. 第 4 步后的错误输出。注意 tls.crt 和 tls.key 的短值(通过 terraform 创建的 secret ):

    *apiVersion: v1
    data:
      tls.crt: bXktdGVzdHMvY2VydHMvc2VydmVyLmNydA==
      tls.key: bXktdGVzdHMvY2VydHMvc2VydmVyLmtleQ==
   `enter code here` kind: Secret
    metadata:
      creationTimestamp: "2019-12-17T16:18:22Z"
      name: dpaas-secret
      namespace: dpaas-prod
      resourceVersion: "9879"
      selfLink: /api/v1/namespaces/dpaas-prod/secrets/dpaas-secret
      uid: d84db7f0-20e8-11ea-aa92-1269ad9fd693
    type: kubernetes.io/tls*  

1. 按照第 2 步正确输出(通过 kubectl 命令创建的 secret )

*apiVersion: v1
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR4RENDQXF5Z0F3SUJBZ0lRZGM0cmVoOHRQNFZPN3NkTWpzc1lTVEFOQmdrcWhraUc5dzBCQVFzRkFEQkUKTVJBd0RnWURWUVFLRXdkbGVHRnRjR3hsTVJRd0VnWURWUVFMRXd0bGVHRnRjR3hsS
UdSd2N6RWFNQmdHQTFVRQpBd3dSS2k1a2NITXVaWGhoYlhCc1pTNWpiMjB3SGhjTk1Ua3hNakUzTVRZeE5UVXpXaGNOTWpreE1qRTNNRE14Ck5UVXpXakJaTVFzd0NRWURWUVFHRXdKVlV6RVVNQklHQTFVRUNoTUxaWGhoYlhCc1pTQmtjSE14R0RBV0JnTlYKQkFzV
EQyVjRZVzF3YkdVdVpIQnpMbU52YlRFYU1CZ0dBMVVFQXd3UktpNWtjSE11WlhoaGJYQnNaUzVqYjIwdwpnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDaDU0enBwVTBod1hscm1qVXpOeVl0Ckp5WG9NSFU4WFpXTzhoVG9KZ09YUDU5N
nZFVmJQRXJlQ1VxM1BsZXB5SkRrcHNWbHo1WWc1TWp4NkVGTnlxNVQKOHVLUlVZUXNPVzNhd1VCbzM2Y3RLZEVvci8wa0JLNXJvYTYyR2ZFcHJmNVFwTlhEWnY3T1Y1YU9VVjlaN2FFTwpNNEl0ejJvNWFVYm5mdHVDZVdqKzhlNCtBS1phVTlNOTFCbFROMzFSUUFSR
3RnUzE4MFRzcVlveGV3YXBoS3FRCmUvTm5TeWF6ejUyTU5jeml6WTRpWXlRUU9EbUdEOEtWRGRJbWxJYXFoYXhiVGVTMldWZFJzdmpTa2xVZ0pGMUUKb2VWaWo1KytBd0FBczYwZkI2M1A4eFB1NEJ3cmdGTmhTV2F2ZXdJV1RMUXJPV1I2V2wvWTY1Q3lnNjlCU0xse
gpBZ01CQUFHamdad3dnWmt3RGdZRFZSMFBBUUgvQkFRREFnV2dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CCkJnZ3JCZ0VGQlFjREFqQU1CZ05WSFJNQkFmOEVBakFBTUI4R0ExVWRJd1FZTUJhQUZPanRBdTJoNDN0WjhkS1YKaHUzc2xVS3VJYTlHTURrR0ExV
WRFUVF5TURDQ0VTb3VaSEJ6TG1WNFlXMXdiR1V1WTI5dGdoc3FMbVJ3Y3k1MQpjeTFsWVhOMExURXVaWGhoYlhCc1pTNWpiMjB3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUlxRVlubHdwQnEyCmNmSnhNUUl0alF4ZTlDK2FDTnZXS1VOZjlwajlhZ3V6YXNxTW9wU
URWTFc1dnZxU21FbHJrVXNBQzJPemZ3K2UKRkxKNUFvOFg3VFcxTHBqbk01Mm1FVjRZYUcvM05hVTg5dWhOb0FHd0ZPbU5TK3ZldU12N3RKQjhsUHpiQ1k3VApKaG9TL2lZVE9jUEZUN1pmNkVycjFtd1ZkWk1jbEZuNnFtVmxwNHZGZk1pNzRFWnRCRXhNaDV3aWU3Q
Wl4Z2tTCmZaVno4QUEzTWNpalNHWFB6YStyeUpJTnpYY0gvM1FRaVdLbzY5SUQrYUlSYTJXUUtxVlhVYmk0bmlZaStDUXcKeTJuaW5TSEVCSDUvOHNSWVZVS1ZjNXBPdVBPcFp0RmdqK1l6d1VsWGxUSytLRTR0R21Ed09teGxvMUNPdGdCUAorLzFXQWdBN1p0QT0KL
S0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBb2VlTTZhVk5JY0Y1YTVvMU16Y21MU2NsNkRCMVBGMlZqdklVNkNZRGx6K2ZlcnhGCld6eEszZ2xLdHo1WHFjaVE1S2JGWmMrV0lPVEk4ZWhCVGNxdVUvTGlrVkdFT
ERsdDJzRkFhTituTFNuUktLLzkKSkFTdWE2R3V0aG54S2EzK1VLVFZ3MmIremxlV2psRmZXZTJoRGpPQ0xjOXFPV2xHNTM3YmdubG8vdkh1UGdDbQpXbFBUUGRRWlV6ZDlVVUFFUnJZRXRmTkU3S21LTVhzR3FZU3FrSHZ6WjBzbXM4K2RqRFhNNHMyT0ltTWtFRGc1C
mhnL0NsUTNTSnBTR3FvV3NXMDNrdGxsWFViTDQwcEpWSUNSZFJLSGxZbytmdmdNQUFMT3RId2V0ei9NVDd1QWMKSzRCVFlVbG1yM3NDRmt5MEt6bGtlbHBmMk91UXNvT3ZRVWk1Y3dJREFRQUJBb0lCQUZaNjFqdmUvY293QytrNwozM3JSMUdSOTZyT1JYcTIxMXpNW
mY2MVkwTVl6Ujc1SlhrcVRjL0lSeUlVRW1kS292U3hGSUY5M2VGdHRtU0FOCnpRUCtaUXVXU3dzUUhhZDVyWUlSZzVRQkVzeis3eWZxaVM1NkNhaVlIamhLdHhScVNkTk5tSmpkSlBHV3UyYWQKZEc4V2pOYUhFTnZqVkh3Q0RjdU5hVGJTSHhFOTAwSjhGQTg0c3d2M
lZFUGhSbExXVjJudVpLTko5aGIrY2IzVQpsZ2JrTnVxMkFsd2Y3MkRaTVRXZ21DM3N1Z004eGYwbWFCRWV3UXdETVdBZis2dWV6MEJ5V0hLdThwNHZRREJvCjBqQVYzOGx6UHppTDU3UTZYbFdnYjIxWUh2QmJMSVVKcEFRMGdrcGthaEFNVmJlbHdiSDJVR25wOXcrb
zU3MnIKTmhWMFJXRUNnWUVBeEtFU3FRWEV5dmpwWU40MGxkbWNhTmdIdFd1d3RpUThMMWRSQitEYXBUbWtZbzFlWXJnWgpzNi9HbStRalFtNlU1cG04dHFjTm1nL21kTDJzdk9uV1Y1dnk5eThmZHo3OFBjOFlobXFJRE5XRE9tZG9wUVJsCmxsVUZ6S0NwRmVIVTJRW
URYYjBsNWNvZzYyUVFUaWZIdjFjUGplWlltc2I5elF0cDd6czJZMGtDZ1lFQTBzcFoKTWRRUUxiRkZkWDlYT05FU2xvQlp3Slg5UjFQZVA0T2F4eUt2a01jQXFFQ0Npa05ZU3FvOU55MkZQNVBmQlplQgpWbzYvekhHR0dqVkFPQUhBczA5My8zZUZxSFRleWVCSzhQR
kJWMHh5em9ZZThxYUhBR1JxVnpjS240Zy9LVjhWClpjVGMwTm5aQzB5b09NZkhYUTVnQm1kWnpBVXBFOHlqZzhucGV0c0NnWUVBd0ZxU1ZxYytEUkhUdk4ranNiUmcKUG5DWG1mTHZ2RDlXWVRtYUc0cnNXaFk1cWUrQ0ZqRGpjOVRSQmsvMzdsVWZkVGVRVlY2Mi82L
3VVdVg2eGhRNwppeGtVWnB2Q3ZIVHhiY1hheUNRUFUvN0xrYWIzeC9hMUtvdWlVTHdhclQxdmE1OW1TNTF1SlkzSEJuK3RNOGZXCnNHZ0szMVluOThJVEp6T3pQa1UrdjRFQ2dZQUpRQkFCKzhocCtPbVBhbk10Yng5ZHMydzg0MWdtRlN3ZnBXcloKYWxCQ0RqbWRLS
mVSOGJxaUxDNWJpWWZiYm1YUEhRTDBCWGV0UlI0WmNGVE5JR2FRZCtCUU9iS0gzZmtZNnRyZgpEL2RLR1hUQVUycHdRNWFSRWRjSTFNV0drcmdTM0xWWHJmZnl3bHlmL2xFempMRFhDSloyTVhyalZTYWtVOHFwCk1lY3BHUUtCZ0c0ZjVobDRtL1EvaTNJdGZJbGw4W
DFRazFUSXVDK0JkL1NGM0xVVW5JUytyODA4dzVuaTNCUnEKNXgvQjFnRUhZbTNjSTROajR5ZzEvcE1CejhPMk1PeFhVbVNVZVh6dit1MG5oOFQxUE96eDJHOTNZaVlOL0cvNQpjMlBMSFMvTTlmVjhkTEVXL0hBVFM3K0hsMDFGQlVlREhrODQrVXlha2V2ZFU2djdVZ
2ErCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
kind: Secret
metadata:
  creationTimestamp: "2019-12-17T16:16:56Z"
  name: dpaas-secret
  namespace: dpaas-prod
  resourceVersion: "9727"
  selfLink: /api/v1/namespaces/dpaas-prod/secrets/dpaas-secret
  uid: a5739134-20e8-11ea-a977-0a0ee9b505f9
type: kubernetes.io/tls*  

问题:为什么在使用 terraform 添加 secret 时，kubernetes secret pem 数据变短(损坏？)？
我们使用 terraform 版本 0.12.8 和 kubernetes 1.13

为了重现这里是文件链接:

• server.crt - https://drive.google.com/open?id=1vFakHWkx9JxyDAEzFE_5fJomEkmQQ7Wt
• server.key - https://drive.google.com/open?id=1wc5Xn-yHWDDY9mFQ2l42k2KqeCObb-L3

Answer 1

问题是您将证书文件的路径编码为 secret ，而不是文件的内容。

如果您对示例中的 secret 字符串进行 base64 解码，您会发现情况就是这样:

$ echo -n bXktdGVzdHMvY2VydHMvc2VydmVyLmNydA== | base64 -d
my-tests/certs/server.crt
$ echo -n bXktdGVzdHMvY2VydHMvc2VydmVyLmtleQ== | base64 -d
my-tests/certs/server.key

所以，而不是这个:

data = {
    "tls.crt" = "${path.module}/certs/server.crt"
    "tls.key" = "${path.module}/certs/server.key"
}

这样做:

data = {
    "tls.crt" = file("${path.module}/certs/server.crt")
    "tls.key" = file("${path.module}/certs/server.key")
}