amazon-web-services - 使用 NLB 和 IAM 身份验证公开 AWS MSK - 主机名验证失败

Question

我们正在努力让 Amazon MSK (Kafka) 与 IAM 身份验证一起使用，然后使用 aws kafka 广告监听器中的更改使其可通过 DNS 公开访问。为了实现这一点，我们遵循与下面相同的基础设施计划，但我们使用网络负载均衡器而不是接口(interface)端点: https://aws.amazon.com/blogs/big-data/how-goldman-sachs-builds-cross-account-connectivity-to-their-amazon-msk-clusters-with-aws-privatelink/

有趣的是，相同的基础设施在使用 SASL/SCRAM 身份验证时可以完美运行，但不能使用 IAM 身份验证。您是否有关于可公开访问的 AWS MSK 和 IAM 身份验证问题的任何信息？

基本上我们遵循指南中的想法，特别是模式 2:使用单个共享接口(interface)端点为所有 MSK 代理提供前端，但改用 IAM 身份验证使用您的 AWS MSK 的 IAM 指南，我们已经使用内部 dns 代理地址成功地与我们的代理进行了通信。当我们稍后根据上述指南更改通告的监听器时，我们无法与代理通信并收到错误消息:

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SaslAuthenticationException: [9d5b944c-df83-4573-9979-4d121f49a533]: Hostname verification failed
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
at kafka.admin.ConfigCommand$.getResourceConfig(ConfigCommand.scala:552)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4(ConfigCommand.scala:512)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4$adapted(ConfigCommand.scala:504)
at scala.collection.immutable.List.foreach(List.scala:431)
at kafka.admin.ConfigCommand$.describeResourceConfig(ConfigCommand.scala:504)
at kafka.admin.ConfigCommand$.describeConfig(ConfigCommand.scala:484)
at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:304)
at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:97)
at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: [9d5b944c-df83-4573-9979-4d121f49a533]: Hostname verification failed

Answer 1

事实证明这不受支持，这是来自 AWS Support 的消息:

Dear Customer,
 
Thank you for you patience while I investigate this issue.
 
After going through our internal resources, I would like to inform you that unfortunately IAM authentication against cluster using a custom domain name through intermediate NLB is not supported as of now. 
 
Also, I could confirm that there is an already existing feature request for this and it is indeed in the backlog of our MSK service team. As you may understand, any new functionality addition goes through regressive testing and analysis to determine feasibility and ensure the stability of the service. It is for this reason that we cannot provide a timeline on when this feature would be available. I sincerely apologise on behalf of AWS for the inconvenience caused. I appreciate your understanding and patience with us as we grow the service. 
 
In the meantime, I would suggest you to keep an eye on our What's New page[1] and AWS Blogs[2] for updates on the latest announcements.
 
In case you require any further assistance kindly feel free to reach out to me and I will be happy to assist you with the same.
 
Stay safe and Have a nice day!