ssl - 使用 SSL 的 Logstash 输出 kafka -> SSL 握手失败

Question

我是 kafka 和 logstash 的新手。我想使用 logstash 作为生产者，并且想使用 SSL。

这是我的 logstash conf

input {
  http {
    port => 5044
    codec => json
  }
}

output {
  kafka {
    #bootstrap_servers => ["localhost:9093"]
    bootstrap_servers => ["kafka broker's IP:9093"]
    topic_id => "test"
    codec => "json"
    ssl_truststore_location => "/etc/logstash/conf.d/test/kafka.client.truststore.jks"
    ssl_truststore_password => "passwd"
    security_protocol => "SSL"
    #ssl_keystore_location => "/etc/logstash/conf.d/test/kafka.client.keystore.jks"
    #ssl_keystore_password => "passwd"
    #ssl_key_password => "passwd"
  }
}

这是 Kafka 服务器的属性。

listeners=PLAINTEXT://:9092, SSL://kafka broker's ip:9093
advertised.listeners=PLAINTEXT://kafka broker's ip:9092,SSL://kafka broker's ip:9093
security.inter.broker.protocol=SSL

#ssl.client.auth=required

ssl.keystore.location=/etc/logstash/conf.d/test/kafka.server.keystore.jks
ssl.keystore.password=dlffpr
ssl.key.password=dlffpr
ssl.truststore.location=/etc/logstash/conf.d/test/kafka.server.truststore.jks
ssl.truststore.password=dlffpr
ssl.endpoint.identification.algorithm=

这是我制作 SSL key 的方法。 I referenced Azure's guide.在制作 kafka.server.keystore.jks 时，我不确定在“CN:”中写什么。没有域名，我只知道它的主机名(它是 kafkamanager)和 ip 地址。我尝试添加 -ext "SAN=dns:hostname,ip:kafka broker's ip"

# each broker
keytool -keystore kafka.server.keystore.jks -alias asd -genkey -keyalg RSA -validity 365 -storepass "dlffpr" -keypass "dlffpr" -dname "**CN=kafka broker's ip**" -storetype pkcs12
keytool -keystore kafka.server.keystore.jks -alias asd -certreq -file cert-file -storepass "dlffpr" -keypass "dlffpr"

# manager
# CA 인증서 및 키 파일 생성
openssl req -new -newkey rsa:2048 -days 365 -x509 -subj "/CN=asdasdasd" -keyout ca-key -out ca-cert -nodes

# broker
#scp root@kafka broker's ip:/etc/logstash/conf.d/test/ca-cert .
#scp root@kafka broker's ip:/etc/logstash/conf.d/test/ca-key .
# CA cert-signed
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:"dlffpr"

# KEY STORE, truststore add ca-cert
keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert -storepass "dlffpr" -keypass "dlffpr" -noprompt
keytool -keystore kafka.server.keystore.jks -alias CARoot -import -file ca-cert -storepass "dlffpr" -keypass "dlffpr" -noprompt
# keystore add cert signed
keytool -keystore kafka.server.keystore.jks -alias asd -import -file cert-signed -storepass "dlffpr" -keypass "dlffpr" -noprompt

# manager(client)
# client keystore
keytool -keystore kafka.client.keystore.jks -alias localhost -genkey -keyalg RSA -validity 365 -storepass "dlffpr" -keypass "dlffpr" -dname "CN=localhost" -storetype pkcs12
# client cert request
keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file client-cert-sign-request -storepass "dlffpr" -keypass "dlffpr"
# CA cert
openssl x509 -req -CA ca-cert -CAkey ca-key -in ./client-cert-sign-request -out client-cert-signed -days 365 -CAcreateserial -passin pass:dlffpr
# keystore, truststore
keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert -storepass "dlffpr" -keypass "dlffpr" -noprompt
keytool -keystore kafka.client.keystore.jks -alias CARoot -import -file ca-cert -storepass "dlffpr" -keypass "dlffpr" -noprompt
# keystore
keytool -keystore kafka.client.keystore.jks -alias localhost -import -file client-cert-signed -storepass "dlffpr" -keypass "dlffpr" -noprompt

当我启动logstash和kafka时，出现了这样的错误。Logstash 错误

[org.apache.kafka.common.network.Selector][main] [Producer clientId=producer-1] Failed authentication with /192.168.1.6 (SSL handshake failed)
[2020-06-26T16:56:59,131][ERROR][org.apache.kafka.clients.NetworkClient][main] [Producer clientId=producer-1] Connection to node -1 (/kafka broker's ip:9093) failed authentication due to: SSL handshake failed
[2020-06-26T16:56:59,131][WARN ][org.apache.kafka.clients.NetworkClient][main] [Producer clientId=producer-1] Bootstrap broker kafka broker's ip:9093 (id: -1 rack: null) disconnected

卡夫卡错误

INFO [SocketServer brokerId=0] Failed authentication with /kafka client's ip (SSL handshake failed) (org.apache.kafka.common.network.Selector)

当我像这样设置 Kafka 的服务器属性并使用“CN:localhost”制作 key 时，它会起作用但是 logstash 和 kafka 不在同一台机器上。请给我任何建议。谢谢。

listeners=PLAINTEXT://:9092, SSL://localhost:9093

Answer 1

今天我们遇到了同样的问题，尝试使用 SASL_SSL 和两个监听器(每个监听器具有不同的 IP 和 DNS 名称)启动 kafka，我们重现了您的问题。当我们尝试启动 Logstash 时，不会出现如下错误:

[ERROR][org.apache.kafka.clients.NetworkClient][main] [Producer clientId=producer-1] Connection to node -1 (/kafka broker's ip:9093) failed authentication due to: SSL handshake failed

我们创建一个多域证书，然后设置选项:

ssl.keystore.location

在server.propierties(kafka)中，重启服务后，logstash正常运行!