apache-kafka - Kafka - 简单的消费者/生产者设置不适用于不同的机器，但可以在本地工作

Question

我已经使用以下简单的 producer/consumer 教程安装并设置了 Kafka:

https://kafka.apache.org/quickstart

我有两台机器，都在使用Ubuntu。

问题重现:

如果我在同一台机器上使用 producer 和 consumer，一切正常。如果我在machine 2上使用producer，而在machine 1上使用其余的，比如kafka，zookeeper 服务器和消费者，我从未在机器 1 上收到任何消息。

Machine 1 has IP: 192.168.1.100

Machine 2 has IP: 192.168.1.101

Working Example using just the Machine 1 only, with 4 console applications

控制台 1 - 启动 zookeeper:

bin/zookeeper-server-start.sh config/zookeeper.properties

控制台 2 - 启动 kafka 服务器

bin/kafka-server-start.sh config/server.properties

控制台 3创建了一个名为 test 的主题:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

测试主题:

bin/kafka-topics.sh --list --zookeeper localhost:2181

启动消费者

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test

控制台 4 - 发送一些消息:

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

有效!

NOT Working Example using Machine 1 and 2

机器 1 - 控制台 1 - 启动 zookeeper:

bin/zookeeper-server-start.sh config/zookeeper.properties

机器 1 - 控制台 2 - 启动了 kafka 服务器

bin/kafka-server-start.sh config/server.properties

机器 1 - 控制台 3创建了一个名为 test 的主题:

bin/kafka-topics.sh --create --zookeeper 192.168.1.100:2181 --replication-factor 1 --partitions 1 --topic test

测试主题:

bin/kafka-topics.sh --list --zookeeper 192.168.1.100:2181

启动消费者

bin/kafka-console-consumer.sh --bootstrap-server 192.168.1.100:9092 --topic test

机器 2 - 控制台 1 - 向 IP 192.168.1.100 上的 kafka 发送一些消息

bin/kafka-console-producer.sh --broker-list 192.168.1.100:9092 --topic test

不工作...

我在这里错过了什么？

编辑:

我现在使用 kafkacat 来测试连接...这是输出。

root@ubuntu:~/kafka/kafka_2.11-0.11.0.1# kafkacat -b 192.168.1.100 -t test -L
Metadata for test (from broker -1: 192.168.1.100:9092/bootstrap):
 1 brokers:
  broker 0 at ubuntu:9092
 1 topics:
  topic "test" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0
%3|1507802180.807|FAIL|rdkafka#producer-1| [thrd:ubuntu:9092/0]: ubuntu:9092/0: Connect to ipv4#127.0.1.1:9092 failed: Connection refused
%3|1507802180.807|ERROR|rdkafka#producer-1| [thrd:ubuntu:9092/0]: ubuntu:9092/0: Connect to ipv4#127.0.1.1:9092 failed: Connection refused

为什么上面设置的是127.0.1.1:9092？不应该是 192.168.1.100:9092 吗？

Answer 1

来自 the FAQ ，一个常见的问题是主机名:

When a broker starts up, it registers its ip/port in ZK. You need to make sure the registered ip is consistent with what's listed in metadata.broker.list in the producer config. By default, the registered ip is given by InetAddress.getLocalHost.getHostAddress(). Typically, this should return the real ip of the host. However, sometimes (e.g., in EC2), the returned ip is an internal one and can't be connected to from outside. The solution is to explicitly set the host ip to be registered in ZK by setting the hostname property in server.properties. In another rare case where the binding host/port is different from the host/port for client connection, you can set advertised.host.name and advertised.port for client connection.

所以，打开你的 server.properties并更改 hostname :

host.name=<your hostname>

如果还是不行，尝试修改advertised.host.name :

advertised.host.name=<your ip>

如果这真的不起作用，请查看 advertised.listeners并确保它是 0.0.0.0:port或 <your.ip>:port .示例:

advertised.listeners=PLAINTEXT://0.0.0.0:9092