gpt4 book ai didi

apache-zookeeper - 将 Kafka 0.8.1 与 Zookeeper 3.4.6 结合使用时遇到 LeaderNotAvailableException

转载 作者:行者123 更新时间:2023-12-03 05:32:37 24 4
gpt4 key购买 nike

我按照他们的网站安装了 kafka 的稳定版本(0.8.1 和 2.9.2 Scala),并使用 3 节点 Zookeeper 整体(3.4.6)运行它。我尝试创建一个测试主题,但一直看到没有领导者分配给该主题的分区:

[kafka_2.9.2-0.8.1]$ ./bin/kafka-topics.sh --zookeeper <zookeeper_ensemble> --describe --topic test-1
Topic:test-1 PartitionCount:1 ReplicationFactor:3 Configs:
Topic: test-1 Partition: 0 **Leader: none** Replicas: 0,1,2 **Isr:**

无论如何,我尝试使用控制台生成器写入该主题,但遇到了 LeaderNotAvailableException 异常:

[kafka_2.9.2-0.8.1]$ ./kafka-console-producer.sh --broker-list <broker_list> --topic test-1

hello world

[2014-04-22 11:58:48,297] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,321] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,322] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test-1 (kafka.producer.async.DefaultEventHandler)

[2014-04-22 11:58:48,445] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,467] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,467] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test-1 (kafka.producer.async.DefaultEventHandler)

[2014-04-22 11:58:48,590] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,612] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,612] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test-1 (kafka.producer.async.DefaultEventHandler)

[2014-04-22 11:58:48,731] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,753] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,754] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test-1 (kafka.producer.async.DefaultEventHandler)

[2014-04-22 11:58:48,876] WARN Error while fetching metadata [{TopicMetadata for topic test-1 ->
No partition metadata for topic test-1 due to kafka.common.LeaderNotAvailableException}] for topic [test-1]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)

[2014-04-22 11:58:48,877] ERROR Failed to send requests for topics test-1 with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)

[2014-04-22 11:58:48,878] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)

我还应该指出,这最初工作了几天,然后突然创建的任何主题都出现了缺少领导者的问题。

最佳答案

Kafka 使用外部协调框架(默认为 Zookeeper)来维护配置。看来配置现在与 Kafka 日志数据不同步。在这种情况下,我将删除受影响的主题数据和相关的 Zookeeper 数据。

对于测试环境:

  1. 停止Kafka服务器Zookeeper服务器
  2. 移除两个服务的数据目录,默认为 /tmp/kafka-log/tmp/zookeeper
  3. 再次启动 Kafka-serverZookeeper-server
  4. 创建新主题

现在您可以再次处理该主题。

对于生产环境:

由于 Kafka 主题存储在不同的目录中,因此您应该删除特定的目录。您还应该使用 Zookeeper 客户端从 Zookeeper 中删除 /brokers/{broker_id}/topics/{broken_topic}

在做任何愚蠢的事情之前,请仔细阅读 Kafka 文档以确保配置结构。 Kafka正在推出删除主题功能( KAFKA-330 ),以便更轻松地解决问题。

关于apache-zookeeper - 将 Kafka 0.8.1 与 Zookeeper 3.4.6 结合使用时遇到 LeaderNotAvailableException,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/23228222/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com