apache-kafka - 尽管配置正确，Kafka transactionLog 仍因 NotEnoughReplicasException 而失败

Question

我在单节点开发环境中从 Kafka 读取失败时遇到问题

我们正在运行单节点 kafka 实例和单节点 Zookeeper——因为这是一个开发环境，我们并不真正关心复制。

我们使用 Kafka 作为 Akka 持久化的事件存储。当我们尝试读取消息时(即使是从一个空主题)，我们会引发 NotEnoughReplicasException:

[2018-05-03 16:57:00,880] ERROR [ReplicaManager broker=0] Error processing append operation on partition __transaction_state-7 (kafka.server.ReplicaManager)
org.apache.kafka.common.errors.NotEnoughReplicasException: Number of insync replicas for partition __transaction_state-7 is [1], below required minimum [2]
[2018-05-03 16:57:00,889] INFO [Transaction State Manager 0]: TransactionalId akka-journal-message-AkkaJavaSpring-4 append transaction log for TxnTransitMetadata(producerId=6000, producerEpoch=0, txnTimeoutMs=60000, txnState=Empty, topicPartitions=Set(), txnStartTimestamp=-1, txnLastUpdateTimestamp=1525366620855) transition failed due to COORDINATOR_NOT_AVAILABLE, resetting pending state from Some(Empty), aborting state transition and returning COORDINATOR_NOT_AVAILABLE in the callback (kafka.coordinator.transaction.TransactionStateManager)

我配置了以下设置:

"min.insync.replicas": 1
"offsets.topic.replication.factor": 1
"transaction.state.log.replication.factor": 1
"transaction.state.log.min.isr" : 1

在启动时登录的配置输出中得到确认:

[2018-05-03 16:33:34,381] INFO KafkaConfig values: 
    ...snip...
    min.insync.replicas = 1
    num.io.threads = 8
    num.network.threads = 3
    num.partitions = 1
    ...snip...
    transaction.state.log.load.buffer.size = 5242880
    transaction.state.log.min.isr = 1
    transaction.state.log.num.partitions = 50
    transaction.state.log.replication.factor = 1
    transaction.state.log.segment.bytes = 104857600

但是，查询 __transactionLog 主题似乎报告它的 min.insync.replicas 设置为 2:

/usr/bin/kafka-topics --zookeeper zookeeper:2181 --describe

Topic:__transaction_state       PartitionCount:50       ReplicationFactor:1     Configs:segment.bytes=104857600,unclean.leader.election.enable=false,compression.type=uncompressed,cleanup.policy=compact,min.insync.replicas=2
        Topic: __transaction_state      Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: __transaction_state      Partition: 1    Leader: 0       Replicas: 0     Isr: 0
        Topic: __transaction_state      Partition: 2    Leader: 0       Replicas: 0     Isr: 0
        Topic: __transaction_state      Partition: 3    Leader: 0       Replicas: 0     Isr: 0

我使用的是 Kafka 1.0.0，如日志所示:

 INFO Kafka version : 1.0.0-cp1 (org.apache.kafka.common.utils.AppInfoParser)

这是什么原因造成的，防止这些异常发生的正确配置是什么？

Answer 1

上面发布的配置结果是正确的。

但是，当主题最初创建时，Zookeeper 实例配置不正确，重放日志中的消息似乎会引发错误。

删除 Zookeeper 实例并重新启动解决了错误