apache-kafka - 中止 kafka 重新分配分区操作

Question

我正在使用 kafka 0.8.2 & 我的一个 kafka 服务器死了(无法恢复磁盘上的数据)。有一个主题，复制为 1，其中一个分区位于死服务器上。我认为重新分配会将那个分区的元数据移动到新服务器而不需要数据，但重新分配被困在 in progress .

我跑了:

$ /opt/kafka/kafka/bin/kafka-reassign-partitions.sh --zookeeper myzookeeper.my.com --reassignment-json-file new_assignment.json --verify
Status of partition reassignment:
Reassignment of partition [topicX,1] is still in progress

这永远不会成功，因为死服务器永远不会回来。

在新服务器的日志中，我看到:

[2015-05-28 06:25:15,401] INFO Completed load of log topicX-1 with log end offset 0 (kafka.log.Log)
[2015-05-28 06:25:15,402] INFO Created log for partition [topicX,1] in /mnt2/data/kafka with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, min.insync.replicas -> 1, cleanup.policy -> delete, unclean.leader.election.enable -> true, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 259200000, segment.jitter.ms -> 0}. (kafka.log.LogManager)
[2015-05-28 06:25:15,403] WARN Partition [topicX,1] on broker 4151132: No checkpointed highwatermark is found for partition [topicX,1] (kafka.cluster.Partition)
[2015-05-28 06:25:15,405] INFO [ReplicaFetcherManager on broker 4151132] Removed fetcher for partitions  (kafka.server.ReplicaFetcherManager)
[2015-05-28 06:25:15,408] INFO [ReplicaFetcherManager on broker 4151132] Added fetcher for partitions List() (kafka.server.ReplicaFetcherManager)
[2015-05-28 06:25:15,411] INFO [ReplicaFetcherManager on broker 4151132] Removed fetcher for partitions  (kafka.server.ReplicaFetcherManager)
[2015-05-28 06:25:15,413] INFO [ReplicaFetcherManager on broker 4151132] Added fetcher for partitions List() (kafka.server.ReplicaFetcherManager)

有没有办法强制它完成或中止重新分配操作？

Answer 1

您可以通过使用zookeeper shell删除zookeeper集群上的“/admin/reassign_partitions”zk节点来中止分配，并将分配给死代理的分区移动到新节点。