apache-kafka - Storm : KafkaSpout - where is my consumer group?

Question

我需要查看storm通过其KafkaSpout读取的偏移值。这是我传入的配置:

SpoutConfig kafkaConfig = new SpoutConfig(brokerHosts, "some_values",  
     "/storm/env_values", "storm_DEBUG");

我尝试使用一些 kafka 工具进行搜索，但还没有找到任何有用的东西:

• kafka.tools.ExportZkOffsets
• kafka.tools.ConsumerOffsetChecker

有没有更好的工具可以用来查找我的偏移量？

Answer 1

看看https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem在管理控制台中有一个用于监控主题和偏移量的工具列表。我记得我用过https://github.com/otoolep/stormkafkamon一个python监控工具。

但是你到底想要什么？，如果你只是试图查看你的偏移量，你可以在 zookeeper 中看到你的偏移量，只连接到存储偏移量的 zookeeper。

例子:

bin/zookeeper-shell.sh localhost:2000

连接到 localhost:2000

Welcome to ZooKeeper! JLine support is disabled

ls /

[storm, brokers, zookeeper]

ls /brokers

[kafka-spout]

ls /brokers/kafka-spout

[partition_0]

get /brokers/kafka-spout/partition_0

{"topology":{"id":"a9be1962-6b4e-4ed4-ae68-155a1948a1f6","name":"consolidate_reports"},"offset":4426029,"partition":0,"broker":{"host":"localhost","port":9092},"topic":"bid_history"} cZxid = 0x50 ctime = Thu May 21 11:00:48 BRT 2015 mZxid = 0x50 mtime = Thu May 21 11:00:48 BRT 2015 pZxid = 0x50 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 182 numChildren = 0