apache-kafka - 为什么 kafka-consumer-groups 不为列出的消费者产生任何输出？

Question

当我运行命令时:

kafka-consumer-groups -bootstrap-server localhost:9092 -list

它生成一个消费者组列表。这是一些冗长的输出:

$ kafka-consumer-groups --bootstrap-server localhost:9092 --list
Note: This will not show information about old Zookeeper-based consumers.
simulate_birdfeeder.e67c034c-a96d-11e9-861a-983b8f0e47c8
xfer_server.e65d3732-a96d-11e9-992f-983b8f0e47c8
xfer_server.e654e596-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6ba65ce-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e695c336-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e681a4c8-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6b936f4-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e6956c2e-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6de0222-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68ec02c-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6b48f82-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6b436cc-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68fed3a-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e658faf0-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e64f28ea-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e691a710-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6889f30-a96d-11e9-861a-983b8f0e47c8
74b6a2e3-efe6-4a62-ad6b-a4782038db43
simulate_birdfeeder.e67ec7bc-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e67b9178-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65b4b2a-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6dcad1e-a96d-11e9-b125-983b8f0e47c8
water_connector
simulate_birdfeeder.e6924904-a96d-11e9-861a-983b8f0e47c8
reporter
xfer_server.e652ee8a-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e68d3446-a96d-11e9-b125-983b8f0e47c8
simulate_flexcharger.e560bef8-a96d-11e9-a249-983b8f0e47c8
511bcf37-7e6b-4a33-9a57-9ad498f9a089
simulator.e5d05a10-a96d-11e9-9375-983b8f0e47c8
trajectory.queue
simulate_birdfeeder.e662497a-a96d-11e9-861a-983b8f0e47c8
scheduler
simulate_birdfeeder.e6874ca2-a96d-11e9-861a-983b8f0e47c8
85f138e2-bb87-4885-98b8-a6536595cf7c
scheduler_node.e69bcd9e-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e69b0c42-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e6612c7a-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e691649e-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e665d02c-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6554ab8-a96d-11e9-861a-983b8f0e47c8
xfer_server.e65b81ee-a96d-11e9-992f-983b8f0e47c8
server.e6159012-a96d-11e9-bbdf-983b8f0e47c8
simulate_birdfeeder.e666c9b4-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e67dffb2-a96d-11e9-861a-983b8f0e47c8
KMOffsetCache-my-computer
simulate_birdfeeder.e697c230-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e683e0bc-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65fbd40-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e69706c4-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e69cc438-a96d-11e9-b125-983b8f0e47c8
xfer_server.e64a93de-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6a55e72-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e685be50-a96d-11e9-861a-983b8f0e47c8
product_monitor
simulate_birdfeeder.e688f89a-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e66b29fa-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e6811242-a96d-11e9-861a-983b8f0e47c8
scheduler_node.e6a62be0-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6b9fa58-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68a2986-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e690480c-a96d-11e9-861a-983b8f0e47c8
server.e60fec02-a96d-11e9-bbdf-983b8f0e47c8
scheduler_node.e6a45856-a96d-11e9-b125-983b8f0e47c8
simulate_birdfeeder.e68385ae-a96d-11e9-861a-983b8f0e47c8
server
simulate_birdfeeder.e668c246-a96d-11e9-861a-983b8f0e47c8
simulate_birdfeeder.e65f11a6-a96d-11e9-861a-983b8f0e47c8
xfer_server.e64f2bb0-a96d-11e9-992f-983b8f0e47c8
scheduler_node.e6b8e834-a96d-11e9-b125-983b8f0e47c8
scheduler_node.e6ddb13c-a96d-11e9-b125-983b8f0e47c8

其中我可以描述它们:

$ kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group server
Note: This will not show information about old Zookeeper-based consumers.

TOPIC                PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                  HOST            CLIENT-ID
reservationist.queue 0          456             456             0               rdkafka-fa9be154-c7ac-471f-a9d1-5bfd3f42da8c /127.0.0.1      rdkafka

但是，对于某些消费群体，如果我描述它们，则根本没有输出。我会得到如下内容:

$ kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group scheduler_node.e6ddb13c-a96d-11e9-b125-983b8f0e47c8
Note: This will not show information about old Zookeeper-based consumers.

除了警告之外，没有任何输出，但消费者确实存在，因为它在前面的 list 命令中列出。现在，如果我使用 zookeeper 选项，该命令会提示该组不存在。

通常看起来像 widget 的消费者有偏移量的描述，而看起来像 widget.some-long-random-string 的消费者没有输出。看起来像 some-long-random-string 的消费者也会产生带有偏移量的描述。

(我不是 100% 确定，但我很确定丢失的消费者组已订阅，否则我的应用程序会爆炸，事实并非如此。)

我在本地使用融合的 kafka。我的代码和汇合都在我的开发箱中运行:

$ confluent version kafka
This CLI is intended for development only, not for production
https://docs.confluent.io/current/cli/index.html

1.1.1-cp1

任何人都可以给我一些关于正在发生的事情的见解吗？谢谢。

Answer 1

kafka-consumer-groups \
  --bootstrap-server localhost:9092 \
  --describe \
  --group your_consumer_group_name

如果消费者组中的消费者没有提交任何偏移量，则不会返回任何输出。

如果输出为

bin/kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --describe \
  --group server

和

bin/kafka-consumer-groups.sh \
  --bootstrap-server localhost:9092 \
  --new-consumer
  --describe \
  --group server 

匹配并且消费组出现在两个消费组列表输出中，我的猜测是您的消费者在消费组 server中没有提交任何偏移量，这就是为什么使用 describe 命令没有信息出现的原因。

确保您的消费者在 server 内组成功提交他们的偏移量(手动或自动)。