apache-kafka - 如何让 kafka 消费者从上次消费的偏移量而不是从头开始读取

Question

我是 kafka 的新手，并试图了解是否有办法从上次消耗的偏移量读取消息，而不是从头读取消息。

我正在写一个示例案例，这样我的意图就不会偏离。

Eg:
1) I produced 5 messages at 7:00 PM and console consumer consumed those.
2) I stopped consumer at 7:10 PM
3) I produced 10 message at 7:20 PM. No consumer had read those messages.
4) Now, i have started console consumer at 7:30 PM, without from-beginning.
5) Now, it Will read the messages produced after it has started. Not the earlier ones, which were produced at 7.20 PM

有没有办法从上次消费的偏移量中获取消息。？

Answer 1

I am new to kafka and trying to understand if there is a way to read messages from last consumed offset, but not from beginning.

是的，可以使用控制台消费者从上次消费的偏移量中读取。您必须添加 消费者.config 在调用 kafka-console-consumer 时标记。

例子:-

[root@sandbox bin]# ./kafka-console-consumer.sh --topic test1 --zookeeper localhost:2181 --consumer.config /home/mrnakumar/consumer.properties

这里 /home/mrnakumar/consumer.properties 是一个包含 的文件group.id .
这是/home/mrnakumar/consumer.properties 的外观:-

group.id=consoleGroup

不使用consumer.config，可以从 开始阅读[通过使用--from-beginning] 或仅记录结束。日志结束是指消费者启动后发布的所有消息。