apache-kafka - 确定哪个主题在 Kafka 消费者中有记录

Question

我一直在关注 Kafka 消费者类(class)。我可以将主题作为列表对象传递。我指的是以下文章https://docs.confluent.io/current/clients/java.html ，但是我需要知道一旦消费者类订阅了主题，我怎么知道哪个主题中有记录。有什么办法可以查到吗？这是代码:

public abstract class ConsumeLoop implements Runnable {
  private final KafkaConsumer<K, V> consumer;
  private final List<String> topics;
  private final CountDownLatch shutdownLatch;

  public BasicConsumeLoop(KafkaConsumer<K, V> consumer, List<String> topics) {
    this.consumer = consumer;
    this.topics = topics;
    this.shutdownLatch = new CountDownLatch(1);
  }

  public abstract void process(ConsumerRecord<K, V> record);

  public void run() {
  try {
      consumer.subscribe(topics);  --> Consuming list of topics

      while (true) {
        ConsumerRecords<K, V> records = consumer.poll(Long.MAX_VALUE);  --> Which topic is returning the records?
        records.forEach(record -> process(record));    
     }  
   } catch (WakeupException e) {
     // ignore, we're closing
    } catch (Exception e) {
      log.error("Unexpected error", e);
    } finally {
       consumer.close();
      shutdownLatch.countDown();
    }
  } 
}

Answer 1

partitions()方法来自 ConsumerRecords将返回一组 TopicPartition s:

partitions - Get the partitions which have records contained in this record set.

然后您可以迭代该集合以获得 topic()姓名和 partition()数字，取决于您的需要。例如:

   for (TopicPartition tp : records.partitions()) {
     System.out.println("Got " + records.records(tp).size() + " records "
       + "from topic:partition " + tp.topic() + ":" + tp.partition());
   }