java - 为什么 KafkaUtils.createStream() 的 "topics"参数是一个 Map 而不是数组？

Question

文档中的定义:

org.apache.spark.streaming.kafka

Class KafkaUtils

static JavaPairReceiverInputDStream<String,String> createStream(JavaStreamingContext jssc, String zkQuorum, String groupId, java.util.Map<String,Integer> topics)

Create an input stream that pulls messages from Kafka Brokers.

为什么主题是 Map(而不是字符串数组)？

我知道字符串键是主题名称。但是整数值呢？我应该填写什么？

Answer 1

阅读Javadoc :

public static JavaPairReceiverInputDStream createStream(JavaStreamingContext jssc, String zkQuorum, String groupId, java.util.Map topics)

Create an input stream that pulls messages from Kafka Brokers. Storage level of the data will be the default StorageLevel.MEMORY_AND_DISK_SER_2.

Parameters: jssc - JavaStreamingContext object

zkQuorum - Zookeeper quorum (hostname:port,hostname:port,..)

groupId - The group id for this consumer

topics - Map of (topic_name -> numPartitions) to consume. Each partition is consumed in its own thread

Returns: DStream of (Kafka message key, Kafka message value)

Map 的值是给定主题名称的分区数，它决定了将用于消费该主题的线程数。