spring-boot - Spring AMQP 多个消费者与更高的预取值

Question

即使阅读了大量 SO 问题(1、2)和文章，仍不清楚为消费者设置哪个选项更好。多个消费者或更高的预取值？

据我了解，当谈到 SimpleRabbitListenerContainerFactory 时，因为它最初设计为每个连接只有一个线程，所以它旨在解决 amqp -client 每个连接只有一个线程，这是否意味着设置多个消费者不会有太大区别，因为实际上只有一个线程从 rabbit 消费，而不是将它交给多个消费者(线程)？或者实际上有几个消费者同时消费？

那么，在涉及预取/消费者的 rabbit spring 实现方面，最佳实践是什么？什么时候应该使用一个而不是另一个？我应该切换到这个新的 DirectRabbitListenerContainerFactory 吗？它是“更好”还是仅取决于用例？

当涉及到高预取时，我看到的一些缺点是，如果应用程序消耗了它可以保存在缓冲区中的更多消息，它可能会导致内存问题？ (还没有实际测试过，TBH)

当涉及到多个消费者时，我看到了在操作系统级别打开更多文件描述符的缺点，我看到了 this关于每个消费者实际上为每个 ack ping rabbit 的文章，这使得它变慢了。

仅供引用，如果相关，我通常会这样设置我的配置:

@Bean
public ConnectionFactory connectionFactory() {
final CachingConnectionFactory connectionFactory = new CachingConnectionFactory(server);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
connectionFactory.setVirtualHost(virtualHost);
connectionFactory.setRequestedHeartBeat(requestedHeartBeat);
return connectionFactory;
}

@Bean
public AmqpAdmin amqpAdmin() {
    AmqpAdmin admin = new RabbitAdmin(connectionFactory());
    admin.declareQueue(getRabbitQueue());
    return admin;
}

@Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
final SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(concurrency);
factory.setMaxConcurrentConsumers(maxConcurrency);
factory.setPrefetchCount(prefetch);
factory.setMissingQueuesFatal(false);
return factory;
}

@Bean
public Queue getRabbitQueue() {
final Map<String, Object> p = new HashMap<String, Object>();
p.put("x-max-priority", 10);
return new Queue(queueName, true, false, false, p);
}

Answer 1

没有； SMLC 不是“为每个连接一个线程设计的”，它旨在解决 amqp-client 每个连接只有一个线程的限制，以便线程通过内存队列传递给消费者线程；这已不再是这种情况。客户端是多线程的，每个消费者有一个专用线程。

拥有多个消费者(增加并发性)是完全有效的(而且曾经是，即使是旧客户端)。

预取实际上是为了减少网络干扰并提高整体吞吐量。您是否真的需要增加并发性与预取是正交的。如果 (a) 您的监听器处理每条消息的速度相对较慢，并且 (b) 严格的消息排序并不重要，您通常会增加并发性。

DirectListenerContainer 的引入是为了提供不同的线程模型，其中监听器直接在 amqp-client 线程上调用。

在 Choosing a Container 中描述了选择一个容器而不是另一个容器的原因。 .

The following features are available with the SMLC, but not the DMLC:

• txSize - with the SMLC, you can set this to control how many messages are delivered in a transaction and/or to reduce the number of acks, but it may cause the number of duplicate deliveries to increase after a failure. (The DMLC does have mesagesPerAck which can be used to reduce the acks, the same as with txSize and the SMLC, but it can’t be used with transactions - each message is delivered and ack’d in a separate transaction).

• maxConcurrentConsumers and consumer scaling intervals/triggers - there is no auto-scaling in the DMLC; it does, however, allow you to programmatically change the consumersPerQueue property and the consumers will be adjusted accordingly.

However, the DMLC has the following benefits over the SMLC:

• Adding and removing queues at runtime is more efficient; with the SMLC, the entire consumer thread is restarted (all consumers canceled and re-created); with the DMLC, unaffected consumers are not canceled.

• The context switch between the RabbitMQ Client thread and the consumer thread is avoided.

• Threads are shared across consumers rather than having a dedicated thread for each consumer in the SMLC. However, see the IMPORTANT note about the connection factory configuration in the section called “Threading and Asynchronous Consumers”.