spring - 是什么导致 - "DLQ support is not available for anonymous subscriptions"？

Question

我正在使用依赖项处理 SpringBootApplication - spring-cloud-stream-binder-kafka-streams 并尝试测试在发生 serdeError 时向 Dlq 发送错误消息。

@Slf4j
@Component
@EnableBinding(KafkaBinding.class)
public class AListener {

      @StreamListener
      public void sink(@Input(KafkaBinding.ABINDING) KStream<String, AnOrder> events) {
        log.info("HERE_BEFORE");
        events.foreach((k, v) -> {
          log.info("HERE_AFTER value: {}", v.toString());
          throw new RuntimeException("Failed, should land in dlq topic");
        });
      }
    }

public interface KafkaBinding {
  String ABINDING = "some.events";

  @Input(ABINDING)
  public KStream<String, AnOrder> incomingOrder();
}

应用程序.yml

spring:
  application:
    name: aprocessor
  cloud:
    stream:
      kafka:
        streams:
          binder:
            brokers: localhost:9092
            serdeError: sendToDlq
            configuration:
              commit.interval.ms: 1000
              default:
                key.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
                value.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
          bindings:
            input:
              consumer:
                enableDlq: true
                dlqName: a-dlq
                autoCommitOnError: true
                autoCommitOffset: true
      bindings:
        input:
          group: a-group
          destination: some.events
        pos:
          destination: some.events
          consumer.header-mode: raw

测试:

@Slf4j
@DirtiesContext
@SpringBootTest
@EmbeddedKafka(
    partitions = 1,
    topics = {"some.events"},
    controlledShutdown = true,
    brokerProperties = {
      "listeners=PLAINTEXT://localhost:9092",
      "port=9092",
      "auto.create.topics.enable=${topics.autoCreate:false}",
      "delete.topic.enable=${topic.delete:true}"
    })
public class AListenerTest {
  private KafkaTemplate<String, String> kafkaTemplate;

  @Autowired private EmbeddedKafkaBroker embeddedKafka;
  @SpyBean private AListener listener;

  private static final String INPUT_TOPIC = "some.events";

  @BeforeEach
  public void setUp() {

    Map<String, Object> senderProperties =
        KafkaTestUtils.senderProps(embeddedKafka.getBrokersAsString());

    ProducerFactory<String, String> producerFactory =
        new DefaultKafkaProducerFactory<>(senderProperties);

    kafkaTemplate = new KafkaTemplate<>(producerFactory);

    kafkaTemplate.setDefaultTopic(INPUT_TOPIC);
  }

  @Test
  public void whenExceptionInConsumer_thenLogToDLQ(){
    String logme = "{\"body\":\"thor\"}";
    kafkaTemplate.sendDefault(logme);
    log.info("<<<<DATA>>>> {}", logme);
  }
}

测试失败，堆栈跟踪如下:

Caused by: org.springframework.context.ApplicationContextException: Failed to start bean 'inputBindingLifecycle'; nested exception is java.lang.IllegalArgumentException: DLQ support is not available for anonymous subscriptions
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185)
    at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
    at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
    at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
    at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:893)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
    at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:127)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
    ... 54 more
Caused by: java.lang.IllegalArgumentException: DLQ support is not available for anonymous subscriptions
    at org.springframework.util.Assert.isTrue(Assert.java:118)
    at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.doProvisionConsumerDestination(KafkaTopicProvisioner.java:186)
    at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.provisionConsumerDestination(KafkaTopicProvisioner.java:161)
    at org.springframework.cloud.stream.binder.kafka.streams.KafkaStreamsBinderUtils.prepareConsumerBinding(KafkaStreamsBinderUtils.java:53)
    at org.springframework.cloud.stream.binder.kafka.streams.KStreamBinder.doBindConsumer(KStreamBinder.java:93)
    at org.springframework.cloud.stream.binder.kafka.streams.KStreamBinder.doBindConsumer(KStreamBinder.java:51)
    at org.springframework.cloud.stream.binder.AbstractBinder.bindConsumer(AbstractBinder.java:142)
    at org.springframework.cloud.stream.binding.BindingService.doBindConsumer(BindingService.java:144)
    at org.springframework.cloud.stream.binding.BindingService.bindConsumer(BindingService.java:112)
    at org.springframework.cloud.stream.binding.BindableProxyFactory.createAndBindInputs(BindableProxyFactory.java:254)
    at org.springframework.cloud.stream.binding.InputBindingLifecycle.doStartWithBindable(InputBindingLifecycle.java:58)
    at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
    at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:48)
    at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:34)
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182)
    ... 66 more

我希望测试成功，控制台日志显示创建了一个 dlq/topic 并查询 dlq 以打印消息。是什么导致 KafkaTopicProvisioner 抛出“IllegalArgumentException:DLQ 支持不适用于匿名订阅”？

我已经尝试过此处帖子中提到的步骤 - “Correctly manage DLQ in Spring Cloud Stream Kafka”。

Answer 1

不允许匿名消费者使用DLQ；你需要一个持久的订阅。

匿名消费者是那些没有指定消费者组的消费者。

来自您引用的答案。

  bindings:
    input:
      group: so51247113

此外，这是开源的，您可以查看 KafkaTopicProvisioner 的源代码......

    boolean anonymous = !StringUtils.hasText(group);
    Assert.isTrue(!anonymous || !properties.getExtension().isEnableDlq(),
            "DLQ support is not available for anonymous subscriptions");