java - Spring amqp 当rabbitmq server和consumer之间出现网络故障时，抛出AmqpIOException，consumer不会重试

Question

我们在项目中使用 Spring rabbitmq。我们目前正在测试不同的故障转移方案，以防止生产出现任何进一步的异常。

我们正在使用以下属性: 听众: 类型:直接 直接的: 确认模式:手动

我们面临的是，当出现任何网络故障时，我们会得到以下日志:

    INFO  [SimpleAsyncTaskExecutor-1][2018-10-22 15:37:55,726] - org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:463) - Created new connection: rabbitConnectionFactory#4c05df3b:0/SimpleConnection@2d361383 [delegate=amqp://workflow@192.168.156.53:5671/, localPort= 49445]
INFO  [SimpleAsyncTaskExecutor-1][2018-10-22 15:37:55,789] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:613) - SimpleConsumer [queue=vishalr_dev.1482820471007_workflow, consumerTag=amq.ctag-TNSIb9kI1SxdO8cXtfOgrg identity=7155ef26] started
WARN  [AMQP Connection 192.168.156.53:5671][2018-10-22 15:38:08,830] - com.rabbitmq.client.impl.ForgivingExceptionHandler.log(ForgivingExceptionHandler.java:115) - An unexpected connection driver error occured (Exception message: Connection reset)
ERROR [AMQP Connection 192.168.156.53:5671][2018-10-22 15:38:08,836] - org.springframework.amqp.rabbit.connection.CachingConnectionFactory$DefaultChannelCloseLogger.log(CachingConnectionFactory.java:1302) - Channel shutdown: connection error
ERROR [null-consumerMonitor-1][2018-10-22 15:38:12,285] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$null$3(DirectMessageListenerContainer.java:397) - Consumer canceled - channel closed SimpleConsumer [queue=vishalr_dev.1482820471007_workflow, consumerTag=amq.ctag-TNSIb9kI1SxdO8cXtfOgrg identity=7155ef26]
INFO  [null-consumerMonitor-1][2018-10-22 15:38:17,282] - org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:447) - Attempting to connect to: [montu.local.com:5671]
ERROR [null-consumerMonitor-1][2018-10-22 15:38:17,290] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$4(DirectMessageListenerContainer.java:416) - Cannot connect to server
org.springframework.amqp.AmqpIOException: java.net.SocketException: Network is unreachable: connect
    at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:476) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:614) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createBareChannel(CachingConnectionFactory.java:564) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getCachedChannelProxy(CachingConnectionFactory.java:538) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getChannel(CachingConnectionFactory.java:520) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.access$1500(CachingConnectionFactory.java:94) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$ChannelCachingConnectionProxy.createChannel(CachingConnectionFactory.java:1171) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:569) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$4(DirectMessageListenerContainer.java:413) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.5.RELEASE.jar:5.0.5.RELEASE]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_141]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_141]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_141]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_141]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_141]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_141]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_141]
Caused by: java.net.SocketException: Network is unreachable: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_141]
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_141]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_141]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_141]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_141]
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_141]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_141]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_141]
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673) ~[na:1.8.0_141]
    at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60) ~[amqp-client-5.1.2.jar:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:955) ~[amqp-client-5.1.2.jar:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:907) ~[amqp-client-5.1.2.jar:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:847) ~[amqp-client-5.1.2.jar:5.1.2]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:449) ~[spring-rabbit-2.0.4.RELEASE.jar:2.0.4.RELEASE]
    ... 16 common frames omitted

在上面的日志之后，它没有重试连接消费者。

但是我们调试到 spring jar 本身，发现对于 org.springframework.amqp.AmqpIOException，它没有为重启添加消费者，这可能是它背后的问题。

任何想法，我们如何解决我们的网络故障切换场景。

这里的网络故障转移意味着我们在 rabbitmq 服务器和消费者之间放置了防火墙作为第一个测试，作为第二个我们从消费者端禁用了网络适配器。

更新:1=============================================== ==============================

我们测试这种类型的网络故障的原因是，在我们遇到一些基础设施问题的前一天，由于 rabbitmq VM 停机了 30 分钟。

但是当 rabbitmq VM 再次运行时，所有消费者都没有自动连接。我们试图在我们的本地重新生成它，但不能。这是来自消费者无法自动连接的实例的日志。

INFO  [SimpleAsyncTaskExecutor-1][2018-10-22 10:15:51,142] - org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:447) - Attempting to connect to: [RABBITMQAIND1.hqdev.india:5671, RABBITMQBIND1.hqdev.india:5671]
ERROR [SimpleAsyncTaskExecutor-1][2018-10-22 10:15:51,142] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:459) - Error creating consumer; retrying in 5000
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused: connect
    at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:476) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:614) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:553) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.consumeFromQueue(DirectMessageListenerContainer.java:535) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:446) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_152]
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_152]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_152]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_152]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_152]
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_152]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_152]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_152]
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673) ~[na:1.8.0_152]
    at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:955) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:907) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:847) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:449) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    ... 5 common frames omitted
INFO  [SimpleAsyncTaskExecutor-1][2018-10-22 10:15:54,472] - org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:463) - Created new connection: rabbitConnectionFactory#40ae8c5:85/SimpleConnection@54d6cde2 [delegate=amqp://workflow@192.168.154.191:5671/, localPort= 50085]
ERROR [SimpleAsyncTaskExecutor-1][2018-10-22 10:16:07,314] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:459) - Error creating consumer; retrying in 5000
org.springframework.amqp.AmqpConnectException: org.springframework.amqp.AmqpIOException: java.io.IOException
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:557) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.consumeFromQueue(DirectMessageListenerContainer.java:535) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:446) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
Caused by: org.springframework.amqp.AmqpIOException: java.io.IOException
    at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.RabbitAccessor.convertRabbitAccessException(RabbitAccessor.java:113) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1834) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1771) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1752) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.initialize(RabbitAdmin.java:540) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$null$9(RabbitAdmin.java:453) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287) ~[spring-retry-1.2.2.RELEASE.jar!/:na]
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164) ~[spring-retry-1.2.2.RELEASE.jar!/:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$afterPropertiesSet$10(RabbitAdmin.java:452) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CompositeConnectionListener.onCreate(CompositeConnectionListener.java:36) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:620) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:553) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    ... 3 common frames omitted
Caused by: java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:144) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:947) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:52) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:991) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at com.sun.proxy.$Proxy180.queueDeclare(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.declareQueues(RabbitAdmin.java:608) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.lambda$initialize$11(RabbitAdmin.java:542) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1828) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    ... 13 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit@RABBITMQAIND1' of durable queue 'collabscrum1.1454137672227_workflow' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:288) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:138) ~[amqp-client-5.1.2.jar!/:5.1.2]
    ... 24 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit@RABBITMQAIND1' of durable queue 'collabscrum1.1454137672227_workflow' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:504) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:346) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:178) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:111) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:643) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:47) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:581) ~[amqp-client-5.1.2.jar!/:5.1.2]
    ... 1 common frames omitted
ERROR [SimpleAsyncTaskExecutor-1][2018-10-22 10:16:07,345] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:593) - Queue not present, scheduling consumer SimpleConsumer [queue=neha_dev_workflow, consumerTag=null identity=5e8ed89c] for restart
java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:144) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:991) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:52) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:991) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at com.sun.proxy.$Proxy180.queueDeclarePassive(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:572) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.consumeFromQueue(DirectMessageListenerContainer.java:535) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:446) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit@RABBITMQAIND1' of durable queue 'neha_dev_workflow' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:288) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:138) ~[amqp-client-5.1.2.jar!/:5.1.2]
    ... 12 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit@RABBITMQAIND1' of durable queue 'neha_dev_workflow' in vhost '/' is down or inaccessible, class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:504) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:346) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:178) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:111) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:643) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:47) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:581) ~[amqp-client-5.1.2.jar!/:5.1.2]
    ... 1 common frames omitted
ERROR [SimpleAsyncTaskExecutor-1][2018-10-22 10:16:07,345] - org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:593) - Queue not present, scheduling consumer SimpleConsumer [queue=autoind9.1526118175953_workflow, consumerTag=null identity=6465a7f8] for restart
java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:144) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:991) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:52) ~[amqp-client-5.1.2.jar!/:5.1.2]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_152]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_152]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_152]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_152]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:991) ~[spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at com.sun.proxy.$Proxy180.queueDeclarePassive(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:572) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.consumeFromQueue(DirectMessageListenerContainer.java:535) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$5(DirectMessageListenerContainer.java:446) [spring-rabbit-2.0.4.RELEASE.jar!/:2.0.4.RELEASE]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_152]

此日志是否对跟踪问题更有用？

Answer 1

您使用的是什么版本？当网络在启动期间不可用时，它对我来说按预期工作 (2.0.8.RELEASE)...

2018-10-22 10:57:50.924  INFO 91651 --- [cTaskExecutor-1] o.s.a.r.c.CachingConnectionFactory       : Attempting to connect to: [10.0.0.88:5672]
2018-10-22 10:57:50.925 ERROR 91651 --- [cTaskExecutor-1] o.s.a.r.l.DirectMessageListenerContainer : Error creating consumer; retrying in 5000

org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Network is unreachable (connect failed)

并且在运行时断开网络...

2018-10-22 11:08:02.729  INFO 93234 --- [nsumerMonitor-1] o.s.a.r.c.CachingConnectionFactory       : Attempting to connect to: [10.0.0.3:5672]
2018-10-22 11:08:02.730 ERROR 93234 --- [nsumerMonitor-1] o.s.a.r.l.DirectMessageListenerContainer : Cannot connect to server

org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Network is unreachable (connect failed)

你在什么硬件上运行？ - 如您所见，我收到了 AmqpConnectException，但您收到了 AmqpIOException。

编辑

Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method(reply-code=404, reply-text=NOT_FOUND - home node 'rabbit@RABBITMQAIND1' of durable queue 'neha_dev_workflow' in vhost '/' is down or inaccessible, class-id=50, method-id=10)

这意味着您正在使用集群，但队列未镜像(HA)；失败后，您连接到另一个节点到拥有队列的节点(并且该实例未运行)。如果该节点正在运行，则可能意味着集群已损坏(您可以在 Admin UI 的概览页面上看到这一点)- 每个节点都认为另一个节点没有运行。

参见 the rabbitmq documentation about recovering from a network partition .

我昨天在测试你的情况时遇到了类似的问题;我通过重新启动其中一个节点来恢复它。