django - celery 错误 'connection reset by peer'

Question

Celery 的连接会定期重置，并显示错误消息:

[Errno 104] Connection reset by peer

目前，我会丢失任务以及之后触发的任何内容，直到重置服务器为止。我有两个问题:(1) 是什么导致了错误/如何防止它？ (2) 如何捕获发生的错误并重试我的任务？

我正在 Heroku 上运行。 Celery 配置为使用 AMQP (CloudAMQP) 作为代理，使用 redis (REDIS_TO_GO) 作为结果后端。

堆栈跟踪:

File "celery/task/trace.py", line 224, in trace_task
  R = retval = fun(*args, **kwargs)
File "myapp/tasks.py", line 310, in mytask1
  mytask2.delay(message)
File "celery/app/task.py", line 343, in delay
  return self.apply_async(args, kwargs)
File "celery/app/task.py", line 458, in apply_async
  with app.producer_or_acquire(producer) as P:
File "python2.7/contextlib.py", line 17, in __enter__
  return self.gen.next()
File "celery/app/base.py", line 256, in producer_or_acquire
  with self.amqp.producer_pool.acquire(block=True) as producer:
File "kombu/connection.py", line 712, in acquire
  R = self.prepare(R)
File "kombu/pools.py", line 57, in prepare
  p.revive(connection.default_channel)
File "kombu/connection.py", line 626, in default_channel
  self._default_channel = self.channel()
File "kombu/connection.py", line 167, in channel
  chan = self.transport.create_channel(self.connection)
File "kombu/transport/amqplib.py", line 324, in create_channel
  return connection.channel()
File "kombu/transport/amqplib.py", line 243, in channel
  return Channel(self, channel_id)
File "kombu/transport/amqplib.py", line 267, in __init__
  super(Channel, self).__init__(*args, **kwargs)
File "amqplib/client_0_8/channel.py", line 82, in __init__
  self._x_open()
File "amqplib/client_0_8/channel.py", line 469, in _x_open
  self._send_method((20, 10), args)
File "amqplib/client_0_8/abstract_channel.py", line 76, in _send_method
  method_sig, args, content)
File "amqplib/client_0_8/method_framing.py", line 252, in write_method
  self.dest.write_frame(1, channel, payload)
File "amqplib/client_0_8/transport.py", line 165, in write_frame
  frame_type, channel, size, payload, 0xce))
File "python2.7/socket.py", line 224, in meth
  return getattr(self._sock,name)(*args)

版本:

Django==1.4.1
django-celery==3.0.9
celery==3.0.9
celery-with-redis==3.0
amqplib==1.0.2
kombu==2.4.7

Answer 1

您的连接似乎已用完。当您查看CloudAMQP Console打开连接时，您可以验证它。

也许看看 celery BROKER_POOL_LIMIT