python - Django Celery 在更改数据库后抛出错误

Question

由于运行 celery 任务时存在高并发写入问题，我最近被迫将数据库从 sqlite3 切换到 mysql(这些任务在 sqlite3 上运行良好)。现在我已经配置了 mysql 数据库，它会抛出一个

C:\Sites\deltafy-massive-feed-backend>celery -A backend worker -l info

-------------- celery@Robert-PC v3.1.11 (Cipater)
---- **** -----
--- * ***  * -- Windows-7-6.1.7601-SP1
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         backend:0x2d69b10
- ** ---------- .> transport:   django://localhost//
- ** ---------- .> results:     djcelery.backends.cache:CacheBackend
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ----
--- ***** ----- [queues]
--------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . backend.celery.debug_task
  . private_api.tasks.gilt_script
  . private_api.tasks.nordstrom_script
  . private_api.tasks.zappos_script

[2014-06-14 01:37:13,625: ERROR/MainProcess] Unrecoverable error: WindowsError(123,             'The filename, directory name, or volume label syntax is incorrect')
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\celery\worker\__init__.py", line 206, in start
    self.blueprint.start(self)
  File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 123, in start
    step.start(parent)
  File "C:\Python27\lib\site-packages\celery\bootsteps.py", line 373, in start
    return self.obj.start()
  File "C:\Python27\lib\site-packages\celery\concurrency\base.py", line 131, in start
    self.on_start()
  File "C:\Python27\lib\site-packages\celery\concurrency\prefork.py", line 117, in on_start
    **self.options)
  File "C:\Python27\lib\site-packages\billiard\pool.py", line 925, in __init__
    self._setup_queues()
  File "C:\Python27\lib\site-packages\billiard\pool.py", line 1267, in _setup_queues
    self._inqueue = SimpleQueue()
  File "C:\Python27\lib\site-packages\billiard\queues.py", line 369, in __init__
    self._reader, self._writer = Pipe(duplex=False)
  File "C:\Python27\lib\site-packages\billiard\__init__.py", line 97, in Pipe
    return Pipe(duplex, rnonblock, wnonblock)
  File "C:\Python27\lib\site-packages\billiard\py2\connection.py", line 233, in Pipe
    1, obsize, ibsize, win32.NMPWAIT_WAIT_FOREVER, win32.NULL
WindowsError: [Error 123] The filename, directory name, or volume label syntax is     incorrect`

此外，我有预感该地址是问题所在并且包含无效字符，因此我进入了connection.py并打印了它。地址是:

c:\\.\pipe\pyc-5532-0-7ix5od

任何关于这是做什么的想法，或者任何问题的解决方案，我们都非常感激!即使经过几个小时的搜索，我也没有任何线索。

Answer 1

此问题是 billiard(Celery 使用的多处理模块 fork)中的一个错误。 Here是问题的链接。

您可以通过运行 python -c "from billiard.connection import Pipe; p = Pipe(duplex=False)" 来确认这一点，这应该会产生相同的错误。

Cogniva刚刚把这个固定件推到了他们的台球叉上。在撰写本文时，此修复已提交，但尚未合并到 celery/billiard 中。 。该问题将在台球3.4.0.0版本中得到解决。

安装修复程序:

$ pip uninstall billiard
$ git clone git@github.com:cogniva/billiard.git
$ cd billiard 
$ git checkout bb6169ff1bdd8bc7c5520f7c47085063bc51d4c7
$ python setup.py install

要确认此修复是否有效，请运行python -c "from billiard.connection import Pipe; p = Pipe(duplex=False)"。这次您应该不会看到错误消息。