python - celery 任务不适用于 gevent

Question

当我将 celery + gevent 用于使用 subprocess 模块的任务时，我得到以下堆栈跟踪:

Traceback (most recent call last):
  File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/webapp/admin/webadmin/apps/loggingquarantine/tasks.py", line 107, in release_mail_task
    res = call_external_script(popen_obj.communicate)
  File "/home/webapp/admin/webadmin/apps/core/helpers.py", line 42, in call_external_script
    return func_to_call(*args, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 740, in communicate
    return self._communicate(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1257, in _communicate
    stdout, stderr = self._communicate_with_poll(input)
  File "/usr/lib64/python2.7/subprocess.py", line 1287, in _communicate_with_poll
    poller = select.poll()
AttributeError: 'module' object has no attribute 'poll'

我的 manage.py 看起来如下(在那里做 monkeypatch):

#!/usr/bin/env python
from gevent import monkey
import sys
import os

if __name__ == "__main__":
    if not 'celery' in sys.argv:
        monkey.patch_all()
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webadmin.settings")
    from django.core.management import execute_from_command_line
    sys.path.append(".")
    execute_from_command_line(sys.argv)

celery 任务表现得好像没有正确修补是有原因的吗？

附注奇怪的是，我在 Macos 上的本地设置工作正常，而我在 Centos 下遇到此类异常(所有包版本都相同，init 和配置脚本也是如此)

Answer 1

在 gevent 中没有模拟轮询，所以 monkey.patch_all 删除了 gevent.select 不模拟的轮询机制:poll、epoll、kqueue、kevent。参见 gevent.monkey – Make the standard library cooperative .