python - 同一个 docker-compose 中的 Celery 和 Flask

Question

我正在尝试使用 docker-compose 来生成我的 Flask/Celery/Redis 服务。

这是我的 docker-compose.yml:

flask:
    build: .
    command: "python3 app.py"
    ports:
        - '5000:5000'
    links:
        - redis
    volumes:
        - .:/usr/src/app:ro

celery:
    build: .
    command: "celery -A app.celery worker --loglevel=info"
    volumes:
        - .:/usr/src/app:ro

redis:
    image: redis
    ports:
        - '6379:6379'

当我运行这个 docker-compose 时，Flask 和 Redis 都可以正常启动并按预期运行。关于 Celery，Docker 报告:flaskcelery_celery_1 exited with code 1，没有其他信息。

如果我在没有 Docker 的情况下运行我的三个服务，并使用 celery -A app.celery worker --loglevel=info 启动 Celery，我的应用程序运行正常。

如有必要，请提供更多信息:

Dockerfile:(此图像在构建时也安装了 requirements.txt)

FROM python:3.5-onbuild
EXPOSE 5000

requirements.txt:

flask==0.11.1
celery==3.1.23

docker-compose up 输出:

Starting flaskcelery_celery_1
Starting flaskcelery_redis_1
Starting flaskcelery_flask_1
Attaching to flaskcelery_celery_1, flaskcelery_redis_1, flaskcelery_flask_1
redis_1  |                 _._                                                  
redis_1  |            _.-``__ ''-._                                             
redis_1  |       _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
redis_1  |   .-`` .-```.  ```\/    _.,_ ''-._                                   
redis_1  |  (    '      ,       .-`  | `,    )     Running in standalone mode
redis_1  |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
redis_1  |  |    `-._   `._    /     _.-'    |     PID: 1
redis_1  |   `-._    `-._  `-./  _.-'    _.-'                                   
redis_1  |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1  |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
redis_1  |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1  |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1  |  |    `-._`-._        _.-'_.-'    |                                  
redis_1  |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1  |       `-._    `-.__.-'    _.-'                                       
redis_1  |           `-._        _.-'                                           
redis_1  |               `-.__.-'                                               
redis_1  | 
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1  | 1:M 23 Aug 10:23:08.409 # Server started, Redis version 3.2.3
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1  | 1:M 23 Aug 10:23:08.409 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1  | 1:M 23 Aug 10:23:08.409 * The server is now ready to accept connections on port 6379
flaskcelery_celery_1 exited with code 1
flask_1  |  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
flask_1  |  * Restarting with stat
flask_1  |  * Debugger is active!
flask_1  |  * Debugger pin code: 196-119-737

Answer 1

解决了我的问题。我最终发现我可以在 Docker 镜像上获得命令提示符:

docker build -t <image name> .
docker run -it <image name> /bin/bash

然后尝试在容器内运行 celery 发现问题:

root@4a6edc5d7372:/usr/src/app# celery -A app.celery worker --loglevel=info
Running a worker with superuser privileges when the
worker accepts messages serialized with pickle is a very bad idea!

If you really want to continue then you have to set the C_FORCE_ROOT
environment variable (but please think about this before you do).

User information: uid=0 euid=0 gid=0 egid=0

Docker 通常以 root 身份运行，而 Celery 出于安全原因不喜欢以 root 身份运行(我相信您可以通过 pickle 反序列化获得代码执行)。更安全的解决方案是将 celery 容器设置为以 nobody 运行。工作 docker-compose.yml:

flask:
    build: .
    command: "python3 app.py"
    ports:
        - '5000:5000'
    links:
        - redis
        - celery
    volumes:
        - .:/usr/src/app:ro

celery:
    build: .
    command: "celery -A app.celery worker --loglevel=info"
    user: nobody
    links:
        - redis
    volumes:
        - .:/usr/src/app:ro

redis:
    image: redis
    ports:
        - '6379:6379'