docker - Gunicorn/Uvicorn worker 在 AWS Fargate 上超时

Question

我尝试在 AWS Fargate 上部署我的 FastAPI 应用程序，前面有一个应用程序负载均衡器。该应用程序调用 AWS Aurora。我的堆栈如下

• python 3.7.4
• uvicorn 0.11.3
• unicorn 19.9.0
• fastapi 0.53.2
• 异步 0.20.1

我的容器在本地或 EC2 实例上部署时运行良好，但在 Fargate 上部署时工作人员在启动期间超时。日志没有给我任何错误的线索(见下文)。

我试图解决这个问题的事情:

• 增加 Fargate 任务的内存
• 放宽安全组设置
• 增加 gunicorn 超时值

这些都没有任何效果。

但是，当尝试部署准系统 FastAPI 应用程序(只有一个根路由，没有依赖项)时，该应用程序设法启动并且我可以通过负载均衡器访问它。一旦我添加了我的自定义路由，就会出现问题。

我还有哪些其他选项可以对此进行调试？如果 guincorn 无法启动我的工作人员，是否有办法记录问题？

03:05:16    Checking for script in /app/prestart.sh
03:05:16    Running script /app/prestart.sh
03:05:16    Running inside /app/prestart.sh, you could add migrations to this file, e.g.:
03:05:16    #! /usr/bin/env bash
03:05:16    # Let the DB start
03:05:16    sleep 10;
03:05:16    # Run migrations
03:05:16    alembic upgrade head
03:05:17    [2020-04-03 03:05:17 +0000] [1] [DEBUG] Current configuration:
03:05:17    config: /app/app/gunicorn_conf.py
03:05:17    bind: ['0.0.0.0:80']
03:05:17    backlog: 2048
03:05:17    workers: 2
03:05:17    worker_class: uvicorn.workers.UvicornWorker
03:05:17    threads: 2
03:05:17    worker_connections: 1000
03:05:17    max_requests: 0
03:05:17    max_requests_jitter: 0
03:05:17    timeout: 120
03:05:17    graceful_timeout: 30
03:05:17    keepalive: 120
03:05:17    limit_request_line: 4094
03:05:17    limit_request_fields: 100
03:05:17    limit_request_field_size: 8190
03:05:17    reload: False
03:05:17    reload_engine: auto
03:05:17    reload_extra_files: []
03:05:17    spew: False
03:05:17    check_config: False
03:05:17    preload_app: False
03:05:17    sendfile: None
03:05:17    reuse_port: False
03:05:17    chdir: /app
03:05:17    daemon: False
03:05:17    raw_env: []
03:05:17    pidfile: None
03:05:17    worker_tmp_dir: None
03:05:17    user: 0
03:05:17    group: 0
03:05:17    umask: 0
03:05:17    initgroups: False
03:05:17    tmp_upload_dir: None
03:05:17    secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
03:05:17    forwarded_allow_ips: ['127.0.0.1']
03:05:17    accesslog: None
03:05:17    disable_redirect_access_to_syslog: False
03:05:17    access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
03:05:17    errorlog: -
03:05:17    loglevel: debug
03:05:17    capture_output: False
03:05:17    logger_class: gunicorn.glogging.Logger
03:05:17    logconfig: None
03:05:17    logconfig_dict: {}
03:05:17    syslog_addr: udp://localhost:514
03:05:17    syslog: False
03:05:17    syslog_prefix: None
03:05:17    syslog_facility: user
03:05:17    enable_stdio_inheritance: False
03:05:17    statsd_host: None
03:05:17    statsd_prefix:
03:05:17    proc_name: None
03:05:17    default_proc_name: app.main:app
03:05:17    pythonpath: None
03:05:17    paste: None
03:05:17    on_starting: <function OnStarting.on_starting at 0x7ff9c4742ef0>
03:05:17    on_reload: <function OnReload.on_reload at 0x7ff9c4757050>
03:05:17    when_ready: <function WhenReady.when_ready at 0x7ff9c4757170>
03:05:17    pre_fork: <function Prefork.pre_fork at 0x7ff9c4757290>
03:05:17    post_fork: <function Postfork.post_fork at 0x7ff9c47573b0>
03:05:17    post_worker_init: <function PostWorkerInit.post_worker_init at 0x7ff9c47574d0>
03:05:17    worker_int: <function WorkerInt.worker_int at 0x7ff9c47575f0>
03:05:17    worker_abort: <function WorkerAbort.worker_abort at 0x7ff9c4757710>
03:05:17    pre_exec: <function PreExec.pre_exec at 0x7ff9c4757830>
03:05:17    pre_request: <function PreRequest.pre_request at 0x7ff9c4757950>
03:05:17    post_request: <function PostRequest.post_request at 0x7ff9c47579e0>
03:05:17    child_exit: <function ChildExit.child_exit at 0x7ff9c4757b00>
03:05:17    worker_exit: <function WorkerExit.worker_exit at 0x7ff9c4757c20>
03:05:17    nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7ff9c4757d40>
03:05:17    on_exit: <function OnExit.on_exit at 0x7ff9c4757e60>
03:05:17    proxy_protocol: False
03:05:17    proxy_allow_ips: ['127.0.0.1']
03:05:17    keyfile: None
03:05:17    certfile: None
03:05:17    ssl_version: 2
03:05:17    cert_reqs: 0
03:05:17    ca_certs: None
03:05:17    suppress_ragged_eofs: True
03:05:17    do_handshake_on_connect: False
03:05:17    ciphers: TLSv1
03:05:17    raw_paste_global_conf: []
03:05:17    [2020-04-03 03:05:17 +0000] [1] [INFO] Starting gunicorn 19.9.0
03:05:17    [2020-04-03 03:05:17 +0000] [1] [DEBUG] Arbiter booted
03:05:17    [2020-04-03 03:05:17 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
03:05:17    [2020-04-03 03:05:17 +0000] [1] [INFO] Using worker: uvicorn.workers.UvicornWorker
03:05:17    [2020-04-03 03:05:17 +0000] [8] [INFO] Booting worker with pid: 8
03:05:17    [2020-04-03 03:05:17 +0000] [9] [INFO] Booting worker with pid: 9
03:05:17    [2020-04-03 03:05:17 +0000] [1] [DEBUG] 2 workers
03:07:17    [2020-04-03 03:07:17 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:8)
03:07:17    [2020-04-03 03:07:17 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:9)
03:07:17    [2020-04-03 03:07:17 +0000] [12] [INFO] Booting worker with pid: 12
03:07:18    [2020-04-03 03:07:18 +0000] [13] [INFO] Booting worker with pid: 13
03:07:54    [2020-04-03 03:07:54 +0000] [1] [INFO] Handling signal: term

Answer 1

我必须在我的 Aurora 数据库安全组的入口规则中将我的 ECS 任务的安全组列入白名单。