python - Flask Redis 队列 (RQ) worker 无法导入名为 app 的模块

Question

我有一个名为 modeling-manager 的 flask 应用程序正在运行，它正在接收来自另一个 flask 应用程序的发布请求。 '直到那里一切正常。 (所有文件都位于同一个文件夹中)

当被 post 请求触发时，Modeling-manager 会在 Redis 中注册一个作业供 worker 执行。要执行的作业称为 run_model()，它存储在 main.py( flask 应用程序)中的不同文件和导入器中

worker 在与建模管理器应用程序相同的 docker 上启动。使用脚本:worker.py

main.py -> flask 应用程序

from flask import Flask, request, jsonify
import logging as lg
from datetime import datetime as dt
import os
import boto3
import redis
import json
from rq import Worker, Queue, Connection
from .model import run_model

app = Flask(__name__)

@app.route('/ModelManager/register',  methods=['POST'])
def register_philjob_to_queue():
    lg.info("triggered register cahce on model manager")
    job_data = request.get_json()
    lg.info("json data " + job_data["InstanceNumber"])
    try:
        with Connection(redis.Redis(host="redis-caching", port="6379")):
            lg.info("adding job to queue")
            q = Queue()
            task = q.enqueue(run_model, job_data)
        response_object = {
            'status': 'pending',
            'data': {
                'task_id': task.get_id()
            }
        }
        return jsonify(response_object), 202
        except Exception as e:
          lg.error(e)

worker.py

import redis
from rq import Worker, Queue, Connection

conn = redis.Redis(host="redis-caching", port="6379")
listen = ['default']

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(list(map(Queue, listen)))
        worker.work()

最后 model.py

import logging as lg
def run_model(job_data):

    # Run phil on started instance
    try:

        lg.info("executing the model") 
        return 'Model Executed'

    except Exception as e:
        print(e)
        print("RUN_MODEL:")
        lg.error(e)

我从工作人员那里得到的错误消息，其中“app”可能指的是我的 flask 应用程序(名为 app)的根文件夹:

10:35:15 RQ worker 'rq:worker:7555cdf9c1ad.9' started, version 0.12.0
10:35:15 *** Listening on default...
10:35:15 Cleaning registries for queue: default
10:35:15 default: app.model.run_model({'InstanceNumber': '002365984', 
'CallBackUrl': 'http://blabla.be'}) (5e83d96b-de11-4ddb-9c8f- 
b7b256580bfc)
10:35:15 ModuleNotFoundError: No module named 'app'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 
793, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 599, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 213, 
in func
return import_attribute(self.func_name)
  File "/usr/local/lib/python3.7/site-packages/rq/utils.py", line 152, in import_attribute
    module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in 
import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in 
_find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'app'
Traceback (most recent call last):
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 793, in perform_job
rv = job.perform()
 File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 599, in perform
self._result = self._execute()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 213, in func 
return import_attribute(self.func_name)
  File "/usr/local/lib/python3.7/site-packages/rq/utils.py", line 152, in import_attribute
module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
  ModuleNotFoundError: No module named 'app'   
10:35:15 Moving job to 'failed' queue

这都是在docker容器上执行的，redis在不同的容器上。这两个容器之间的通信工作正常。

使用命令在 docker 上启动 flask 应用程序: flask 运行 --host=0.0.0.0 --port=8081

Answer 1

我正在运行一个 flask 应用程序，并在这里遇到了同样的问题。上面的解决方案对我不起作用。但是，我的问题与不正确的导入有关。

当在终端的 Windows 上调试时，我的 redis 服务器在 cygwin 中运行。我刚刚用我的静态项目路径启动了我的工作人员。现在我的导入错误解决了。之前:

rq worker flask-task

之后:

rq worker flask-task --path /mnt/c/Users/<USER>/Desktop/projects/flask_app