gpt4 book ai didi

Airflow in Docker: Unable to import an installed Python module(Docker中的气流:无法导入已安装的Python模块)

转载 作者:bug小助手 更新时间:2023-10-25 09:15:25 32 4
gpt4 key购买 nike



I'm new to both docker and airflow and I'm trying to create an airflow pipeline in docker on my windows desktop. I have used the latest airflow image and extended it to include some additional python libraries that are needed. All the libraries were installed, image extended and container up and running.

我对docker和airflow都是新手,我正试图在我的Windows桌面上的docker中创建一个气流管道。我使用了最新的Airflow图像,并对其进行了扩展,以包括所需的一些附加的Python库。所有的库都已安装,映像已扩展,容器已启动并运行。


Next, I write the .py file which contains all DAG's and have imported the following modules.

接下来,我编写.py文件,该文件包含所有DAG并导入了以下模块。


from airflow import DAG
from airflow.operators.bash import BashOperator
import time as tm
import pickle, os, datetime, json, requests, locale
from datetime import datetime, date, time, timedelta
from yahoo_fin import options as op
import yfinance as yf
import pandas as pd, numpy as np
from polygon import RESTClient
from polygon.rest.models import Sort
from urllib3 import HTTPResponse

After saving this file, I see the following error message on airflow UI:

保存此文件后,我在AirFlow界面上看到以下错误消息:


Broken DAG: [/opt/airflow/dags/our_first_dag.py] Traceback (most recent call last):
File "", line 219, in _call_with_frames_removed
File "/opt/airflow/dags/our_first_dag.py", line 9, in
from polygon import RESTClient
ImportError: cannot import name 'RESTClient' from 'polygon' (/home/airflow/.local/lib/python3.8/site-packages/polygon/init.py)

中断的DAG:[/opt/airflow/dgs/our_first_dag.py]回溯(最近一次调用):文件“”,第219行,In_Call_With_Frame_Remoted文件“/opt/airflow/dgs/our_first_dag.py”,第9行,从多边形导入RESTClient导入错误:无法从‘POLYGON’(/home/airflow/.local/lib/python3.8/site-packages/polygon/init.py)导入名称‘RESTClient’


From this message it appears that python is unable to import RESTClient from Polygon module, so I check if this package is installed in the image by by going to the container terminal (from docker UI) confirm that all the packages I added while extending the image are actually installed. I also tried running the line which produced error i.e. "from polygon import RESTClient" in the local python and it worked perfectly fine.

从这条消息中可以看出,Python无法从Polygon模块导入RESTClient,所以我通过转到容器终端(从docker UI)确认我在扩展映像时添加的所有包都已实际安装来检查映像中是否安装了此包。我还尝试在本地的python中运行产生错误的代码行,即“from POLYGON IMPORT RESTClient”,它工作得非常好。


I'd appreciate any suggestions on how to fix this.

如果有任何关于如何解决这个问题的建议,我将不胜感激。


Below are the contents of my directory:

以下是我的目录的内容:


dockerfile

扩展坞文件


FROM apache/airflow:2.7.1
COPY requirements.txt /
USER airflow
RUN pip install -r /requirements.txt
USER root

requirements.txt

Requirements.txt


polygon
html5lib
yahoo_fin
yfinance
polygon-api-client

.env

.env


AIRFLOW_UID=50000

docker-compose.yaml
https://pastebin.com/SPNsk1f4

Docker-pose.yaml https://pastebin.com/SPNsk1f4


更多回答
优秀答案推荐
更多回答

32 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com