gpt4 book ai didi

python - Airflow BashOperator 找不到 Bash

转载 作者:行者123 更新时间:2023-12-04 02:42:14 28 4
gpt4 key购买 nike

我在 Centos 7 中使用 Airflow,使用 Python 3.7。

当我通过 BashOperator 运行 Bash 命令时,我遇到了以下问题:

[2019-11-13 23:20:08,238] {taskinstance.py:1058} ERROR - [Errno 2] No such file or directory: 'bash': 'bash'
Traceback (most recent call last):
File "/home/airflow/virtualenvs/airflow_env/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
result = task_copy.execute(context=context)
File "/home/airflow/virtualenvs/airflow_env/lib/python3.7/site-packages/airflow/operators/bash_operator.py", line 120, in execute
preexec_fn=pre_exec)
File "/home/airflow/python/Python-3.7.5/Lib/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/home/airflow/python/Python-3.7.5/Lib/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'bash': 'bash'

我是否需要将变量传递给 BashOperator,以便它知道查找 /bin/bash?在源代码中,似乎 BashOperator 使用 bash 打开了一个子进程;我需要修改它才能使用 /bin/bash 吗?

最佳答案

原来我必须修改 systemctl 文件中的 PATH 变量。

:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin 添加到 PATH 解决了我的问题.

我的设置是使用 Centos 7 上通过 Systemctl 管理的 Airflow + virtualenv。

Airflow 调度程序 systemctl 文件

[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
EnvironmentFile=/etc/sysconfig/airflow
Environment=VIRTUAL_ENV=/home/airflow/virtualenvs/airflow_env
Environment=PATH=/home/airflow/virtualenvs/airflow_env/bin:$PATH:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin
User=airflow
Group=airflow
Type=simple
ExecStart=/home/airflow/virtualenvs/airflow_env/bin/airflow scheduler
Restart=always
RestartSec=5s
RuntimeDirectory=airflow
RuntimeDirectoryMode=0775

[Install]
WantedBy=multi-user.target

关于python - Airflow BashOperator 找不到 Bash,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/58847124/

28 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com