- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
我正在尝试在 CentOS8 机器上安装带有 ansible 的 Airflow 2.0.1。 Python 版本 3.8.1。我按照 Airflow 文档中的建议制作了 pip 20.2.4。
我正在使用 postgresql 和 airflow db check
是成功的。但是 db init 任务给出了以下错误。我试过airflow db init
手动,但结果是一样的:
ERROR - Failed to add operation for GET /api/v1/connections
Traceback (most recent call last):
File "/opt/airflow/lib/python3.8/site-packages/connexion/apis/abstract.py", line 209, in add_paths
self.add_operation(path, method)
File "/opt/airflow/lib/python3.8/site-packages/connexion/apis/abstract.py", line 162, in add_operation
operation = make_operation(
File "/opt/airflow/lib/python3.8/site-packages/connexion/operations/__init__.py", line 8, in make_operation
return spec.operation_cls.from_spec(spec, *args, **kwargs)
File "/opt/airflow/lib/python3.8/site-packages/connexion/operations/openapi.py", line 128, in from_spec
return cls(
File "/opt/airflow/lib/python3.8/site-packages/connexion/operations/openapi.py", line 75, in __init__
super(OpenAPIOperation, self).__init__(
File "/opt/airflow/lib/python3.8/site-packages/connexion/operations/abstract.py", line 96, in __init__
self._resolution = resolver.resolve(self)
File "/opt/airflow/lib/python3.8/site-packages/connexion/resolver.py", line 40, in resolve
return Resolution(self.resolve_function_from_operation_id(operation_id), operation_id)
File "/opt/airflow/lib/python3.8/site-packages/connexion/resolver.py", line 66, in resolve_function_from_operation_id
raise ResolverError(str(e), sys.exc_info())
connexion.exceptions.ResolverError: <ResolverError: columns>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/airflow/bin/airflow", line 8, in <module>
sys.exit(main())
File "/opt/airflow/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/opt/airflow/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/opt/airflow/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
db.initdb()
File "/opt/airflow/lib/python3.8/site-packages/airflow/utils/db.py", line 549, in initdb
upgradedb()
File "/opt/airflow/lib/python3.8/site-packages/airflow/utils/db.py", line 684, in upgradedb
command.upgrade(config, 'heads')
File "/opt/airflow/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
script.run_env()
File "/opt/airflow/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
util.load_python_file(self.dir, "env.py")
File "/opt/airflow/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "/opt/airflow/lib/python3.8/site-packages/alembic/util/compat.py", line 182, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/airflow/lib/python3.8/site-packages/airflow/migrations/env.py", line 108, in <module>
run_migrations_online()
File "/opt/airflow/lib/python3.8/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/opt/airflow/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
self.get_context().run_migrations(**kw)
File "/opt/airflow/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
step.migration_fn(**kw)
File "/opt/airflow/lib/python3.8/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 314, in upgrade
remap_permissions()
File "/opt/airflow/lib/python3.8/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 289, in remap_permissions
appbuilder = create_app(config={'FAB_UPDATE_PERMS': False}).appbuilder
File "/opt/airflow/lib/python3.8/site-packages/airflow/www/app.py", line 120, in create_app
init_api_connexion(flask_app)
File "/opt/airflow/lib/python3.8/site-packages/airflow/www/extensions/init_views.py", line 171, in init_api_connexion
api_bp = connexion_app.add_api(
File "/opt/airflow/lib/python3.8/site-packages/connexion/apps/flask_app.py", line 57, in add_api
api = super(FlaskApp, self).add_api(specification, **kwargs)
File "/opt/airflow/lib/python3.8/site-packages/connexion/apps/abstract.py", line 144, in add_api
api = self.api_cls(specification,
File "/opt/airflow/lib/python3.8/site-packages/connexion/apis/abstract.py", line 111, in __init__
self.add_paths()
File "/opt/airflow/lib/python3.8/site-packages/connexion/apis/abstract.py", line 216, in add_paths
self._handle_add_operation_error(path, method, err.exc_info)
File "/opt/airflow/lib/python3.8/site-packages/connexion/apis/abstract.py", line 231, in _handle_add_operation_error
raise value.with_traceback(traceback)
File "/opt/airflow/lib/python3.8/site-packages/connexion/resolver.py", line 61, in resolve_function_from_operation_id
return self.function_resolver(operation_id)
File "/opt/airflow/lib/python3.8/site-packages/connexion/utils.py", line 111, in get_function_from_name
module = importlib.import_module(module_name)
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/airflow/lib/python3.8/site-packages/airflow/api_connexion/endpoints/connection_endpoint.py", line 26, in <module>
from airflow.api_connexion.schemas.connection_schema import (
File "/opt/airflow/lib/python3.8/site-packages/airflow/api_connexion/schemas/connection_schema.py", line 42, in <module>
class ConnectionSchema(ConnectionCollectionItemSchema): # pylint: disable=too-many-ancestors
File "/opt/airflow/lib/python3.8/site-packages/marshmallow/schema.py", line 121, in __new__
klass._declared_fields = mcs.get_declared_fields(
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/schema/sqlalchemy_schema.py", line 94, in get_declared_fields
fields.update(mcs.get_auto_fields(fields, converter, opts, dict_cls))
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/schema/sqlalchemy_schema.py", line 104, in get_auto_fields
{
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/schema/sqlalchemy_schema.py", line 105, in <dictcomp>
field_name: field.create_field(
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/schema/sqlalchemy_schema.py", line 28, in create_field
return converter.field_for(model, column_name, **self.field_kwargs)
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/convert.py", line 171, in field_for
return self.property2field(prop, **kwargs)
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/convert.py", line 146, in property2field
field_class = field_class or self._get_field_class_for_property(prop)
File "/opt/airflow/lib/python3.8/site-packages/marshmallow_sqlalchemy/convert.py", line 210, in _get_field_class_for_property
column = prop.columns[0]
File "/opt/airflow/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 1220, in __getattr__
return self._fallback_getattr(key)
File "/opt/airflow/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 1194, in _fallback_getattr
raise AttributeError(key)
AttributeError: columns
最佳答案
我有同样的问题,临时修复是使用 sqlalchemy < 1.4。也许这对你有用。
关于centos - Airflow db init 错误 - 无法为 GET/api/v1/connections 添加操作,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/67437595/
在Airflow中,我一直在使用“airflow run”和“airflow test”,但不完全理解它们有何不同。他们有什么区别? 最佳答案 我自己通读了文档,发现它是多么令人困惑。 Airflow
我使用 Airflow 已经有一段时间了,它是由一位同事创建的。最近我遇到了一些错误,这需要我更深入地了解如何修复 Airflow 中的某些问题。 我确实理解这三个进程是什么,但我只是不明白运行它们时
AIRFLOW_HOME=/path/to/my/airflow_home 我收到这个警告... >airflow trigger_dag python_dag3 /Users/alexryan/mi
有没有人报告过他们在他们的公司中让 Airflow 扩展了多少?我正在考虑实现 Airflow 来执行 5,000 多个任务,每个任务每小时运行一次,有一天可以将其扩展到 20,000 多个任务。在检
问题 :我想使用 Github 上最新版本的 Apache-Airflow 安装 apache-airflow 以及所有依赖项? 我怎样才能使用 pip 做到这一点? 在生产环境中使用它是否安全? 最
我们在 AWS ECS 上运行 Airflow,并将所有 DAG 捆绑在一个 Docker 镜像中。我们不时更新 DAGS,并部署新版本的 Docker Image。当我们这样做时,ECS 将终止正在
问题很简单。我需要限制 Airflow 网络用户仅查看和执行某些 DAG 和任务。 如果可能,我宁愿不使用 Kerberos也不是 OAuth . Multi-tenancy option 似乎是一个
我们正在使用 Airflow 2.00。我正在尝试实现一个做两件事的 DAG: 通过 API 触发报告 从源到目标下载报告。 任务 1 和任务 2 之间至少需要 2-3 小时的间隔。根据我的研究,我有
对于一项任务,有许多辅助任务 - 从文件/数据库中获取/保存属性、验证、审计。这些辅助方法并不耗时。 一个示例 DAG 流, fetch_data >> actual_processing >> va
有什么方法可以重新加载作业而不必重新启动服务器吗? 最佳答案 在airflow.cfg中,您具有以下两种配置来控制此行为: # after how much time a new DAGs shoul
我们可以通过将任务/dag 超时设置为 None 并手动触发其运行来使用 Airflow dag 来定义永无止境的作业(即具有无条件循环以消耗流数据的任务)吗?让 Airflow 监测永无止境的任务会
我是 Airflow 的新手,最近开始探索这个工具。我在 18.4 版本的 ubuntu 机器上安装了 1.10.10 版。从设置和安装的角度来看,一切正常,但是我在任何 DAG 中的任务都没有运行,
我主要看到Airflow被用于ETL / Bid数据相关的工作。我正在尝试将其用于业务工作流,其中用户操作将来会触发一组相关任务。其中某些任务可能需要根据某些其他用户操作来清除(删除)。 我认为最好的
我有一个 DAG,只要 FileSensor 检测到文件,它就会使用它,为每个文件生成任务,以 (1) 将文件移动到暂存区域,(2) 触发单独的 DAG 来处理文件。 FileSensor -> Mo
我需要手动或以编程方式执行的管道,可以使用 Airflow 吗?看起来现在每个工作流程都必须与时间表绑定(bind)。 最佳答案 只需在创建 DAG 时将 schedule_interval 设置为
所以这是一个愚蠢的想法...... 我在 Airflow 中创建了(许多)DAG...并且它有效...但是,我想以某种方式将其打包,以便我可以在不安装 Airflow 的情况下运行单个 DAG 运行;
我使用“pip install 'apache-airflow[statsd]' 安装了 airflow[statsd] 并安装了 statsd_exporter。现在我可以看到来自 Promethe
我们正在尝试将 MongoHook 和 GCSToLocalFilesystemOperator 导入到我们的 Airflow 项目中: docs for MongoHook docs for GCS
启动 Airflow 网络服务器时出现以下错误 balajee@Balajees-MacBook-Air.local:~$ Airflow 网络服务器 -p 8080 [2018-12-03 00:2
运行pip install airflow[postgres]命令后出现以下错误: > raise RuntimeError("By default one of Airflow's dependen
我是一名优秀的程序员,十分优秀!