python - 来自 Airflow 调度程序的奇怪类型错误——在 v1.9 中是否更改了 @once 用于调度程序间隔的用法？

Question

我有一个 super 简单的测试 DAG，如下所示:

from datetime import datetime

from airflow.models import DAG
from airflow.operators.python_operator import PythonOperator


DAG = DAG(
  dag_id='scheduler_test_dag',
  start_date=datetime(2017, 9, 9, 4, 0, 0, 0), #..EC2 time. Equal to 11pm hora México
  max_active_runs=1,
  schedule_interval='@once' #externally triggered
  )

def ticker_function():
    with open('/tmp/ticker', 'a') as outfile:
        outfile.write('{}\n'.format(datetime.now()))

time_ticker = PythonOperator(
    task_id='time_ticker',
    python_callable=ticker_function,
    dag=DAG
)

自升级到 apache-airflow v1.9 后，此 DAG 已挂起且无法运行。深入研究调度程序日志，我发现了错误跟踪:

[2018-02-12 17:03:06,259] {jobs.py:1754} INFO - DAG(s) dict_keys(['scheduler_test_dag']) retrieved from /home/ubuntu/airflow/dags/scheduler_test_dag.py
[2018-02-12 17:03:06,315] {jobs.py:1386} INFO - Processing scheduler_test_dag
[2018-02-12 17:03:06,320] {jobs.py:379} ERROR - Got an exception! Propagating...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 371, in helper
pickle_dags)
  File "/usr/local/lib/python3.5/dist-packages/airflow/utils/db.py", line 50, in wrapper
result = func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 1792, in process_file
self._process_dags(dagbag, dags, ti_keys_to_schedule)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 1388, in _process_dags
dag_run = self.create_dag_run(dag)
  File "/usr/local/lib/python3.5/dist-packages/airflow/utils/db.py", line 50, in wrapper
result = func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 807, in create_dag_run
if next_start <= now:
TypeError: unorderable types: NoneType() <= datetime.datetime()

这个错误是从哪里来的？我唯一能想到的是 scheduler_interval='@once' 的用法发生了变化，这是自从v1.9升级。否则它是有史以来最基本的 DAG——似乎不应该有问题。以前，在切换到 apache-airflow 存储库之前，我使用的是基本的 pip 安装。

这是 Web UI 的屏幕截图。一切似乎都工作正常，除了顶部和底部的 DAGS，它们的调度间隔设置为 @once 并且无限期挂起:

有什么想法吗？

Answer 1

您是否在 airflow.cfg 中将 catch up 定义为 True？然后这是固定在主人。禁用此 dag 的追赶，它应该开始工作。