python - 成功运行任务但不跳过

Question

我在 AirFlow 中有几个任务，我只想在前面的任务成功且未跳过时触发。根据 AirFlow 文档，如果我跳过一个任务并且下一个任务设置为“all_success”，它无论如何都会被触发。有什么办法可以避免这种情况吗？

my_task = PythonOperator(
    task_id='task_id',
    python_callable=my_func,
    provide_context=True,
    trigger_rule='all_success',
    dag=my_dag)

http://airflow.apache.org/concepts.html#trigger-rules

One must be aware of the interaction between trigger rules and skipped tasks in schedule level. Skipped tasks will cascade through trigger rules all_success and all_failed but not all_done, one_failed, one_success, none_failed, none_skipped and dummy.

谢谢!

Answer 1

我认为您可以修改文档中给出的分支示例。在一个分支上，首先在 none_skipped 上触发，然后在 none_skipped 和 all_success 上触发你想要的任务。默认情况下，另一个分支将只是 all_success，如果没有 none_skipped，它将围绕您只想完成的分支进行分支。

run_this_first = DummyOperator(task_id='run_this_first', dag=dag)
branching = BranchPythonOperator(
    task_id='branching', dag=dag,
    python_callable=lambda: 'none_skipped_branch'
)

none_skipped_branch = DummyOperator(task_id='none_skipped_branch',trigger_rule='none_skipped', dag=dag)
none_skipped_and_all_success = DummyOperator(task_id='none_skipped_and_all_success', dag=dag)

branch_false = DummyOperator(task_id='branch_false', dag=dag)

join = DummyOperator(task_id='join', dag=dag)

run_this_first >> branching
branching >> none_skipped_branch >> none_skipped_and_all_success >> join
branching >> branch_false >> join