Trigger rules for cleanup and partial failure

Trigger rules determine the conditions under which a downstream task is cleared to execute based on the completion states of its directly upstream parent tasks.

Default behavior and the all_done rule

The default trigger rule in Airflow is all_success: the downstream task runs only if every single parent task completes successfully. If any upstream task fails, downstream tasks are immediately marked as upstream_failed and skipped.

When creating cleanup or notification tasks that must run regardless of whether prior work succeeded or failed, use trigger_rule="all_done":

from airflow.operators.bash import BashOperator
from airflow.utils.trigger_rule import TriggerRule

cleanup_task = BashOperator(
    task_id="cleanup_temp_files",
    bash_command="rm -rf /tmp/staging/*",
    trigger_rule=TriggerRule.ALL_DONE,
)

upstream_extract >> upstream_transform >> cleanup_task

Under all_done, the cleanup task begins execution as soon as all upstream tasks finish, whether they succeeded, failed, or were skipped.

Common trigger rules in production pipelines

Other key trigger rules include:

• one_failed: Triggers as soon as at least one parent task fails, without waiting for remaining sibling tasks to finish. This is ideal for fast failure notifications that alert on-call engineers immediately during a long-running batch.
• none_failed_min_one_success: Requires all parent tasks to finish, with zero failures, but permits some parents to be skipped.

The branching skip propagation gotcha

A classic bug occurs when combining BranchPythonOperator with downstream convergence tasks. If a branching operator picks Branch A, Branch B is marked skipped.

If a downstream reporting task connects to both branches using the default all_success rule, the task is skipped because Branch B was skipped. To allow the reporting task to proceed after either branch finishes, set its trigger rule to none_failed_min_one_success.

Note that in Airflow 2.7 and later, setup and teardown tasks provide a dedicated, cleaner syntax for infrastructure cleanup that avoids complex trigger rule gymnastics.