Cross-DAG dependencies

Managing cross-DAG dependencies is necessary when data pipelines are partitioned into separate DAGs owned by different teams or running on distinct cadences.

Three dependency patterns

Airflow provides three mechanisms to coordinate execution across DAG boundaries:

• Assets (Data-aware scheduling): The modern, decoupled standard. DAG A declares an Asset as an output outlet. DAG B defines that Asset in its schedule parameter. When DAG A finishes writing data, Airflow records the asset update and automatically schedules DAG B. The two DAGs do not need to know each other's DAG IDs or schedule timing.
• TriggerDagRunOperator (Push model): A task at the end of upstream DAG A explicitly triggers downstream DAG B using the Airflow API. It can pass configuration dictionaries via the conf parameter. This is simple, but creates a tight coupling where DAG A must know DAG B's exact identifier.
• ExternalTaskSensor (Pull model): Downstream DAG B waits until a specific task in upstream DAG A reaches a successful state for an aligned execution timestamp.

# Pulling state using ExternalTaskSensor
wait_for_ingest = ExternalTaskSensor(
    task_id="wait_for_ingest",
    external_dag_id="upstream_raw_pipeline",
    external_task_id="finalize_upload",
    mode="reschedule",
    timeout=7200,
)

Operational trade-offs and traps

ExternalTaskSensor is notoriously fragile in production. It checks for exact alignment between the downstream DAG's logical date and the upstream DAG's logical date. If DAG A runs hourly while DAG B runs daily, the sensor will never find a matching run unless you write custom date offset functions using execution_delta or execution_date_fn. In addition, if an engineer changes the schedule of DAG A, DAG B silently stalls waiting for timestamps that no longer exist.

Whenever possible, prefer data-aware Assets over sensors. Assets decouple pipeline boundaries, eliminate slot-wasting sensor polling, and trigger downstream work immediately when data arrives.