Sensor modes and timeouts

Airflow sensors wait for an external condition to become true before letting downstream tasks proceed. The choice between poke and reschedule mode determines whether the sensor hoards worker capacity during that wait.

Poke versus reschedule mode

Sensors operate in one of two modes:

• poke (default): The sensor task occupies a worker slot continuously. It checks the condition, sleeps for poke_interval seconds, and repeats. If an expected data file is delayed by 5 hours, that worker slot is completely blocked for 5 hours doing nothing.
• reschedule: The sensor checks the condition. If false, the task frees its worker slot, updates its state to up_for_reschedule, and exits. The scheduler re-queues it when poke_interval expires.

# Safe sensor configuration
wait_for_file = S3KeySensor(
    task_id="wait_for_file",
    bucket_name="landing-bucket",
    bucket_key="incoming/orders.csv",
    mode="reschedule",
    poke_interval=300,       # check every 5 minutes
    timeout=3600 * 4,        # fail after 4 hours
    soft_fail=True,          # mark skipped rather than failed
)

The pool deadlock hazard

If ten DAGs each have two sensors running in poke mode against a shared 20-slot pool, those sensors consume all 20 worker slots. If the tasks that actually generate the incoming data also need slots in that pool, they cannot start because the sensors are occupying every slot. The cluster enters a deadlocked state where waiting tasks block the very work they are waiting on. Always default to reschedule for any sensor that might wait longer than a couple of minutes.

Timeouts and soft fail

Every sensor must declare a realistic timeout. Without an explicit timeout, some sensors default to 7 days, holding resources indefinitely if an upstream file never lands.

The soft_fail=True flag changes the timeout behavior from marking the task failed to marking it skipped. This allows downstream workflows with conditional branching to handle missing optional files gracefully without triggering emergency on-call alerts.