Airflow & DAGs · Architecture & Scalability

What is a Pool?

Mediumpoolsconcurrency

Interview question

What is an Airflow Pool, and how does it differ from max_active_runs?

How to answer

The short answer

A Pool limits how many tasks can use a shared resource at once.

Example: only 5 API connections allowed → api_pool with 5 slots; tasks assigned to that pool share those slots.

Pool vs max_active_runs

  • max_active_runs → how many runs can be active
  • Pool → how many tasks can consume a specific resource

Interview tip: Use pools for rate-limited APIs, warehouses, and shared credentials.