Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. What is the Metadata Database?

Airflow & DAGs · Core Concepts & DAGs

What is the Metadata Database?

Easyairflow-09
metadatadatabase

Question

What is the Airflow Metadata Database, and why can it become a bottleneck?

Solution

The metadata DB stores Airflow internal state: DAG runs, task instances, states, connections, variables, XCom metadata.

Production: PostgreSQL / MySQL. SQLite: local learning only.

It becomes a bottleneck with thousands of DAGs, millions of task instances, heavy scheduler traffic, and many UI users. Treat it like a critical production database: size it, tune it, monitor it, and back it up.

PreviousNext