What is Apache Airflow, and what problem does it solve?

Apache Airflow is an open-source workflow orchestration platform.

It helps you define:

• What tasks need to run
• In what order
• When they should run
• What should happen if a task fails
• How many times it should retry
• How to monitor the pipeline

In one sentence

> Airflow schedules, orchestrates, monitors, and manages workflows.

Example

Say your company gets sales data every day:

Download CSV
     ↓
Validate CSV
     ↓
Load into database
     ↓
Run transformations
     ↓
Send report

Instead of manually running these tasks, Airflow orchestrates them automatically.

Airflow vs cron

Cron is mainly a scheduler:

0 2 * * * python pipeline.py

That means: run this script every day at 2 AM.

But if you have Task A → B → C → D and Task B fails, cron alone won't give you dependency management, retries, monitoring, logs, parallel runs, backfills, or a workflow graph.

Interview tip:

> Cron tells you *when* to run something. Airflow helps manage the *entire* workflow.

Airflow is not a compute engine. It starts and watches work that usually runs elsewhere (Spark, dbt, warehouses).