Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. How does Airflow handle logs?

Airflow & DAGs · Architecture & Scalability

How does Airflow handle logs?

Mediumairflow-19
loggings3ops

Question

How does Airflow handle logs in distributed deployments?

Solution

With many workers (or short-lived K8s pods), logs shouldn't live only on one machine.

Remote logging sends logs to centralized storage (S3, GCS, Azure Blob) so they remain available after workers disappear.

Interview tip: Remote logging is especially important with KubernetesExecutor.

PreviousNext