Cloud · GCP

Cloud Composer

gcpEasygcpcomposerairfloworchestration

Interview question

What is Cloud Composer?

How to answer

The short answer

Cloud Composer is Google's managed Apache Airflow service. Google runs the Airflow control plane (webserver, scheduler, metadata DB) on GKE under the hood; you write and deploy them.

Problem first. You still need orchestration: "wait for GCS file → run Dataflow → run dbt in BigQuery → notify Slack." Airflow is the standard DAG tool; Composer removes most of the cluster babysitting.

Output
Composer (managed Airflow)
   DAG: daily_sales
     1. GCSSensor
     2. DataflowStartFlexTemplateOperator
     3. BigQueryInsertJobOperator
     4. Email / Pub/Sub notify

Why DE teams use it

  • Native GCP operators (BigQuery, Dataflow, GCS, Dataproc)
  • + VPC integration with the rest of GCP
  • Less DIY Airflow ops than self-hosting

Trade-offs

  • Cost of a managed environment (even when idle-ish)
  • Version upgrades / environment sizing still need attention
  • Same Airflow concepts: DAGs, tasks, retries, sensors, XCom

Interview tip: "Composer = managed Airflow on GCP." Map one DAG step to BigQuery/Dataflow so it sounds concrete.