Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. What is XCom?

Airflow & DAGs · Advanced Development

What is XCom?

Mediumairflow-21
xcomtaskflow

Question

What is XCom in Airflow?

Solution

XCom = Cross-communication. Tasks exchange small pieces of information (IDs, file paths, timestamps, small configs).

It is not for large datasets. Don't put a 500 MB DataFrame in XCom.

Instead:

Save DataFrame → S3
XCom → "s3://bucket/path/file.parquet"

How big you can go depends on the metadata DB/backend. Treat XCom as small metadata only.

PreviousNext