Pipelines & scenarios · Core Pipeline Concepts

What is CDC?

MediumCDCDebeziumWALincrementalKafka

Interview question

What is Change Data Capture (CDC), and why do data engineers use it?

How to answer

The short answer

(Change Data Capture) continuously captures inserts, updates, and deletes from a source database and streams those changes downstream.

Instead of dumping the whole table every night, you ship only what changed.

Why CDC beats full reloads

  • Lower load on the source DB
  • Fresher analytics (minutes, not next day)
  • Captures deletes and updates that a "SELECT *" snapshot can miss between runs

Event shapes you will see

  • op = c/u/d (create/update/delete)
  • before/after images of the row
  • source commit timestamp / LSN

Gotchas

Schema changes, tombstones for deletes, ordering across tables, and idempotent merges into the sink.

Interview tip: "CDC reads the database log and emits row-level changes." Name /Kafka or a managed CDC tool, then mention merge into lakehouse tables.