Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. What is CDC?

Pipelines & scenarios · Core Pipeline Concepts

What is CDC?

Mediumpipe-05
CDCDebeziumWALincrementalKafka

Question

What is Change Data Capture (CDC), and why do data engineers use it?

Solution

CDC (Change Data Capture) continuously captures inserts, updates, and deletes from a source database and streams those changes downstream.

Instead of dumping the whole table every night, you ship only what changed.

OLTP Postgres
  orders table
      |
  WAL / binlog / redo log
      |
  Debezium / DMS / Fivetran CDC
      |
  Kafka topic: orders.cdc
      |
  +---> lake bronze (append change events)
  +---> warehouse merge into dim/fact
  +---> search / cache invalidation

Why CDC beats full reloads

  • Lower load on the source DB
  • Fresher analytics (minutes, not next day)
  • Captures deletes and updates that a "SELECT *" snapshot can miss between runs

Event shapes you will see

  • op = c/u/d (create/update/delete)
  • before/after images of the row
  • source commit timestamp / LSN

Gotchas

Schema changes, tombstones for deletes, ordering across tables, and idempotent merges into the sink.

Interview tip: "CDC reads the database log and emits row-level changes." Name Debezium/Kafka or a managed CDC tool, then mention merge into lakehouse tables.

PreviousNext