Kafka · Advanced Kafka

Delete vs compact retention

Mediumretentioncleanup.policydeletecompact

Interview question

What is the difference between delete retention and compact retention in Kafka?

How to answer

The short answer

have a cleanup policy that decides how old data is reclaimed.

delete (time/size retention)

  • Default for many event topics.
  • Segments older than retention.ms (or exceeding retention bytes) are deleted.
  • Entire old segments go away, regardless of keys.
  • Use for clickstreams, logs, immutable event history where old data can expire.
Output
delete policy:
  keep last 7 days of events, then drop old segments

compact (key-based retention)

  • Cleanup policy compact.
  • Retains latest record per key.
  • Older keyed records may be removed even if "recent" history of that key is superseded.
  • Use for changelogs / current-state topics.

Both (compact,delete)

Many production changelog topics use both:

  • Compact to keep latest per key
  • Also delete very old tombstones / enforce a retention ceiling

Choosing in interviews

Data shapePolicy
Immutable events (orders placed)delete
Upserts / entity statecompact (often + delete)
Need full replay foreverincrease retention / archive out of Kafka

Interview tip: "delete removes old segments by time/size; compact keeps the last value per key."