Kafka · Advanced Kafka

What is log compaction?

Mediumlog compactionchangelogtombstonecdc

Interview question

What is Kafka log compaction, and when would you use it?

How to answer

The short answer

Log compaction retains at least the last known value for each key in a , instead of (or in addition to) deleting only by time/size.

Mental model

Think of a compacted topic as a changelog of a table keyed by .

Older values for user-1 can be removed. Latest per key remains.

Great use cases

  • changelogs (Debezium-style)
  • Kafka Streams / ksqlDB state store backups (changelog topics)
  • "Current entity snapshot" streams (customers, account_status)

Tombstones

A record with value null for a key is a tombstone. Compaction eventually removes the key entirely after the delete retention window. That models deletes.

What compaction does not mean

  • It is not a full database.
  • Ordering still matters; compaction is per partition by key.
  • Keys must be meaningful; null keys are not useful for compaction semantics.

Interview tip: "Compaction keeps the latest value per key. Perfect for changelogs and materialized views, not for pure event history."