Kafka · Advanced Kafka

Exactly-once in Kafka Streams

Hardexactly-oncekafka streamstransactionseos

Interview question

How does exactly-once semantics (EOS) work in Kafka Streams at a high level?

How to answer

The short answer

Kafka Streams can be configured for exactly-once processing so that a read → update state → write output cycle does not create duplicate side effects under retries.

Config flag (classic naming)

processing.guarantee=exactly_once_v2 (modern) or older exactly_once.

What "exactly-once" covers here

Within Kafka:

  • Consuming from input topics
  • Updating state stores / changelogs
  • Producing to output topics

…as one atomic transactional unit per commit.

High-level mechanism

  1. 1Streams uses idempotent producers + transactions.
  2. 2Offsets, state store changelogs, and output records are committed together.
  3. 3On failure/restart, unfinished transactions abort; committed results appear once.

Important boundaries

  • EOS protects the Kafka-to-Kafka processing loop.
  • Writing to an external DB still needs idempotent sinks or transactional outbox patterns unless you carefully bridge systems.
  • Exactly-once is not free: more coordination, potential latency/throughput cost.

Interview-ready sentence

"Kafka Streams EOS uses transactions so input offset commits, state updates, and output produces commit atomically, preventing duplicate outputs on retry."

Interview tip: Distinguish broker/transaction EOS from end-to-end including external systems.