Batch & Streaming · Extra High-Value

Exactly-once vs idempotency

Mediumexactly-onceidempotencyretriessemantics

Interview question

How do exactly-once processing and idempotency relate, and how do they differ?

How to answer

The short answer

Exactly-once is a *system guarantee* (or end-to-end protocol): after failures and retries, each input event contributes once to the output effect.

is a *property of your write*: applying the same operation N times leaves the same correct state as applying it once. You design sinks (merge keys, overwrite, dedupe table) so duplicates are harmless.

Output
at-least-once delivery + idempotent sink  ≈  practical "exactly once effect"

Kafka redelivers offset 100
  non-idempotent INSERT  -> duplicate revenue row
  idempotent MERGE on order_id -> still 1 row

How they work together

True engine exactly-once (Flink checkpoints + transactional sink) reduces duplicates. Idempotent application logic still saves you when any layer retries.

Interview tip: "Exactly-once is the promise; idempotency is how DEs often achieve the business outcome under at-least-once reality." Give overwrite-by-partition or merge-on-key as examples.