Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Exactly-once vs idempotency

Batch & Streaming · Extra High-Value

Exactly-once vs idempotency

Mediumstream-33
exactly-onceidempotencyretriessemantics

Question

How do exactly-once processing and idempotency relate, and how do they differ?

Solution

Exactly-once is a *system guarantee* (or end-to-end protocol): after failures and retries, each input event contributes once to the output effect.

Idempotency is a *property of your write*: applying the same operation N times leaves the same correct state as applying it once. You design sinks (merge keys, partition overwrite, dedupe table) so duplicates are harmless.

at-least-once delivery + idempotent sink  ≈  practical "exactly once effect"

Kafka redelivers offset 100
  non-idempotent INSERT  -> duplicate revenue row
  idempotent MERGE on order_id -> still one row

How they work together

True engine exactly-once (Flink checkpoints + transactional sink) reduces duplicates. Idempotent application logic still saves you when any layer retries.

Interview tip: "Exactly-once is the promise; idempotency is how DEs often achieve the business outcome under at-least-once reality." Give overwrite-by-partition or merge-on-key as examples.

PreviousNext