Batch & Streaming · Core Concepts

At-least-once vs exactly-once

Mediumexactly-onceat-least-onceduplicatessemantics

Interview question

What do at-least-once and exactly-once delivery/processing mean?

How to answer

The short answer

At-least-once: every event is processed ≥1 time. After a crash you may reprocess, so duplicates are possible unless *you* make writes idempotent.

Exactly-once: each event affects the sink as if it ran once, even when the job retries. Teams often get there with dedupe, transactional sinks, or idempotent upserts (sometimes called effectively once).

Output
crash mid-write
  at-least-once:  may write again  --> duplicates unless idempotent
  exactly-once:   replay + commit protocol / idempotent sink --> one effect

Process ≤1 time; may lose events on failure. Rarely acceptable for money/audit data.

Reality check

True end-to-end exactly-once across Kafka → Flink → warehouse is hard. Many teams aim for at-least-once + idempotent merges.

Interview tip: Separate *delivery* (transport) from *processing effect* (business result). Say "exactly-once semantics" carefully and mention as the practical twin.