The interview setup
Design a strategy for late-arriving data in streaming. Discuss watermarks (allowed lateness), reprocessing (nightly batch), and append-only design with deduplication at read time.
Late data is not a Flink footnote. It is a product policy question: when a phone was in airplane mode for two hours, do you revise the dashboard, ignore the events, or land corrections beside immutable closes?
Background concepts from first principles
Late means "after we thought we were done"
A window for hour 10 closes in your stream job. An event with event_time inside hour 10 arrives at wall-clock 13:00. That event is late relative to your watermark decision.
Watermarks and allowed lateness
Watermarks estimate event-time progress. Allowed lateness keeps windows open a bit longer for updates. Then windows close and further events need another policy.
Batch reconcile
Many teams use stream for speed and nightly batch from raw logs for truth. Late events naturally appear in the batch rebuild.
Append-only facts with read-time resolution
Store every observation with versions. Views pick the latest per key. Writers stay simple. Readers become smarter. This pattern shows up in ledgers and some lakehouse designs.
The expanded problem
Provide a coherent late-data policy that spans:
- Stream watermarks and allowed lateness
- Side outputs for very late events
- Nightly or on-demand reprocessing
- Append-only + dedupe-at-read as an alternative
- Communication of correctness expectations to consumers
Stream windows (fast, may revise within lateness) Raw log --> nightly job --> curated daily tables (truth)
What good looks like
- Airplane-mode story up front
- Explicit product policy
- Watermark + lateness with state bound
- Batch truth path
- Answer for immutable finance closes
Clarifying questions
- How late is late for this business: minutes, hours, days?
- Can downstream tolerate revisions?
- Do finance closes need immutability?
- What generates lateness: mobile, multi-region, upstream batch?
Out of scope
- Perfect prediction of all late distributions
- Building a new stream engine
- Ignoring business policy in favor of only tooling
How to use clarifying questions
Ask a few questions that change architecture: SLA definition, source of truth, retention, and tolerance for approximates. If told to decide, state assumptions on the board and proceed.
What good sounds like
Narrate trade-offs while drawing. Name the waiting room for bursts, the transform, the serving surface, and the replay path. Explicitly list out-of-scope items.
Before you draw
Find entry, buffer, transform, serve, and replay. If any is missing, your operational story has a hole.
How to use clarifying questions
Ask a few questions that change architecture: SLA definition, source of truth, retention, and tolerance for approximates. If told to decide, state assumptions on the board and proceed.
What good sounds like
Narrate trade-offs while drawing. Name the waiting room for bursts, the transform, the serving surface, and the replay path. Explicitly list out-of-scope items.
Before you draw
Find entry, buffer, transform, serve, and replay. If any is missing, your operational story has a hole.
How to use clarifying questions
Ask a few questions that change architecture: SLA definition, source of truth, retention, and tolerance for approximates. If told to decide, state assumptions on the board and proceed.
What good sounds like
Narrate trade-offs while drawing. Name the waiting room for bursts, the transform, the serving surface, and the replay path. Explicitly list out-of-scope items.
Before you draw
Find entry, buffer, transform, serve, and replay. If any is missing, your operational story has a hole.
How to use clarifying questions
Ask a few questions that change architecture: SLA definition, source of truth, retention, and tolerance for approximates. If told to decide, state assumptions on the board and proceed.
What good sounds like
Narrate trade-offs while drawing. Name the waiting room for bursts, the transform, the serving surface, and the replay path. Explicitly list out-of-scope items.