Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Keyed vs non-keyed streams

Batch & Streaming · Core Concepts

Keyed vs non-keyed streams

Mediumstream-07
keyed streamstateparallelismskew

Question

What is the difference between keyed and non-keyed stream processing?

Solution

A keyed stream partitions events by a key (user_id, device_id, order_id). Operators maintain per-key state, and events with the same key are processed in order on one parallel instance.

A non-keyed (or global) stream has no key. Aggregations are global, parallelism is limited, and state is shared carefully (often a single parallelism of 1 for correctness).

Keyed by user_id
  key=u1 --> operator instance A   (state for u1, u4, ...)
  key=u2 --> operator instance B
  key=u3 --> operator instance A

Non-keyed global count
  all events --> one logical aggregator (harder to scale)

Why keys matter

  • Parallelism: different keys on different workers
  • Correctness: per-user sessions need keyed state
  • Hot keys: one celebrity user_id can overload one worker (skew)

Interview tip: "Key = how we shard state and parallelism." Contrast with Kafka partitions: similar idea of ordered shards.

PreviousNext