System design interview

IoT Sensor Data Pipeline (1M Devices)

HardLocked55 min read

Streaming analytics for 1M devices (~200KB/s aggregate): rolling averages, anomalies, time-series storage.

system-designiotstreamingtime-series

Start here

The problem in plain words

A company sells one million smart thermostats. Each thermostat sends a tiny reading, about 200 bytes, every 5 seconds. The company wants three things: a rolling average temperature for each device, an alert when a device behaves strangely, and a chart of any device's last 24 hours.

People react to the number "one million" in two opposite ways. Some panic and build something huge. Others hear "only 40 MB a second" and put it all into an ordinary warehouse table, which then struggles. Both are wrong. The data volume is modest. What is hard is having a million separate little histories to track, keeping a small amount of memory for each one, and coping when thousands of devices reconnect at the same moment.

This problem teaches you to separate the real difficulty, which is the many devices, from the apparent difficulty, which is the large count.

Words you will meet

IoT
Internet of Things: everyday devices such as thermostats and meters that send data over the internet.
Time series
A list of values for one thing over time, such as one thermostat's temperature each 5 seconds.
Series cardinality
The number of separate time series. One million devices means one million series, which is hard for naive databases.
Gateway
The front door that devices connect to. It checks who they are and whether their messages look valid.
Partition key
The value that decides which lane of a message log an event goes to. Using device_id keeps each device's events in order.
Keyed state
Memory a stream job keeps for each key, such as each device's recent readings, so it can compute a rolling average.
Rolling average
The average of the most recent period, such as the last 5 to 15 minutes, which moves forward as time passes.
Anomaly
A reading that is unusual, for example a sudden temperature spike.
Hysteresis
Requiring a condition to hold for a while before raising or clearing an alert, so a noisy sensor does not flip it on and off.
Downsampling
Keeping coarser data for old periods, such as one value per hour instead of one per 5 seconds, to save space.
Time-series database (TSDB)
A database designed to store and query values over time per series, such as InfluxDB or TimescaleDB.
A million small histories in one ordinary table
200K a secone deviceslow1,000,000 devicesone reading every 5secPlain warehousetableone giant heapDevice 42 chartlast 24 hours
Press Play to watch it move

Each arrow is one thing happening. Play it step by step, or read the list below.

  1. 1.About 200,000 readings arrive every second. By volume that is fine, only about 40 MB a second.
  2. 2.A user wants just one device for 24 hours, but in a heap of mixed rows the database has to hunt through everything to find them.
  3. 3.Charts are slow and costs grow. The real issue is the million separate series, not the number of bytes.

The numbers, as things you can picture

  • 1 million devices, one reading every 5 secondsThat is 200,000 readings every second.
  • 200 bytes each, about 40 MB a secondA modest data rate. A single good server could receive several times that. Message logs handle it easily.
  • 1 million time seriesThis is the number that stresses simple databases, because each series needs its own state and its own index entries.

What the interviewer wants

Design intake, per-device live calculations, alerting, and storage for 1 million devices, including what happens when devices go offline and reconnect together, and how you avoid an alert storm.

The interview setup

You are designing streaming analytics for 1 million IoT devices. Each device sends about 200 bytes every 5 seconds. You must compute per-device rolling averages, detect anomalies, and store time-series data for historical charts.

This interview is a trap for candidates who only flex Kafka knowledge. Bandwidth is modest. The real bosses are series cardinality, keyed state size, downsample policy, and reconnect storms after outages.

What the interviewer wants:

  • You notice cardinality before you brag about throughput
  • You design ingest that survives thundering herds
  • You pick storage that fits time-series access patterns
  • You prevent alert fatigue
  • You can explain how rolling windows live in stream state

What is an IoT telemetry pipeline?

Devices emit readings (temperature, vibration, GPS, voltage). A cloud pipeline ingests them, computes short-horizon analytics, alerts on bad behavior, and stores history for dashboards and forensics.

Unlike a clickstream ad-tech firehose, many IoT fleets are "chatty keys, medium bytes." 1 million devices means 1 million (or more) active series.

Why cardinality hurts

Every distinct series key (device_id, or device_id × metric_name) needs indexing and often memory for recent state. A warehouse table that is fine for 10k products can become painful with 1M hot devices if every query filters one device across billions of rows without the right layout.

Time-series databases and careful lake clustering exist because of this access pattern: append by time, fetch by series and range.

Rolling averages need state

A 15-minute rolling average is not a stateless map. The job must remember recent points or a compact summary per device. In Flink or similar engines, that is keyed state. RocksDB state backends are common. Unbounded state for dead devices is a slow-moving outage.

Downsampling is a product decision

Ops debugging yesterday wants fine resolution. A monthly capacity chart wants coarse aggregates. Keeping 5-second raw forever at fleet scale becomes a storage and compaction tax. Healthy designs tier: raw for days, minutes for months, hours for years.

Burst reconnect

After a network partition, devices come back and flush buffers. Your gateway and Kafka must absorb a herd. Without admission control, you create a second outage during recovery.

The expanded problem

Build a system that:

  • Ingests about 200K events/sec (1M devices × 1 event every 5 seconds)
  • Computes per-device rolling averages over windows such as 5 to 15 minutes
  • Detects anomalies or threshold breaches and alerts
  • Stores time-series for charts
  • Handles offline devices and reconnect bursts

Constraints and scale prompts

Do the math out loud:

  • 200K events/sec × 200 bytes ≈ 40 MB/s raw ingest
  • Per day raw ≈ 40 MB/s × 86,400 ≈ 3.5 TB/day before compression and downsample
  • Cardinality ≈ 1M series baseline; multiply by metrics per device
  • Alert fanout must be bounded; noisy sensors can page humans to death

What good looks like

  • Lead with cardinality and reconnects, not only MB/s
  • Partition or key by device_id carefully, with hotspot controls
  • State TTL for inactive devices
  • Explicit retention tiers
  • Alert hysteresis and cooldown
  • Optional edge aggregation if devices can do it

Clarifying questions

  • How many metrics per device?
  • Alert latency SLA: seconds or minutes?
  • Retention at raw resolution versus downsampled?
  • Can devices aggregate on the edge?
  • How long may devices buffer offline?
  • Single region or global fleet with regional gateways?

Out of scope

  • Designing sensor firmware in hardware detail
  • Publishing a research-grade anomaly model paper
  • Building the entire device twin UI
  • Multi-cloud networking deep dive unless invited

A concrete story to keep in your head

A cold-chain pharmacy fridge fleet loses connectivity for 20 minutes in one city. When the radios return, 50,000 devices dump buffered temperature samples. Without gateway admission control, Kafka request latency spikes, stream jobs fall behind, and alert rules think the entire city is on fire because catch-up data looks like sudden change.

Your design must make recovery boring: absorb, meter, compute, and alert with cooldown. That story is what this problem is really about.