It is Friday at 5 pm. A team that produces data adds a new required field to a message, or renames a column from amt to amount_usd, and ships the change. A few minutes later, three other teams' programs, which read that data, start failing. The people who made the change have gone home.
In a company, many programs read the same stream or table. They are written by different teams on different schedules, and they all assume the data has a particular shape. That shape is called a schema. When it changes without care, the readers break.
Evolving a schema without downtime is mostly discipline: make changes that old readers can survive, check compatibility automatically, and replace risky changes, such as renames, with a slow, safe sequence.
Words you will meet
Schema
The agreed shape of the data: field names, types and which fields are required.
Producer and consumer
A producer writes data. A consumer reads it. One table or topic often has many consumers.
Schema registry
A service that stores every version of a schema and refuses changes that break compatibility rules.
Backward compatibility
Readers on the new schema can still read data written under the previous schema.
Default value
What a reader uses when a field is missing. A new field with a default is safe, a required field without one is not.
Expand and contract
Change in two safe phases: first add the new thing alongside the old (expand), then remove the old after everyone has moved (contract).
Dual-write
Writing both the old and the new field at the same time during a migration.
SELECT *
Selecting every column without naming them. It silently changes meaning whenever a column is added or reordered.
mergeSchema (Delta)
A Delta Lake option that automatically adds new columns when data arrives with them.
One unannounced change, three broken readers
Press Play to watch it move
Each arrow is one thing happening. Play it step by step, or read the list below.
1.On Friday afternoon the producer renames amt to amount_usd and deploys. Nothing stops them.
2.Consumer A expects a field called amt, cannot find it and crashes.
3.Consumer B uses SELECT * and gets different columns than before, so its output is silently wrong or its job fails.
The numbers, as things you can picture
Several independent consumersEach reads the same topic or table on its own schedule, so you cannot change all of them at the same moment.
What the interviewer wants
Show how to change a schema across producing, processing and storing without downtime, including how you would rename a field safely.
Interview room
Design schema evolution with registry compatibility, explicit columns (no SELECT *), and deliberate lake evolution without downtime.
Basics
Schemas are producer/consumer contracts. Compatibility modes: backward, forward, full. Expand/contract for safe change. SELECT * couples you to upstream whims. Blind mergeSchema can accept junk columns.
Registry mode named; expand/contract; dual fields for renames; explicit projections; support windows.
Clarify/scope
Format choice, approvers, consumer count, lake format. Out of scope: rewriting company to one stack Monday.
Scoring lens: Schema Evolution Without Downtime
Interviewers reward incident-first teaching, a clear control loop, and named owners. Tool logos without that loop score poorly.
Weak vs strong answers on Schema Evolution Without Downtime
Weak: list products. Strong: tell the failure story, define the mechanism, draw the path, state rollout and monitors.
Scope control
Say what is out of scope for v1. Come back only if follow-ups demand it. This is a seniority signal for Schema Evolution Without Downtime.
Clarifying then committing
Ask a few high-use questions about SLAs, owners, and scale, then sketch. Endless questions without a diagram look evasive.
Compatibility matrix on the board
Add optional field: usually OK. Add required without default: break. In-place rename: break. Delete while readers exist: break. Write this matrix before debating tools.
Dual-write operations
During rename windows, monitor both fields for null rates and mismatch rates. Alert if amount_usd is null while amt is present after the dual-write deploy. Migration is a data problem, not only a schema problem.
Consumer EOL playbook
Publish EOL date, list catalog consumers, page lagging owners weekly, then remove. Support windows without inventory are theater.
Interview framing detail 1
For lb-sd-26, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.