The interview room
Design a data quality framework for 500 tables and 200 daily pipelines with contracts-as-code, pipeline execution, monitoring, and circuit breakers that halt critical publishes.
Interviewers have watched optional DQ dashboards while finance ate bad data. They want trust tied to publish.
Basics
Quality dimensions: freshness, completeness, uniqueness, validity, consistency, accuracy. Schema-only is insufficient.
Wiki checklists do not scale to 200 pipelines at 2 AM. Expectations must be executable and owned.
Contracts
YAML/dbt/GX declarations with owner, SLA, tests, severity. Example halt contract on gold.fact_orders for PK/not-null/freshness.
Breakers and severity
Critical fail blocks consumer-visible publish; staging keeps evidence. Halt tier-0; warn lower tiers.
Prompt
Author, execute, monitor, gate, severity, gradual rollout, SLA-safe runtime, domain-authored tests.
Good looks like
Tier-0 first; contract→runner→results→gate→alerts; sampling strategy; flaky quarantine.
Clarify / scope
Tier-0 list, policy owners, engines, sampling. Out of scope: commercial DQ product fantasy.
Scoring lens: Data Quality Framework for 500 Tables
Interviewers reward incident-first teaching, a clear control loop, and named owners. Tool logos without that loop score poorly.
Weak vs strong answers on Data Quality Framework for 500 Tables
Weak: list products. Strong: tell the failure story, define the mechanism, draw the path, state rollout and monitors.
Scope control
Say what is out of scope for v1. Come back only if follow-ups demand it. This is a seniority signal for Data Quality Framework for 500 Tables.
Clarifying then committing
Ask a few high-use questions about SLAs, owners, and scale, then sketch. Endless questions without a diagram look evasive.
Results store and API
Store every test result with run_id, table, partition, test name, severity, status, metric, owner, and duration. Expose POST /runs and GET /runs/{id} so Airflow can gate publish on failed_critical. Without a results API, gates become brittle shell scripts.
Author experience
Ship tier templates. Domain engineers copy a Gold halt template, change column names, open a PR, and get CI validation. If authoring requires a platform ticket for every test, coverage dies.
Finance-grade example battery
For fact_orders: unique order_id on partition; not_null amount/currency; amount >= 0; freshness 4h; volume within weekday band; sample FK to dim_customer. That set prevents most silent revenue disasters without boiling the ocean.
Interview framing detail 1
For lb-sd-25, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 2
For lb-sd-25, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 3
For lb-sd-25, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 4
For lb-sd-25, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.