Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Circuit breaker for data pipelines

Data quality · Contracts & Ops

Circuit breaker for data pipelines

Hardquality-15
circuit breakerquality gatefail fastpublish

Question

What is a circuit breaker pattern in data pipelines / data quality?

Solution

A circuit breaker stops downstream work when upstream data looks dangerously bad, so you do not publish poison into marts and dashboards.

Inspired by resilience patterns in services: after repeated failures (or a severe quality breach), open the circuit, fail fast, and wait for recovery instead of hammering a broken dependency.

CLOSED (normal):  ingest -> transform -> publish
OPEN (tripped):   ingest fails quality gate -> STOP publish
                  alert owners; do not refresh marts
HALF-OPEN:        allow a probe run after fix / cooldown

Data examples that trip the breaker

  • Freshness breach beyond critical threshold
  • Row count drop of 90% vs baseline
  • Contract / schema validation failure
  • Reconciliation mismatch above tolerance

Why it matters

Publishing a half-empty fct_orders can be worse than serving slightly stale yesterday data. Executive dashboards and ML features will amplify the bad batch.

Interview tip: "Circuit breaker = stop the blast radius when quality is critically bad." Contrast with silent publish + hope someone notices.

PreviousNext