Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. System Design

System design interview

Data Lineage Tracking

MediumPro55 min read

Column-level lineage from source to dashboard for impact analysis and compliance; DataHub/OpenMetadata.

system-designlineagecataloggovernance

Interview room

Design column-level lineage source→dashboard for impact analysis and audits; discuss DataHub/OpenMetadata.

Basics

Lineage is a graph of datasets and jobs. Table-level vs column-level. Wiki lineage rots. OpenLineage emitters feed catalogs. Impact analysis and provenance are the killer apps.

Prompt

Automate capture, answer impact/provenance, catalog UX, freshness of lineage, include BI.

Good

Emitters, column edges on tier-0, owners, BI connectors, honest notebook gaps.

Clarify/scope

Stack coverage, column-level priority, existing catalog, PII tags.

Scoring lens: Data Lineage Tracking

Interviewers reward incident-first teaching, a clear control loop, and named owners. Tool logos without that loop score poorly.

Weak vs strong answers on Data Lineage Tracking

Weak: list products. Strong: tell the failure story, define the mechanism, draw the path, state rollout and monitors.

Scope control

Say what is out of scope for v1. Come back only if follow-ups demand it. This is a seniority signal for Data Lineage Tracking.

Clarifying then committing

Ask a few high-use questions about SLAs, owners, and scale, then sketch. Endless questions without a diagram look evasive.

Impact ticket template

When changing a column, the catalog should generate: downstream models, dashboards, owners, last lineage emit time, and suggested notification list. Engineers paste that into the change ticket.

Emitter SRE

Lineage is only as fresh as emitters. Alert when a tier-0 job has not emitted OpenLineage in N hours. A stale graph is worse than no graph because it creates false confidence.

BI inclusion

If Looker/Tableau/Power BI datasets are missing, impact analysis lies. Budget connectors for the top BI tool first.

Interview framing detail 1

For lb-sd-27, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.

Interview framing detail 2

For lb-sd-27, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.

Interview framing detail 3

For lb-sd-27, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.

Interview framing detail 4

For lb-sd-27, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.

Interview framing detail 5

For lb-sd-27, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.