Interview room
2000 small 1MB Parquet files after streaming writes: fix via compaction and prevent via batching/auto-optimize.
Basics
Many tiny files hurt listing/planning. Streaming triggers create them. Compaction rewrites larger files under snapshots. Prevention: longer triggers, coalesce carefully, scheduled rewrites.
Prompt
Explain pain, compact safely, prevent, monitor.
Good
Planner pain; OPTIMIZE; knobs; target size; vacuum.
Clarify/scope
Table format; freshness vs size.
Scoring lens: Small Files Problem in Lakehouse
Interviewers reward incident-first teaching, a clear control loop, and named owners. Tool logos without that loop score poorly.
Weak vs strong answers on Small Files Problem in Lakehouse
Weak: list products. Strong: tell the failure story, define the mechanism, draw the path, state rollout and monitors.
Scope control
Say what is out of scope for v1. Come back only if follow-ups demand it. This is a seniority signal for Small Files Problem in Lakehouse.
Clarifying then committing
Ask a few high-use questions about SLAs, owners, and scale, then sketch. Endless questions without a diagram look evasive.
Target size as a monitored SLO
Track avg file size and files per partition. Alert when a daily partition exceeds a file-count budget. File metrics belong next to lag metrics.
Sealed vs open partitions
Compact yesterday aggressively. Treat the open streaming partition gently to reduce commit conflicts. Say this split schedule.
Vacuum discipline
Compaction without vacuum/retention cleanup burns storage. Pair rewrite jobs with expiration policy.
Interview framing detail 1
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 2
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 3
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 4
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 5
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.
Interview framing detail 6
For lb-sd-33, spend the first minutes making the problem concrete: who gets hurt when this fails, what SLA is implied, and what is explicitly out of scope. Then teach the prerequisite concept before proposing boxes. A candidate who names vendors first usually loses the plot. Write two clarifying questions on the board and answer them with assumptions if the interviewer shrugs. Keep the first diagram small enough to redraw when a follow-up changes a constraint.