Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. A query got 10x slower overnight

SQL · Performance & Internals

A query got 10x slower overnight

Hardsql-63
scenarioperformancequery-plantroubleshooting

Question

A dashboard query that ran in 20 seconds yesterday takes 4 minutes today. Nothing in the SQL changed. How do you investigate?

Solution

Start by finding out what changed, not by guessing. If the SQL is the same, then something around it changed: the data, the statistics, the plan, or the resources. The cheapest evidence comes first.

Step 1: compare yesterday's run with today's

Pull the query profile or plan for both runs (query history in Snowflake or BigQuery, EXPLAIN ANALYZE in Postgres). Look at three numbers: rows and bytes scanned, time per stage, and the shape of the plan. If bytes scanned went from 20 GB to 2 TB, you already know it is a pruning or data issue and not a compute issue.

Step 2: the usual suspects, in order

  • Data volume or skew. Did a table suddenly double? Did one customer_id or a NULL key become huge after a bad upstream load? A skewed join key makes one worker do most of the work.
  • Lost pruning. An upstream change turned a DATE column into a STRING, so WHERE order_date >= '2025-01-01' can no longer use the partition. The query is correct and scans everything. Check the partitions or micro-partitions scanned versus total.
  • Stale statistics (Postgres, Redshift, SQL Server). After a big load, the planner may still think the table is small and choose a nested loop join. ANALYZE and rerun.
  • Cache. Yesterday's 20 seconds may have been a result or data cache hit. Today the cache was cold or invalidated by new data.
  • Contention. The warehouse or slot pool is busy with other queries. In OLTP databases look for lock waits and a long-running transaction blocking you.

Step 3: change one thing at a time

If the plan changed, force the old shape on a copy (hint, or rewrite), and see if the time returns. If bytes scanned is the problem, fix the filter. If it is queueing, look at concurrency, not at the SQL.

Keep it from happening again

Add a data volume check and a freshness check on the inputs, so a surprise table doubling alerts you before the dashboard does. Keep query history so "yesterday's profile" exists. Alert on bytes scanned per query for the top dashboards, which catches lost pruning quickly.

Say out loud that you check bytes scanned before touching the SQL. That one habit separates a methodical answer from a guess.

🎯 Put this concept into practice

Solidify this answer with real hands-on interview drills in the browser studio.

Open related drill →
PreviousNext