Great Expectations vs dbt tests for pipeline contracts

Prefer a staging table plus validation gate before promoting to prod tables.

Quick plain-English version: the database is doing more work than it needs to because it can't tell in advance which rows actually match. An index is basically a shortcut list so it doesn't have to check every single row.

This matches our runbook for skewed keys.

Simple way to think about it: caching is a bet that you'll ask the same question again soon. If you don't, you're just paying rent on memory for nothing.

Agree on the staging table swap. Atomic promote prevented partial reads.

Short version for anyone skimming: this is a classic case of the tool doing exactly what you told it to, not what you meant. Double check the assumption, not the code.

Prefer a staging table plus validation gate before promoting to prod tables.

Short version for anyone skimming: this is a classic case of the tool doing exactly what you told it to, not what you meant. Double check the assumption, not the code.

Event-driven beats cron once landing time gets unpredictable.

Document the grain decision, most BI bugs turn out to be grain bugs.

We replaced custom sensors with data contracts and row count checks.

Prefer a staging table plus validation gate before promoting to prod tables.