Day 21Phase 04 · Cloud
Warehouses: Redshift, Synapse & BigQuery
The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.
By the end of today you can
- Explain Massively Parallel Processing (MPP) architecture and how data distributes across compute slices.
- Choose distribution styles (KEY, EVEN, ALL) and sort keys to avoid network data movement.
- Execute high-speed bulk ingestion via staging tables and atomic MERGE transactions.
MPP Architecture: Why Warehouses Scale Beyond Single Nodes
Joining 500 million orders against 20 million customers on one relational server eventually runs out of RAM and disk bandwidth, even with 64 CPU cores.
Cloud warehouses (Amazon Redshift, Azure Synapse Dedicated, Google BigQuery) use massively parallel architectures to execute queries across distributed nodes, distributions, and slots.
This is the first section of the lesson. The rest unlocks with a pass.
Core topics: Warehouse MPP & serverless architectures, Distribution & clustering, Sort keys & zone maps, columnar compression, bulk loading patterns, atomic staging merges
Build: Cloud warehouse engine: bulk parallel load → staging deduplication → atomic idempotent merge transaction.
Warehouses don't enforce primary key constraints on write. Guarantee idempotency using staging tables and atomic transactions.
Already have a pass? Go to your account. Need a pass? See pricing.