Day 17Phase 03 · Data Engineering Fundamentals
Phase 3 checkpoint: build a mini warehouse
The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.
By the end of today you can
- Build an end-to-end medallion pipeline: raw CSV → bronze → silver clean → gold dimensional star schema.
- Implement an SCD Type 2 dimension update with deterministic surrogate keys.
- Run automated reconciliation queries verifying row counts, uniqueness, and referential integrity.
The Medallion Warehouse Flow: Bronze to Silver to Gold
Raw data from apps, files and partner APIs has duplicates, negative amounts, missing IDs and badly formatted dates.
The Medallion Architecture cleans it in 3 steps, so no step has to do everything.
This is the first section of the lesson. The rest unlocks with a pass.
Core topics: medallion pipeline, bronze standardization, silver cleaning & dedup, gold star schema, SCD2 dimension update, reconciliation checks
Build: Mini warehouse platform: raw CSV → bronze → silver clean → gold star schema + SCD2 dimension in DuckDB.
Phase 3 Checkpoint: build the complete local analytical warehouse with automated acceptance tests.
Already have a pass? Go to your account. Need a pass? See pricing.