Day 17Phase 03 · Data Engineering Fundamentals

Phase 3 checkpoint: build a mini warehouse

The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.

By the end of today you can

  • Build an end-to-end medallion pipeline: raw CSV → bronze → silver clean → gold dimensional star schema.
  • Implement an SCD Type 2 dimension update with deterministic surrogate keys.
  • Run automated reconciliation queries verifying row counts, uniqueness, and referential integrity.

The Medallion Warehouse Flow: Bronze to Silver to Gold

Raw data from apps, files and partner APIs has duplicates, negative amounts, missing IDs and badly formatted dates.

The Medallion Architecture cleans it in 3 steps, so no step has to do everything.

This is the first section of the lesson. The rest unlocks with a pass.

Core topics: medallion pipeline, bronze standardization, silver cleaning & dedup, gold star schema, SCD2 dimension update, reconciliation checks

Build: Mini warehouse platform: raw CSV → bronze → silver clean → gold star schema + SCD2 dimension in DuckDB.

Phase 3 Checkpoint: build the complete local analytical warehouse with automated acceptance tests.

Already have a pass? Go to your account. Need a pass? See pricing.