Day 13Phase 03 · Data Engineering Fundamentals

How data moves: ETL, ELT, batch and streaming

The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.

By the end of today you can

  • Explain the architectural differences between ETL and ELT and identify where compute executes in each.
  • Decide between batch processing, micro-batching, and streaming based on latency requirements and cost.
  • Trace the flow of raw data through bronze, silver, and gold medallion storage layers.

What is a data pipeline?

A data pipeline is a set of automatic steps that takes data from the place where it is created, cleans it on the way, and delivers it to the place where people use it.

By hand you could download one CSV file a week. A company with new orders every second cannot.

This is the first section of the lesson. The rest unlocks with a pass.

Core topics: ETL vs ELT, batch vs streaming, micro-batching, medallion stages, raw immutable storage, replay recovery

Build: Multi-source ingestion → immutable partitioned raw storage → dual ETL vs ELT comparison → analytics datamart.

System trade-offs: compute costs of pre-load transforms vs in-warehouse SQL, and raw storage as replay insurance.

Already have a pass? Go to your account. Need a pass? See pricing.