Learn data engineering

From complete beginner to job-ready data engineer.

12 tracks in five phases, from your first Python line to designing a pipeline. Every lesson has code you run and an exercise that checks your answer, right here in the browser. When a lesson is done, it points you to the problems that test it.

Not sure where to begin?

Answer two quick questions and LakeBench picks your first lesson and one free practice ticket. No mentor, no guesswork, no account needed.

  1. 1Pick your goal
  2. 2Read one lesson
  3. 3Solve one ticket

01Foundations

Python basics and SQL against a warehouse. Start here if you are new.

python

Python from Zero

Start from zero. Python basics, file handling, APIs and a production ingest capstone, with data engineering explained along the way. No prior experience needed.

43 lessons · 11 modules · ~544 min

What is inside · 11 modules
  • Getting Started(3)
  • Foundations(3)
  • Flow, functions & files(5)
  • APIs, streams & objects(3)
  • Time & validation(2)
  • Text & Patterns(2)
  • Reliable Pipelines(4)
  • Packaging & Config(3)
  • Testing & Capstone(2)
  • Databases, Cloud & Profiling(3)
  • DSA for Data Engineers(13)

sql

SQL from Zero to Advanced

Start with what data and databases are, then learn SELECT, JOINs, GROUP BY, window functions and star schemas, with a fact-table capstone, all in the SQL editor.

33 lessons · 6 modules · ~389 min

What is inside · 6 modules
  • Understanding Data(4)
  • Relational core(6)
  • Windows, cleaning & ops(5)
  • Advanced Querying(6)
  • Warehouse & Dimensional Modeling(6)
  • Performance & Production SQL(6)

pandas

Data Analysis Foundations

Learn to think like an analyst: turn a vague question into a table question, check the data, learn the statistics behind a trustworthy number, compare groups fairly, test whether a difference is real and write a finding. All in the pandas editor, no setup.

18 lessons · 5 modules · ~238 min

What is inside · 5 modules
  • Think like an analyst(2)
  • Trust the numbers(3)
  • Statistics: spread, shape and links(4)
  • Compare, spot trends, report(3)
  • Statistics: chance, samples and tests(6)

02Data processing

Single-machine pandas, then distributed PySpark when 1 node is not enough.

pandas

Pandas: Data Analysis with Python

Analyse data with DataFrames from scratch: reading, filtering, grouping, merging, reshaping and cleaning, with a messy-data capstone. SQL comparisons at every step.

20 lessons · 5 modules · ~214 min

What is inside · 5 modules
  • Getting Started with Pandas(3)
  • Cleaning, filtering and the pandas way(4)
  • Transforms & shape(6)
  • Scaling Up(3)
  • Production Pandas(4)

pyspark

PySpark from Basics to Big Data

Start with PySpark basics, then handle data too big for one machine: Spark architecture, DataFrames, joins, the medallion pattern and a bronze-to-gold capstone. Pandas translations included.

21 lessons · 5 modules · ~273 min

What is inside · 5 modules
  • PySpark Architecture(5)
  • Mental model & DataFrames(3)
  • Aggregations, windows & nested data(6)
  • Performance at Scale(3)
  • Production Spark(4)

03Production pipelines

Schedulers, analytics engineering, and quality gates on real queries.

orchestration

Orchestration & Reliable Pipelines

Why pipelines need schedulers, how DAGs work, retries, idempotent loads, backfills, and CDC. Simulated runner (no Airflow scheduler in this tab).

14 lessons · 6 modules · ~180 min

What is inside · 6 modules
  • What is a Pipeline(3)
  • DAG Mental Model(3)
  • Retries, Idempotency, Dedup(3)
  • Backfills, Catchup, Intervals(2)
  • CDC Concepts(2)
  • Capstone(1)

dbt

dbt & Analytics Engineering

What analytics engineering is, how dbt compiles and tests SQL models, staging layers, marts, and incrementals. SQL runs in the editor; no dbt CLI needed.

14 lessons · 6 modules · ~180 min

What is inside · 6 modules
  • Analytics Engineering(3)
  • What dbt Is(2)
  • Staging(2)
  • unique and relationships(2)
  • Marts and Incremental(4)
  • Capstone(1)

quality

Data Quality & Observability

Why data quality matters, how to catch bad data before it reaches dashboards, quality gates, lineage, SLAs, and a publish-suite capstone.

12 lessons · 6 modules · ~148 min

What is inside · 6 modules
  • What is Data Quality(2)
  • Quality Gates(3)
  • More checks(3)
  • Suites and lineage(2)
  • SLA and SLI(1)
  • Quality capstone(1)

04Modern data platforms

Cloud storage and warehouses, streaming concepts, and infra practices.

cloud

Cloud Platforms for Data Engineers

Provider-neutral cloud fundamentals with balanced AWS, Azure, and GCP depth: IAM, object storage, warehouses, orchestration, compute, and cost governance.

14 lessons · 4 modules · ~178 min

What is inside · 4 modules
  • What is the Cloud(4)
  • Cloud Mental Model(3)
  • Managed Data Services(4)
  • Cost, Security, and Shipping It(3)

streaming

Streaming & Message Queues

Why streaming exists, how message queues work, partitions, offsets, consumer groups, late data, and replay. Simulated topics (no broker in this tab).

13 lessons · 6 modules · ~174 min

What is inside · 6 modules
  • Batch versus Streaming(2)
  • Partitions, Offsets and Groups(3)
  • Delivery Promises(2)
  • Time(2)
  • Architectures and Replay(3)
  • Capstone(1)

python

Infrastructure & Engineering Practices

Git, Docker, Kubernetes, Terraform, and CI/CD explained from scratch with hands-on critique exercises. The tools run on your laptop; the concepts run here.

13 lessons · 5 modules · ~176 min

What is inside · 5 modules
  • Terminal, Git and Docker basics(6)
  • Docker(2)
  • Kubernetes and Terraform(2)
  • CI/CD(2)
  • Capstone(1)

05System design for interviews

Pipeline design tradeoffs and interview-style reasoning, not staff-level architecture.

design

Data Engineering System Design

How to design data systems at scale: batch vs streaming, 500M events/day case studies, cost analysis, and interview preparation. Diagrams and tradeoffs.

9 lessons · 5 modules · ~158 min

What is inside · 5 modules
  • How to design(2)
  • 500M events/day(3)
  • Serving and cost(2)
  • Interview(1)
  • Capstone(1)