Phase 01 · Foundations · python

Python from Zero

The language you will write almost every pipeline in.Start from zero. Python basics, file handling, APIs and a production ingest capstone, with data engineering explained along the way. No prior experience needed.
Start lesson 1Your first Python program
lessons
43
modules
11
stages
7
of learning
9h 4m

Why this track exists

A company has data in 100 places: a CSV a vendor emails every morning, a payments API, a database another team owns. Somebody has to fetch it, fix the broken rows, and put it somewhere useful, on a schedule, without being woken up at 3 a.m. every time it fails. Doing that by hand does not scale past a week. Python is how you write it down once and let a machine repeat it.

Most data engineering job posts list Python first. In practice you use a small slice of it constantly: read a file, loop over records, call an API, handle the error, write a log line, exit with the right code. This track teaches that slice properly instead of teaching the whole language badly.

What you need before starting

  • Nothing about programming

    This track starts at what a variable is. If you can use a computer and type, you have enough.

  • Nothing about data engineering

    The first lesson explains what the job actually is before any code appears.

The roadmap

7 stages, in the order they build on each other. Each stage lists the modules and lessons it covers, and what you should be able to do by the end of it.

01Orientation

Before writing code, understand what a data engineer is paid to do and why Python is the tool. Skipping this is why beginners write scripts with no idea what problem they serve.

Getting StartedFree

0/3

What data engineering is, why Python is the language of choice, and how this track works.

  1. Your first Python program8m
  2. What is data engineering?10m
  3. Why Python for data engineers?8m

By the end of this stage

You can explain in your own words what a pipeline is and what part of it Python handles.

02The language core

Every transformation you will ever write is built from four things: values, decisions, repetition, and functions. Get these solid and the rest of the track is just vocabulary.

FoundationsFree

0/3

Variables, types, data structures, and the building blocks of every Python program.

  1. Variables, types & type hints8m
  2. DE data structures12m
  3. Decisions and loops: if, for and while12m

Module checkpoint

2 conceptual questions to verify mastery

Flow, functions & filesFree preview

0/5

Loops, retries, reusable utilities, and reading warehouse files safely.

  1. Control flow & error handlingFree10m
  2. Functions, modules & importsFree10m
  3. Strings and textFree10m
  4. File I/O & data formatsFree12m
  5. Working with JSONFree10m

By the end of this stage

You can read a CSV off disk, loop through it, clean each row, and write the result back out.

Where people get stuck

Dictionaries and lists of dictionaries are the shape almost all pipeline data takes. If that shape still feels fuzzy, redo the exercises before moving on. Everything later assumes it.

03Real data sources

Files on your laptop are the easy case. Real sources are HTTP APIs that paginate, time zones that lie, and fields that are sometimes null and sometimes the string 'null'.

APIs, streams & objectsFree preview

0/3

Talk to services, stream huge files, and model connectors as classes.

  1. Working with REST APIs12m
  2. Iterators & generators (yield)Free12m
  3. OOP for pipeline engineering12m

Module checkpoint

1 conceptual questions to verify mastery

Time & validation

0/2

UTC intervals and rejecting dirty rows before they hit the lake.

  1. Working with dates & timestamps10m
  2. Data validation with Pydantic12m

By the end of this stage

You can pull data from a REST API page by page, parse timestamps correctly, and reject records that fail validation instead of silently corrupting the output.

Where people get stuck

Time zones and naive datetimes cause more production bugs than any other single thing in this stage. Slow down on that lesson.

04Code that survives production

A script that works once is not a pipeline. A pipeline runs unattended, fails safely, retries the transient errors, and does not double-write when it runs twice.

Text & PatternsFree preview

0/2

Parse logs and survive encoding surprises at the ingest boundary.

  1. Regular expressions for logsFree12m
  2. String encoding & unicode gotchas12m

Reliable Pipelines

0/4

Logging, cleanup, retries, and an honest mental model of concurrency.

  1. Logging instead of print-debugging12m
  2. Context managers & resource cleanup10m
  3. Retries, backoff, and idempotency14m
  4. Concurrency, asyncio, and the GIL14m

Module checkpoint

1 conceptual questions to verify mastery

Packaging & Config

0/3

12-factor config, pinning, and a CLI entrypoint, including what this browser cannot do.

  1. Config & secrets management10m
  2. Dependency management & pinning10m
  3. Building a pipeline CLI12m

By the end of this stage

You can write a job with logging, retries with backoff, config separated from code, and idempotent behaviour on rerun.

Where people get stuck

Idempotency is the concept most people nod along to and then get wrong. If you cannot explain what happens when your job runs twice on the same input, you have not got it yet.

05Databases, cloud, and profiling

Pipelines read from databases and write to object storage, and the slow ones get fixed by measuring instead of guessing. This stage covers sqlite3, boto3, and the profiling tools.

Databases, Cloud & Profiling

0/3

Connect to databases with sqlite3, interact with cloud object storage via boto3, and profile performance with timeit and cProfile.

  1. Databases from Python: sqlite3 & SQL engines15m
  2. Cloud SDK from Python: S3 and object stores15m
  3. Profiling: timeit, cProfile & memory tracking15m

By the end of this stage

You can query a database from Python, move a file to and from cloud storage, and find which line is making a job slow.

06Algorithms for pipelines

Interview rounds and real pipelines both punish quadratic loops. This stage applies complexity, buffers, windowing, and graph ordering to data work.

By the end of this stage

You can say what a piece of code costs as the input grows, and replace a nested loop with a hash lookup or a heap when it matters.

07Tests and the capstone

The difference between a script and something a team will let near production is usually a test suite and a clean entrypoint.

Testing & Capstone

0/2

Prove transforms with tests, then assemble an ingest script that uses the whole track.

  1. Unit testing data transforms12m
  2. Capstone: ingest script end to end18m

By the end of this stage

You finish with an ingest job you can put in a repo and talk about in an interview: it takes a real source, validates, transforms, loads, logs, and has tests.

Finish this track, get a certificate

Complete all 43 lessons and claim a certificate with a link you can add to LinkedIn or send to a recruiter.

How certificates work

Mistakes to avoid on this track

Common mistakes on this track and what to do instead
Common mistakeWhat to do instead
Learning Python from a general tutorial that spends 3 weeks on classes and decorators.Data engineering Python is mostly files, dicts, requests, and error handling. Learn that first, and go deeper only when a real problem needs it.
Reading lessons without running the exercises.The exercises are where the learning happens. Reading code gives you recognition, writing it gives you recall, and interviews test recall.
Catching every exception with a bare except and moving on.That turns a loud failure into silent data loss. Catch the error you expect, and let the ones you did not expect crash the job so you find out.

Where to practise this

Where to go after this

Python from Zero reviews & rating

Finished a few lessons? Tell us what worked and what was confusing. Reviews are read by hand and help decide what gets rewritten next.