Phase 02 · Data processing · pandas

Pandas: Data Analysis with Python

Table transformations in Python, for data that fits on one machine.Analyse data with DataFrames from scratch: reading, filtering, grouping, merging, reshaping and cleaning, with a messy-data capstone. SQL comparisons at every step.
Start lesson 1What is pandas?
lessons
20
modules
5
stages
5
of learning
3h 34m

Why this track exists

You have a 200 MB export with inconsistent country codes, three date formats, and duplicate customer rows. Writing nested loops over that in plain Python takes 50 lines and runs slowly. The work is really table shaped: filter these rows, group by that column, join on this key. Pandas gives you the table operations of SQL inside Python, which is why most real pipeline transformation code looks like this.

Pandas is what you use for the awkward middle: too messy for SQL, too small to justify a Spark cluster. It is also the default tool for investigating a data quality complaint, because you can load a sample and poke at it in minutes.

What you need before starting

  • Python basics

    You need to be comfortable with variables, lists, dicts, and functions. If not, do the first three modules of Core Python first.

  • Helpful: any SQL

    Every operation here is compared to its SQL equivalent, so knowing SQL makes this track much faster. Not required.

The roadmap

5 stages, in the order they build on each other. Each stage lists the modules and lessons it covers, and what you should be able to do by the end of it.

01The DataFrame itself

Almost every pandas error a beginner hits comes from not knowing what a Series is, what the index is doing, or when they are looking at a copy instead of the original.

Getting Started with PandasFree

0/3

What pandas is, what a Series and a DataFrame are, and how to load data from files.

  1. What is pandas?10m
  2. Series & DataFrames8m
  3. Reading CSV, Excel, and JSON10m

By the end of this stage

You can load a file into a DataFrame, select rows and columns deliberately, and predict what an operation returns.

Where people get stuck

The index is the thing people ignore and then fight for weeks. Pay attention to it early and later joins and groupbys stop being mysterious.

02Cleaning real data

Real files have missing values, wrong types, duplicate rows, and inconsistent strings. This stage is the daily work of the job.

Cleaning, filtering and the pandas wayFree

0/4

Fix types and missing values, filter rows, then learn when apply is the wrong tool and how pandas compares with SQL, all on df_orders / df_events.

  1. Data cleaning & type casting10m
  2. Filtering & slicing8m
  3. apply and map12m
  4. Pandas versus SQL10m

By the end of this stage

You can take a messy export and produce a typed, deduplicated frame, and say what you did to every bad row rather than dropping them silently.

03Reshaping and combining

Data almost never arrives in the shape the question needs. Grouping, merging, pivoting, and long versus wide are the transformations that get it there.

By the end of this stage

You can group and aggregate, merge frames without silently multiplying rows, and reshape between long and wide on purpose.

Where people get stuck

A merge that duplicates rows is the same bug as a bad SQL join, and just as easy to miss. Check the shape before and after.

04When pandas starts to hurt

Pandas holds everything in memory. There is a size where your laptop stops coping, and knowing where that line is stops you from either wasting a cluster or crashing a job.

Scaling Up

0/3

Parquet and chunks, advanced groupby, and MultiIndex, still on a laptop-sized warehouse.

  1. Parquet, chunks, and out-of-core12m
  2. Advanced groupby: transform, filter, custom agg12m
  3. MultiIndex / hierarchical indexing12m

By the end of this stage

You can reduce memory with dtypes and chunking, and say clearly when a dataset has outgrown pandas and needs Spark.

05Production pandas

Notebook pandas and pipeline pandas are different disciplines. One is for exploring, the other has to run unattended and produce the same answer every time.

Production Pandas

0/4

Schema contracts, SQL round-trip thinking, text cleaning, and a messy-data capstone.

  1. Data validation & schema contracts12m
  2. SQL ⟷ pandas round-trip10m
  3. The .str accessor for text cleaning12m
  4. Capstone: messy data to analysis-ready16m

By the end of this stage

You finish with a silver layer transformation that handles messy input reproducibly, which is exactly the shape of real transformation code.

Practice what Pandas: Data Analysis with Python teaches

Each card is one idea. Watch it move, read the one-minute explanation, then work through problems from easiest.

Finish this track, get a certificate

Complete all 20 lessons and claim a certificate with a link you can add to LinkedIn or send to a recruiter.

How certificates work

Mistakes to avoid on this track

Common mistakes on this track and what to do instead
Common mistakeWhat to do instead
Using pandas for something SQL should do in the warehouse.If the data is already in the warehouse and the operation is a join or an aggregate, do it there. Pulling it out to a laptop is slower and harder to schedule.
Dropping nulls with dropna() without checking what was lost.Count first. Silently deleting 30 percent of your rows is a data incident, not a cleaning step.
Writing loops over rows with iterrows.Vectorised operations are both faster and clearer. If you are looping, there is usually a column operation you have not found yet.

Where to practise this

Where to go after this

Pandas: Data Analysis with Python reviews & rating

Finished a few lessons? Tell us what worked and what was confusing. Reviews are read by hand and help decide what gets rewritten next.