Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing

Pandas for Data Manipulation

Progress0/20
x

Getting Started with Pandas

  • What is pandas?10m
  • Pandas versus SQL10m
  • Reading CSV, Excel, and JSON10m
  • apply and map12m

Frames & cleaning

  • Series & DataFrames8m
  • Data cleaning & type casting10m
  • Filtering & slicing8m

Transforms & shape

  • Transformations & derived columns10m
  • Grouping & aggregations10m
  • Combining datasets10m
  • Reshaping & pivot tables10m
  • Time series in pandas10m
  • Memory optimization10m

Scaling Up

  • Parquet, chunks, and out-of-core12m
  • Advanced groupby: transform, filter, custom agg12m
  • MultiIndex / hierarchical indexing12m

Production Pandas

  • Data validation & schema contracts12m
  • SQL ⟷ pandas round-trip10m
  • The .str accessor for text cleaning12m
  • Capstone: messy data to analysis-ready16m
Back to track
  1. Learn
  2. Pandas for Data Manipulation
  3. Getting Started with Pandas
  4. Reading CSV, Excel, and JSON

Lesson 3 of 20 · Theory first, then run it

Reading CSV, Excel, and JSON

pandasbeginner10 min

Overview

pd.read_csv, read_excel, and read_json load files into DataFrames. In this tab the frames are already loaded.

On this page7 sections›
  1. 1What you will do
  2. 2Why this skill
  3. 3How the code works
  4. 4Worked examples
  5. 5Common beginner questions
  6. 6What comes next
  7. 7Practice

What you will do

Before you can filter or group, the table has to exist in Python. On your machine you create a DataFrame by reading a file. Pandas has a reader for each common format: CSV, Excel, and JSON.

pd.read_csv opens a text file of rows and commas. pd.read_excel opens a workbook and a sheet. pd.read_json opens a JSON file and tries to line the objects up as rows. Each function returns a DataFrame.

In this tab you do not read a file from disk. The studio already built df_orders and the other frames. You still need to know the readers, because that is what you will type at work. Here you practice inspecting a frame that is already loaded.

Why this skill

A payments vendor emails a CSV every morning. Finance sends an Excel workbook with three sheets. The website team dumps events as JSON. If you cannot load those files, you cannot clean them. The rest of pandas is useless until the table is in memory.

Wrong load options produce silent garbage: a header treated as a data row, a date column left as text, or a semicolon-separated European CSV split on the wrong character. Learning the readers now saves hours of 'why is everything object dtype' later.

How the code works

File on disk to DataFrame in memory
orders.csv / .xlsx /.jsonpd.read_csv /read_excel /read_jsonDataFrame in memoryInspect shape andcolumns

Pick the reader that matches the format. The output is always a DataFrame.

CSV is the default interchange format: one row per line, commas between fields, a header row of column names. Excel is a workbook. You name the sheet. JSON is nested. A list of objects becomes rows; nested keys may need flattening later.

Same destination (a DataFrame). Different readers because the files are different.

FunctionInputTypical extra argumentsReturns
pd.read_csv(path)Text table (.csv, .tsv)sep, header, dtype, parse_dates, encodingDataFrame
pd.read_excel(path)Workbook (.xlsx)sheet_name, header, dtypeDataFrame
pd.read_json(path)JSON file or stringorient, lines (JSON Lines)DataFrame
  1. Choose the reader that matches the file type.
  2. Pass options for separator, header row, encoding, and types when the defaults are wrong.
  3. Print shape, columns, and dtypes before you transform anything.
  4. Assign the frame you want to display to result.

Those four steps are the laptop workflow. In this editor the first two steps already happened. df_orders is waiting. Your job is the last two: inspect, then assign a preview to result.

What a successful load looks like
order_idorder_statusorder_totalO-104paid84.20O-109pending31.00O-116paid112.40

Named columns, typed values, one order per row. That is the goal of read_csv.

Worked examples

Run the example below in this tab. Read the input, follow the code, then check the output matches what you expect.

PythonHow read_csv looks at work, then inspect the preloaded frame here
import pandas as pd

# Conceptual laptop workflow (there is no orders.csv in this tab):
# orders = pd.read_csv("orders.csv")
# sheet = pd.read_excel("orders.xlsx", sheet_name="Sheet1")
# events = pd.read_json("events.json")

print("Use the preloaded frame instead:")
print(df_orders.columns.tolist())
result = df_orders.head(5)

The commented lines are what you will write on your machine. Uncommenting them here would fail because those paths do not exist. The live lines print column names and keep five rows so you can confirm the load (in this tab, the preload) worked.

PythonInspect a loaded frame: columns, shape, types, preview
print(df_orders.columns.tolist())
print(df_orders.shape)
print(df_orders.dtypes)
result = df_orders.head(5)

columns.tolist() is the contract of the table: the names you can select. shape is the size. dtypes tells you whether a 'number' column actually loaded as text. head(5) is a short preview for the grid.

Headers, separators, and encoding

If the first row of data becomes the column names, you passed the wrong header. If one column contains commas and everything else, the separator is not a comma (try sep=';'). If you see odd characters, name the encoding (often utf-8 or latin-1). Pandas cannot guess a file you have not described.

SQL connection

pd.read_csv is the laptop cousin of loading a file into a warehouse table (COPY, LOAD, or an external table). SQL then queries the table. Pandas then queries the DataFrame. Different engine, same sequence: land the file, then select from it.

Copy-paste without reading the output

Run Sample first. If the numbers or row count look wrong, stop and re-read the previous section before changing code.

Common beginner questions

Why can't I read a CSV in this tab?

The studio already loaded the lesson frames. There is no orders.csv path to open. Practice inspection on df_orders. On your laptop, pd.read_csv("orders.csv") is the first line.

What is the difference between CSV and Excel?

CSV is plain text: one sheet, no formatting, easy to version. Excel can have multiple sheets, types, and formatting. Prefer CSV for pipelines. Use read_excel when the source only sends workbooks.

Will read_json always give a clean table?

Only when the JSON is already row-shaped (a list of objects with the same keys). Nested payloads often need extra flattening. The PySpark track covers exploding nested structs when the file is huge.

What comes next

You can load and preview a table. The next lesson covers how to derive new columns: map, apply, and the faster vectorized comparisons you should prefer.

Practice

Run Sample to print columns and preview five rows. Then complete Exercise: print df_orders.columns.tolist() and assign df_orders.head(5) to result.

The grid should show exactly five rows. result must stay a DataFrame.

Practicals · load into the editor

After you read the theory, run these in the pane on the right. They execute in this tab, no cluster.

Rate:
Was this useful?
Pandas versus SQLapply and map