Projects
Build projects you can put on your CV.
Every project explains the problem, the data and the steps, then leaves the building to you. Smaller ones run in your browser and check your work. The bigger ones you build on your own machine.
- 58 projects
- 49 run in your browser
- 6 final-year builds
Showing 58 of 58 projects
- PythonGuided build
CLI Expense Tracker with File I/O
Build a structured expense tracker in Python that parses transaction records, computes categorical subtotals, and exports financial summaries.
- Core Python
CV: “Built a Python script that reads expense records with the csv module and prints totals and shares by category.”
Beginner45 minRuns in your browserStart building - PythonGuided build
Web Server Log Parser & Metrics Aggregator
Parse Common Log Format web server access logs using regex, extract status codes and IP addresses, and calculate error rates.
- Core Python
CV: “Parsed web server access logs in Python with a regular expression and collections.Counter to report error rates and the busiest clients.”
Beginner45 minRuns in your browserStart building - PythonGuided build
HTTP API Client with Truncated Exponential Backoff
Build a resilient HTTP client wrapper with truncated exponential backoff and jitter to safely retry transient network errors.
- Core Python
CV: “Wrote a Python retry function with exponential backoff and a cap that retries 429 and 5xx responses and stops at once on other 4xx errors.”
Beginner45 minRuns in your browserStart building - PythonGuided build
Order Lifecycle Finite State Machine
Build a deterministic Finite State Machine (FSM) in Python to enforce valid order transitions, guard against invalid mutations, and maintain audit trails.
- Core Python
CV: “Modelled an order lifecycle as a state machine in Python with an allowed-moves table, an error for illegal moves and an audit log.”
Beginner45 minRuns in your browserStart building - PythonGuided build
CSV Ingestion Validator with Custom Exceptions
Build a robust schema and data validator in Python that categorizes bad records using a custom exception hierarchy and routes defects to a quarantine log.
- Core Python
CV: “Built a CSV validator in Python with custom exception classes that loads good rows and quarantines bad ones with a reason.”
Beginner45 minRuns in your browserStart building - PandasGuided build
Data Cleansing Pipeline for Messy Sales Exports
Clean dirty e-commerce CSV exports in Pandas: strip currency symbols, standardize mixed dates, drop duplicates, and fill missing categories.
- Pandas Data Wrangling
CV: “Cleaned a messy sales export in pandas: removed duplicate orders, turned currency text into numbers and standardised dates.”
Beginner45 minRuns in your browserStart building - PandasGuided build
Multi-Level Executive Pivot Table Dashboard
Reshape flat transaction rows into multi-dimensional executive cross-tabulations in Pandas using pivot_table().
- Pandas Data Wrangling
CV: “Turned a long sales table into a region by category summary in pandas with pivot_table and row and column totals.”
Beginner40 minRuns in your browserStart building - PySparkGuided build
Distributed Log Tokenizer & Stopword Counter
Tokenize distributed server log files, strip punctuation, filter stopwords, and compute term frequency counts using native PySpark DataFrame transformations.
- PySpark & Distributed Compute
CV: “Counted word frequencies in raw log lines with PySpark DataFrame functions (split, explode, filter, groupBy) instead of collecting to the driver.”
Beginner45 minRuns in your browserStart building - Data EngineeringProTrack capstone
Capstone: ingest script end to end
Bring the track together in one small ingest job: fetch API pages, validate each row, log accepted and rejected counts, and return clean order IDs.
- Python from Zero
Beginner18 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: model a star from orders
Declare grain, aggregate payments first, and project a fact table from source headers.
- SQL from Zero to Advanced
Beginner18 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: messy data to analysis-ready
Duplicate keys, mixed types, gnarly dates, nulls: bronze to silver in pandas, same instinct as Spark medallion.
- Pandas: Data Analysis with Python
Beginner16 minRuns in your browserOpen capstone - Data EngineeringFinal-year
Daily air quality ETL pipeline
Pull data from a public API every day, load it safely, and chart it.
- Python
- REST API
- SQLite or PostgreSQL
- Airflow or cron
CV: “Built a scheduled ETL pipeline that ingests hourly air quality data from a public API for multiple cities, with retries, logging and idempotent loads, and a dashboard of trends and threshold breaches.”
Beginner4 weeksBuild on your laptopOpen the plan - AnalyticsFinal-year
College placement analytics
Clean messy student records and show placement rates by branch, skill and company.
- Python
- pandas
- SQL
- A dashboard tool such as Streamlit or Power BI
CV: “Built a placement analytics project in Python and SQL that cleans multi-source student records with documented rules and reports placement rates by branch, CGPA band and skill on an interactive dashboard.”
Beginner3 weeksBuild on your laptopOpen the plan - Data EngineeringFinal-year
Data quality monitor for incoming files
Check every new file against rules and send a clear pass or fail report.
- Python
- pandas
- Pydantic (optional)
- SQL
CV: “Built a data quality monitor in Python that validates incoming files with configurable rules (uniqueness, nulls, allowed values, ranges, row counts) and produces pass/fail reports, tested against planted errors.”
Beginner3 weeksBuild on your laptopOpen the plan - PythonGuided build
Warehouse Inventory Manager with Invariant Validation
Design object-oriented inventory management classes in Python enforcing stock constraints, replenishment, and order fulfillment.
- Core Python
CV: “Wrote a Product class in Python with a custom InsufficientStockError so stock can never go below zero, and processed a batch of orders with it.”
Intermediate45 minRuns in your browserStart building - PythonGuided build
Sliding Window Token Bucket Rate Limiter
Implement a stateful token bucket rate limiter in Python that throttles high-volume API requests and enforces tenant quotas.
- Core Python
CV: “Implemented a per-client token bucket rate limiter in Python that refills based on elapsed time.”
Intermediate45 minRuns in your browserStart building - PythonGuided build
Custom LRU Cache with TTL Expiration
Build a custom Least Recently Used (LRU) function decorator in Python with time-to-live (TTL) expiration and capacity eviction.
- Core Python
CV: “Wrote a Python decorator that caches results with least recently used eviction, a time to live, and hit and miss counts.”
Intermediate45 minRuns in your browserStart building - PythonGuided build
Dynamic Environment Config Loader with Secret Masking
Build a robust JSON configuration parser in Python that overrides settings with environment variables and masks secrets for safe logging.
- Core Python
CV: “Built a Python config loader that applies environment variable overrides and masks secret values before the config is logged.”
Intermediate45 minRuns in your browserStart building - PythonGuided build
Streaming N-Gram Frequency Text Analyzer
Build a streaming text analyzer in Python that computes n-gram word frequencies, handles empty inputs safely, and filters stop words.
- Core Python
CV: “Built a Python text analyser that cleans reviews, counts words and two-word phrases without stop words, and handles empty input.”
Intermediate45 minRuns in your browserStart building - SQLGuided build
User Retention and Churn Cohort Matrix
Build a monthly cohort grid in SQL that shows how many customers from each first-purchase month come back in later months.
- SQL & Warehousing
CV: “Built a monthly cohort retention grid in SQL with CTEs, calendar month offsets and conditional aggregation.”
Intermediate50 minRuns in your browserStart building - SQLGuided build
Clickstream Sessionization with Window Functions
Transform unstructured web clickstream event logs into discrete user sessions using LAG() window functions and cumulative sums.
- SQL & Warehousing
CV: “Split a clickstream into sessions in SQL using LAG() to measure idle gaps and a running sum to assign session ids.”
Intermediate45 minRuns in your browserStart building - SQLGuided build
Multi-Step E-Commerce Conversion Funnel
Quantify customer conversion rates and stage drop-offs across a 4-step e-commerce funnel using conditional aggregation and chronological ordering.
- SQL & Warehousing
CV: “Built a four-step conversion funnel in SQL that only counts users who reached each step in order, with drop-off percentages.”
Intermediate45 minRuns in your browserStart building - SQLGuided build
Financial Ledger Running Totals and Moving Averages
Calculate cumulative year-to-date net revenue and rolling 3-day moving averages in SQL using window frame specifications.
- SQL & Warehousing
CV: “Computed daily net revenue, a running total and a 3-day moving average in SQL with explicit window frames.”
Intermediate40 minRuns in your browserStart building - SQLGuided build
Top-N Selling Products per Category with Dense Rank
Rank and isolate top-performing catalog items within each retail category in SQL using DENSE_RANK() and CTE filtering.
- SQL & Warehousing
CV: “Listed the top 2 products by revenue in each category in SQL with DENSE_RANK() in a CTE, and compared it with ROW_NUMBER() on ties.”
Intermediate40 minRuns in your browserStart building - SQLGuided build
Idempotent Event Deduplication with Window Functions
Eliminate duplicate webhook events and resolve status conflicts deterministically using ROW_NUMBER() in SQL.
- SQL & Warehousing
CV: “Wrote a SQL query that keeps one authoritative row per payment from duplicated webhook events, using ROW_NUMBER() and an explicit tie-break order.”
Intermediate45 minRuns in your browserStart building - PandasGuided build
Customer RFM Segmentation and Scoring
Segment retail customer transaction histories into Recency, Frequency, and Monetary tiers in Pandas using groupby and qcut scoring.
- Pandas Data Wrangling
CV: “Scored customers on recency, frequency and monetary value in pandas and grouped them into simple segments.”
Intermediate50 minRuns in your browserStart building - PandasGuided build
IoT Sensor Telemetry Resampling and Imputation
Resample irregular high-frequency IoT temperature telemetry into uniform 1-minute time series intervals with forward-fill and interpolation in Pandas.
- Pandas Data Wrangling
CV: “Resampled irregular sensor readings to one value per minute in pandas and filled gaps with time interpolation.”
Intermediate45 minRuns in your browserStart building - PandasGuided build
Financial Reconciliation and Ledger Matching
Perform automated two-way financial ledger reconciliation in Pandas using outer joins, merge indicators, and discrepancy reporting.
- Pandas Data Wrangling
CV: “Reconciled a ledger against a bank statement in pandas with an outer merge and indicator=True, and reported every mismatch.”
Intermediate45 minRuns in your browserStart building - PandasGuided build
Statistical Outlier Detection & Truncation Report
Detect and isolate numerical transaction anomalies in Pandas using the robust non-parametric Interquartile Range (IQR) technique.
- Pandas Data Wrangling
CV: “Flagged unusual transaction amounts in pandas with interquartile range fences, and showed why a three standard deviation rule missed them.”
Intermediate40 minRuns in your browserStart building - PandasGuided build
DataFrame Memory Optimization & Downcasting
Compress massive DataFrames by downcasting 64-bit numerical types and converting high-cardinality strings to categorical dtypes.
- Pandas Data Wrangling
CV: “Reduced a DataFrame's memory in pandas by downcasting numbers and converting repeated text to category, and checked no values changed.”
Intermediate45 minRuns in your browserStart building - PandasGuided build
Runtime Schema Contract & Column Type Assertion Gate
Enforce strict runtime data quality contracts on incoming Pandas DataFrames before triggering downstream ingestion jobs.
- Pandas Data Wrangling
CV: “Wrote a reusable pandas check that stops a load when required columns are missing or have the wrong type.”
Intermediate40 minRuns in your browserStart building - PySparkGuided build
Skewed Transaction Join Optimization with Broadcast
Optimize slow SortMergeJoins between large fact tables and small dimension lookups by broadcasting the dimension to worker executors.
- PySpark & Distributed Compute
CV: “Changed a PySpark join from a sort merge join to a broadcast hash join with a broadcast hint, and confirmed it in the physical plan.”
Intermediate45 minRuns in your browserStart building - PySparkGuided build
Multi-Level Window Ranking & Deduplication
Deduplicate at-least-once distributed payment event streams by partitioning by merchant ID and ordering by timestamp with row_number().
- PySpark & Distributed Compute
CV: “Kept the newest event per merchant in PySpark with a window ordered by time and row_number(), instead of an arbitrary dropDuplicates.”
Intermediate45 minRuns in your browserStart building - PySparkGuided build
Incremental Micro-Batch Deduplication
Deduplicate streaming micro-batch events and retain the latest record per business key using PySpark window partitions.
- PySpark & Distributed Compute
CV: “Kept the latest status per order from duplicated, out-of-order events in PySpark using a window and row_number().”
Intermediate45 minRuns in your browserStart building - PySparkGuided build
Lakehouse Medallion Bronze-to-Silver ETL
Validate raw IoT sensor bronze data, quarantine corrupted records with dead-letter routing, and promote clean silver telematics.
- PySpark & Distributed Compute
CV: “Wrote validation rules in PySpark that promote clean sensor rows from Bronze to Silver and keep failing rows in a quarantine table with the reason.”
Intermediate45 minRuns in your browserStart building - PySparkGuided build
Nested Semi-Structured JSON Hierarchy Flattener
Parse deeply nested semi-structured JSON payloads with StructType schemas, explode item arrays, and extract customer metadata.
- PySpark & Distributed Compute
CV: “Flattened nested order JSON in PySpark with an explicit schema, from_json and explode, giving one row per ordered item.”
Intermediate45 minRuns in your browserStart building - PySparkGuided build
ACID Delta Lake Upsert & Merge Pipeline
Execute idempotent upsert merges on hospital patient records in PySpark, updating existing records and inserting new profiles.
- PySpark & Distributed Compute
CV: “Applied a daily change batch to a patient table in PySpark so updated rows are replaced, new rows added and every id appears once.”
Intermediate45 minRuns in your browserStart building - AnalyticsGuided build
Executive SaaS KPI Scorecard (MRR, ARPU, Churn)
Compute executive subscription health metrics in SQL: Monthly Recurring Revenue (MRR), ARPU, and subscriber churn percentage.
- Data Analysis & Metrics
CV: “Calculated MRR, ARPU and churn rate in SQL from a subscriptions table, counting only active subscriptions in revenue.”
Intermediate45 minRuns in your browserStart building - AnalyticsGuided build
Rigorous A/B Experimentation Readout & Significance
Perform two-sample hypothesis testing in Python evaluating conversion rates, relative uplift, z-scores, and p-values.
- Data Analysis & Metrics
CV: “Analysed an A/B test in Python: conversion rates, relative uplift, pooled standard error, z-score and p-value, with a plain-English conclusion.”
Intermediate45 minRuns in your browserStart building - AnalyticsGuided build
Multi-Factor Revenue Waterfall Bridge (Price vs Volume)
Decompose quarterly top-line revenue variances into volume, price, and mix effects using Pandas dataframes.
- Data Analysis & Metrics
CV: “Split a quarter-on-quarter revenue change into volume and price effects per product in pandas, and checked they add up to the total.”
Intermediate45 minRuns in your browserStart building - AnalyticsGuided build
Longitudinal Customer Lifetime Value & Cohort Decay
Model customer spend decay curves and project 36-month cumulative Customer Lifetime Value (LTV) using Pandas.
- Data Analysis & Metrics
CV: “Estimated 36-month cumulative customer value in pandas from six months of cohort spend, using an average monthly retention factor.”
Intermediate45 minRuns in your browserStart building - Data EngineeringProTrack capstone
Capstone part 2: incremental MERGE
Keep bronze → silver → gold, then apply a second micro-batch with a Delta-style latest-wins MERGE.
- PySpark from Basics to Big Data
Intermediate16 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Medallion pipeline project
A medallion pipeline moves from raw bronze data to cleaned silver data and then to a business-ready gold result. Build the 3 steps and check the grain at each one.
- PySpark from Basics to Big Data
Intermediate14 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: a reliable night job
Walk extract → load in order, skip duplicate keys, retry a flaky extract. Still a simulation. Still the right habits.
- Orchestration & Reliable Pipelines
Intermediate16 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: staging, mart, test
Layer a staging CTE, project a paid-orders mart, and return unique-test violators on order_id (zero in this warehouse).
- dbt & Analytics Engineering
Intermediate16 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: a publish suite
Wire not-null, unique, and row-count on df_orders. Print a summary dict of booleans. Project the frame you would actually publish.
- Data Quality & Observability
Intermediate16 minRuns in your browserOpen capstone - Data EngineeringFinal-year
Retail sales data warehouse with a dashboard
Turn a raw sales file into a star schema and a dashboard a manager can read.
- SQL
- DuckDB or PostgreSQL
- Python
- A dashboard tool such as Metabase or Power BI
CV: “Built a retail data warehouse (star schema) from 1M+ raw invoice rows using SQL and Python, with repeatable loads and row-count checks, and a dashboard showing revenue by month, product and country.”
Intermediate5 weeksBuild on your laptopOpen the plan - AnalyticsFinal-year
UPI transaction risk analytics
Spot unusual payments with SQL window functions and show them on a simple report.
- SQL
- DuckDB or PostgreSQL
- Python
- Streamlit or Metabase
CV: “Built a payment risk analytics pipeline in SQL that flags bursts of transactions and unusually large amounts using window functions, evaluated against planted cases, with a daily flag report.”
Intermediate4 weeksBuild on your laptopOpen the plan - Data EngineeringFinal-year
Website clickstream pipeline with sessions
Turn a stream of page-view events into sessions, funnels and a daily report.
- Python
- SQL
- Kafka (optional)
- DuckDB or PostgreSQL
CV: “Designed an event pipeline that groups clickstream events into sessions (30-minute rule), de-duplicates repeated events and produces daily funnel reports, with unit tests for session boundaries.”
Intermediate5 weeksBuild on your laptopOpen the plan - SQLGuided build
Slowly Changing Dimension Type 2 History Rebuilder
Reconstruct immutable temporal audit timelines with valid_from, valid_to, and current flags using LEAD() in SQL.
- SQL & Warehousing
CV: “Rebuilt SCD Type 2 address history in SQL from an audit log using LEAD(), with valid_from, valid_to and a current flag, and joined a past date to the right row.”
Advanced50 minRuns in your browserStart building - SQLGuided build
Consecutive Login Streak Detector (Gaps and Islands)
Detect consecutive daily user workout streaks and breaks using the classic SQL gaps-and-islands difference technique.
- SQL & Warehousing
CV: “Found consecutive daily login streaks in SQL with the gaps-and-islands technique (ROW_NUMBER() and date arithmetic).”
Advanced50 minRuns in your browserStart building - SQLGuided build
Hierarchical Management Org Chart Rollup
Traverse multi-level management reporting hierarchies and calculate organizational depth using recursive CTEs in SQL.
- SQL & Warehousing
CV: “Walked a manager hierarchy of any depth in SQL with a recursive CTE, adding each employee's level and full reporting path.”
Advanced50 minRuns in your browserStart building - SQLGuided build
Warehouse Query Plan Analysis and Filter Pushdown
Read a slow query's plan with EXPLAIN, move the date filter before the join, and check that the faster query returns the same numbers.
- SQL & Warehousing
CV: “Read a slow query's plan with EXPLAIN, moved the date filter before the join, and checked the faster query returned the same results.”
Advanced45 minRuns in your browserStart building - PySparkGuided build
Data Skew Mitigation with Key Salting
Eliminate distributed join stragglers and executor OOMs by splitting heavily skewed keys across parallel partitions using random salting.
- PySpark & Distributed Compute
CV: “Spread a skewed key across partitions in PySpark with salting and a two-stage aggregation, and checked the totals matched the unsalted result.”
Advanced50 minRuns in your browserStart building - Data EngineeringProTrack capstone
Capstone: deploy ingest to the cloud
A numbered GCP free-tier checklist: bucket, bronze file, BigQuery, a query, a README screenshot. The last mile is still outside this tab.
- Cloud Platforms for Data Engineers
Advanced16 minChecklistOpen capstone - Data EngineeringProTrack capstone
Capstone: late messages
3 orders on FakeKafkaTopic, one behind the watermark. Keep the on-time keys and return the offsets you would commit.
- Streaming & Message Queues
Advanced16 minRuns in your browserOpen capstone - Data EngineeringProTrack capstone
Capstone: ship checklist
Commit, test, build, plan, deploy. Put the whole track together as one safe release, and work out which step is ready next.
- Infrastructure & Engineering Practices
Advanced16 minChecklistOpen capstone - Data EngineeringProTrack capstone
Capstone: full pipeline design
One path from click to dashboard at 500M events/day. Every hop points at a lesson you already did. The whiteboard is the work.
- Data Engineering System Design
Advanced22 minWhiteboardOpen capstone
Can I publish these to GitHub?
Passing every LakeBench lesson means you can write the transforms, the DAG habits, the tests, and the design argument. It does not publish a repo. The Cloud capstone is a numbered GCP free-tier checklist this tab cannot execute. Do those steps outside, then copy the solutions you wrote, add a README that states grain, schedule, and failure modes, and push that.