Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
45-Day Plan
Day 1

SQL Foundations

DuckDB WASM
Day 1 of 45/Phase 01: SQL MasterclassDuckDB WASM Engine

SQL Foundations & Logical Query Processing Order

SELECT, WHERE, ORDER BY, and NULL look simple until a filter quietly drops half your rows. Today walks through how a query engine actually runs those clauses, why `WHERE col = NULL` never matches, and how CASE turns messy status codes into labels you can report on.

Prerequisite:None: Start here with zero prior knowledge
Estimated time:~2 hours

What you'll learn

By the end of this lesson, you will be able to:

  • Understand what relational tables and columns represent in an e-commerce platform.
  • Write clean, readable SELECT queries with WHERE filtering, aliases, and CASE expressions.
  • Get solid on Three-Valued Logic and explain why WHERE column = NULL drops every row.
  • Sort and paginate large result sets using ORDER BY, DISTINCT, and LIMIT.
  • Internalize the 8-step logical execution order: why SELECT runs almost last in the database engine.

Why Does SQL Exist? The ShopKart Scenario

Imagine you just joined ShopKart, an e-commerce platform processing hundreds of thousands of customer purchases every day. Every time a shopper in Bengaluru, Mumbai, or Hyderabad buys an item on their phone, the transaction is recorded across tables in a database:

customers  ·  orders  ·  products  ·  payments

At 9:00 AM on Monday, ShopKart's Head of Operations asks you:

“Show me every customer from Bengaluru who completed an order worth more than ₹500 yesterday, sorted by the highest spend.”

In a traditional programming language like Python, you would open a file, write a for loop, check each row with an if statement, sort the list, and write out the results. That is imperative programming: you specify every low-level instruction yourself.

SQL (Structured Query Language) is declarative. You describe what data you need, and you hand that request to the database engine. The database engine contains a sophisticated component called a query optimizer that looks at table sizes, memory, and indexes on disk to figure out the fastest physical way to find your answer.

Before SQL: What is a Relational Table?

A relational table is a structured collection of data organized into rows and columns. Each row represents a single discrete entity or event, and each column represents a specific attribute with a declared data type (such as integers, decimals, text, or dates).

Here is a sample slice from ShopKart's orders table loaded into your interactive workspace:

Sample Table: orders12 rows in DuckDB workspace
order_id (PK)customer_id (FK)amount (INR)order_statusorder_date
1011₹500COMPLETED2026-09-01
1022₹800CANCELLED2026-09-01
1031₹300COMPLETED2026-09-02
1043₹1,200COMPLETED2026-09-02
1052₹450COMPLETED2026-09-03

Notice two critical column labels:

  • Primary Key (order_id): A column guaranteed to be unique for every single row. No two orders ever share the same order_id.
  • Foreign Key (customer_id): A reference pointing to the primary key in another table (customers.customer_id). This is how relational databases link disparate business events without duplicating names and addresses on every order.

Retrieving & Filtering: SELECT and WHERE

Every SQL query starts with two fundamental clauses: FROM (which table to read from disk) and SELECT (which columns to return to the caller).

Retrieve the order identifier, customer reference, and purchase amount from the orders table.

Never use SELECT * in production pipelines. Selecting only the required columns reduces disk I/O and network transfer by up to 90%.

Query 1: Select Specific Columns
SELECT
    order_id,
    customer_id,
    amount
FROM orders;
Expected Result
order_idcustomer_idamount
1011500
1022800
1031300
10431200

Next, add a WHERE clause to filter out rows that don't meet your business criteria. For example, ShopKart's finance team only recognizes revenue from completed transactions:

Filter for completed orders where the transaction amount is strictly greater than ₹500.

Filter as early as possible in your pipeline to discard irrelevant rows before spending compute resources on downstream transformations.

Query 2: Filter by Status and Value
SELECT
    order_id,
    customer_id,
    amount,
    order_status
FROM orders
WHERE order_status = 'COMPLETED'
  AND amount > 500;
Expected Result
order_idcustomer_idamountorder_status
10431200COMPLETED
10641500COMPLETED
1086950COMPLETED
11012100COMPLETED
1128800COMPLETED

Three-Valued Logic: Why NULL is Not Zero or Empty String

In computer science, boolean logic is binary: an expression evaluates to either TRUE or FALSE.

SQL is fundamentally different. SQL implements Three-Valued Logic (3VL):

TRUEFALSEUNKNOWN

In SQL, NULL does not mean zero, false, or an empty string "". It represents unknown, missing, or inapplicable data.

Because NULL represents an unknown value, comparing anything to NULL with an equality operator (=) cannot be determined:

NULL = NULL       → UNKNOWN (NOT TRUE!)
phone = NULL       → UNKNOWN
phone != NULL      → UNKNOWN
phone IS NULL      → TRUE or FALSE (The correct predicate)

WHERE phone = NULL

When a database engine evaluates a WHERE clause, it only retains rows where the condition evaluates to TRUE. If the condition evaluates to UNKNOWN, the row is discarded just as if it were FALSE.

If you write WHERE phone = NULL or WHERE phone != NULL, every single row evaluates to UNKNOWN, and the query returns 0 rows! Always use IS NULL or IS NOT NULL.

Find all ShopKart customers who did not provide a contact phone number during registration.

Data quality pipelines check for NULLs daily to identify unverified accounts or incomplete customer profiles.

Query 3: Finding Missing Contact Numbers
SELECT
    customer_id,
    name,
    city,
    phone
FROM customers
WHERE phone IS NULL;
Expected Result
customer_idnamecityphone
2RaviHyderabadNULL
5KarthikMumbaiNULL
8MeeraPuneNULL

Sorting, Deduplication & Pagination

Databases store rows as an unordered mathematical set. Unless you explicitly provide an ORDER BY clause, the order in which rows return is non-deterministic. Never assume rows will return in insertion order.

Find the 3 highest completed order amounts across all customers.

Executive leaderboards and high-value customer audits rely on explicit descending sorts paired with row limits.

Query 4: Top 3 Highest Value Orders
SELECT
    order_id,
    customer_id,
    amount,
    order_date
FROM orders
WHERE order_status = 'COMPLETED'
ORDER BY amount DESC
LIMIT 3;
Expected Result
order_idcustomer_idamountorder_date
1101₹2,1002026-09-05
1064₹1,5002026-09-03
1043₹1,2002026-09-02

Conditional Logic with CASE Expressions

The CASE expression is SQL's inline conditional. It evaluates conditions sequentially from top to bottom and returns the result of the first branch that evaluates to TRUE.

Tag each order with a business category: Premium (> ₹1000), Standard (₹500 to ₹1,000), or Low Value (< ₹500).

Data engineers use CASE statements in Silver and Gold pipeline transformations to create business dimensions and segment customers.

Query 5: Segmenting Orders into Spend Tiers
SELECT
    order_id,
    amount,
    CASE
        WHEN amount >= 1000 THEN 'Premium'
        WHEN amount >= 500 THEN 'Standard'
        ELSE 'Low Value'
    END AS order_tier
FROM orders
ORDER BY amount DESC
LIMIT 5;
Expected Result
order_idamountorder_tier
110₹2,100Premium
106₹1,500Premium
104₹1,200Premium
108₹950Standard
102₹800Standard

The 8-Step Logical Query Processing Order

When you write code in Python, code executes top to bottom. But in SQL, the order you write clauses is completely different from the order the database engine executes them.

Visual Explanation: How the Database Engine Physically Evaluates a Query

Why can't you use a column alias defined in SELECT inside your WHERE clause?

What You WRITE:                      How the Engine EXECUTES:                                                                                                                          1. SELECT column, ALIAS               [Step 1] FROM orders                                 2. FROM orders                         │  (Locates and opens table data on disk)           3. WHERE condition                     ▼                                                   4. GROUP BY column                    [Step 2] WHERE order_status = 'COMPLETED'            5. HAVING agg_condition                │  (Discards non-matching rows before grouping)     6. ORDER BY column                     ▼                                                   7. LIMIT n                            [Step 3] GROUP BY customer_id                                                               │  (Buckets rows by grouping key)                                                          ▼                                                                                         [Step 4] HAVING SUM(amount) > 1000                                                          │  (Filters aggregated groups)                                                             ▼                                                                                         [Step 5] SELECT customer_id, SUM(amount) AS total                                           │  (Evaluates expressions and generates aliases!)                                          ▼                                                                                         [Step 6] DISTINCT                                                                           │  (Eliminates duplicate output rows)                                                      ▼                                                                                         [Step 7] ORDER BY total DESC                                                                │  (Sorts output - NOW alias 'total' is available!)                                        ▼                                                                                         [Step 8] LIMIT 10                                                                              (Transmits first 10 rows to client)             
  1. [1]FROM: Identify and read tables from disk or object storage.
  2. [2]WHERE: Filter out rows before any grouping or calculations occur.
  3. [3]GROUP BY: Partition surviving rows into aggregation buckets.
  4. [4]HAVING: Filter aggregated groups after functions execute.
  5. [5]SELECT: Evaluate scalar expressions and assign column aliases.
  6. [6]DISTINCT: Deduplicate identical projected rows.
  7. [7]ORDER BY: Sort output rows (can now safely use SELECT aliases).
  8. [8]LIMIT: Slice the top N records and send them over the wire.
Notice that SELECT is step 5, not step 1! This explains why aliases created in SELECT cannot be referenced in WHERE or GROUP BY.

Why Data Engineers Must Memorize This Order

In production systems with 500-million row tables, understanding this order prevents catastrophic query mistakes. If a junior engineer writes an expensive scalar function in the SELECT list, thinking it filters data, the database has already read and filtered every row in steps 1 and 2.

Always push your filtering down into the WHERE clause so the engine discards rows before memory-intensive grouping or projection takes place.

Hands-On Practice Drills

Click "Run in Editor" on any drill to load it directly into your live DuckDB WASM workspace.

Query 1: Show all customersBeginner

Show all customers from the customers table.

Solution: SELECT * returns all columns and all rows.
SELECT *
FROM customers;
Query 2: Show only customer names and citiesBeginner

Show only customer names and cities.

Solution: Column projection (picking specific attributes).
SELECT
    name,
    city
FROM customers;
Query 3: Find completed ordersBeginner

Find completed orders from the orders table.

Solution: WHERE clause for row filtering with string equality.
SELECT *
FROM orders
WHERE status = 'COMPLETED';
Query 4: Find orders greater than ₹500Beginner

Find orders greater than ₹500.

Solution: Numeric comparison operators in WHERE.
SELECT *
FROM orders
WHERE amount > 500;
Query 5: Show orders from highest amount to lowestBeginner

Show orders from highest amount to lowest.

Solution: ORDER BY DESC (descending sort).
SELECT *
FROM orders
ORDER BY amount DESC;

Can you explain this without looking?

Test your conceptual understanding before moving on. Can you explain these out loud without looking?

Interview Practice

Scenario questions common in technical data engineering interviews.

Key points covered today:

  • Understood the declarative nature of SQL and how tables represent business entities in ShopKart.
  • Wrote structured SELECT queries with WHERE filtering, aliases, and CASE expressions.
  • Mastered SQL Three-Valued Logic and how to safely handle NULL with IS NULL predicates.
  • Internalized the 8-step logical execution order and why SELECT aliases cannot be used in WHERE.
Next in curriculum: Day 2: SQL Aggregation, GROUP BY & Grain
Continue to Day 2
Day 1 of 45/Phase 01: SQL MasterclassDuckDB WASM Engine

SQL Foundations & Logical Query Processing Order

SELECT, WHERE, ORDER BY, and NULL look simple until a filter quietly drops half your rows. Today walks through how a query engine actually runs those clauses, why `WHERE col = NULL` never matches, and how CASE turns messy status codes into labels you can report on.

Prerequisite:None: Start here with zero prior knowledge
Estimated time:~2 hours

What you'll learn

By the end of this lesson, you will be able to:

  • Understand what relational tables and columns represent in an e-commerce platform.
  • Write clean, readable SELECT queries with WHERE filtering, aliases, and CASE expressions.
  • Get solid on Three-Valued Logic and explain why WHERE column = NULL drops every row.
  • Sort and paginate large result sets using ORDER BY, DISTINCT, and LIMIT.
  • Internalize the 8-step logical execution order: why SELECT runs almost last in the database engine.

Why Does SQL Exist? The ShopKart Scenario

Imagine you just joined ShopKart, an e-commerce platform processing hundreds of thousands of customer purchases every day. Every time a shopper in Bengaluru, Mumbai, or Hyderabad buys an item on their phone, the transaction is recorded across tables in a database:

customers  ·  orders  ·  products  ·  payments

At 9:00 AM on Monday, ShopKart's Head of Operations asks you:

“Show me every customer from Bengaluru who completed an order worth more than ₹500 yesterday, sorted by the highest spend.”

In a traditional programming language like Python, you would open a file, write a for loop, check each row with an if statement, sort the list, and write out the results. That is imperative programming: you specify every low-level instruction yourself.

SQL (Structured Query Language) is declarative. You describe what data you need, and you hand that request to the database engine. The database engine contains a sophisticated component called a query optimizer that looks at table sizes, memory, and indexes on disk to figure out the fastest physical way to find your answer.

Before SQL: What is a Relational Table?

A relational table is a structured collection of data organized into rows and columns. Each row represents a single discrete entity or event, and each column represents a specific attribute with a declared data type (such as integers, decimals, text, or dates).

Here is a sample slice from ShopKart's orders table loaded into your interactive workspace:

Sample Table: orders12 rows in DuckDB workspace
order_id (PK)customer_id (FK)amount (INR)order_statusorder_date
1011₹500COMPLETED2026-09-01
1022₹800CANCELLED2026-09-01
1031₹300COMPLETED2026-09-02
1043₹1,200COMPLETED2026-09-02
1052₹450COMPLETED2026-09-03

Notice two critical column labels:

  • Primary Key (order_id): A column guaranteed to be unique for every single row. No two orders ever share the same order_id.
  • Foreign Key (customer_id): A reference pointing to the primary key in another table (customers.customer_id). This is how relational databases link disparate business events without duplicating names and addresses on every order.

Retrieving & Filtering: SELECT and WHERE

Every SQL query starts with two fundamental clauses: FROM (which table to read from disk) and SELECT (which columns to return to the caller).

Retrieve the order identifier, customer reference, and purchase amount from the orders table.

Never use SELECT * in production pipelines. Selecting only the required columns reduces disk I/O and network transfer by up to 90%.

Query 1: Select Specific Columns
SELECT
    order_id,
    customer_id,
    amount
FROM orders;
Expected Result
order_idcustomer_idamount
1011500
1022800
1031300
10431200

Next, add a WHERE clause to filter out rows that don't meet your business criteria. For example, ShopKart's finance team only recognizes revenue from completed transactions:

Filter for completed orders where the transaction amount is strictly greater than ₹500.

Filter as early as possible in your pipeline to discard irrelevant rows before spending compute resources on downstream transformations.

Query 2: Filter by Status and Value
SELECT
    order_id,
    customer_id,
    amount,
    order_status
FROM orders
WHERE order_status = 'COMPLETED'
  AND amount > 500;
Expected Result
order_idcustomer_idamountorder_status
10431200COMPLETED
10641500COMPLETED
1086950COMPLETED
11012100COMPLETED
1128800COMPLETED

Three-Valued Logic: Why NULL is Not Zero or Empty String

In computer science, boolean logic is binary: an expression evaluates to either TRUE or FALSE.

SQL is fundamentally different. SQL implements Three-Valued Logic (3VL):

TRUEFALSEUNKNOWN

In SQL, NULL does not mean zero, false, or an empty string "". It represents unknown, missing, or inapplicable data.

Because NULL represents an unknown value, comparing anything to NULL with an equality operator (=) cannot be determined:

NULL = NULL       → UNKNOWN (NOT TRUE!)
phone = NULL       → UNKNOWN
phone != NULL      → UNKNOWN
phone IS NULL      → TRUE or FALSE (The correct predicate)

WHERE phone = NULL

When a database engine evaluates a WHERE clause, it only retains rows where the condition evaluates to TRUE. If the condition evaluates to UNKNOWN, the row is discarded just as if it were FALSE.

If you write WHERE phone = NULL or WHERE phone != NULL, every single row evaluates to UNKNOWN, and the query returns 0 rows! Always use IS NULL or IS NOT NULL.

Find all ShopKart customers who did not provide a contact phone number during registration.

Data quality pipelines check for NULLs daily to identify unverified accounts or incomplete customer profiles.

Query 3: Finding Missing Contact Numbers
SELECT
    customer_id,
    name,
    city,
    phone
FROM customers
WHERE phone IS NULL;
Expected Result
customer_idnamecityphone
2RaviHyderabadNULL
5KarthikMumbaiNULL
8MeeraPuneNULL

Sorting, Deduplication & Pagination

Databases store rows as an unordered mathematical set. Unless you explicitly provide an ORDER BY clause, the order in which rows return is non-deterministic. Never assume rows will return in insertion order.

Find the 3 highest completed order amounts across all customers.

Executive leaderboards and high-value customer audits rely on explicit descending sorts paired with row limits.

Query 4: Top 3 Highest Value Orders
SELECT
    order_id,
    customer_id,
    amount,
    order_date
FROM orders
WHERE order_status = 'COMPLETED'
ORDER BY amount DESC
LIMIT 3;
Expected Result
order_idcustomer_idamountorder_date
1101₹2,1002026-09-05
1064₹1,5002026-09-03
1043₹1,2002026-09-02

Conditional Logic with CASE Expressions

The CASE expression is SQL's inline conditional. It evaluates conditions sequentially from top to bottom and returns the result of the first branch that evaluates to TRUE.

Tag each order with a business category: Premium (> ₹1000), Standard (₹500 to ₹1,000), or Low Value (< ₹500).

Data engineers use CASE statements in Silver and Gold pipeline transformations to create business dimensions and segment customers.

Query 5: Segmenting Orders into Spend Tiers
SELECT
    order_id,
    amount,
    CASE
        WHEN amount >= 1000 THEN 'Premium'
        WHEN amount >= 500 THEN 'Standard'
        ELSE 'Low Value'
    END AS order_tier
FROM orders
ORDER BY amount DESC
LIMIT 5;
Expected Result
order_idamountorder_tier
110₹2,100Premium
106₹1,500Premium
104₹1,200Premium
108₹950Standard
102₹800Standard

The 8-Step Logical Query Processing Order

When you write code in Python, code executes top to bottom. But in SQL, the order you write clauses is completely different from the order the database engine executes them.

Visual Explanation: How the Database Engine Physically Evaluates a Query

Why can't you use a column alias defined in SELECT inside your WHERE clause?

What You WRITE:                      How the Engine EXECUTES:                                                                                                                          1. SELECT column, ALIAS               [Step 1] FROM orders                                 2. FROM orders                         │  (Locates and opens table data on disk)           3. WHERE condition                     ▼                                                   4. GROUP BY column                    [Step 2] WHERE order_status = 'COMPLETED'            5. HAVING agg_condition                │  (Discards non-matching rows before grouping)     6. ORDER BY column                     ▼                                                   7. LIMIT n                            [Step 3] GROUP BY customer_id                                                               │  (Buckets rows by grouping key)                                                          ▼                                                                                         [Step 4] HAVING SUM(amount) > 1000                                                          │  (Filters aggregated groups)                                                             ▼                                                                                         [Step 5] SELECT customer_id, SUM(amount) AS total                                           │  (Evaluates expressions and generates aliases!)                                          ▼                                                                                         [Step 6] DISTINCT                                                                           │  (Eliminates duplicate output rows)                                                      ▼                                                                                         [Step 7] ORDER BY total DESC                                                                │  (Sorts output - NOW alias 'total' is available!)                                        ▼                                                                                         [Step 8] LIMIT 10                                                                              (Transmits first 10 rows to client)             
  1. [1]FROM: Identify and read tables from disk or object storage.
  2. [2]WHERE: Filter out rows before any grouping or calculations occur.
  3. [3]GROUP BY: Partition surviving rows into aggregation buckets.
  4. [4]HAVING: Filter aggregated groups after functions execute.
  5. [5]SELECT: Evaluate scalar expressions and assign column aliases.
  6. [6]DISTINCT: Deduplicate identical projected rows.
  7. [7]ORDER BY: Sort output rows (can now safely use SELECT aliases).
  8. [8]LIMIT: Slice the top N records and send them over the wire.
Notice that SELECT is step 5, not step 1! This explains why aliases created in SELECT cannot be referenced in WHERE or GROUP BY.

Why Data Engineers Must Memorize This Order

In production systems with 500-million row tables, understanding this order prevents catastrophic query mistakes. If a junior engineer writes an expensive scalar function in the SELECT list, thinking it filters data, the database has already read and filtered every row in steps 1 and 2.

Always push your filtering down into the WHERE clause so the engine discards rows before memory-intensive grouping or projection takes place.

Hands-On Practice Drills

Click "Run in Editor" on any drill to load it directly into your live DuckDB WASM workspace.

Query 1: Show all customersBeginner

Show all customers from the customers table.

Solution: SELECT * returns all columns and all rows.
SELECT *
FROM customers;
Query 2: Show only customer names and citiesBeginner

Show only customer names and cities.

Solution: Column projection (picking specific attributes).
SELECT
    name,
    city
FROM customers;
Query 3: Find completed ordersBeginner

Find completed orders from the orders table.

Solution: WHERE clause for row filtering with string equality.
SELECT *
FROM orders
WHERE status = 'COMPLETED';
Query 4: Find orders greater than ₹500Beginner

Find orders greater than ₹500.

Solution: Numeric comparison operators in WHERE.
SELECT *
FROM orders
WHERE amount > 500;
Query 5: Show orders from highest amount to lowestBeginner

Show orders from highest amount to lowest.

Solution: ORDER BY DESC (descending sort).
SELECT *
FROM orders
ORDER BY amount DESC;

Can you explain this without looking?

Test your conceptual understanding before moving on. Can you explain these out loud without looking?

Interview Practice

Scenario questions common in technical data engineering interviews.

Key points covered today:

  • Understood the declarative nature of SQL and how tables represent business entities in ShopKart.
  • Wrote structured SELECT queries with WHERE filtering, aliases, and CASE expressions.
  • Mastered SQL Three-Valued Logic and how to safely handle NULL with IS NULL predicates.
  • Internalized the 8-step logical execution order and why SELECT aliases cannot be used in WHERE.
Next in curriculum: Day 2: SQL Aggregation, GROUP BY & Grain
Continue to Day 2
SQL Editor (PostgreSQL / DuckDB dialect)Press Cmd+Enter to execute
Loading editor…
Initializing DuckDB engine…