SQL Foundations & Logical Query Processing Order
SELECT, WHERE, ORDER BY, and NULL look simple until a filter quietly drops half your rows. Today walks through how a query engine actually runs those clauses, why `WHERE col = NULL` never matches, and how CASE turns messy status codes into labels you can report on.
What you'll learn
By the end of this lesson, you will be able to:
- Understand what relational tables and columns represent in an e-commerce platform.
- Write clean, readable SELECT queries with WHERE filtering, aliases, and CASE expressions.
- Get solid on Three-Valued Logic and explain why WHERE column = NULL drops every row.
- Sort and paginate large result sets using ORDER BY, DISTINCT, and LIMIT.
- Internalize the 8-step logical execution order: why SELECT runs almost last in the database engine.
Why Does SQL Exist? The ShopKart Scenario
Imagine you just joined ShopKart, an e-commerce platform processing hundreds of thousands of customer purchases every day. Every time a shopper in Bengaluru, Mumbai, or Hyderabad buys an item on their phone, the transaction is recorded across tables in a database:
customers · orders · products · payments
At 9:00 AM on Monday, ShopKart's Head of Operations asks you:
“Show me every customer from Bengaluru who completed an order worth more than ₹500 yesterday, sorted by the highest spend.”
In a traditional programming language like Python, you would open a file, write a for loop, check each row with an if statement, sort the list, and write out the results. That is imperative programming: you specify every low-level instruction yourself.
SQL (Structured Query Language) is declarative. You describe what data you need, and you hand that request to the database engine. The database engine contains a sophisticated component called a query optimizer that looks at table sizes, memory, and indexes on disk to figure out the fastest physical way to find your answer.
Before SQL: What is a Relational Table?
A relational table is a structured collection of data organized into rows and columns. Each row represents a single discrete entity or event, and each column represents a specific attribute with a declared data type (such as integers, decimals, text, or dates).
Here is a sample slice from ShopKart's orders table loaded into your interactive workspace:
| order_id (PK) | customer_id (FK) | amount (INR) | order_status | order_date |
|---|---|---|---|---|
| 101 | 1 | ₹500 | COMPLETED | 2026-09-01 |
| 102 | 2 | ₹800 | CANCELLED | 2026-09-01 |
| 103 | 1 | ₹300 | COMPLETED | 2026-09-02 |
| 104 | 3 | ₹1,200 | COMPLETED | 2026-09-02 |
| 105 | 2 | ₹450 | COMPLETED | 2026-09-03 |
Notice two critical column labels:
- Primary Key (order_id): A column guaranteed to be unique for every single row. No two orders ever share the same order_id.
- Foreign Key (customer_id): A reference pointing to the primary key in another table (
customers.customer_id). This is how relational databases link disparate business events without duplicating names and addresses on every order.
Retrieving & Filtering: SELECT and WHERE
Every SQL query starts with two fundamental clauses: FROM (which table to read from disk) and SELECT (which columns to return to the caller).
Retrieve the order identifier, customer reference, and purchase amount from the orders table.
Never use SELECT * in production pipelines. Selecting only the required columns reduces disk I/O and network transfer by up to 90%.
SELECT
order_id,
customer_id,
amount
FROM orders;| order_id | customer_id | amount |
|---|---|---|
| 101 | 1 | 500 |
| 102 | 2 | 800 |
| 103 | 1 | 300 |
| 104 | 3 | 1200 |
Next, add a WHERE clause to filter out rows that don't meet your business criteria. For example, ShopKart's finance team only recognizes revenue from completed transactions:
Filter for completed orders where the transaction amount is strictly greater than ₹500.
Filter as early as possible in your pipeline to discard irrelevant rows before spending compute resources on downstream transformations.
SELECT
order_id,
customer_id,
amount,
order_status
FROM orders
WHERE order_status = 'COMPLETED'
AND amount > 500;| order_id | customer_id | amount | order_status |
|---|---|---|---|
| 104 | 3 | 1200 | COMPLETED |
| 106 | 4 | 1500 | COMPLETED |
| 108 | 6 | 950 | COMPLETED |
| 110 | 1 | 2100 | COMPLETED |
| 112 | 8 | 800 | COMPLETED |
Three-Valued Logic: Why NULL is Not Zero or Empty String
In computer science, boolean logic is binary: an expression evaluates to either TRUE or FALSE.
SQL is fundamentally different. SQL implements Three-Valued Logic (3VL):
In SQL, NULL does not mean zero, false, or an empty string "". It represents unknown, missing, or inapplicable data.
Because NULL represents an unknown value, comparing anything to NULL with an equality operator (=) cannot be determined:
WHERE phone = NULL
When a database engine evaluates a WHERE clause, it only retains rows where the condition evaluates to TRUE. If the condition evaluates to UNKNOWN, the row is discarded just as if it were FALSE.
If you write WHERE phone = NULL or WHERE phone != NULL, every single row evaluates to UNKNOWN, and the query returns 0 rows! Always use IS NULL or IS NOT NULL.
Find all ShopKart customers who did not provide a contact phone number during registration.
Data quality pipelines check for NULLs daily to identify unverified accounts or incomplete customer profiles.
SELECT
customer_id,
name,
city,
phone
FROM customers
WHERE phone IS NULL;| customer_id | name | city | phone |
|---|---|---|---|
| 2 | Ravi | Hyderabad | NULL |
| 5 | Karthik | Mumbai | NULL |
| 8 | Meera | Pune | NULL |
Sorting, Deduplication & Pagination
Databases store rows as an unordered mathematical set. Unless you explicitly provide an ORDER BY clause, the order in which rows return is non-deterministic. Never assume rows will return in insertion order.
Find the 3 highest completed order amounts across all customers.
Executive leaderboards and high-value customer audits rely on explicit descending sorts paired with row limits.
SELECT
order_id,
customer_id,
amount,
order_date
FROM orders
WHERE order_status = 'COMPLETED'
ORDER BY amount DESC
LIMIT 3;| order_id | customer_id | amount | order_date |
|---|---|---|---|
| 110 | 1 | ₹2,100 | 2026-09-05 |
| 106 | 4 | ₹1,500 | 2026-09-03 |
| 104 | 3 | ₹1,200 | 2026-09-02 |
Conditional Logic with CASE Expressions
The CASE expression is SQL's inline conditional. It evaluates conditions sequentially from top to bottom and returns the result of the first branch that evaluates to TRUE.
Tag each order with a business category: Premium (> ₹1000), Standard (₹500 to ₹1,000), or Low Value (< ₹500).
Data engineers use CASE statements in Silver and Gold pipeline transformations to create business dimensions and segment customers.
SELECT
order_id,
amount,
CASE
WHEN amount >= 1000 THEN 'Premium'
WHEN amount >= 500 THEN 'Standard'
ELSE 'Low Value'
END AS order_tier
FROM orders
ORDER BY amount DESC
LIMIT 5;| order_id | amount | order_tier |
|---|---|---|
| 110 | ₹2,100 | Premium |
| 106 | ₹1,500 | Premium |
| 104 | ₹1,200 | Premium |
| 108 | ₹950 | Standard |
| 102 | ₹800 | Standard |
The 8-Step Logical Query Processing Order
When you write code in Python, code executes top to bottom. But in SQL, the order you write clauses is completely different from the order the database engine executes them.
Why can't you use a column alias defined in SELECT inside your WHERE clause?
- [1]FROM: Identify and read tables from disk or object storage.
- [2]WHERE: Filter out rows before any grouping or calculations occur.
- [3]GROUP BY: Partition surviving rows into aggregation buckets.
- [4]HAVING: Filter aggregated groups after functions execute.
- [5]SELECT: Evaluate scalar expressions and assign column aliases.
- [6]DISTINCT: Deduplicate identical projected rows.
- [7]ORDER BY: Sort output rows (can now safely use SELECT aliases).
- [8]LIMIT: Slice the top N records and send them over the wire.
Why Data Engineers Must Memorize This Order
In production systems with 500-million row tables, understanding this order prevents catastrophic query mistakes. If a junior engineer writes an expensive scalar function in the SELECT list, thinking it filters data, the database has already read and filtered every row in steps 1 and 2.
Always push your filtering down into the WHERE clause so the engine discards rows before memory-intensive grouping or projection takes place.
Hands-On Practice Drills
Click "Run in Editor" on any drill to load it directly into your live DuckDB WASM workspace.
Show all customers from the customers table.
SELECT *
FROM customers;Show only customer names and cities.
SELECT
name,
city
FROM customers;Find completed orders from the orders table.
SELECT *
FROM orders
WHERE status = 'COMPLETED';Find orders greater than ₹500.
SELECT *
FROM orders
WHERE amount > 500;Show orders from highest amount to lowest.
SELECT *
FROM orders
ORDER BY amount DESC;Can you explain this without looking?
Test your conceptual understanding before moving on. Can you explain these out loud without looking?
Interview Practice
Scenario questions common in technical data engineering interviews.