The short answer
(also called projection pushdown) means the engine reads only the columns required by the query, not every column in the file.
Problem first. A wide might have 80 columns. A dashboard needs order_id, amount, and region. In a row store or SELECT * CSV mindset, you still pay for the other 77 fields. In Parquet/ORC, those other column chunks can stay on disk.
File columns: id | user | amount | notes | json_blob | ... Query needs: amount, region Column pruning → open only those column chunks
# Good: project early so the scan can prune columns
df = spark.read.parquet("/lake/orders").select("order_id", "amount", "region")
# Bad habit: pull everything then drop later
df = spark.read.parquet("/lake/orders") # may still prune if plan is smart,
df = df.select("order_id") # but SELECT * patterns and UDFs hurtWhy it matters financially
Less I/O, less CPU decompression, less memory, cheaper cloud scans.
Interview tip: "Column pruning = do not read unused columns." Contrast with (skip rows/chunks) and (skip directories).
Seeing it in a real file. I wrote a 200,000-row Parquet file with DuckDB: order_id, user_name, amount, region, and one long text column notes. Parquet stores each column in its own chunk, so the compressed size per column can be read from the file's metadata:
amount 379,265 bytes 9.4%
notes 2,825,135 bytes 70.0%
order_id 801,181 bytes 19.9%
region 2,555 bytes 0.1%
user_name 26,305 bytes 0.7%
query needs amount+region: 381,820 of 4,034,441 bytes = 9.5%A query that needs only amount and region reads about 9.5% of the file. The saving depends on how wide the skipped columns are: here notes was made deliberately long. A CSV cannot do this, because the engine must scan every line to find the fields.