Sorting data before writing to Parquet concentrates similar values into the same row groups, creating narrow, non-overlapping min and max statistics in the file footer. When query engines evaluate filtering predicates, these tight bounds allow readers to skip entire row groups rather than scanning every block. Co-locating identical values also dramatically improves compression by letting run-length and dictionary encodings represent long sequences of identical data with minimal bytes.
Why ordering transforms file statistics
When data is written in random order, high-cardinality or temporal columns will have min and max values in every single row group that span almost the entire range of the dataset:
Unsorted Table (Every row group spans full range): Row Group 0: min_date='2025-01-01', max_date='2025-12-31' -> MUST SCAN Row Group 1: min_date='2025-01-02', max_date='2025-12-30' -> MUST SCAN Sorted Table (Each row group covers narrow window): Row Group 0: min_date='2025-01-01', max_date='2025-01-15' -> SKIPPED Row Group 1: min_date='2025-01-16', max_date='2025-01-31' -> SKIPPED Row Group 2: min_date='2025-02-01', max_date='2025-02-15' -> MATCHES QUERY
A query filtering for February 2025 on unsorted data must open and inspect every row group because every group contains at least one matching row. On sorted data, the reader eliminates almost the entire file without fetching the underlying data pages.
Compression improvements
Encoding algorithms inside Parquet benefit directly from sorted data:
- Run-length encoding stores repeated consecutive values as a count and a value pair, turning thousands of identical status flags into just a few bytes.
- Dictionary encoding limits the distinct values present in any single page, preventing the dictionary page from overflowing its size limit and falling back to uncompressed plain encoding.
- Bit-packing algorithms compress sequential integers far more densely when delta values between sorted keys remain small.
Practical clustering guidance
Order columns intentionally when preparing analytical tables:
- Sort by the most frequently filtered column first, prioritizing columns that appear in high-selectivity WHERE clauses like tenant identifiers or event dates.
- Avoid sorting by more than two or three hierarchical columns, as data entropy increases rapidly on lower-priority sort columns.
- In modern lakehouse tables, this concept is mechanized through clustering or Z-order curves, which sort across multiple dimensions simultaneously to provide multi-column data skipping without strict hierarchical ordering.