The short answer
In traditional on-premises databases, compute processors and disk drives reside within the same physical server chassis. Google BigQuery pioneered a completely decoupled architecture, separating the compute execution engine from the physical storage layer across Google's planetary network.
The Colossus distributed file system
All BigQuery tables reside on Colossus, Google's global distributed cluster file system. Colossus handles physical replication, automatic disk failure recovery, and cross-data-center durability.
Because storage is completely separate from compute:
- You never provision disk drives, configure RAID arrays, or resize volume storage.
- Tables can scale from megabytes to petabytes without resizing clusters.
- Multiple independent compute queries can scan the exact same underlying table simultaneously without resource locking.
The Capacitor columnar format
Within Colossus, BigQuery stores table records in a proprietary, highly optimized columnar file format called Capacitor.
Unlike row-based storage (which stores all columns of a record side by side), Capacitor stores each column in separate, contiguous storage blocks.
Capacitor provides advanced physical optimizations:
- Intelligent compression algorithms: Capacitor inspects the distribution and of each column, dynamically choosing optimal compression algorithms including run-length encoding, encoding, and frame-of-reference integer packing.
- Embedded min-max statistics: Every columnar block contains header metadata recording the minimum and maximum values of the data in that block. When a query contains a filter , the query engine reads the header and skips reading the entire block if the target value falls outside the range.
- Nested and repeated structures: Capacitor natively stores hierarchical JSON-like structures using Google Protocol Buffers definitions without flattening them, allowing high-speed analytical queries on nested record arrays.
Understanding Capacitor explains why selecting specific columns in SQL reduces query costs: the execution engine reads only the storage blocks for the columns requested, leaving unreferenced columns untouched on Colossus disk arrays.