Multi-tenant SaaS, largest tenant is 40x the size of a typical one. Single Postgres instance still handles current load fine, but sharding by tenant_id keeps coming up as "the eventual answer." When do you actually pull that trigger?
Accepted answer
Shard when a single instance's vertical scaling ceiling is visibly approaching, not preemptively. Sharding adds real operational complexity, cross-shard queries, rebalancing, migrations times N, that's expensive to carry before you actually need it.
Our dimension is slowly changing, does that change the join order?
Idempotent writes with merge keys saved us during backfills.
A middle ground: isolate your largest few tenants onto dedicated instances or schemas while keeping the long tail on a shared instance. That partial sharding solves the actual hot-tenant problem without building full infrastructure for everyone.
This matches our runbook for skewed keys.
Worth measuring the serialized size before choosing broadcast.
Document the grain decision, most BI bugs turn out to be grain bugs.
Check whether AQE is disabled in your Spark conf, skew join handling helped us a lot here.
Prefer a staging table plus validation gate before promoting to prod tables.
Prefer a staging table plus validation gate before promoting to prod tables.
Sign in to reply.
© 2026 Lakebench, operated by Hunnurji Rao. Bengaluru, Karnataka, India.
No cluster. No install. Just the tab.
