12
Accepted answer
Document the grain decision, most BI bugs turn out to be grain bugs.
This worked in dev on sample data but fails at full volume. Details:
Context: Predicate pushdown not happening in Spark SQL join
Happy to share schema snippets or metrics if useful.
Accepted answer
Document the grain decision, most BI bugs turn out to be grain bugs.
Start with the execution plan, numbers beat guesses.
+1, saw identical behaviour after upgrading Spark 3.4 to 3.5.
Document the grain decision, most BI bugs turn out to be grain bugs.
Agree on the staging table swap. Atomic promote prevented partial reads.
Idempotent writes with merge keys saved us during backfills.
Sign in to reply.
© 2026 Lakebench, operated by Hunnurji Rao. Bengaluru, Karnataka, India.
No cluster. No install. Just the tab.