1
Accepted answer
Document the grain decision, most BI bugs turn out to be grain bugs.
Migrated stacks last quarter and this regression appeared. Stack trace and plan attached:
Context: Readable CTE chain vs one giant query, performance difference?
Happy to share schema snippets or metrics if useful.
Accepted answer
Document the grain decision, most BI bugs turn out to be grain bugs.
Idempotent writes with merge keys saved us during backfills.
We saw the same issue, fixing the partition filter dropped runtime 60%.
+1, saw identical behaviour after upgrading Spark 3.4 to 3.5.
Sign in to reply.