26
In our case the root cause was an implicit cast preventing pushdown.
This worked in dev on sample data but fails at full volume. Details:
Context: Dataflow streaming with BigQuery sink, duplicate rows on retry
Happy to share schema snippets or metrics if useful.
In our case the root cause was an implicit cast preventing pushdown.
Start with the execution plan, numbers beat guesses.
+1, saw identical behaviour after upgrading Spark 3.4 to 3.5.
Our dimension is slowly changing, does that change the join order?
Could you share a sketch of the salting logic?
Sign in to reply.
© 2026 Lakebench, operated by Hunnurji Rao. Bengaluru, Karnataka, India.
No cluster. No install. Just the tab.