5
Our dimension is slowly changing, does that change the join order?
This worked in dev on sample data but fails at full volume. Details:
Context: Snowflake warehouse auto-suspend vs always-on dev cost
Happy to share schema snippets or metrics if useful.
Our dimension is slowly changing, does that change the join order?
Check whether AQE is disabled in your Spark conf, skew join handling helped us a lot here.
Tried that, partial improvement but lag still spikes on redeploy.
+1, saw identical behaviour after upgrading Spark 3.4 to 3.5.
Sign in to reply.