10
Accepted answer
We saw the same issue, fixing the partition filter dropped runtime 60%.
This worked in dev on sample data but fails at full volume. Details:
Context: Predicate pushdown not happening in Spark SQL join
Happy to share schema snippets or metrics if useful.
Accepted answer
We saw the same issue, fixing the partition filter dropped runtime 60%.
We saw the same issue, fixing the partition filter dropped runtime 60%.
Note that merge on Delta still needs unique keys defined correctly.
Document the grain decision, most BI bugs turn out to be grain bugs.
Sign in to reply.