Manoj Sharma

@manojsharma

Free member

Former analyst, now building pipelines with Airflow. Based in Madurai.

Madurai, Tamil Nadu0–2 years1 questions2 answers

AirflowGCPIcebergFlinkBigQuery

Streak: 0d
XP: 394
Lessons done: 4
Studio passes: 2

First LessonFirst Ticket

Share on X Share on LinkedIn

Questions

votes

answers

views

spark.sql.shuffle.partitions default of 200, how do you actually tune it

Every tuning guide says "don't leave it at 200" but picking the actual number feels like folklore. How do you arrive at a real value instead of guessing?

Apache Sparksparkshuffle