PySpark data engineering interview problem. Difficulty: intermediate. Pattern: Explain Plans. About 16 minutes. Part of the Pro drill bank.
Replace a slow UDF banding path with when. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.
A colleague banded amounts with a Python UDF and the job crawled. Produce the same Low/Medium/High bands with when/otherwise. Return user_id, amount, band ordered by user_id. Assign result.
Input: amounts Output: user_id | amount | band 1 | 20.0 | Low 4 | 250.0 | High 6 | 200.0 | Medium Same labels without a UDF.
Topics: lakebench, pyspark, when, performance.
More PySpark interview questions · All interview problems · Learn data engineering
Interview-style drill: Replace a slow UDF banding path with when.
A colleague banded amounts with a Python UDF and the job crawled. Produce the same Low/Medium/High bands with `when`/`otherwise`. Return `user_id`, `amount`, `band` ordered by `user_id`. Assign `result`.