Filter and Select Columns

PySpark data engineering interview problem. Difficulty: beginner. Pattern: Filtering. About 12 minutes. Free to practice.

Keep purchase rows with amount > 100 and project user_id, amount. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

You have a table of events called df. Each row says which user did what and how much money was involved. You only care about big purchases. Keep the rows where event_type is 'purchase' and amount is greater than 100. Then keep only 2 columns: user_id and amount. Sort by user_id, and by amount when the user is the same. Save your final table in result. The editor already creates df for you. At the bottom of the starter code, result = df is a placeholder. Replace it with your own code.

Requirements

• Keep only rows where event_type == 'purchase'.
• Keep only rows where amount > 100.
• Both rules must be true for the same row.
• Return only the columns user_id and amount, in that order.
• Sort by user_id and then by amount.

Constraints

• df has the columns user_id, event_type, amount and ts.
• Join the two filters with & (and put each one in brackets), not with the word and.
• Return only the columns user_id and amount.

Examples

Input: df events fixture Output: user_id | amount 1 | 150.0 2 | 220.0 3 | 110.0 Views, refunds, and purchases <= 100 are dropped.

Topics: lakebench, pyspark, filter, select.

More PySpark interview questions · All interview problems · Learn data engineering