GroupBy with Filter (HAVING)

PySpark data engineering interview problem. Difficulty: intermediate. Pattern: GroupBy. About 16 minutes. Part of the Pro drill bank.

Users with more than 2 transactions. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

In SQL, HAVING filters groups after you have counted or added them. In PySpark you do the same by filtering after the agg. Group df by user_id and count the rows of each user as txn_count. Keep only the users with more than 2 transactions. Return user_id and txn_count, sorted by user_id. The editor creates the table df and ends with result = df. Change the last line to build your answer.

Requirements

• Group by user_id and count the rows as txn_count.
• Keep only groups where txn_count is greater than 2.
• Return user_id and txn_count, sorted by user_id.
• Store the DataFrame in result.

Constraints

• The filter must come after the aggregation, because the count only exists after grouping.
• Use a strict comparison: > 2, which means 3 or more.
• The count is the number of rows, F.count("*").

Examples

Input: transactions Output: user_id | txn_count 1 | 3 Only user 1 clears the > 2 bar.

Topics: lakebench, pyspark, having, count.

More PySpark interview questions · All interview problems · Learn data engineering