Anti-Join

PySpark data engineering interview problem. Difficulty: beginner. Pattern: Joins. About 12 minutes. Free to practice.

Find orders whose user_id is missing from users. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

Data checks often need to find rows that have no partner in another table. Here you want the orders that point to a user who does not exist, which usually means broken or missing data. A left anti join returns the rows of the left table that have no match in the right table. It returns only the columns of the left table. Use a left anti join to get the orders in orders whose user_id is not in users. Select order_id, user_id and amount, and sort by order_id. The editor creates orders and users and ends with result = orders. Change the last line so result holds your answer.

Requirements

• Use how="left_anti" to keep orders with no matching user.
• Select order_id, user_id, amount.
• Sort by order_id.
• Store the DataFrame in result.

Constraints

• Join on user_id, with orders as the left table.
• A left anti join returns only columns from the left table, so you cannot select name or email.
• The result contains no rows from users.

Examples

Input: orders + users Output: order_id | user_id | amount 104 | 9 | 40.0 Only user_id 9 is absent from users.

Topics: lakebench, pyspark, left_anti.

More PySpark interview questions · All interview problems · Learn data engineering