Sort and Limit

PySpark data engineering interview problem. Difficulty: beginner. Pattern: Filtering. About 12 minutes. Free to practice.

Return the top 3 rows by amount descending. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

You want a quick look at the biggest transactions. From the table df, sort the rows by amount from largest to smallest and return only the top 3 rows. All the columns stay. The table here is tiny, so we ask for 3. In real work you would use a bigger number such as 10. Save your final table in result.

Requirements

• Sort by amount from largest to smallest.
• Return only the first 3 rows after sorting.
• Keep all the columns of df.

Constraints

• The input table is df with the columns user_id, event_type, amount and ts.
• Use orderBy(F.col('amount').desc()) and limit(3).

Examples

Input: events by amount Output: user_id | event_type | amount | ts 2 | purchase | 220.0 | 2024-01-02 12:00:00 1 | purchase | 150.0 | 2024-01-01 10:00:00 3 | purchase | 110.0 | 2024-01-03 09:30:00 Descending amount then limit 3.

Topics: lakebench, pyspark, orderBy, limit.

More PySpark interview questions · All interview problems · Learn data engineering