PySpark data engineering interview problem. Difficulty: advanced. Pattern: Window Functions. About 24 minutes. Part of the Pro drill bank.
For each employee, the longest run of consecutive calendar days marked present. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.
attendance has employee, day (a date string) and present (1 or 0). A day with no row is not a present day. For each employee, return longest_streak: the length of the longest run of consecutive calendar days on which present = 1. An employee who was never present has a streak of 0. Columns: employee, longest_streak. Assign the DataFrame to result.
Input: attendance employee | day | present e1 | 2024-05-01 | 1 e1 | 2024-05-02 | 1 e1 | 2024-05-03 | 1 e1 | 2024-05-04 | 0 e1 | 2024-05-05 | 1 e1 | 2024-05-06 | 1 e2 | 2024-05-01 | 1 e2 | 2024-05-03 | 1 e2 | 2024-05-04 | 1 e3 | 2024-05-02 | 0 Output: employee | longest_streak e1 | 3 e2 | 2 e3 | 0 e1 has runs of 3 and 2 days, so 3. e2 is present on May 1, then May 3 and 4: the longest run is 2. e3 was never present.
Topics: lakebench, pyspark, gaps and islands, row_number, datediff.
More PySpark interview questions · All interview problems · Learn data engineering
Interview-style drill: For each employee, the longest run of consecutive calendar days marked present.
`attendance` has `employee`, `day` (a date string) and `present` (1 or 0). A day with no row is not a present day. For each employee, return `longest_streak`: the length of the longest run of **consecutive calendar days** on which `present = 1`. An employee who was never present has a streak of 0. Columns: `employee`, `longest_streak`. Assign the DataFrame to `result`.