Forward-fill per device

Pandas data engineering interview problem. Difficulty: intermediate. Pattern: Missing Data. About 14 minutes. Free to practice.

Fill missing readings from the same device only, never across devices. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

Sensors sometimes miss a reading and the value is empty (NaN). A forward fill replaces each gap with the last known value, but only for the same device. df has device, ts (a time as text) and reading. The rows are not in time order. Fill each missing reading with the most recent earlier reading of the same device. A device that has no earlier reading keeps the missing value. Keep the 3 columns, sort by device and then ts, and give the result a clean index. The editor builds df and ends with result = df. Change the last line. Use groupby("device")["reading"].ffill() after sorting.

Requirements

• Sort the rows by device and ts first.
• Fill the gaps inside each device with groupby("device")["reading"].ffill().
• Keep device, ts, reading, with a clean index.
• Store the DataFrame in result.

Constraints

• The fill must not carry a value from one device to the next.
• ts is a time as text (09:00) that sorts in time order.
• Leading gaps stay empty, because there is nothing earlier to copy.

Examples

Input: df device | ts | reading d1 | 09:20 | NULL d1 | 09:00 | 10.5 d1 | 09:10 | NULL d2 | 09:10 | 7.5 d2 | 09:00 | NULL d2 | 09:20 | NULL d3 | 09:00 | NULL Output: device | ts | reading d1 | 09:00 | 10.5 d1 | 09:10 | 10.5 d1 | 09:20 | 10.5 d2 | 09:00 | NULL d2 | 09:10 | 7.5 d2 | 09:20 | 7.5 d3 | 09:00 | NULL d1's gaps take 10.5. d2's first reading is missing and has nothing earlier; its later gap takes 7.5. d3 only has a missing value.

Topics: lakebench, pandas, ffill, groupby, missing data.

More interview problems · All interview problems · Learn data engineering