Fill with the group median and flag it

Pandas data engineering interview problem. Difficulty: intermediate. Pattern: Missing Data. About 14 minutes. Part of the Pro drill bank.

Fill each missing temperature with the median of its site and mark which values were filled. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

df has sensor readings: reading_id, site and temp. Some temp values are missing. Fill each missing temp with the median temp of the same site (the middle value when sorted; with an even number of values, the average of the two middle ones), computed from the readings that are present. Add a column was_imputed that is True for a row whose value was missing and got filled, and False otherwise. A site with no readings at all keeps its missing value and is not flagged. Keep the rows in their original order and the columns reading_id, site, temp, was_imputed. Assign the DataFrame to result.

Requirements

• Return the columns reading_id, site, temp, was_imputed.
• Keep the original row order.

Examples

Input: df reading_id | site | temp 1 | A | 20 2 | A | NULL 3 | A | 22 4 | B | 30 5 | B | 34 6 | B | NULL 7 | C | NULL Output: reading_id | site | temp | was_imputed 1 | A | 20 | False 2 | A | 21 | True 3 | A | 22 | False 4 | B | 30 | False 5 | B | 34 | False 6 | B | 32 | True 7 | C | NULL | False Site A fills 21 (the median of 20 and 22), site B fills 32. Site C has no readings, so reading 7 stays empty and unflagged.

Topics: lakebench, pandas, fillna, groupby, median, imputation.

More interview problems · All interview problems · Learn data engineering