Duplicate grain check

Pandas data engineering interview problem. Difficulty: intermediate. Pattern: Data Quality. About 12 minutes. Part of the Pro drill bank.

Report the order lines whose order number and line number appear more than once. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

df holds order lines with order_id, line_no, sku and qty. The grain of the table is 1 row per order_id and line_no pair, so no pair may appear twice. Return the pairs that break the rule, with the number of rows each one has, in the columns order_id, line_no and rows. Sort by order_id, then line_no, and reset the index. If the table is clean, return an empty frame with those columns. Assign the DataFrame to result.

Requirements

• Return the columns order_id, line_no, rows.
• Sort by order_id, line_no and reset the index.

Examples

Input: df order_id | line_no | sku | qty 1 | 1 | A | 1 1 | 2 | B | 2 1 | 2 | B | 2 2 | 1 | C | 1 2 | 1 | C | 3 3 | 1 | D | 1 3 | 2 | E | 1 Output: order_id | line_no | rows 1 | 2 | 2 2 | 1 | 2 Pair (1, 2) and pair (2, 1) each appear twice. Order 3 has two different lines, which is fine.

Topics: lakebench, pandas, duplicated, composite key, quality gate.

More interview problems · All interview problems · Learn data engineering