Skip duplicate events
Python data engineering interview problem. Difficulty: beginner. Pattern: Streaming. About 10 minutes. Free to practice.
Remove events that were delivered more than once, keeping the first copy of each id. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.
A stream can deliver the same event more than once. Each event is a dict with a unique-looking id. Implement dedupe_events(events). Keep the first event for each id and drop later events with the same id. Keep the remaining events in their original order. Two events with different ids are never duplicates, even if everything else is equal.
Requirements
- The first copy of each id is kept.
- The original order is kept.
Constraints
- Every event has an id.
- The input list is not changed.
Examples
Input: dedupe_events([{'id': 1, 'v': 'a'}, {'id': 2, 'v': 'b'}, {'id': 1, 'v': 'c'}]) Output: [{'id': 1, 'v': 'a'}, {'id': 2, 'v': 'b'}] The second copy of id 1 is dropped, and the first copy is kept.
Topics: lakebench, python, stream, dedup, event.
More Python interview questions · All interview problems · Learn data engineering
Skip duplicate events
Interview-style drill: Remove events that were delivered more than once, keeping the first copy of each id.
A stream can deliver the same event more than once. Each event is a dict with a unique-looking `id`. Implement `dedupe_events(events)`. - Keep the first event for each `id` and drop later events with the same `id`. - Keep the remaining events in their original order. - Two events with different ids are never duplicates, even if everything else is equal.