Assign partitions across a consumer group

Python data engineering interview problem. Difficulty: beginner. Pattern: Hashing. About 12 minutes. Free to practice.

In a streaming system such as Kafka, a topic is split into partitions so many consumers can read it in parallel. A group of consumers shares the work: each partition must be read by exactly one consumer, so no message is read twice and none is skipped.

A topic has num_partitions partitions, numbered from 0 to num_partitions - 1. A consumer group has the members listed in consumer_ids. Write a function assign_partitions(num_partitions, consumer_ids) that returns a dictionary. Each key is a consumer id, and each value is the list of partitions it owns. Deal the partitions out in turns (round-robin): partition 0 goes to the first consumer, partition 1 to the second, and so on. When you reach the last consumer, go back to the first. For 6 partitions and the consumers ['c1', 'c2', 'c3'], the answer is {'c1': [0, 3], 'c2': [1, 4], 'c3': [2, 5]}. Keep the print lines at the bottom of the editor. The grader reads what your code prints.

Requirements

• Give every partition to exactly one consumer.
• Deal partitions out in turns, in the order of consumer_ids.
• Include every consumer id as a key, even if it ends up with no partitions.
• Keep each consumer's partition list in ascending order.

Constraints

• Partitions are numbered 0 to num_partitions - 1.
• num_partitions >= len(consumer_ids) in the tested cases, so every consumer gets at least one.
• Keep the print lines, because the grader reads the printed output.

Examples

Input: assign_partitions(6, ["c1", "c2", "c3"]) Output: {'c1': [0, 3], 'c2': [1, 4], 'c3': [2, 5]} This is the expected return value for the given call.

Topics: streaming, kafka, consumer-group.

More Python interview questions · All interview problems · Learn data engineering