Python: one problem shape at a time

Look for

• A line of text that must become fields
• Trim, lowercase or standardise before comparing
• Lines with a missing or bad field

Smallest shape

def parse_line(line):
    # strip each piece, then unpack into three names
    sku, qty, price = [part.strip() for part in line.split("|")]
    return {"sku": sku, "qty": int(qty), "price": float(price)}

The usual trap

split keeps empty fields. 'A1||9.5'.split('|') gives ['A1', '', '9.5'], and int('') raises ValueError: invalid literal for int() with base 10: ''. Decide what a missing field means before you convert it.

Not this

If the pattern is irregular (many formats in 1 column), a regular expression fits better than split. Use split when one separator always divides the same fields.

Look for

• How many times does each thing appear
• Group items by a key
• Find a record by its id

Smallest shape

def count_words(text):
    counts = {}
    for word in text.lower().split():
        counts[word] = counts.get(word, 0) + 1
    return counts

The usual trap

A defaultdict creates a key when you only look. After d = defaultdict(int), d['a'] += 1, and then reading d['zzz'], len(d) is 2 and the dict is {'a': 1, 'zzz': 0}. Use `in` or .get when you only want to check.

Not this

If you only need to know whether you have seen an item, a set is enough. A dict is for when each key carries a value, such as a count or a list.

Look for

• Sort records by one field, then another
• Break ties alphabetically
• The second largest, the closest, the top few

Smallest shape

def sort_products(rows):
    # sort by price, then by name when prices tie
    return sorted(rows, key=lambda r: (r["price"], r["name"]))

The usual trap

reverse=True reverses the tie-break too. For prices 5, 5, 9 and names b, a, c, sorting by (price, name) with reverse=True gives c, b, a. A descending price with an ascending name needs the key (-price, name), which gives c, a, b.

Not this

If you only need the single biggest or smallest, max or min with a key is enough and does not sort the whole list.

Look for

• Remove duplicates but keep the order
• Have I seen this before
• 2 items that add up to a target

Smallest shape

def unique_in_order(items):
    seen = set()
    out = []
    for item in items:
        if item not in seen:
            seen.add(item)
            out.append(item)
    return out

The usual trap

Set members must be hashable (in practice, unchangeable, like numbers, strings and tuples). set([[1], [2]]) raises TypeError: unhashable type: 'list'. Turn lists into tuples first. And list(set(items)) throws away the order.

Not this

Counting how often things appear is a dictionary job. Use a set when the only question is yes or no, and a dict when each key needs a value.

Look for

• Two sorted lists to combine
• Walk two lists, or both ends of one list
• One pass, no re-sorting

Smallest shape

def merge_sorted(a, b):
    i = j = 0
    out = []
    while i < len(a) and j < len(b):
        if a[i] <= b[j]:
            out.append(a[i])
            i += 1
        else:
            out.append(b[j])
            j += 1
    return out + a[i:] + b[j:]

The usual trap

The loop stops when one list runs out, and the other still has items. Without the last line's a[i:] + b[j:], merge_sorted([1, 4, 9], [2, 3]) returns [1, 2, 3] and drops 4 and 9.

Not this

A sliding window keeps a block of neighbouring values that moves along one list. Two pointers are two separate positions, over two sorted lists or from both ends of one list.

Look for

• Best 3-day or 7-day stretch
• A block of neighbouring values that moves along the data

Smallest shape

def best_window(values, k):
    window = sum(values[:k])
    best = window
    for i in range(k, len(values)):
        window += values[i] - values[i - k]
        best = max(best, window)
    return best

The usual trap

Off by one on the last window. For [3, 1, 4, 1, 5, 9, 2, 6] and k=3, taking max(sum(values[i:i + 3]) for i in range(len(values) - 3)) misses the last window and returns 16. The correct answer is 17, from 9 + 2 + 6.

Not this

Prefix sums (running totals stored in a list) answer many range questions after one pass. A sliding window is for one pass when the window moves in a single direction.

Look for

• Running total
• Sum of any range, asked many times
• Carry a value forward as you scan

Smallest shape

from itertools import accumulate

def running_totals(values):
    return list(accumulate(values))

The usual trap

Range sums need a leading 0. With values [5, 3, 8, 2], the totals are [5, 8, 16, 18]. To get the sum from index l to r you compute prefix[r] - prefix[l - 1]. For l = 0 that reads prefix[-1], the last item, and gives -2 instead of 16. Start the prefix list with 0 and use prefix[r + 1] - prefix[l].

Not this

A sliding window recomputes only what enters and leaves a fixed block. Prefix sums store totals so any block can be answered later.

Look for

• Overlapping time ranges
• Merge windows into the fewest
• Do these two bookings clash

Smallest shape

def merge_intervals(intervals):
    out = []
    for start, end in sorted(intervals):
        if out and start <= out[-1][1]:
            out[-1][1] = max(out[-1][1], end)
        else:
            out.append([start, end])
    return out

The usual trap

Whether touching ranges merge is a rule you must ask about. With [(1, 2), (2, 3)], comparing start <= previous end gives [[1, 3]], and start < previous end gives [[1, 2], [2, 3]]. Also sort first: [(5, 6), (1, 3), (2, 4)] only merges correctly once it is sorted.

Not this

If the ranges are already sorted and you are combining two lists, two pointers is enough. Intervals need a sort when the input order is unknown.

Look for

• A list inside a list, to any depth
• A function that has to call itself (recursion)
• Collect every value of one kind from nested data

Smallest shape

def flatten(items):
    out = []
    for item in items:
        if isinstance(item, list):
            out.extend(flatten(item))
        else:
            out.append(item)
    return out

The usual trap

A string is iterable, and each character is again a string. If you test for 'can I loop over it' instead of isinstance(item, list), flatten([1, ['ab']]) never reaches a plain value and fails with RecursionError.

Not this

If the structure is always one level deep, a list comprehension is simpler. Use recursion when the depth is not known.

Look for

• Too much data to hold in memory
• Process in batches of a fixed size
• Read one item at a time

Smallest shape

from itertools import islice

def batches(iterable, size):
    iterator = iter(iterable)
    while True:
        batch = list(islice(iterator, size))
        if not batch:
            return  # the stream is empty, so the generator ends
        yield batch  # hand back one batch, then pause until asked again

The usual trap

A generator can be read only once. After g = (x * x for x in range(3)), list(g) is [0, 1, 4] and a second list(g) is []. If you need the data twice, store it or build the generator again.

Not this

A list is better when the data is small and you need it more than once. A generator is for one pass over data that may not fit in memory.

Look for

• Run this step again if it fails
• Add timing or logging around many functions
• Add the same behaviour around many functions

Smallest shape

import functools
import time

def retry(times, delay=0.0):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, times + 1):
                try:
                    return func(*args, **kwargs)
                except ConnectionError:
                    if attempt == times:
                        raise
                    time.sleep(delay)
        return wrapper
    return decorator

The usual trap

Without functools.wraps, the decorated function takes the wrapper's name. After a decorator written without it, the wrapped function's __name__ is the wrapper's name ('w' in our check) instead of 'load', which makes logs and tracebacks hard to read. Also retry only temporary errors such as ConnectionError, not every Exception.

Not this

A decorator wraps a whole function, so the same behaviour applies to every call. To guard one block with cleanup that always runs, even on an error, use a with statement and a context manager.

Look for

• A job must rerun for every date it missed (a backfill)
• Which dates, partitions or messages a job should handle
• A small exact rule behind a pipeline ticket

Smallest shape

from datetime import date, timedelta

def dates_between(start, end):
    return [start + timedelta(days=d) for d in range((end - start).days + 1)]

The usual trap

Ranges are easy to get one short. For 2026-10-01 to 2026-10-04, (end - start).days is 3, but a backfill must cover 4 dates, so add 1 when both ends are included. State whether the end is included before writing the loop.

Not this

If the task is about the order jobs must run in (job A before job B), it is a different problem from which dates to run. Read which one the ticket asks for.