Lazily read and filter a large log iterator

Python data engineering interview problem. Difficulty: intermediate. Pattern: Strings. About 15 minutes. Part of the Pro drill bank.

A log file is too large to load at once. A helper should iterate lines and yield only the ones that look like errors.

A service writes thousands of log lines, and you only care about the ones that start with ERROR:. Go through the lines and, for each one that starts with ERROR:, make a dictionary with level set to 'ERROR' and msg set to the rest of the line with spaces trimmed. Return the dictionaries in order. Write parse_error_lines(lines). From ['INFO: ok', 'ERROR: disk', 'ERROR: timeout '] the result is [{'level': 'ERROR', 'msg': 'disk'}, {'level': 'ERROR', 'msg': 'timeout'}]. The starter file already has the function name and its print lines. Keep them and fill in the body.

Requirements

• Keep only lines that start with ERROR:.
• Set level to 'ERROR'.
• Set msg to the text after the prefix, with spaces trimmed.
• Keep the line order.

Constraints

• lines is a list of strings (or any iterable of strings).
• Other levels, like INFO, are ignored.

Examples

Input: parse_error_lines(['INFO: ok', 'ERROR: disk', 'ERROR: timeout ']) Output: [{'level': 'ERROR', 'msg': 'disk'}, {'level': 'ERROR', 'msg': 'timeout'}] This input follows the stated rules and produces this output.

Topics: lakebench, python, iterators.

More Python interview questions · All interview problems · Learn data engineering