Parse a file with mixed delimiters

PySpark data engineering interview problem. Difficulty: beginner. Pattern: String Functions. About 12 minutes. Free to practice.

Split raw text lines containing both pipes and commas into structured columns with cast types. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

Given a DataFrame df with a single string column raw_line: Each row contains customer information formatted with mixed delimiters (pipes | and commas ,): 101|Alice,250.50|NY Parse each line into four structured columns: user_id: integer name: string amount: double state: string Sort by user_id ascending. Assign the final DataFrame to result.

Requirements

• Output columns must be: user_id, name, amount, state.
• Sort by user_id ascending.

Constraints

• raw_line contains exactly 3 delimiters.
• Delimiters are either pipe or comma.

Examples

Input: df raw_line 101|Alice,250.50|NY 102|Bob,120.00|CA 103|Charlie,45.25|WA 104|David,89.00|TX Output: user_id | name | amount | state 101 | Alice | 250.5 | NY 102 | Bob | 120 | CA 103 | Charlie | 45.25 | WA 104 | David | 89 | TX Each row is divided across pipes and commas into individual typed columns.

Topics: lakebench, pyspark, regex-split, schema.

More PySpark interview questions · All interview problems · Learn data engineering