Rename Columns

PySpark data engineering interview problem. Difficulty: intermediate. Pattern: Schema Drift. About 16 minutes. Part of the Pro drill bank.

Different teams name the same column in different ways. One uses user_id, another uses uid. When you load data from another source you often have to rename its columns to match your own naming. PySpark does this with withColumnRenamed.

The table df has the columns user_id, event_type and amount. Rename them to uid, etype and amt. The data stays the same. Only the names change. Sort the result by uid and store it in result. The editor creates the table df and ends with result = df. Change the last line to build your answer.

Requirements

• Rename user_id to uid.
• Rename event_type to etype.
• Rename amount to amt.
• Sort by uid and store the DataFrame in result.

Constraints

• Use withColumnRenamed(old_name, new_name) once for each column.
• The values and the number of rows do not change.
• Sort by the new name, uid, because the old name no longer exists.

Examples

Input: events Output: uid | etype | amt 1 | purchase | 10.0 2 | view | 0.0 3 | purchase | 20.0 Exact rename map.

Topics: lakebench, pyspark, withColumnRenamed.

More PySpark interview questions · All interview problems · Learn data engineering