Profile every column of a frame

Pandas data engineering interview problem. Difficulty: beginner. Pattern: Data Quality. About 12 minutes. Free to practice.

Summarise each column: its kind, how many values are present, missing and distinct. Treat this as a production helper: match the contracted return shape, including empty and duplicate inputs.

df is a small extract of an orders feed. Before using it you want a profile. Return 1 row per column of df, in the same order as the columns of df, with: column: the column name kind: number if pandas treats the column's data type as numeric, text otherwise (decide from the data type, not from the values) non_null: how many values are present nulls: how many values are missing unique: how many different values there are, not counting missing ones Assign the DataFrame to result.

Requirements

• Return the columns column, kind, non_null, nulls, unique.
• 1 row per column, in the order of df.columns.

Constraints

• df has between 1 and 20 columns.
• A column is either numeric or holds text.

Examples

Input: df order_id | customer | amount | channel 1 | Ann | 10.5 | web 2 | Bo | 20 | app 3 | NULL | NULL | web 4 | Ann | 7.25 | NULL 5 | Cy | 20 | NULL 6 | Bo | 3 | web Output: column | kind | non_null | nulls | unique order_id | number | 6 | 0 | 6 customer | text | 5 | 1 | 3 amount | number | 5 | 1 | 4 channel | text | 4 | 2 | 2 customer has 5 present values and 3 distinct names, amount has 5 present values of which 20.0 appears twice, and channel is missing twice.

Topics: lakebench, pandas, profiling, nulls, nunique.

More interview problems · All interview problems · Learn data engineering