Intermediate · 25 min · 5 steps
Updating old rows and adding new rows in one safe step
Rajeshwari Iyer, Lakehouse Systems Engineer at BharatInventory Systems
It is Thursday, 11:15 AM. Rajeshwari Iyer is a data engineer at BharatInventory Systems, a company whose software helps over 60,000 supplier shops across India manage stock. She has been called into an urgent meeting about a data problem with the head of architecture. A job that runs at night copies each shop's subscription plan and wallet balance from billing into the analytics warehouse (the lakehouse).
The question
How do we label the new batch of rows, count what will happen, build the merged table, and check that it has no repeated accounts?
- Upsert merge
- SCD Type 1
- FULL OUTER JOIN
- Data validation
The rest of this case study is locked
The data, every step with its code and result, and the final answer open with a pass. A few case studies stay free so you can see how they work.