Beginner · 25 min · 5 steps
Which incoming events are broken, and how do we read the good ones?
Tarun Banerjee, Data Quality Lead at SensorGrid IoT
It is Friday, 9:30 AM. Tarun Banerjee checks the quality of data at SensorGrid IoT, a company that makes smart sensors for factories and vehicles in India. These sensors send millions of small messages every hour. At 4:00 AM the night job that loads this data crashed, and the factory dashboards are now empty.
The question
Which of the incoming JSON messages are valid, what percentage are broken, and how do we pull clean columns out of the good ones?
- JSON validation
- Data quarantine
- JSON extraction
- Data hygiene
The rest of this case study is locked
The data, every step with its code and result, and the final answer open with a pass. A few case studies stay free so you can see how they work.