Day 11Phase 02 · Python for Data Engineering
Production Python
The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.
By the end of today you can
- Write automated unit tests using pytest to verify data transformation logic and edge cases.
- Implement exponential backoff retries with jitter for unreliable network and database calls.
- Design data transformation functions to be deterministic and idempotent on reruns.
Core topics: config management, structured logging, custom exceptions, retries & backoff, idempotency, pytest
Build: Modular pipeline (extract, transform, load, config, utils) with idempotent upserts and automated tests.
Production pipelines survive network blips, malformed payloads, and repeated Airflow retries safely.
Already have a pass? Go to your account. Need a pass? See pricing.