Day 11Phase 02 · Python for Data Engineering

Production Python

The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.

By the end of today you can

  • Write automated unit tests using pytest to verify data transformation logic and edge cases.
  • Implement exponential backoff retries with jitter for unreliable network and database calls.
  • Design data transformation functions to be deterministic and idempotent on reruns.

Core topics: config management, structured logging, custom exceptions, retries & backoff, idempotency, pytest

Build: Modular pipeline (extract, transform, load, config, utils) with idempotent upserts and automated tests.

Production pipelines survive network blips, malformed payloads, and repeated Airflow retries safely.

Already have a pass? Go to your account. Need a pass? See pricing.