Day 42Phase 10 · Job-ready engineering
Shipping code safely
The curriculum for this day stays visible. The lesson, code, and workspace unlock with a pass.
By the end of today you can
- Structure Git workflows with feature branches, descriptive pull requests, and atomic commits.
- Configure automated CI workflows using GitHub Actions to run linters, type checks, and pytest suites.
- Containerize data applications with Docker and manage configuration and secrets through environment variables.
Trunk-Based Development & The CI/CD Deployment Airlock
In amateur setups, engineers log into live database servers over SSH and modify Python code directly with text editors.
When an unversioned hotfix runs on a server, nobody knows who made the change or why. The next automated deployment wipes out that edit, re-breaking the pipeline unexpectedly.
This is the first section of the lesson. The rest unlocks with a pass.
Core topics: Git trunk-based development & feature branching, Data pipeline CI/CD with automated testing stages, Docker containerization for reproducible environments, Secrets management & environment variables, Infrastructure as Code principles, Observability: Metrics, logs, traces, and SLAs
Build: Production CI/CD & validation pipeline: GitHub Actions simulator → linting → SQL fluff → PySpark unit tests → Docker build → staging deployment gate.
Never test data pipelines in production. Enforce automated pre-merge testing with mock datasets to catch bugs before they corrupt production tables.
Already have a pass? Go to your account. Need a pass? See pricing.