26
Accepted answer
If you need runtime validation of values coming from YAML or env vars, pydantic earns its keep. If the dict is already trusted and you just want IDE autocomplete, a frozen dataclass is enough.
Config is currently a plain dict passed around every function. Want to add structure without over-engineering it. Options on the table: dataclass, TypedDict, or pulling in pydantic.
What tips the decision one way for a small internal ETL tool?
Accepted answer
If you need runtime validation of values coming from YAML or env vars, pydantic earns its keep. If the dict is already trusted and you just want IDE autocomplete, a frozen dataclass is enough.
We saw the same issue, fixing the partition filter dropped runtime 60%.
Our dimension is slowly changing, does that change the join order?
Prefer a staging table plus validation gate before promoting to prod tables.
Sometimes the real fix is a product change so you stop needing that join at all.
Sign in to reply.
© 2026 Lakebench, operated by Hunnurji Rao. Bengaluru, Karnataka, India.
No cluster. No install. Just the tab.