7
Accepted answer
Delta Lake or Iceberg table history gives you a physical version log for free. Pair that with a changelog doc for logical or semantic changes that don't show up in the schema diff.
Git handles our transform code fine, but when a dataset's schema or business logic changes, downstream consumers get no signal beyond "it broke." How do teams actually version data, not just code?
Accepted answer
Delta Lake or Iceberg table history gives you a physical version log for free. Pair that with a changelog doc for logical or semantic changes that don't show up in the schema diff.
How do you handle consumers who never migrate off v1?
Set an explicit sunset date in the deprecation notice and actually enforce it. An unmaintained v1 that never dies just becomes permanent technical debt.
Semantic versioning on table schemas: a MAJOR bump for breaking column changes, a deprecation window, and a new dataset version like orders_v2 rather than mutating in place.
Sign in to reply.