Home › Key concepts › Schema evolution
Key concept · Walk (intermediate) · the same on every platform
Schema evolution
How a table's columns can change over time without breaking readers.
What it is
Adding a nullable column is usually safe. Renaming, removing or changing a column's type usually breaks someone. Table formats can allow safe changes automatically and block unsafe ones.
See it happen: Problem Lab: a renamed column, and why auto-evolve is not the fix (simulation), recorded from a real run.
Why it matters
Sources change all the time. Deciding what is allowed, up front, keeps pipelines running.
An example
A new 'discount_code' column appears in the source and is added to bronze automatically; a type change from number to text is blocked and alerts the owner.
The common mistake
Letting every schema change through silently, and finding the broken report a month later.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Open table formatfiles that behave like a database table, with history | Apache Iceberg (S3 Tables) | Delta Lake in OneLake | Iceberg on BigLake | Delta Lake | Delta Lake or Iceberg |
Related ideas
Where it shows up
Part of the Walk level on these paths: Data Engineering (trusting data).