Home › Key concepts › Open table formats and ACID
Key concept · Walk (intermediate) · the same on every platform
Open table formats and ACID
Delta Lake, Apache Iceberg and Apache Hudi make files in storage behave like database tables.
What it is
A table format adds a transaction log on top of Parquet files. That gives ACID transactions (all-or-nothing writes that readers never see half-done), updates and deletes, schema changes, and time travel to read the table as it was yesterday.
See it happen: Delta Lake internals, recorded (simulation), recorded from a real run.
Why it matters
It turns a cheap data lake into something you can trust like a warehouse, and lets many engines read the same tables.
An example
A failed job wrote half its files, but readers never see them because the commit was never written to the log.
The common mistake
Writing plain Parquet folders by hand and then building your own fragile locking and cleanup.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Open table formatfiles that behave like a database table, with history | Apache Iceberg (S3 Tables) | Delta Lake in OneLake | Iceberg on BigLake | Delta Lake | Delta Lake or Iceberg |
Related ideas
Where it shows up
Part of the Walk level on these paths: Data Engineering (storing data).