Home › Data engineering › Delta vs Iceberg
Data engineering · free guide · one page
Delta Lake vs Apache Iceberg, and what differs on Databricks
Delta Lake and Apache Iceberg both turn a folder of Parquet files into a table you can update safely. This page shows how each does it, then the Databricks features that behave differently for each (change data feed, streaming, pipelines, lineage, clustering, masks), and which to pick on AWS.
In plain words
A contents page over the same files
A data lake on its own is a pile of Parquet files. To know which files make up the table right now, which ones were replaced, and what the columns are, you need a contents page. That contents page is the table format. It is what makes updates all-or-nothing, lets two writers work without corrupting each other, and lets you read the table as it was last Tuesday.
Delta keeps its contents page as an ordered diary of changes next to the data. Iceberg keeps it as a tree of metadata files, and a catalog holds the pointer to the current top of the tree. The data files underneath are the same kind of Parquet, which is why Databricks can write Iceberg metadata for a Delta table (UniForm) without copying any data.
Same Parquet files, two different contents pages. The catalog matters more for Iceberg: its pointer decides which version is current.
The formats
Side by side
| Delta Lake | Apache Iceberg | |
|---|---|---|
| How the table is described | An ordered log of commits (JSON) plus checkpoints, next to the data | A tree: metadata file, manifest list per snapshot, manifests, data files |
| What makes a commit atomic | Writing the next log file; Unity Catalog can coordinate commits (catalog-managed commits) | The catalog swaps its pointer to the new metadata file, so a catalog is required |
| Data files | Parquet | Parquet (also ORC, Avro) |
| Partitioning | Liquid clustering (recommended) or classic partitions | Hidden partitioning with transforms (day, bucket) and partition evolution; on Databricks, liquid clustering |
| Renaming and dropping columns | Needs column mapping turned on | Built in: every column has an id |
| Deletes and updates | Deletion vectors mark deleted rows; files rewritten later | v2: position or equality delete files; v3: deletion vectors |
| Time travel | VERSION AS OF, TIMESTAMP AS OF; shallow and deep clones | Snapshots, plus named branches and tags |
| Row-level change tracking | Change data feed; row tracking (row ids) | v3: row lineage (a permanent id per row); no change feed in the spec |
| Semi-structured data | VARIANT | VARIANT (v3) |
| Streaming | First-class source and sink for Spark Structured Streaming and Lakeflow pipelines | Strong with Flink; Spark reads new appends incrementally |
| Who reads and writes it | Databricks, Microsoft Fabric (OneLake), Spark, Trino, DuckDB, delta-rs; others through Iceberg reads (UniForm) | Snowflake, AWS (Athena, Glue, EMR, Redshift, S3 Tables), BigQuery, Trino, Flink, Dremio, DuckDB, Databricks |
| Housekeeping | OPTIMIZE, VACUUM, ANALYZE (predictive optimization on Databricks) | Rewrite data files, expire snapshots, remove orphan files (the catalog or predictive optimization runs them) |
On Databricks
Which Databricks features differ by format
Databricks offers four kinds of table. Green works, yellow works with a limit, rose does not. Checked against the Databricks docs in November 2026; Iceberg support changes quickly, so check items marked "check" before you rely on them.
| Feature | Managed Delta | Delta with Iceberg reads (UniForm) | Managed Iceberg in Unity Catalog | Foreign Iceberg (Glue, Snowflake, Hive) |
|---|---|---|---|---|
| Status on Databricks | Generally available | Generally available (Runtime 14.3+) | Public Preview in the docs (Runtime 16.4+); announced as GA. Check before production | Read: generally available |
| Who can write | Databricks and Delta writers | Databricks writes; Iceberg engines read only | Databricks and any Iceberg engine through the Unity Catalog REST catalog | Its own catalog's engines; Databricks reads |
| Change data feed (table_changes, readChangeFeed) | Yes: delta.enableChangeDataFeed = true, or the automatic feed from row tracking | Yes for Databricks readers; Iceberg readers cannot see it | Only on Iceberg v3 (automatic feed from row lineage, Runtime 18+), only for Databricks readers | No |
| Streaming read with Structured Streaming | Yes | Yes, as Delta | Through the automatic change feed on v3; check | No (older docs: not supported) |
| Lakeflow streaming tables and materialized views | They are Delta tables | Iceberg reads cannot be turned on for them; Compatibility Mode (preview) writes a read-only copy | Pipelines write Delta | No |
| Data lineage in Unity Catalog | Yes, for work run on Databricks | Databricks work yes; reads by outside Iceberg engines are not recorded | Databricks work yes; writes by outside engines through the REST catalog are not recorded | Reads on Databricks only |
| Row ids (row tracking, row lineage) | Row tracking, a table property | Row tracking | Always on for v3 | Depends on the source |
| Liquid clustering | Yes | Yes | Yes; no expression transforms (years(), bucket()) through Databricks SQL; partition evolution only from outside engines | Owned by the other catalog |
| Predictive optimization | Yes | Yes | Required: tables can only be created with it on | The other catalog maintains them |
| Deletion vectors | Yes | Not with Iceberg v2 reads; fine with v3 | On by default for v3 | Depends on the source |
| Row filters and column masks | Yes | Yes | Not supported | No |
| Primary and foreign keys (informational) | Yes | Yes | Not supported | No |
| Time travel | Yes | Yes (Delta side) | Yes (snapshots) | Read |
| Upgrade to v3 / new features | Table features can be added | Can be downgraded | Upgrading to v3 is one way | Source decides |
Change data feed
Change data feed (the "enable CDC" switch)
A change data feed lets a reader ask "what changed in this table since version 12?" and get the inserted, updated and deleted rows, instead of re-reading the whole table. Gold builds use it to find which hours changed.
-- Delta: turn it on per table
ALTER TABLE silver.press_readings SET TBLPROPERTIES (delta.enableChangeDataFeed = true);
-- Read the changes since version 12 (batch)
SELECT * FROM table_changes('silver.press_readings', 12);
-- Or as a stream
spark.readStream.option("readChangeFeed", "true").option("startingVersion", 12).table("silver.press_readings")The same table_changes and readChangeFeed calls work on managed Iceberg v3 tables, where Databricks computes the feed from row lineage.
Outside Databricks, Iceberg has no change feed in its spec; Spark's Iceberg library can build a changelog view between two snapshots, and Flink reads Iceberg changes natively.
Lineage
Two different things called lineage
| Kind | What it is | Used for |
|---|---|---|
| Data lineage | Which job, notebook or query read which tables and columns to produce which table. Unity Catalog records it for work run on Databricks compute, for any format. | Impact analysis, audit, 'where did this number come from' |
| Row lineage (Iceberg v3) or row tracking (Delta) | A permanent id and a version number on every row, kept by the table itself. | Cheap change feeds, incremental materialized views, matching a row across rewrites |
Unity Catalog lineage is format-neutral but only sees work done on Databricks compute. If Trino or Flink writes a managed Iceberg table through the REST catalog, that write is missing from the lineage graph; record it with OpenLineage from the other engine if you need the full picture.
Choosing
Which to pick, by platform
| Situation | Choice | Verdict |
|---|---|---|
| AWS + Databricks (this site's default) | Managed Delta for bronze, silver and gold. Turn on Iceberg reads (UniForm) only for the gold tables that Athena, Redshift or Snowflake must read. | Pursue |
| AWS + Databricks, another engine must write the same table | Managed Iceberg in Unity Catalog, written through its REST catalog. Mind: no row filters or masks, no keys, change feed only on v3 and only for Databricks readers. | Pursue only where needed, after checking it is GA |
| Tables owned by Glue, S3 Tables or Snowflake | Foreign Iceberg: read in place through catalog federation; do not copy. | Pursue |
| AWS only (no Databricks) | Iceberg in S3 Tables, with Athena, Glue and Redshift. | Pursue; Delta here would be a second-class citizen |
| Azure with Fabric | Delta: OneLake is Delta-native. | Pursue |
| Streaming lane on Databricks | Keep streaming tables and materialized views Delta; Iceberg reads cannot be turned on for them, and the change feed drives the gold window recompute. | Pursue Delta |
| Convert everything to Iceberg to be 'open' | Delta is open source too, and UniForm already serves Iceberg readers. Converting adds risk and loses masks and keys today. | Drop |
FAQ
Questions people ask
Is Delta Lake or Iceberg better?
Neither in general. Both put the same Parquet files under a different contents page. Delta is the native format on Databricks and Fabric; Iceberg is the native format on AWS analytics services and Snowflake. On Databricks, keep Delta and turn on Iceberg reads where other engines need it.
Does change data feed work on Iceberg tables in Databricks?
Only on Iceberg v3 managed tables, where Databricks computes the feed from row lineage, and only Databricks readers can use it. Delta tables have it with delta.enableChangeDataFeed = true, or automatically with row tracking.
Can Lakeflow streaming tables be Iceberg?
No. Pipelines write Delta, and Iceberg reads (UniForm) cannot be turned on for streaming tables or materialized views. Compatibility Mode, in preview, can write a read-only copy for other engines.
Does Unity Catalog lineage work for Iceberg tables?
For work run on Databricks, yes. Reads and writes by outside engines through the Iceberg REST catalog are not recorded, so lineage has gaps when other engines write.
What is the difference between data lineage and row lineage?
Data lineage says which job made which table from which tables. Row lineage (Iceberg v3) or row tracking (Delta) gives every row a permanent id inside the table, used for change feeds and incremental updates.
Sources
Checked against
Related
Where it connects
Medallion architecture · Streaming on AWS and Databricks · Data platform on AWS + Databricks · Delta Lake internals, recorded · Who maintains your tables · All data engineering