Home › Data engineering › Delta vs Iceberg

Data engineering · free guide · one page

Delta Lake vs Apache Iceberg, and what differs on Databricks

Delta Lake and Apache Iceberg both turn a folder of Parquet files into a table you can update safely. This page shows how each does it, then the Databricks features that behave differently for each (change data feed, streaming, pipelines, lineage, clustering, masks), and which to pick on AWS.

In plain words

A contents page over the same files

A data lake on its own is a pile of Parquet files. To know which files make up the table right now, which ones were replaced, and what the columns are, you need a contents page. That contents page is the table format. It is what makes updates all-or-nothing, lets two writers work without corrupting each other, and lets you read the table as it was last Tuesday.

Delta keeps its contents page as an ordered diary of changes next to the data. Iceberg keeps it as a tree of metadata files, and a catalog holds the pointer to the current top of the tree. The data files underneath are the same kind of Parquet, which is why Databricks can write Iceberg metadata for a Delta table (UniForm) without copying any data.

Delta LakeApache Iceberg Catalog (Unity Catalog) _delta_log/ 00000.json 00001.json 00002.json … one file per commit, in order; a checkpoint every so often the newest log file is the truth Catalog (REST, UC, Glue) metadata.json: schema, snapshotsmanifest list (one snapshot) manifests: which data files, with column stats Parquet data files (+ deletion vectors)the same kind of files in both formats Parquet data files (+ delete files or deletion vectors)the catalog's pointer to metadata.json is the truth

Same Parquet files, two different contents pages. The catalog matters more for Iceberg: its pointer decides which version is current.

The formats

Side by side

Delta LakeApache Iceberg
How the table is describedAn ordered log of commits (JSON) plus checkpoints, next to the dataA tree: metadata file, manifest list per snapshot, manifests, data files
What makes a commit atomicWriting the next log file; Unity Catalog can coordinate commits (catalog-managed commits)The catalog swaps its pointer to the new metadata file, so a catalog is required
Data filesParquetParquet (also ORC, Avro)
PartitioningLiquid clustering (recommended) or classic partitionsHidden partitioning with transforms (day, bucket) and partition evolution; on Databricks, liquid clustering
Renaming and dropping columnsNeeds column mapping turned onBuilt in: every column has an id
Deletes and updatesDeletion vectors mark deleted rows; files rewritten laterv2: position or equality delete files; v3: deletion vectors
Time travelVERSION AS OF, TIMESTAMP AS OF; shallow and deep clonesSnapshots, plus named branches and tags
Row-level change trackingChange data feed; row tracking (row ids)v3: row lineage (a permanent id per row); no change feed in the spec
Semi-structured dataVARIANTVARIANT (v3)
StreamingFirst-class source and sink for Spark Structured Streaming and Lakeflow pipelinesStrong with Flink; Spark reads new appends incrementally
Who reads and writes itDatabricks, Microsoft Fabric (OneLake), Spark, Trino, DuckDB, delta-rs; others through Iceberg reads (UniForm)Snowflake, AWS (Athena, Glue, EMR, Redshift, S3 Tables), BigQuery, Trino, Flink, Dremio, DuckDB, Databricks
HousekeepingOPTIMIZE, VACUUM, ANALYZE (predictive optimization on Databricks)Rewrite data files, expire snapshots, remove orphan files (the catalog or predictive optimization runs them)

On Databricks

Which Databricks features differ by format

Databricks offers four kinds of table. Green works, yellow works with a limit, rose does not. Checked against the Databricks docs in November 2026; Iceberg support changes quickly, so check items marked "check" before you rely on them.

FeatureManaged DeltaDelta with Iceberg reads (UniForm)Managed Iceberg in Unity CatalogForeign Iceberg (Glue, Snowflake, Hive)
Status on DatabricksGenerally availableGenerally available (Runtime 14.3+)Public Preview in the docs (Runtime 16.4+); announced as GA. Check before productionRead: generally available
Who can writeDatabricks and Delta writersDatabricks writes; Iceberg engines read onlyDatabricks and any Iceberg engine through the Unity Catalog REST catalogIts own catalog's engines; Databricks reads
Change data feed (table_changes, readChangeFeed)Yes: delta.enableChangeDataFeed = true, or the automatic feed from row trackingYes for Databricks readers; Iceberg readers cannot see itOnly on Iceberg v3 (automatic feed from row lineage, Runtime 18+), only for Databricks readersNo
Streaming read with Structured StreamingYesYes, as DeltaThrough the automatic change feed on v3; checkNo (older docs: not supported)
Lakeflow streaming tables and materialized viewsThey are Delta tablesIceberg reads cannot be turned on for them; Compatibility Mode (preview) writes a read-only copyPipelines write DeltaNo
Data lineage in Unity CatalogYes, for work run on DatabricksDatabricks work yes; reads by outside Iceberg engines are not recordedDatabricks work yes; writes by outside engines through the REST catalog are not recordedReads on Databricks only
Row ids (row tracking, row lineage)Row tracking, a table propertyRow trackingAlways on for v3Depends on the source
Liquid clusteringYesYesYes; no expression transforms (years(), bucket()) through Databricks SQL; partition evolution only from outside enginesOwned by the other catalog
Predictive optimizationYesYesRequired: tables can only be created with it onThe other catalog maintains them
Deletion vectorsYesNot with Iceberg v2 reads; fine with v3On by default for v3Depends on the source
Row filters and column masksYesYesNot supportedNo
Primary and foreign keys (informational)YesYesNot supportedNo
Time travelYesYes (Delta side)Yes (snapshots)Read
Upgrade to v3 / new featuresTable features can be addedCan be downgradedUpgrading to v3 is one waySource decides
In short: everything works on managed Delta. Iceberg reads (UniForm) add outside readers without giving anything up, except on streaming tables and materialized views. Managed Iceberg lets other engines write, but today gives up row filters, column masks and keys, and has a change feed only on v3 and only for Databricks readers.

Change data feed

Change data feed (the "enable CDC" switch)

A change data feed lets a reader ask "what changed in this table since version 12?" and get the inserted, updated and deleted rows, instead of re-reading the whole table. Gold builds use it to find which hours changed.

-- Delta: turn it on per table
ALTER TABLE silver.press_readings SET TBLPROPERTIES (delta.enableChangeDataFeed = true);

-- Read the changes since version 12 (batch)
SELECT * FROM table_changes('silver.press_readings', 12);

-- Or as a stream
spark.readStream.option("readChangeFeed", "true").option("startingVersion", 12).table("silver.press_readings")

The same table_changes and readChangeFeed calls work on managed Iceberg v3 tables, where Databricks computes the feed from row lineage.

Outside Databricks, Iceberg has no change feed in its spec; Spark's Iceberg library can build a changelog view between two snapshots, and Flink reads Iceberg changes natively.

Lineage

Two different things called lineage

KindWhat it isUsed for
Data lineageWhich job, notebook or query read which tables and columns to produce which table. Unity Catalog records it for work run on Databricks compute, for any format.Impact analysis, audit, 'where did this number come from'
Row lineage (Iceberg v3) or row tracking (Delta)A permanent id and a version number on every row, kept by the table itself.Cheap change feeds, incremental materialized views, matching a row across rewrites

Unity Catalog lineage is format-neutral but only sees work done on Databricks compute. If Trino or Flink writes a managed Iceberg table through the REST catalog, that write is missing from the lineage graph; record it with OpenLineage from the other engine if you need the full picture.

Choosing

Which to pick, by platform

SituationChoiceVerdict
AWS + Databricks (this site's default)Managed Delta for bronze, silver and gold. Turn on Iceberg reads (UniForm) only for the gold tables that Athena, Redshift or Snowflake must read.Pursue
AWS + Databricks, another engine must write the same tableManaged Iceberg in Unity Catalog, written through its REST catalog. Mind: no row filters or masks, no keys, change feed only on v3 and only for Databricks readers.Pursue only where needed, after checking it is GA
Tables owned by Glue, S3 Tables or SnowflakeForeign Iceberg: read in place through catalog federation; do not copy.Pursue
AWS only (no Databricks)Iceberg in S3 Tables, with Athena, Glue and Redshift.Pursue; Delta here would be a second-class citizen
Azure with FabricDelta: OneLake is Delta-native.Pursue
Streaming lane on DatabricksKeep streaming tables and materialized views Delta; Iceberg reads cannot be turned on for them, and the change feed drives the gold window recompute.Pursue Delta
Convert everything to Iceberg to be 'open'Delta is open source too, and UniForm already serves Iceberg readers. Converting adds risk and loses masks and keys today.Drop

FAQ

Questions people ask

Is Delta Lake or Iceberg better?

Neither in general. Both put the same Parquet files under a different contents page. Delta is the native format on Databricks and Fabric; Iceberg is the native format on AWS analytics services and Snowflake. On Databricks, keep Delta and turn on Iceberg reads where other engines need it.

Does change data feed work on Iceberg tables in Databricks?

Only on Iceberg v3 managed tables, where Databricks computes the feed from row lineage, and only Databricks readers can use it. Delta tables have it with delta.enableChangeDataFeed = true, or automatically with row tracking.

Can Lakeflow streaming tables be Iceberg?

No. Pipelines write Delta, and Iceberg reads (UniForm) cannot be turned on for streaming tables or materialized views. Compatibility Mode, in preview, can write a read-only copy for other engines.

Does Unity Catalog lineage work for Iceberg tables?

For work run on Databricks, yes. Reads and writes by outside engines through the Iceberg REST catalog are not recorded, so lineage has gaps when other engines write.

What is the difference between data lineage and row lineage?

Data lineage says which job made which table from which tables. Row lineage (Iceberg v3) or row tracking (Delta) gives every row a permanent id inside the table, used for change feeds and incremental updates.

Sources

Checked against

Related

Where it connects

Medallion architecture · Streaming on AWS and Databricks · Data platform on AWS + Databricks · Delta Lake internals, recorded · Who maintains your tables · All data engineering