Home › Learn › Medallion architecture

Data engineering · free guide · the same on every platform

Medallion architecture, drawn right

Part of Data platform architecture, drawn right: this page zooms into its middle row.

The most common way to organise a data platform: raw data in, clean data in the middle, business-ready data out, and each step simple enough to check and redo. This guide shows each layer, follows one order through all of them with real rows, and lists the rules, the mistakes and the products on each platform.

In plain words

Think of a kitchen

A restaurant never cooks straight from the delivery bag. Groceries go into a store room as they arrived, labelled with when they came. On the prep counter they are washed, chopped and the bad bits thrown out. Then they are cooked into a dish for one purpose, and the menu gives each dish one name and one price. If a dish goes wrong, the cook goes back to the store room, not to the supplier.

Groceriesas delivered,still in bagsStore roomunpacked, labelled,nothing thrown awayPrep counterwashed, chopped,bad bits removedThe dishcooked forone purposeThe menuone name per dish,one priceyou never cook straight from the delivery bag, and you can always go back to the store room
The same idea in a kitchen. Landing is the delivery bag, bronze the store room, silver the prep counter, gold the dish, semantic the menu.

A data platform works the same way. Bronze is the store room: everything as it arrived. Silver is the prep counter: cleaned, one of each thing. Gold is the dish: shaped for a use. The semantic layer is the menu: one meaning per number. The name "medallion" just means bronze, silver and gold, in order of polish.

The whole picture

The spine, with what sits above and below it

Read it left to right. Data moves one way. Above the layers sits the control plane, which turns a written description of each table (the manifest) into pipelines. Below them sit the cross-cutting jobs such as quality, lineage and cost, which run on every hop, not as a box at the end.

Control plane: the manifest (YAML) → validator → generator → pipelines · orchestration · catalog · run logAgents may only propose manifest changes by pull request; they never touch productionSourcesdatabases, files,APIs, eventsLandingfiles as received,never changedBronzeraw, append-only,+ audit columnsSilvertyped, deduped,one row per thingGoldshaped for a use:facts, dims, martsSemanticmetrics definedonce; no tablesExperiencesdashboards, AI,APIs, MLdata moves one way: each layer is built only from the one before itCross-cutting, on every hop: contracts and quality · reconciliation · lineage · freshness · cost · access and PII · auditdeclared once in the manifest, switched on by default, never bolted on at the end
The medallion, drawn right: a spine of layers, a control plane above it, and the cross-cutting jobs below it.

Each layer

Its job and its promise

Landing

Files exactly as they arrived, so a bad parse can be redone.

Holds
Files from the mover: CSV, JSON, Parquet, exports, API pages, untouched.
Writes
New files only. Never edited, never deleted by a pipeline.
Read by
The bronze loader only.
Keep for
Short: days or weeks. Long enough to replay a bad load.
Skip it when
Change data capture writes straight into bronze.

Bronze

Raw data in the source's own shape, append-only, with notes on where each row came from.

Holds
Every row ever received, in the source's columns and types (often all strings).
Adds
_ingested_at, _source, _batch_id, _file: when, from where, in which batch, from which file.
Writes
Append only. Loading the same batch again is detected by _batch_id and skipped.
Read by
Silver jobs only. No dashboard, no analyst, no model reads bronze.
Promise
You can always rebuild silver from bronze.

Silver

The first layer with an opinion: one clean row per real thing.

Holds
One row per business entity (an order, a customer), typed, de-duplicated, with conformed keys.
Does
Cast types, trim, standardise codes, fix time zones, handle nulls, keep the latest row per key, keep history (SCD Type 1 or 2).
Enforces
The data contract. A row that fails goes to quarantine with its reason; it is never silently dropped.
Writes
MERGE on the business key, so re-running changes nothing.
Read by
Gold jobs, data scientists, feature pipelines.

Gold

Tables shaped for a use: a sales mart, a star schema, a feature table, a data product.

Holds
Facts and dimensions, aggregates, marts, published data products.
Built from
Silver only, never bronze.
Checked by
Reconciliation: counts and sums agree with silver and with the source, within a tolerance.
Read by
The semantic layer, dashboards, sharing, ML.

Semantic

Definitions, not tables: each metric means one thing.

Holds
Metric definitions (net revenue = gross minus refunds, excluding test orders), entities, joins, governed names.
Read by
Dashboards and conversation UIs alike, so both give the same answer. The model never writes raw SQL.
Why
It ends the 'why do two dashboards show two revenues' meeting.

Experiences

Where people and programs use the data.

Includes
Dashboards, conversation UIs, APIs, data sharing, ML features, documents for AI (RAG).
Reads
Only from semantic or gold.

Worked example

One order, through every layer

A clothing shop sends a small file at 9 a.m. It has three rows, and two problems in it. Watch what each layer does with them.

1 · Landing: the file as it arrived

orders_2026-10-27_0900.json
{"order_id":"1001","customer":"C-77","amount":"1,250.00","currency":"inr","placed":"27/10/2026 08:58","status":"PAID "}
{"order_id":"1002","customer":null,"amount":"-40","currency":"INR","placed":"27/10/2026 08:59","status":"paid"}
{"order_id":"1001","customer":"C-77","amount":"1,250.00","currency":"inr","placed":"27/10/2026 08:58","status":"PAID "}

Order 1001 was sent twice. Order 1002 has no customer and a negative amount. Landing keeps all of it, untouched.

2 · Bronze: the same rows, plus where they came from

order_idcustomeramountcurrencyplacedstatus_ingested_at_source_batch_id_file
1001C-771,250.00inr27/10/2026 08:58"PAID "2026-10-27T03:31:02Zshop_dbb-0912orders_…0900.json
1002null-40INR27/10/2026 08:59paid2026-10-27T03:31:02Zshop_dbb-0912orders_…0900.json
1001C-771,250.00inr27/10/2026 08:58"PAID "2026-10-27T03:31:02Zshop_dbb-0912orders_…0900.json

Still strings (note the stray space after "PAID"), still the duplicate, still the bad row. Bronze is a faithful record, not a clean one. The green columns are the audit columns the loader adds.

3 · Silver: one clean, typed row per order

order_idcustomer_keyamount_inrplaced_at_utcstatus_batch_id
1001C-771250.002026-10-27 03:28:00paidb-0912
quarantine: order_idrule failed_batch_id
1002customer_key is required; amount must be ≥ 0b-0912

The duplicate collapsed to one row (latest per key). Text became numbers and timestamps, India time became UTC, codes were standardised. Order 1002 failed the contract and went to quarantine with its reason, where the owner can see it.

4 · Gold: shaped for the question "how much did we sell?"

fact_sales: date_keystore_keycustomer_keyorder_idamount_inr
20261027S-04C-7710011250.00
daily_sales: dateordersrevenue_inrreconciled
2026-10-2711250.00✓ 2 distinct orders in the source = 1 in silver + 1 in quarantine

5 · Semantic: one meaning for "revenue"

metric: net_revenue
  meaning: paid order amounts minus refunds, excluding test orders
  from: gold.fact_sales
  owner: finance

The dashboard and the conversation UI both ask for net_revenue, so both say ₹1,250 for 27 October, and both can say why order 1002 is missing.

Rules

Nine rules that keep it honest

Each rule can be checked by a machine, which is what keeps a medallion from slowly turning back into a swamp.

1
Data moves one way. Each layer is built only from the one before it. Gold never reads bronze; a dashboard never reads silver directly.
2
Bronze is append-only and replayable. Nothing in bronze is edited. If silver is wrong, fix the job and rebuild silver from bronze.
3
Every row says where it came from. Audit columns in bronze, lineage on every hop, the batch id carried into silver.
4
Silver is one row per real thing. De-duplicated on the business key, typed, with history kept as decided (SCD Type 1 or 2).
5
Bad rows are quarantined, never dropped. Each with the rule it failed, visible to its owner. A rule is never loosened just to make a job pass.
6
Every load can run twice. Bronze skips a batch it has seen; silver and gold use MERGE or partition overwrite. Run twice = run once.
7
Numbers reconcile across hops. Counts and sums agree from source to gold within a tolerance, every run.
8
Metrics are defined once. In the semantic layer, not in each dashboard.
9
Cross-cutting jobs are on by default. Quality, lineage, freshness, cost, access and PII are declared per table; turning one off needs a written waiver.

Mistakes

Common mistakes, and the fix

MistakeWhy it hurtsFix
Dashboards reading bronzeEvery report cleans the data its own way, so numbers disagreePoint reports at gold or semantic; block bronze access for analysts
Cleaning before landing a raw copyA bug in the cleaning has nothing to re-run fromLand and keep the raw copy first (ELT)
Dropping bad rows silentlyTotals are quietly wrong and nobody knows whyQuarantine with a reason and alert the owner
INSERT instead of MERGE in silverEvery retry adds duplicatesMERGE on the business key; skip seen batches in bronze
One giant flat table per report in goldThe same product is spelled three waysFacts and conformed dimensions; marts built from them
Too many layers ('platinum', 'diamond', 'staging2')Nobody knows which one to readKeep the meaning of each layer; add tables, not layers
Medallion per team, each with its own rulesThe same customer cleaned five waysOne silver per business entity, owned and shared
Metric logic in dashboardsTwo revenues in one meetingDefine the metric once in the semantic layer

Concept vs product

The same layers on each platform

The layers and rules never change. Only the products that do each job change. Learn the job once; the product is a binding.

JobAWSAzure / FabricGoogle CloudDatabricksOpen source
StorageAmazon S3OneLake / ADLS Gen2Cloud StorageUnity Catalog volumes on cloud storageMinIO or local disk
Table formatApache Iceberg (S3 Tables)Delta LakeIceberg on BigLakeDelta LakeDelta Lake or Iceberg
Into landingAWS DMS, AppFlowData Factory copy, mirroringDatastreamLakeflow Connectdlt or Airbyte
Landing → bronzeGlue (Spark) or EMRFabric Spark / notebooksDataproc (Spark)Auto Loader / Lakeflow pipelinesPySpark
Bronze → silver → goldGlue, EMR, or dbt on Athena/RedshiftFabric Spark or dbt on FabricDataform or dbt on BigQueryLakeflow Declarative Pipelines or dbtPySpark or dbt + DuckDB
Quality checksGlue Data QualityPurview data qualityDataplex data qualityPipeline expectationsGreat Expectations or Soda
Catalog, access, lineageGlue Data Catalog + Lake FormationMicrosoft PurviewDataplexUnity CatalogOpenMetadata + OpenLineage
Semantic layerdbt semantic layerPower BI semantic modelsLooker (LookML)Metric viewsCube Core
OrchestrationMWAA (Airflow), Step FunctionsData Factory / Fabric pipelinesCloud ComposerLakeflow JobsAirflow or Dagster

Product names change often; check the current docs before you build.

FAQ

Questions people ask

What is medallion architecture?

A way to organise a data platform into layers of increasing quality: bronze keeps raw data as it arrived, silver holds clean, typed, de-duplicated data with one row per real thing, and gold holds tables shaped for a business use. Each layer is built only from the one before it.

Is medallion architecture only for Databricks?

No. The name was popularised by Databricks, and Microsoft Fabric uses it too, but the idea (raw, then cleaned, then modelled) is much older and works on AWS, Google Cloud, Snowflake or open source. Only the product names change.

Is medallion a data model?

No. It says how data moves and what each layer promises. You still choose a data model inside the layers: for example dimensional (star schema) in gold, or data vault in silver.

How many layers should there be?

Three named layers (bronze, silver, gold), often with a landing area before bronze and a semantic layer after gold. If you feel you need a fourth medallion, you usually need more tables, not another layer.

Where should data quality checks go?

Everywhere, but the contract is enforced at the bronze-to-silver hop: rows that break it go to quarantine with a reason. Gold adds reconciliation (counts and sums agree with silver and the source).

Where does SCD Type 2 history live?

In silver, for entities whose history matters (a customer's city, a product's price). Gold dimensions read that history.

Does streaming need a different architecture?

Usually not: one medallion, two speeds. Hot tables flow through the same layers continuously; the rules (one-way flow, contracts, MERGE) stay the same.

Medallion vs Kimball vs data vault?

They answer different questions. Medallion is the flow and the promise of each layer. Kimball (star schemas) is how to shape gold for analysis. Data vault is one way to model an integrated, historised silver. Many platforms use medallion with Kimball in gold.

Next

Where to go next

Zoom out: Data platform architecture, drawn right: the control plane, the three services, streaming, 22 cross-cutting concerns and the agents around the medallion.

The ideas behind each layer, one short page each:

Learn it step by step: the Data Engineering path (Crawl, Walk, Run on AWS, Azure, Google Cloud, Databricks or open source).

Build one: the Ecom simple data platform project lands a shop's data through bronze, silver and gold. Join it, free.

Run it: DataTwin Studio follows the same three orders through every layer, recorded from a real Spark and Delta Lake run. Then send the same file again and watch a naive pipeline double the revenue.