Home › Learn › Data systems › Data platform › Layout
Data platform · concern X23 · Platform-wide
Layout: catalogs, schemas and naming
From a table's full name you can tell its environment, layer and domain, and who may read it.
Where landing, bronze, silver, gold and semantic physically live: what becomes a catalog, what becomes a schema, how tables are named, and how that turns access, retention and isolation into simple rules.
For leaders
- Every table has a three-part name: catalog, schema, table. How those three parts are used decides how environments are kept apart, who can read what, and where data is stored.
- We recommend one catalog per environment and layer (for example prod_silver), one schema per source system in bronze, and one schema per business domain in silver, gold and semantic.
- That turns the most important rules into one grant each: analysts read prod_gold and prod_semantic; only the platform reads bronze; nothing in dev can write to prod.
- Decide this before the first table. Renaming later breaks every job, report and agent tool that uses the old names.
In plain words
Like a library: branch, floor, shelf, book
A city library system has a branch in each area, floors inside each branch, shelves by subject on each floor, and books on the shelves. Members may use the reading room, but not the receiving room where new deliveries are unpacked. Anyone can find a book from its address.
A data platform names tables the same way. The environment is the branch (dev, test, prod). The layer is the floor (bronze is the receiving room; gold and semantic are the reading rooms). The domain is the shelf (sales, finance). The table is the book. Most platforms have three name levels (catalog, schema, table), so two of these four have to share one level. Which two share is the main decision on this page.
The picture
One catalog per environment and layer
What goes where
Catalog, schema, table, and the things that aren't tables
| Level | What it holds | How we split it | Example |
|---|---|---|---|
| Workspace or account | Compute, people and jobs for one environment | One per environment | the prod workspace |
| Catalog | One layer of one environment | Environment and layer | prod_silver |
| Schema in bronze | Tables from one source system, in its own shape | By source system | prod_bronze.sap_ecc |
| Schema in silver and gold | Tables for one business domain | By business domain | prod_silver.sales, prod_gold.finance |
| Shared dimensions | Customer, product, calendar used by every domain | One shared schema | prod_gold.shared.dim_customer |
| Schema in semantic | Metric definitions for one domain | By business domain | prod_semantic.sales |
| Landing | Files as received, not tables | A storage volume per source | prod_bronze.sap_ecc.landing (volume) |
| Quarantine | Rows that failed a check, with the reason | Next to the table they failed | prod_silver.sales.orders_quarantine |
| Ops | Metadata store, run log, quality results, cost | By function | prod_ops.run_log |
| Sandboxes | Personal experiments | One schema per person, dev only, with expiry | dev_sandbox.asha |
Naming
Table names by layer
| Where | Rule | Example | Why |
|---|---|---|---|
| Bronze tables | The source object's own name, lower case | prod_bronze.sap_ecc.vbak | Anyone can find the source table |
| Silver tables | The business entity, plural | prod_silver.sales.orders | Names the real thing, not the source |
| Gold facts and dimensions | fact_ and dim_ prefixes | prod_gold.sales.fact_order_line | The grain and role are clear |
| Gold aggregates and marts | agg_ prefix with the grain | prod_gold.sales.agg_sales_daily | The time grain is in the name |
| Semantic | Domain and _metrics | prod_semantic.sales.sales_metrics | One place for each domain's metrics |
| Views for consumers | The stable name consumers use | prod_gold.sales.orders_v | Tables can change behind the view |
Access
One grant per layer
Because each layer is its own catalog, the main access rules are short. Personal data inside silver and gold is further protected with column masks by role.
| Catalog | Who writes | Who reads |
|---|---|---|
| *_bronze | The ingestion service only | Silver jobs only |
| *_silver | Silver jobs only | Gold jobs, data engineers, data scientists (personal data masked) |
| *_gold | Gold jobs only | The semantic layer, BI service, analysts, ML |
| *_semantic | The deployment pipeline | Everyone with rights to the domain: dashboards, conversation UIs, agent tools |
| *_ops | Platform services | The platform team and auditors |
| dev_* | Engineers | Engineers |
Concept vs product
The same layout on each platform
Every platform has a name hierarchy; only the words and the number of levels differ.
| Platform | Name levels | How the layout maps |
|---|---|---|
| Databricks | catalog.schema.table | Unity Catalog; volumes for landing files; bind each catalog to its environment's workspace so dev can't write prod. |
| Microsoft Fabric | workspace / lakehouse / schema / table | A workspace per environment (and often per layer), a lakehouse per layer, schemas by domain; OneLake shortcuts instead of copies. |
| Snowflake | database.schema.table | A database per environment and layer; zero-copy clones help fill test. |
| Google BigQuery | project.dataset.table | A project per environment; with only two levels inside, name datasets layer_domain (silver_sales). |
| AWS (Glue, Lake Formation, S3 Tables) | catalog / database / table | An account per environment; databases named layer_domain; Lake Formation grants per database. |
| Open source (Iceberg REST, Polaris, Unity OSS) | catalog / namespace / table | Namespaces can nest (silver.sales), so the recommended layout fits directly. |
Product features change often; check the current docs before you build.
The question list
Every question, why we ask, and what breaks if nobody does
| # | Question to chase | Why we ask | What breaks in production if nobody asks | Do this (not this) |
|---|---|---|---|---|
| A · Isolationkeeping environments apart | ||||
| A1 | Can a job running in dev write to a production table? | One wrong parameter shouldn't be able to touch production. | A developer's test run overwrites the production orders table. | A workspace per environment, with each catalog bound to its own workspace. Not: environments kept apart only by a name prefix. |
| A2 | Where do people experiment, and can experiments leak into production? | Scratch tables become dependencies the moment someone builds a report on them. | A personal table in a production schema feeds a board report; its owner leaves. | A dev sandbox schema per person, with expiry. Not: scratch tables in production schemas. |
| B · Namingnames people can read and trust | ||||
| B1 | Is there one written naming standard, and is it checked automatically? | Names are the first thing people use to find and trust data. | orders, Orders_v2 and ord_final all exist, and nobody knows which one is right. | Write the standard into the manifest validator, so a bad name fails the build. Not: a wiki page. |
| B2 | Is the environment name written into code anywhere? | Code is promoted unchanged; the environment must come from outside it. | Code promoted to production still writes to the dev catalog. | The catalog name comes from the deployment target's settings. Not: if env == 'prod' in code. |
| B3 | Do consumers read tables directly, or stable views and the semantic layer? | Tables get rebuilt and renamed; consumers shouldn't notice. | A table rename breaks 40 dashboards and three agent tools overnight. | Publish through views and the semantic layer; change tables behind them. Not: dashboards on raw gold tables. |
| C · Splittinghow catalogs and schemas are divided | ||||
| C1 | Do we split schemas by source, by domain, or by team? | The split should outlive reorganisations. | Team-named schemas have to be moved after every reorg. | Bronze by source system; silver, gold and semantic by business domain. Not: by team. |
| C2 | Which domains are there, and who owns each one? | A schema without an owner collects orphan tables. | Nobody can say whether a table is safe to drop, so nothing is ever dropped. | A short domain list with a named owner each, kept in metadata. Not: domains invented per project. |
| C3 | Where do shared dimensions such as customer, product and calendar live? | Every domain needs them; there should be one copy. | Three customer dimensions with three different customer counts. | One shared schema in gold that all domains read. Not: a copy per domain. |
| D · Accesswho reads which layer | ||||
| D1 | Who may read each layer? | Layers are promises; reading the wrong one breaks them. | Reports built on bronze break whenever a source changes. | Grant at catalog level by layer, as in the access table above. Not: one 'data readers' group for everything. |
| D2 | Where does personal data sit, masked and unmasked? | Masked copies drift from the original. | A 'PII-free' copy of a table misses a new column with email addresses. | One table with column masks and row filters by role. Not: separate copies with columns removed. |
| E · Storage and retentionwhere data sits and for how long | ||||
| E1 | Does each catalog have its own storage location, retention and encryption key? | Layers need different retention and cost tiers. | Landing files can't be cleaned after 7 days without touching gold; storage cost can't be split. | A storage location per catalog, with its own lifecycle rules. Not: one bucket for everything. |
| E2 | How long is each layer kept, including table history (time travel)? | Too short and you can't recover from a bad load; too long and storage grows forever. | A bad load from last week can't be undone, or the storage bill doubles each year. | Retention per layer in the manifest: landing days, bronze long and cheap, silver and gold by business need. Not: the default. |
| F · Changemoving and renaming once in use | ||||
| F1 | How will we move or rename a table that people already use? | Something will need to move eventually. | A move breaks unknown consumers, found one complaint at a time. | Use lineage to find consumers, keep the old name as a view for a deprecation period, then remove it. Not: rename and wait for complaints. |
Design decisions
The decisions to take
How do catalogs split?
| Option | Verdict |
|---|---|
| One per environment and layer (prod_silver) | Pursue · Pursue. One grant per layer, one storage location per layer, environments fully apart. |
| One per environment; schemas carry layer and domain (prod.silver_sales) | Pursue when · Pursue only for a small platform with a few domains, where fewer catalogs matter more. |
| One per domain, with layers as schemas (sales.silver) | Pursue when · Pursue only when domain teams own their data end to end (data mesh), with environments kept apart by workspace. |
| One catalog, environments as name prefixes | Drop · Drop. Nothing stops dev from writing to prod. |
How do schemas split?
| Option | Verdict |
|---|---|
| Bronze by source; silver, gold and semantic by business domain | Pursue · Pursue. Bronze mirrors the sources, the rest mirrors the business. |
| By team | Drop · Drop. Teams change more often than the business does. |
How do consumers read data?
| Option | Verdict |
|---|---|
| Through views and the semantic layer | Pursue · Pursue. Tables can change without breaking anyone. |
| Directly from gold tables | Pursue when · Pursue only for a few known consumers, such as ML training jobs. |
Where does landing go?
| Option | Verdict |
|---|---|
| A storage volume per source inside the bronze catalog | Pursue · Pursue. Same grants and lifecycle as bronze, short retention. |
| Its own catalog | Pursue when · Pursue only when landing is filled by a separate team or tool with separate access. |
FAQ
Questions people ask
Should a catalog be per environment or per layer?
Both, combined: one catalog per environment and layer, such as prod_silver. That keeps environments apart and lets one grant control who reads each layer. Smaller platforms can use one catalog per environment with layer and domain in the schema name.
Should bronze be organised by source system or by domain?
By source system. Bronze keeps data in the source's own shape, so grouping by source makes it easy to find and replay. From silver onwards, data is shaped around the business, so it is grouped by domain.
Where do metric views or the semantic model live?
In their own semantic catalog per environment, by domain, so consumers can be given the semantic layer and gold without seeing silver or bronze.
Next
Where to go next
Next concern: Environments and CI/CD, which decides how changes move between the dev, test and prod rows above.
Related: Unity Catalog done right (blog) and the metadata store, where the naming rules are checked.
Back up: The data platform, whole, with all 24 concerns.