Data PlatformZoom level 2
What each layer has: tests, golden data, incidents, and what is missing
This page answers one question exactly: for each layer of the platform, what proves it? It lists the test files, says whether the layer is compared with the oracle's golden data, names the planted incidents it is judged on, and says what is missing.
How to read the counts
The number after each file name is the number of test functions in the file. It was counted when this page was built, by searching each file for lines that start a test function. It is a count of functions, not of cases. A table-driven function with twenty rows counts as one, so the number of cases that actually run is larger.
One row per layer
| Layer | Unit and contract test files | Golden data from the oracle | Incidents it is judged on | What is missing |
|---|---|---|---|---|
| Metadata store | test_meta_store.py (4)test_meta_loader.py (9)test_lakebase.py (5) | No. It holds settings and logs, not business data. Every test that runs the simulated company loads the tyre manifests through it. | One, indirectly. The 12 promised-date corrections must be found in the store's corrections table and not in a file. | The PostgreSQL rows of the contract table are skipped when no local server answers. |
| Ingestion: sources to landing | test_source_ports.py (9)test_ingest_steps.py (11)test_ingest_pipelines.py (14)test_ingest_formats.py (5)test_resilience.py (8)test_cloud_adapters.py (8) | Yes, through bronze. test_ingest_world.py (6) runs 14 days at scale 0.05. | All eleven get a verdict. Handled: DUP-01, SCH-02, DEL-04. Flagged: LATE-03, DEL-01, SCD-02, DUP-04. Deferred: LATE-01, HIST-05, LATE-02, REC-05. | The world test needs the Delta package and is skipped without it. It is marked slow. |
| Bronze | test_ingest_ports.py (6)test_profiling.py (3)test_retention.py (4)test_housekeeping.py (12)test_baseline_stage.py (4) | Yes, by row count only. Rows appended per dataset by each day's run equal the oracle's count on each of the 14 days. The same test also reloads a past day and checks that no bronze table changes. | The file sent twice adds 0 rows. The short copy adds 0 rows. The renamed-column files are stopped. | Sums and keys are not compared at bronze. Content is first compared at silver. |
| Bronze to silver | test_silver_plan.py (5)test_silver_stages.py (37)test_silver_ports.py (13)test_silver_runner.py (12)test_silver_engines.py (5)test_silver_declarative.py (5)test_privacy_stage.py (1)test_live_recon.py (3)test_reconcile_buckets.py (3)test_reconcile_engine.py (1)test_table_store.py (2) | Yes. test_silver_world.py (3) runs 14 days at scale 0.05 and requires that no number differs from the oracle on any day. Recorded at full volume on 4 November 2026: 294 numbers, all equal. | DEL-01, SCD-02, LATE-03 and DUP-04 must be handled. DEL-04 must still be held. | The test needs Spark and is marked slow. The declarative engine is not run against the oracle. REC-05 is not judged in this test. |
| Silver to gold | test_gold_plan.py (13)test_gold_engine.py (11)test_gold_harness.py (12)test_gold_ports.py (2)test_gold_calendar.py (5) | Partly in tests, fully in a recorded run. test_twin_engine.py (2) runs gold each day at scale 0.02 and checks the verdicts and the days its scenario files name. The run script compares gold with the oracle every day. Recorded on 5 November 2026 at scale 0.1: equal on all 14 days. | DEL-01, LATE-01, DUP-01, HIST-05 and LATE-02. | No test asserts gold equals the oracle on all 14 days. The full-volume 14-day run is owed. The scenario test needs Spark and is marked slow. |
| Semantic | test_semantic_metrics.py (14)test_semantic_experiences.py (5)test_semantic_governance.py (17) | Yes, five known answers. One test checks that the oracle fills every known answer's address. One reproduces all five from a full-volume world, and runs only when asked. Two more compare the metric views' numbers from built gold with the file and with the oracle, and skip unless a gold run's output is named. | None directly. | The gold comparison is a separate command after a run. Recorded at scale 0.1: 12 passed and 4 failed, the 4 being answers that depend on volume. At full volume it is owed for the 14-day run. |
| Operations | 18 files, 81 test functions in all: the files named test_obs_, then test_ops_export.py (3), test_ops_restatements.py (1), test_hop_reports.py (4), test_twin_scenarios.py (8) | No. There is no oracle for alerts or screens. They are checked against the incident calendar on a stand-in run log. | All eleven: each must raise one alert, and no clean day may raise any. | The checks do not run on the run log the engine writes. Delivery to a real workspace is untested. |
The simulated company has its own tests, which every golden comparison rests on: test_world_tyre.py (12) and test_world_calendar.py (5). The first needs only Python.
Which of these run without Spark
A check of the public snapshot on 19 November 2026 ran the folders tests/dataplatform and tests/aws_databricks without Spark. 1316 tests passed and 219 were skipped. The skipped ones need Spark. That figure counts cases, so it is much larger than the function counts in the table.
| Golden-data test | Needs | In a run without Spark |
|---|---|---|
test_world_tyre.py | Python and the catalog beside the repository | Runs |
test_ingest_world.py | The Delta package | Runs only if that package is installed |
test_silver_world.py | Spark and the Delta package | Skipped |
test_twin_engine.py, silver and gold part | Spark | Skipped |
| Known answers against built gold | A finished run of the platform | Skipped |
I did not see the list of which tests were skipped in that check, so I cannot say from it whether the bronze world test ran. The table above is from the skip conditions written in each test file.
tests/dataplatform/test_silver_world.py:70The silver golden test: one case per day, and no difference allowed.@pytest.mark.parametrize("day", range(1, DAYS + 1))
def test_silver_equals_the_oracle(platform, day):
s = platform["silver"][day - 1]
assert s.report.status == "succeeded", [d.reason for d in s.report.datasets if d.status == "failed"]
assert s.differences == {}tests/dataplatform/test_ingest_world.py:80The bronze golden test: one case per day.@pytest.mark.parametrize("day", range(1, DAYS + 1))
def test_bronze_equals_the_oracle(platform, day):
d = platform["days"][day - 1]
assert d.bronze_ok, {k: v for k, v in d.bronze.items() if v[0] != v[1]}The obligation coverage check
The specification catalog holds a list of obligations: rules the platform must keep, each with an id such as DEL-01. The coverage check asks a plain question of each one. Does any test file cite its id?
| State | Meaning |
|---|---|
| tested | A test file contains the id. |
| waived | A waiver names the id, with a reason of at least 10 characters, an approver and an end date that has not passed. |
| built | Only code contains the id. The rule is kept somewhere but no test says so. |
| none | Nothing cites it. |
src/dataplatform/governance/coverage.py:53The whole check: a test beats a waiver, a waiver beats code.def coverage(rules: list[Rule], tests: Mapping[str, str], code: Mapping[str, str],
waivers: list[Mapping[str, Any]], today: date) -> list[Covered]:
waived = {str(w["id"]): str(w["reason"]) for w in waivers if valid_waiver(w, today)}
out = []
for r in rules:
t = _cited(r.id, tests)
if t:
out.append(Covered(r, "tested", t))
elif r.id in waived:
out.append(Covered(r, "waived", (waived[r.id],)))
else:
c = _cited(r.id, code)
out.append(Covered(r, "built" if c else "none", c))
return outsrc/dataplatform/governance/coverage.py:42What cited means: the id as a whole word in the file's text.def _cited(rule_id: str, files: Mapping[str, str]) -> tuple[str, ...]:
pat = re.compile(rf"(?<![A-Z0-9-]){re.escape(rule_id)}(?![0-9])")
return tuple(sorted(name for name, text in files.items() if pat.search(text)))A worked example
The rule about deletes at the source has the id DEL-01. The table shows what three pieces of text do to its state. The cases are the check's own test cases.
| Text found | Where | State of DEL-01 |
|---|---|---|
# DEL-01 proven here | A test file | tested |
DEL-011 and XDEL-01 | A test file | none. A longer id does not count for a shorter one. |
DEL-04 guard | A code file | DEL-04 is built. DEL-01 is untouched. |
I ran the script on 19 November 2026 against the public snapshot and the catalog beside it. The catalog holds 105 obligations. 59 are cited by a test, which is 56%. None is waived. 14 are cited only in code. 32 are cited nowhere.
| State | Obligations | Share of 105 |
|---|---|---|
| tested | 59 | 56% |
| waived | 0 | 0% |
| built | 14 | 13% |
| none | 32 | 30% |
The waiver file lists six proposed waivers. Each has an empty approver, so by the rule above none of them counts yet.
docs/dac-waivers.yaml:4One proposed waiver. The empty approver is why it does not count.- id: HEAL-01 reason: Self-healing is phase P7; no self-healing fix exists yet to gate approved_by: "" until: "2027-03-31"
tests/dataplatform/test_dac_coverage.py:16The coverage check's own cases.@pytest.mark.parametrize(("case", "tests", "code", "waivers", "want"), [
("cited in a test is tested", {"t.py": "# DEL-01 proven here"}, {}, [], {"DEL-01": "tested"}),
("cited only in code is built", {}, {"c.py": "DEL-04 guard"}, [], {"DEL-04": "built"}),
("a longer id does not count for a shorter one", {"t.py": "DEL-011 and XDEL-01"}, {}, [], {"DEL-01": "none"}),
("a valid waiver counts", {}, {}, [OK_WAIVER], {"SEC-03": "waived"}),
("an expired waiver does not", {}, {}, [{**OK_WAIVER, "until": "2026-01-01"}], {"SEC-03": "none"}),
("a waiver without an approver does not", {}, {}, [{**OK_WAIVER, "approved_by": " "}], {"SEC-03": "none"}),
("a test beats a waiver", {"t.py": "SEC-03"}, {}, [OK_WAIVER], {"SEC-03": "tested"}),
])Cited is a weak word. A test that mentions an id in a comment counts as much as a test that proves the rule. The check finds obligations nobody has written a test for. It does not measure how well the tested ones are tested, and it does not run the tests.
- The success criterion for coverage says 89 of 89 rules. The catalog now holds 105, and 59 are cited by a test.
- The six waivers are proposed and unsigned, so the count of waived obligations is zero.
- Coverage counts a citation, not a passing test. A cited test that is skipped still counts.
- No golden-data test exists for the metadata store or for operations. Both are proven by contract tables and, for operations, by a stand-in run log.
- Gold has no test file of its own that runs the 14 days and asserts equality with the oracle each day, the way silver and bronze have.
- The script reads workflow files as tests too. The public snapshot has no workflow folder, so any obligation that only a workflow proves shows as untested here.