cleandataengineering.aiData Platform, from the big picture to the code
You are here: Start here › Tests and proof › Proof by layer

Data PlatformZoom level 2

What each layer has: tests, golden data, incidents, and what is missing

This page answers one question exactly: for each layer of the platform, what proves it? It lists the test files, says whether the layer is compared with the oracle's golden data, names the planted incidents it is judged on, and says what is missing.

Metadatastorecontract tableIngestiontables, worldBronzerow countsBronze tosilverfull compareSilver togoldrun scriptSemantic5 answersOperationsfixtureUnder each layer: the strongest proof it has against known truth.
Bronze to silver has the most complete comparison in a test. The metadata store and operations have no golden data.

How to read the counts

The number after each file name is the number of test functions in the file. It was counted when this page was built, by searching each file for lines that start a test function. It is a count of functions, not of cases. A table-driven function with twenty rows counts as one, so the number of cases that actually run is larger.

One row per layer

LayerUnit and contract test filesGolden data from the oracleIncidents it is judged onWhat is missing
Metadata storetest_meta_store.py (4)
test_meta_loader.py (9)
test_lakebase.py (5)
No. It holds settings and logs, not business data. Every test that runs the simulated company loads the tyre manifests through it.One, indirectly. The 12 promised-date corrections must be found in the store's corrections table and not in a file.The PostgreSQL rows of the contract table are skipped when no local server answers.
Ingestion: sources to landingtest_source_ports.py (9)
test_ingest_steps.py (11)
test_ingest_pipelines.py (14)
test_ingest_formats.py (5)
test_resilience.py (8)
test_cloud_adapters.py (8)
Yes, through bronze. test_ingest_world.py (6) runs 14 days at scale 0.05.All eleven get a verdict. Handled: DUP-01, SCH-02, DEL-04. Flagged: LATE-03, DEL-01, SCD-02, DUP-04. Deferred: LATE-01, HIST-05, LATE-02, REC-05.The world test needs the Delta package and is skipped without it. It is marked slow.
Bronzetest_ingest_ports.py (6)
test_profiling.py (3)
test_retention.py (4)
test_housekeeping.py (12)
test_baseline_stage.py (4)
Yes, by row count only. Rows appended per dataset by each day's run equal the oracle's count on each of the 14 days. The same test also reloads a past day and checks that no bronze table changes.The file sent twice adds 0 rows. The short copy adds 0 rows. The renamed-column files are stopped.Sums and keys are not compared at bronze. Content is first compared at silver.
Bronze to silvertest_silver_plan.py (5)
test_silver_stages.py (37)
test_silver_ports.py (13)
test_silver_runner.py (12)
test_silver_engines.py (5)
test_silver_declarative.py (5)
test_privacy_stage.py (1)
test_live_recon.py (3)
test_reconcile_buckets.py (3)
test_reconcile_engine.py (1)
test_table_store.py (2)
Yes. test_silver_world.py (3) runs 14 days at scale 0.05 and requires that no number differs from the oracle on any day. Recorded at full volume on 4 November 2026: 294 numbers, all equal.DEL-01, SCD-02, LATE-03 and DUP-04 must be handled. DEL-04 must still be held.The test needs Spark and is marked slow. The declarative engine is not run against the oracle. REC-05 is not judged in this test.
Silver to goldtest_gold_plan.py (13)
test_gold_engine.py (11)
test_gold_harness.py (12)
test_gold_ports.py (2)
test_gold_calendar.py (5)
Partly in tests, fully in a recorded run. test_twin_engine.py (2) runs gold each day at scale 0.02 and checks the verdicts and the days its scenario files name. The run script compares gold with the oracle every day. Recorded on 5 November 2026 at scale 0.1: equal on all 14 days.DEL-01, LATE-01, DUP-01, HIST-05 and LATE-02.No test asserts gold equals the oracle on all 14 days. The full-volume 14-day run is owed. The scenario test needs Spark and is marked slow.
Semantictest_semantic_metrics.py (14)
test_semantic_experiences.py (5)
test_semantic_governance.py (17)
Yes, five known answers. One test checks that the oracle fills every known answer's address. One reproduces all five from a full-volume world, and runs only when asked. Two more compare the metric views' numbers from built gold with the file and with the oracle, and skip unless a gold run's output is named.None directly.The gold comparison is a separate command after a run. Recorded at scale 0.1: 12 passed and 4 failed, the 4 being answers that depend on volume. At full volume it is owed for the 14-day run.
Operations18 files, 81 test functions in all: the files named test_obs_, then test_ops_export.py (3), test_ops_restatements.py (1), test_hop_reports.py (4), test_twin_scenarios.py (8)No. There is no oracle for alerts or screens. They are checked against the incident calendar on a stand-in run log.All eleven: each must raise one alert, and no clean day may raise any.The checks do not run on the run log the engine writes. Delivery to a real workspace is untested.

The simulated company has its own tests, which every golden comparison rests on: test_world_tyre.py (12) and test_world_calendar.py (5). The first needs only Python.

Which of these run without Spark

A check of the public snapshot on 19 November 2026 ran the folders tests/dataplatform and tests/aws_databricks without Spark. 1316 tests passed and 219 were skipped. The skipped ones need Spark. That figure counts cases, so it is much larger than the function counts in the table.

Golden-data testNeedsIn a run without Spark
test_world_tyre.pyPython and the catalog beside the repositoryRuns
test_ingest_world.pyThe Delta packageRuns only if that package is installed
test_silver_world.pySpark and the Delta packageSkipped
test_twin_engine.py, silver and gold partSparkSkipped
Known answers against built goldA finished run of the platformSkipped

I did not see the list of which tests were skipped in that check, so I cannot say from it whether the bronze world test ran. The table above is from the skip conditions written in each test file.

tests/dataplatform/test_silver_world.py:70The silver golden test: one case per day, and no difference allowed.
@pytest.mark.parametrize("day", range(1, DAYS + 1))
def test_silver_equals_the_oracle(platform, day):
    s = platform["silver"][day - 1]
    assert s.report.status == "succeeded", [d.reason for d in s.report.datasets if d.status == "failed"]
    assert s.differences == {}
tests/dataplatform/test_ingest_world.py:80The bronze golden test: one case per day.
@pytest.mark.parametrize("day", range(1, DAYS + 1))
def test_bronze_equals_the_oracle(platform, day):
    d = platform["days"][day - 1]
    assert d.bronze_ok, {k: v for k, v in d.bronze.items() if v[0] != v[1]}

The obligation coverage check

The specification catalog holds a list of obligations: rules the platform must keep, each with an id such as DEL-01. The coverage check asks a plain question of each one. Does any test file cite its id?

StateMeaning
testedA test file contains the id.
waivedA waiver names the id, with a reason of at least 10 characters, an approver and an end date that has not passed.
builtOnly code contains the id. The rule is kept somewhere but no test says so.
noneNothing cites it.
src/dataplatform/governance/coverage.py:53The whole check: a test beats a waiver, a waiver beats code.
def coverage(rules: list[Rule], tests: Mapping[str, str], code: Mapping[str, str],
             waivers: list[Mapping[str, Any]], today: date) -> list[Covered]:
    waived = {str(w["id"]): str(w["reason"]) for w in waivers if valid_waiver(w, today)}
    out = []
    for r in rules:
        t = _cited(r.id, tests)
        if t:
            out.append(Covered(r, "tested", t))
        elif r.id in waived:
            out.append(Covered(r, "waived", (waived[r.id],)))
        else:
            c = _cited(r.id, code)
            out.append(Covered(r, "built" if c else "none", c))
    return out
src/dataplatform/governance/coverage.py:42What cited means: the id as a whole word in the file's text.
def _cited(rule_id: str, files: Mapping[str, str]) -> tuple[str, ...]:
    pat = re.compile(rf"(?<![A-Z0-9-]){re.escape(rule_id)}(?![0-9])")
    return tuple(sorted(name for name, text in files.items() if pat.search(text)))

A worked example

The rule about deletes at the source has the id DEL-01. The table shows what three pieces of text do to its state. The cases are the check's own test cases.

Text foundWhereState of DEL-01
# DEL-01 proven hereA test filetested
DEL-011 and XDEL-01A test filenone. A longer id does not count for a shorter one.
DEL-04 guardA code fileDEL-04 is built. DEL-01 is untouched.

I ran the script on 19 November 2026 against the public snapshot and the catalog beside it. The catalog holds 105 obligations. 59 are cited by a test, which is 56%. None is waived. 14 are cited only in code. 32 are cited nowhere.

StateObligationsShare of 105
tested5956%
waived00%
built1413%
none3230%

The waiver file lists six proposed waivers. Each has an empty approver, so by the rule above none of them counts yet.

docs/dac-waivers.yaml:4One proposed waiver. The empty approver is why it does not count.
- id: HEAL-01
  reason: Self-healing is phase P7; no self-healing fix exists yet to gate
  approved_by: ""
  until: "2027-03-31"
tests/dataplatform/test_dac_coverage.py:16The coverage check's own cases.
@pytest.mark.parametrize(("case", "tests", "code", "waivers", "want"), [
    ("cited in a test is tested", {"t.py": "# DEL-01 proven here"}, {}, [], {"DEL-01": "tested"}),
    ("cited only in code is built", {}, {"c.py": "DEL-04 guard"}, [], {"DEL-04": "built"}),
    ("a longer id does not count for a shorter one", {"t.py": "DEL-011 and XDEL-01"}, {}, [], {"DEL-01": "none"}),
    ("a valid waiver counts", {}, {}, [OK_WAIVER], {"SEC-03": "waived"}),
    ("an expired waiver does not", {}, {}, [{**OK_WAIVER, "until": "2026-01-01"}], {"SEC-03": "none"}),
    ("a waiver without an approver does not", {}, {}, [{**OK_WAIVER, "approved_by": " "}], {"SEC-03": "none"}),
    ("a test beats a waiver", {"t.py": "SEC-03"}, {}, [OK_WAIVER], {"SEC-03": "tested"}),
])
What this check does and does not say

Cited is a weak word. A test that mentions an id in a comment counts as much as a test that proves the rule. The check finds obligations nobody has written a test for. It does not measure how well the tested ones are tested, and it does not run the tests.

Not built yet, or known gaps
  • The success criterion for coverage says 89 of 89 rules. The catalog now holds 105, and 59 are cited by a test.
  • The six waivers are proposed and unsigned, so the count of waived obligations is zero.
  • Coverage counts a citation, not a passing test. A cited test that is skipped still counts.
  • No golden-data test exists for the metadata store or for operations. Both are proven by contract tables and, for operations, by a stand-in run log.
  • Gold has no test file of its own that runs the 14 days and asserts equality with the oracle each day, the way silver and bronze have.
  • The script reads workflow files as tests too. The public snapshot has no workflow folder, so any obligation that only a workflow proves shows as untested here.