cleandataengineering.aiData Platform, from the big picture to the code
You are here: Start here › Tests and proof

Data PlatformZoom level 1

Tests and proof: how we know every layer is right

Where this starts in the code
PartStarting pointCalled by (found by reading the code)
The whole platform and the simulated company, day by daymain scripts/run_platform.py:315module level · scripts/run_platform.py:745

A data platform can run without an error and still be wrong. A row can be counted twice. A delete can be missed. So the platform is proven in three different ways, and each catches what the others cannot.

Read the three boxes left to right, from the smallest thing proven to the largest. Click a box to zoom in.

The three kinds of proof

KindWhat it isWhat it provesWhat it cannot prove
1. Table-driven unit and contract testsOne test function and a table of cases. Each case is a row: a name, an input and the expected output. A contract table is the same idea for a port: every adapter of the port must pass the same rows.That one function, or one adapter, does what each row says.That the parts add up to a correct table at the end.
2. The simulated company with an oracleA generated tyre company runs for simulated days and writes into mock source systems. The generator also records the true answer for every day. That record is the oracle, and it is the golden data.That each layer's output equals the truth on every simulated day.Anything the simulated company does not do.
3. Planted incidentsEleven faults put into the simulated company's data on purpose, on known days: a late file, a file sent twice, a renamed column, deleted rows and more.That the platform catches or handles each fault at the right layer, and raises nothing on a clean day.Faults nobody thought to plant.

An oracle in testing is a source of true answers that does not depend on the thing being tested. Here it is computed from the simulated company's own record of what it did, never from the platform's logs. Golden data means data whose correct result is known in advance.

The direct answer: is every layer tested against golden data?

Not every layer, and not all in the same way. The table is the short version. Proof by layer has the test files, the counts and what is missing.

LayerCompared with the oracle?How
Metadata storeNoIt holds settings and logs, not business data. It is proven by a contract table run against two adapters.
Ingestion and bronzeYes, by row countRows appended to bronze per dataset per day equal the oracle's count, in a slow test and in the recorded runs.
Bronze to silverYesCurrent rows, versions, deletes marked, each dataset's reconciliation measures and readings per hour, in a slow test that needs Spark and in the recorded runs.
Silver to goldYes in the recorded runs, partly in testsThe run script compares gold with the oracle every day. One slow test runs gold on the simulated company and checks the days its scenarios name. There is no test that asserts gold equals the oracle on all fourteen days.
SemanticYes, five known answersFive certified metrics each carry one known answer. The oracle reproduces all five. Asking the same five of built gold is a separate command, not part of the default test run.
OperationsNoAlerts and screens are checked against the incident calendar on a stand-in run log: one alert per incident, none on a clean day.
One thing to keep in mind

The golden-data tests for silver and gold need Spark, and the one for bronze needs the Delta package. A test run without them skips those tests. A check of the public snapshot on 19 November 2026 ran the two main test folders without Spark: 1316 passed and 219 skipped, and the skipped ones need Spark. So a green run without Spark proves the first kind of proof and not the second.

The pages under this one

PageWhat it explains
The simulated companyThe simulated tyre company: its sources, its clock, its volumes and how it is run.
The oracleWhat truth the oracle records for each day and layer, and how each layer is compared with it.
Planted incidentsEvery planted incident: its day, the rule that detects it, the layer that handles it, and the recorded result.
Tables of casesHow a case table looks, the contract tables every adapter must pass, the history contract, and the same tests on two engines.
Proof by layerOne table with a row per layer, and the check of how many catalog obligations a test cites.

The design decisions behind the proof

DecisionWhy
The truth never comes from the platformA platform checked against its own logs agrees with itself even when it is wrong.
The simulated company is deterministicThe same seed gives the same data on any machine, so a failure can be repeated.
Incidents live in a file, not in codeAnyone can move an incident to another day or change its size by editing one line.
A check on a layer that does not exist is pending, never passingNothing counts as proven by default.
Tests are tablesA new case is a new row. The reader sees every case at a glance.
Not built yet, or known gaps
  • No test asserts that gold equals the oracle on every one of the fourteen days. The run script does that comparison, and its result is recorded in a design note.
  • The fourteen-day run at full volume through gold has not been completed. The recorded attempt ran out of memory on day 12 on a laptop.
  • The golden-data tests are slow and need Spark, so a run without Spark does not include them.
  • The declarative silver engine has not been run against the oracle in any test in this repository.
  • Operations has no oracle of its own. Its checks use a stand-in run log, not the one the engine writes.

Look inside