Data PlatformZoom level 3
Who calls what: gold
Every class, function and method in the gold package, with who calls it. Found by reading the code. A call written as object.method() is matched by the method's name, so a same-named method elsewhere can appear: those rows say by name.
109 definitions in 8 modules. Jump to a module: calendar.py · daybyday.py · harness.py · inspect.py · keys.py · plan.py · ports.py · windows.py
gold/calendar.py
The date dimension (B15; gold modelling: one shared, role-playing date dimension). Pure and product-free. Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
fiscalfunction, line 26 | (fiscal year, fiscal quarter, fiscal month) for a fiscal year starting on the 1st of start_month; the fiscal year is named by the calendar year it ends in (India: April 2026 to March 2027 is fiscal year 2027). | calendar_rows · gold/calendar.py:46 | test_gold_calendar.py |
calendar_rowsfunction, line 36 | One row per day from start to end inclusive. `weekend` holds weekday numbers (Monday = 0). | write_calendar · scripts/run_platform.py:203 | test_gold_calendar.py |
gold/daybyday.py
Gold per world day: after the day's silver run, one gold run for the day's business date, then gold is compared with the oracle's gold for the day and the incidents gold owns are judged. Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
GoldDayclass, line 24 | judge_gold · gold/daybyday.py:65 | test_world_clinical.py | |
GoldDay.differencesmethod, line 33 | show_silver · scripts/run_platform.py:250 (value)show_gold · scripts/run_platform.py:267 (value) | test_end_to_end.py, test_explain.py and 1 more | |
GoldDay.okmethod, line 38 | write_ops_page · scripts/run_platform.py:134 (value)success_scorecard · scripts/run_platform.py:182 (value) | no test names it | |
GoldJudgeclass, line 42 | none found in the code | no test names it | |
GoldJudge.comparemethod, line 43 | (gold against the oracle per table and measure, notes kept for later days). | none found in the code | test_profiling.py, test_silver_stages.py and 4 more |
GoldJudge.judgemethod, line 47 | Verdicts for the incidents gold owns that the day closes (some are judged days after they are planted). | none found in the code | test_end_to_end.py, test_explain.py and 3 more |
gold_dayfunction, line 52 | One gold run for the day, then gold against the oracle and the day's gold incidents. | main.on_day · scripts/run_platform.py:598 | test_twin_engine.py |
judge_goldfunction, line 61 | The day's gold run judged from its report; the run itself may have happened in another process. | main.on_day · scripts/run_platform.py:595gold_day · gold/daybyday.py:58 | no test names it |
gold/harness.py
The gold harness (ADR-003): one run builds the domain's gold tables, dimensions first, then facts. Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
Restatementclass, line 39 | _fact · gold/harness.py:330_gold · hop_reports.py:49 | test_hop_reports.py | |
GoldTableclass, line 48 | run_gold · gold/harness.py:143_fail · gold/harness.py:175_dimension · gold/harness.py:190_fact · gold/harness.py:343_gold · hop_reports.py:49GoldReport.by_name · gold/harness.py:67 (value) | test_hop_reports.py | |
GoldReportclass, line 61 | run_gold · gold/harness.py:156_gold · hop_reports.py:51kind_of · hop_reports.py:30 (value) | test_hop_reports.py | |
GoldReport.by_namemethod, line 67 | run_days · ingest/daybyday.py:131 (by name)judge · silver/daybyday.py:148 (by name)ClinicalGoldJudge._enrolment · world/domains/clinical_gold.py:130 (by name)ClinicalGoldJudge._later · world/domains/clinical_gold.py:206 (by name)TyreGoldJudge._sales · world/domains/tyre_gold.py:146 (by name)TyreGoldJudge._later · world/domains/tyre_gold.py:223 (by name) | test_gold_harness.py, test_hop_reports.py and 11 more | |
Ctxclass, line 72 | run_gold · gold/harness.py:131 | no test names it | |
CheckFailedclass, line 83 | A check before publish failed: the last good windows stay (ADR-006). | _dimension · gold/harness.py:185_fact · gold/harness.py:295_key_lookup · gold/harness.py:384_ledger · gold/harness.py:414_checks · gold/harness.py:428run_gold · gold/harness.py:148 (value) | no test names it |
plansfunction, line 88 | Every enabled gold dataset's plan, from the store; dimensions first. | run_gold · gold/harness.py:130reproduce · gold/inspect.py:62 | test_gold_plan.py, test_silver_runner.py |
silver_watermarkfunction, line 102 | How far an input has loaded in source time (ADR-018): the end of the newest delivery window silver has taken (silver's bookmark counts the file records it took; an empty delivery still moves it). | _inputs · gold/harness.py:251 | no test names it |
_timefunction, line 111 | feed_marks · gold/harness.py:123_fact · gold/harness.py:321 | no test names it | |
feed_marksfunction, line 119 | An input fed per feed (one file per dealer): ingestion's per-feed bookmarks, each the feed's newest data time. | _inputs · gold/harness.py:247 | no test names it |
run_goldfunction, line 127 | Build every gold table of the domain for business date `as_of` (or only the named tables). | run_gold · scripts/run_hop.py:78gold_day · gold/daybyday.py:57 | test_explainer.py, test_gold_harness.py and 1 more |
_columnsfunction, line 159 | An input's contract columns as (name, engine type), for an input silver has not written yet. | _dimension · gold/harness.py:179_inputs · gold/harness.py:233 | no test names it |
_stepfunction, line 166 | _dimension · gold/harness.py:182_trace · gold/harness.py:359_key_lookup · gold/harness.py:380_ledger · gold/harness.py:411_checks · gold/harness.py:425_record · gold/harness.py:456 | no test names it | |
_failfunction, line 171 | run_gold · gold/harness.py:149 | no test names it | |
_dimensionfunction, line 178 | run_gold · gold/harness.py:147 | no test names it | |
_Inputsclass, line 195 | _inputs · gold/harness.py:230 | no test names it | |
_active_plansfunction, line 206 | _inputs · gold/harness.py:234 | no test names it | |
_correctionsfunction, line 211 | Active corrections for one input. When a correction moves a row between windows (it corrects the fact's own window column), it is applied only if its plan names both windows: a correction never restates a window by surprise (ADR-017). Those | _inputs · gold/harness.py:242 | no test names it |
_inputsfunction, line 229 | _fact · gold/harness.py:290 | no test names it | |
_expected_feedsfunction, line 256 | The members a per-feed input owes a file for: those of the fact's dimension the feed keys belong to, from their first version in the source (a dealer that joined later owes nothing for earlier months). Without such a dimension, the feeds se | _inputs · gold/harness.py:249 | no test names it |
_completefunction, line 270 | ADR-018: every input caught up past the window's end (per-feed inputs: every expected feed delivered it). A row the build says still waits for an input (`_input_complete` false) is flagged on its own (finish). | _fact · gold/harness.py:304 | no test names it |
_factfunction, line 286 | run_gold · gold/harness.py:147 | no test names it | |
_numbersfunction, line 346 | _fact · gold/harness.py:331 | no test names it | |
_tracefunction, line 354 | INSP-01: the exact inputs (silver table and version) and the query this run built from, so one window can be rebuilt later from the run log alone (gold/inspect.py). An unknown version is recorded as -1. | _inputs · gold/harness.py:252 | no test names it |
_rangefunction, line 364 | The windows this run recomputes: the lookback from as_of, plus the windows active plans name and those with rows on the unknown member; every window when a withdrawn batch's plan says so (VOID-01). | _fact · gold/harness.py:299 | no test names it |
_key_lookupfunction, line 375 | Surrogate keys by the as-of joins (ADR-005), the rows-in = rows-out check and the row ledger (LEDG-01). | _fact · gold/harness.py:297 | no test names it |
_which_joinfunction, line 388 | The first dimension join that changed the row count (run only when the total moved): names the step without a rerun by hand (LEDG-01). | _key_lookup · gold/harness.py:379 | no test names it |
_ledgerfunction, line 399 | LEDG-01: rows read per input, rows the build gave, and per dimension join rows matched, on the unknown member and not applicable. Each join's three counts must add up to the rows after the lookup, else the run fails. | _key_lookup · gold/harness.py:382 | no test names it |
_checksfunction, line 418 | Checks before publish (ADR-006): grain unique (KEY-01), orphan rate (LATE-02), reconcile with what the build gave (no row or quantity lost by the harness). A failure raises: nothing is written. | _fact · gold/harness.py:309 | no test names it |
_recordfunction, line 453 | _fact · gold/harness.py:336 | no test names it | |
summaryfunction, line 476 | The run in plain numbers (for reports). | none found in the code | test_bundle.py, test_dac_coverage.py and 1 more |
gold/inspect.py
Inspect mode (INSP-01): rebuild one window of a past gold run from what its run log names. Product-free. Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
NotInspectableclass, line 28 | The run log does not name what is needed (an older run, a dimension, a table the run did not build). | reproduce · gold/inspect.py:64 | no test names it |
Reproducedclass, line 33 | reproduce · gold/inspect.py:83 | no test names it | |
Reproduced.samemethod, line 46 | none found in the code | no test names it | |
_plainfunction, line 50 | Reproduced.same · gold/inspect.py:47 | no test names it | |
_stepsfunction, line 54 | reproduce · gold/inspect.py:66 | no test names it | |
reproducefunction, line 60 | none found in the code | test_gold_harness.py, test_demo_end_to_end.py |
gold/keys.py
Deterministic hash surrogate keys with reserved members (ADR-004, DPC-0081, DPC-0046). Landing to bronze
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
canonical_timefunction, line 26 | A version start as the recipe writes it (a naive time is taken as UTC). | key_text · gold/keys.py:35 | no test names it |
key_textfunction, line 32 | surrogate_key · gold/keys.py:42 | test_gold_plan.py | |
surrogate_keyfunction, line 40 | The member's key; the same inputs always give the same number. | none found in the code | test_gold_engine.py, test_gold_plan.py |
memberfunction, line 46 | The key a fact row points at: not applicable without a business key, unknown when no member matched. | none found in the code | test_gold_engine.py, test_gold_harness.py and 3 more |
gold/plan.py
A gold table's plan, read from its DPM gold manifest (manifest/gold/<name>.yaml) and its ODCS contract. Silver to gold
gold/ports.py
The ports gold talks to (dpc/ports.yaml: table_writer for gold, and the engine operations the harness asks for). Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
GoldTablesclass, line 20 | The read and write edges of gold. | none found in the code | test_gold_ports.py |
GoldTables.silvermethod, line 23 | Every row silver holds for `source.dataset` (every version, deleted ones marked). A table never written (an input that has delivered nothing yet) is an empty frame of its contract `columns` (name, engine type) and silver's own columns: an i | _dimension · gold/harness.py:179 (by name)_inputs · gold/harness.py:233 (by name)reproduce · gold/inspect.py:76 (by name) | test_bundle.py, test_semantic_governance.py and 62 more |
GoldTables.silver_versionmethod, line 28 | The silver table's current version (commit or snapshot), recorded per gold run (INSP-01); None when the table was never written or the format cannot say. | _inputs · gold/harness.py:232 (by name) | no test names it |
GoldTables.silver_atmethod, line 32 | The silver table as it was at `version` (time travel), to rebuild a past run's window (INSP-01). | reproduce · gold/inspect.py:76 (by name) | no test names it |
GoldTables.goldmethod, line 35 | The gold table as published, or None before its first write. | build · realms/databricks/semantic_governance/generate.py:110 (value)main · scripts/run_platform.py:459 (value)_expected_feeds · gold/harness.py:261 (by name)_fact · gold/harness.py:292 (by name)reproduce · gold/inspect.py:79 (by name) | no test names it |
GoldTables.write_dimensionmethod, line 38 | Replace the whole dimension in one transaction (its keys are deterministic, so a rebuild changes nothing that did not change). Returns the rows written. | _dimension · gold/harness.py:186 (by name) | test_gold_ports.py |
GoldTables.replace_windowsmethod, line 42 | Replace exactly these windows of a fact with `rows` in one transaction: rows of other windows are never touched, rows of these windows not in `rows` are removed (ADR-006: replace, never append). Writing the same rows twice changes nothing. | _fact · gold/harness.py:318 (by name) | test_gold_ports.py |
GoldEngineclass, line 48 | Frame operations of the harness, pure: frames in, frames or numbers out. | none found in the code | no test names it |
GoldEngine.dimensionmethod, line 51 | The common one-to-one pipeline: contract columns, hash surrogate keys per version (ADR-004), valid from and to (the first version from the beginning of time so an early fact finds it, the current one to the end of time), and the reserved me | _where · realms/databricks/semantic_governance/render.py:131 (value)_tile_query · realms/databricks/semantic_governance/render.py:169 (value)_widget · realms/databricks/semantic_governance/render.py:191 (value)scorecard · obs/success.py:157 (value)_where · semantic/answers.py:21 (value)_dimension · gold/harness.py:180 (by name)and 1 more | test_semantic_governance.py, gold_helpers.py and 10 more |
GoldEngine.overlaymethod, line 56 | Lay active corrections over silver rows (ADR-017): each correction's field takes its value on every row of its business key. Silver itself is never changed. | _inputs · gold/harness.py:245 (by name) | test_gold_engine.py |
GoldEngine.query_textmethod, line 60 | What the build runs, for people (INSP-01): the SQL model with its parameter values shown, or the name of the dataframe function. Never executed: the build binds the parameters itself. | _trace · gold/harness.py:360 (by name) | test_lineage_end_to_end.py, test_adf_generator.py and 1 more |
GoldEngine.buildmethod, line 64 | Call the fact's declared build() (SQL model or dataframe function) on its input frames. | cmd_code · scripts/explain.py:101 (by name)_fact · gold/harness.py:291 (by name)reproduce · gold/inspect.py:78 (by name)Monitor.tables · obs/monitor.py:122 (by name) | test_semantic_governance.py, conftest.py and 17 more |
GoldEngine.add_keysmethod, line 67 | Each dimension's surrogate key by a left join as of the event (ADR-005): valid_from <= t < valid_to; no member -> -1, no business key -> -2. Never drops or multiplies a row. | _key_lookup · gold/harness.py:377 (by name)_which_join · gold/harness.py:392 (by name)reproduce · gold/inspect.py:80 (by name) | test_gold_engine.py |
GoldEngine.finishmethod, line 71 | Harness columns (_is_complete: the window's flag, and false on a row whose `_input_complete` is false; _run_id) and the contract's column order. | none found in the code | test_gold_engine.py, test_meta_store.py and 2 more |
GoldEngine.only_windowsmethod, line 75 | The rows of windows from `first` on, plus the windows named in `also`. | _fact · gold/harness.py:307 (by name)_checks · gold/harness.py:443 (by name)reproduce · gold/inspect.py:80 (by name) | no test names it |
GoldEngine.window_numbersmethod, line 78 | Per window from `first` on (plus `also`): its numbers and a digest of every row's content but _run_id. | _fact · gold/harness.py:300 (by name)reproduce · gold/inspect.py:81 (by name) | test_gold_engine.py |
GoldEngine.holdmethod, line 82 | The same rows, computed once and kept for the operations that follow (released with `release`). | _fact · gold/harness.py:296 (by name)_key_lookup · gold/harness.py:377 (by name) | no test names it |
GoldEngine.countmethod, line 85 | none found in the code | test_contract_rules.py, test_dac_coverage.py and 29 more | |
GoldEngine.duplicatesmethod, line 87 | Rows beyond the first per value of `columns` (0 when unique). | _dimension · gold/harness.py:181 (by name)_checks · gold/harness.py:424 (by name) | test_contract_rules.py, test_gold_engine.py and 6 more |
GoldEngine.key_tallymethod, line 90 | Per dimension key: rows matched to a member, on the unknown member (-1), not applicable (-2) (LEDG-01). | _ledger · gold/harness.py:406 (by name) | no test names it |
GoldEngine.unknown_rowsmethod, line 93 | Rows on the unknown member, per dimension key (orphans, LATE-02). | _checks · gold/harness.py:429 (by name) | test_gold_engine.py |
GoldEngine.windows_with_unknownmethod, line 96 | Published windows still holding rows on the unknown member: re-keyed on the next recompute (ADR-004). | _range · gold/harness.py:372 (by name) | no test names it |
GoldEngine.rows_incompletemethod, line 99 | Windows where a build said a row's inputs have not all arrived (its `_input_complete` column is false). | _fact · gold/harness.py:301 (by name) | no test names it |
GoldEngine.first_seenmethod, line 102 | Each real member's first version start in its source (for which feeds a window expects). | _expected_feeds · gold/harness.py:264 (by name) | no test names it |
GoldEngine.totalsmethod, line 105 | Sum of each column (for reconciliation). | _checks · gold/harness.py:443 (by name) | test_ingest_world.py, test_reconcile_engine.py and 3 more |
GoldEngine.releasemethod, line 108 | Free what the operations held for these frames (cached rows) once the table is written. | none found in the code | test_meta_store.py, test_silver_ports.py and 2 more |
GoldReaderclass, line 112 | Read-only questions to published gold (checks, the oracle comparison, known answers): one SQL query over the gold tables, each visible by its table name (fact_otif, dim_dealer, ...). Rows come back as plain values. | none found in the code | test_gold_ports.py, test_world_clinical.py |
GoldReader.rowsmethod, line 116 | `time_zone`: the session time zone the query runs in (default: the engine's, UTC). | none found in the code | no test names it |
gold/windows.py
Windows of a fact (ADR-006), when one is complete (ADR-018) and when a rebuild is a restatement (ADR-017). Pure. Silver to gold
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|---|---|---|
_month_addfunction, line 20 | end · gold/windows.py:37range_start · gold/windows.py:53 | no test names it | |
startfunction, line 25 | end · gold/windows.py:35 | silver_helpers.py, test_gold_calendar.py and 15 more | |
endfunction, line 32 | The first instant after the window. | day_end · gold/windows.py:62caught_up · gold/windows.py:67 | no test names it |
offunction, line 41 | The window an instant falls in. | range_start · gold/windows.py:57feeds_caught_up · gold/windows.py:74 | no test names it |
range_startfunction, line 49 | The first window the recompute rebuilds on business date `as_of`: `lookback` units back (ADR-006). | none found in the code | test_gold_plan.py |
day_endfunction, line 60 | How far an input has loaded in source time once a business day's delivery window is in silver. | none found in the code | test_gold_plan.py |
caught_upfunction, line 65 | One input has loaded past the window's end (ADR-018). No watermark: it has loaded nothing yet. | none found in the code | test_gold_plan.py |
feeds_caught_upfunction, line 70 | A per-feed input (one file per dealer): each expected feed's newest data time falls in this window or a later one. Returns (complete, feeds still missing). | none found in the code | test_gold_plan.py |
Numbersclass, line 79 | What readers of a window see: its rows and the sums of its measures, and whether it was marked complete. | SparkGoldEngine.window_numbers · adapters/spark_gold.py:236SparkGoldEngine.window_numbers · adapters/spark_gold.py:225 (value)GoldEngine.window_numbers · gold/ports.py:79 (value) | test_gold_plan.py |
Numbers.same_numbersmethod, line 86 | is_restatement · gold/windows.py:105 (by name) | no test names it | |
is_restatementfunction, line 91 | A rebuild restates a window when numbers readers were entitled to treat as final change: - the window was published complete and its numbers change (a correction, a late change), or - the window, already over when it was published, was flag | none found in the code | test_gold_plan.py |