Every class, function and method in the obs package, with who calls it. Found by reading the code. A call written as object.method() is matched by the method's name, so a same-named method elsewhere can appear: those rows say by name.
262 definitions in 22 modules. Jump to a module: alerts.py · clock.py · dashboards.py · explain.py · explain_meta.py · fixture.py · freshness.py · lineage.py · monitor.py · native.py · ops.py · ops_export.py · page.py · pipeline_events.py · recorder.py · runlog.py · steps.py · success.py · system.py · trace.py · traced_store.py · views.py
Alerts that people still read (OPS-01, OPS-06, DPC-0019, DPC-0056). Rules are configuration; evaluation is pure. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
AlertConfigError class, line 37 | An alert rule, route or snooze the config may not hold. | AlertConfig.rule · obs/alerts.py:77
_need · obs/alerts.py:140
_aware · obs/alerts.py:147
parse_snooze · obs/alerts.py:155
parse_config · obs/alerts.py:169 | test_obs_alerts.py |
Rule class, line 42 | | parse_config · obs/alerts.py:166
module level · obs/alerts.py:66 (value) | no test names it |
Route class, line 50 | | parse_config · obs/alerts.py:178
module level · obs/alerts.py:67 (value) | no test names it |
Snooze class, line 56 | | parse_snooze · obs/alerts.py:159
module level · obs/alerts.py:69 (value) | no test names it |
AlertConfig class, line 65 | | parse_config · obs/alerts.py:188 | no test names it |
AlertConfig.rule method, line 73 | | payload · adapters/databricks_notify.py:30 (value)
counts_per_rule_per_day · obs/alerts.py:295 (value)
alert_tables · obs/ops.py:287 (value)
unexplained_alerts · twin/runner.py:197 (value) | no test names it |
Signal class, line 81 | One subject's verdict for one rule's signal at one check. | _maker.make · obs/ops.py:328
evaluate · obs/alerts.py:238 (value) | test_obs_alerts.py |
Alert class, line 94 | One message to one route: a rule's alert for a business day, with every subject that joined it. | _fail · obs/alerts.py:276
_recover · obs/alerts.py:286 | test_obs_alerts.py, test_obs_notify.py |
Event class, line 106 | | _fail · obs/alerts.py:266
_recover · obs/alerts.py:285 | test_resilience.py, test_lineage_dbt_manifest.py |
Scope class, line 117 | What one evaluation speaks for: these signal kinds, and (if given) only subjects of these datasets. A subject outside the scope keeps its state; inside it, a subject absent from the signals has recovered. | evaluate · obs/alerts.py:234
Monitor._lateness · obs/monitor.py:92
Monitor.run_finished · obs/monitor.py:103 | test_obs_alerts.py |
Scope.covers method, line 124 | | evaluate · obs/alerts.py:252 (by name) | test_access_model.py, test_obs_dashboards.py and 1 more |
Evaluation class, line 129 | | evaluate · obs/alerts.py:254 | no test names it |
_need function, line 138 | | parse_snooze · obs/alerts.py:153
parse_config · obs/alerts.py:165 | no test names it |
_aware function, line 144 | | parse_snooze · obs/alerts.py:156 | no test names it |
parse_snooze function, line 151 | | parse_config · obs/alerts.py:184 | no test names it |
parse_config function, line 163 | | load_config · obs/alerts.py:195 | test_obs_alerts.py |
load_config function, line 192 | | main · scripts/run_platform.py:424
Harness.build · twin/runner.py:96 | test_obs_alerts.py, test_obs_dashboards.py |
_zone function, line 203 | | business_day · obs/alerts.py:208 | no test names it |
business_day function, line 207 | | evaluate · obs/alerts.py:236
counts_per_rule_per_day · obs/alerts.py:295
runs_table · obs/ops.py:115
dataset_runs_table · obs/ops.py:146
steps_table · obs/ops.py:191
restatements_table · obs/ops.py:214 and 9 more | no test names it |
route_for function, line 211 | | _fail · obs/alerts.py:263 | no test names it |
snoozed function, line 219 | | _fail · obs/alerts.py:265 | test_obs_alerts.py |
_key function, line 226 | | _fail · obs/alerts.py:259 | no test names it |
evaluate function, line 230 | One check. Subjects a rule knew as failing that are absent from this check's signals have recovered (within the scope: by default every signal and every subject). | Monitor._evaluate · obs/monitor.py:78 | test_obs_alerts.py, test_config_sync.py and 3 more |
_fail function, line 257 | | evaluate · obs/alerts.py:249 | no test names it |
_recover function, line 280 | | evaluate · obs/alerts.py:253 | no test names it |
counts_per_rule_per_day function, line 290 | OPS-06 evidence: alerts sent per rule per business day (fired only; grouped and snoozed are not sent). | alert_tables · obs/ops.py:289 | test_obs_alerts.py |
The clock port (DPC-0032): observability reads time only through this, so a simulated week runs in a second. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Clock class, line 13 | | none found in the code | test_resilience.py |
Clock.now method, line 14 | An aware datetime. | step_prepare · realms/databricks/tasks/main.py:71 (by name)
module level · realms/databricks/v1/tasks/hop.py:28 (by name)
main · scripts/dac_coverage.py:40 (by name)
WallClock.now · scripts/run_platform.py:106 (by name)
write_ops_page · scripts/run_platform.py:130 (by name)
main.pushed · scripts/run_platform.py:408 (by name) and 20 more | no test names it |
SystemClock class, line 19 | | none found in the code | no test names it |
SystemClock.now method, line 20 | | step_prepare · realms/databricks/tasks/main.py:71 (by name)
module level · realms/databricks/v1/tasks/hop.py:28 (by name)
main · scripts/dac_coverage.py:40 (by name)
WallClock.now · scripts/run_platform.py:106 (by name)
write_ops_page · scripts/run_platform.py:130 (by name)
main.pushed · scripts/run_platform.py:408 (by name) and 19 more | no test names it |
ManualClock class, line 24 | | Harness.build · twin/runner.py:89 | test_obs_recorder.py, test_obs_runlog.py and 1 more |
ManualClock.__init__ method, line 25 | | none found in the code | no test names it |
ManualClock.now method, line 30 | | step_prepare · realms/databricks/tasks/main.py:71 (by name)
module level · realms/databricks/v1/tasks/hop.py:28 (by name)
main · scripts/dac_coverage.py:40 (by name)
WallClock.now · scripts/run_platform.py:106 (by name)
write_ops_page · scripts/run_platform.py:130 (by name)
main.pushed · scripts/run_platform.py:408 (by name) and 20 more | no test names it |
ManualClock.set method, line 33 | | none found in the code | no test names it |
ManualClock.advance method, line 38 | | FixtureRunner._rerun · obs/fixture.py:138 (by name)
FixtureRunner._dataset · obs/fixture.py:149 (by name) | test_obs_recorder.py, test_obs_runlog.py and 3 more |
The ops dashboards as definitions (G8, DPC-0057, ADR-024). Product-free: a realm compiles them to its BI tool's file (a dashboard-file adapter writes them into the realm's dashboards folder), and `evaluate` runs any of them on the ops table Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Measure class, line 35 | | module level · des/dashboards.py:21
_m · obs/dashboards.py:173
module level · obs/dashboards.py:46 (value) | test_obs_dashboards.py |
Query class, line 42 | | module level · des/dashboards.py:21
module level · obs/dashboards.py:183
module level · obs/dashboards.py:75 (value) | test_obs_dashboards.py, test_mssql_connector.py |
Query.fields method, line 54 | | none found in the code | test_ingest_ports.py, test_meta_store.py and 16 more |
Widget class, line 59 | | module level · des/dashboards.py:34
module level · obs/dashboards.py:197
module level · obs/dashboards.py:76 (value) | no test names it |
Dashboard class, line 71 | | module level · des/dashboards.py:15
module level · obs/dashboards.py:176
bundle_resources · adapters/lakeview.py:144 (value)
every · obs/dashboards.py:297 (value) | test_obs_dashboards.py |
Dashboard.query method, line 80 | | _handler.Handler.do_GET · adapters/http_api.py:73 (value)
validate · obs/dashboards.py:116 (value)
render · obs/dashboards.py:313 (value)
check_on_dashboard · twin/runner.py:143 (value)
validate · obs/dashboards.py:116 (by name)
render · obs/dashboards.py:313 (by name) and 1 more | silver_helpers.py, test_contract_rules.py and 12 more |
DefinitionError class, line 87 | | validate · obs/dashboards.py:94 | test_obs_dashboards.py |
validate function, line 91 | | compile_dashboard · adapters/lakeview.py:132 | test_obs_dashboards.py, test_slice1_local.py and 3 more |
_match function, line 124 | | evaluate · obs/dashboards.py:153 | no test names it |
_agg function, line 132 | | evaluate · obs/dashboards.py:163 | no test names it |
_sort_key function, line 145 | | evaluate · obs/dashboards.py:155 | no test names it |
_sort_key.key function, line 146 | | none found in the code | no test names it |
evaluate function, line 152 | | render · obs/dashboards.py:313 | test_obs_alerts.py, test_config_sync.py and 3 more |
_m function, line 172 | | module level · obs/dashboards.py:183 | no test names it |
every function, line 297 | Observability's dashboards and the DES-owned ones (generated side by side, owned apart). | files · realms/databricks/dashboards/generate.py:30
by_id · obs/dashboards.py:305 | test_bundle.py, test_lakeformation.py and 56 more |
by_id function, line 304 | | none found in the code | no test names it |
render function, line 311 | Every widget's rows, by widget id. | none found in the code | test_semantic_governance.py, test_obs_dashboards.py and 3 more |
A run explained in plain words, deterministically: the run's report and ops tables in, a Markdown narrative out. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
facts function, line 59 | The non-zero facts of one dataset or table, in the table's order. | ingest_lines · obs/explain.py:73
silver_lines · obs/explain.py:85
gold_lines · obs/explain.py:94 | test_explain.py, test_gold_engine.py and 2 more |
sentence function, line 65 | | ingest_lines · obs/explain.py:76
silver_lines · obs/explain.py:82
gold_lines · obs/explain.py:97 | test_explain.py |
ingest_lines function, line 69 | | explain · obs/explain.py:160 | test_explain.py |
silver_lines function, line 80 | | explain · obs/explain.py:161 (value) | test_explain.py |
gold_lines function, line 91 | | explain · obs/explain.py:162 (value) | test_explain.py |
differences function, line 101 | | explain · obs/explain.py:161 | test_end_to_end.py, test_explain.py and 1 more |
bronze_differences function, line 110 | | explain · obs/explain.py:160 (value) | test_end_to_end.py, test_explain.py |
verdict_lines function, line 118 | | explain · obs/explain.py:173 | no test names it |
held_lines function, line 128 | | explain · obs/explain.py:175 | no test names it |
layer_block function, line 136 | | explain · obs/explain.py:173 | no test names it |
explain function, line 148 | | cmd_run · scripts/explain.py:59 | test_explain.py, test_twin_gate.py |
A dataset's metadata and the repository's checks explained in plain words, deterministically: no model, no network. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
listed function, line 59 | | _promise · obs/explain_meta.py:95
_history · obs/explain_meta.py:110
_checked_and_owned · obs/explain_meta.py:167
_workflow · obs/explain_meta.py:209 | test_azure_adapters.py, test_cloud_adapters.py and 7 more |
_arrival function, line 64 | | explain_dataset · obs/explain_meta.py:185 | no test names it |
_promise function, line 91 | | explain_dataset · obs/explain_meta.py:185 | no test names it |
_history function, line 99 | | explain_dataset · obs/explain_meta.py:185 | no test names it |
_loading function, line 126 | | explain_dataset · obs/explain_meta.py:185 | no test names it |
_bad_rows function, line 146 | | explain_dataset · obs/explain_meta.py:186 | no test names it |
_checked_and_owned function, line 160 | | explain_dataset · obs/explain_meta.py:186 | no test names it |
explain_dataset function, line 178 | The Markdown narrative of one dataset manifest (and its contract, when given). | cmd_metadata · scripts/explain.py:76 | test_explain.py |
_triggers function, line 197 | | _workflow · obs/explain_meta.py:209 | no test names it |
_step_name function, line 204 | | _workflow · obs/explain_meta.py:211 | no test names it |
_workflow function, line 208 | | explain_cicd · obs/explain_meta.py:226 | no test names it |
explain_cicd function, line 216 | The checks a change passes. help_lines: the Makefile's `make <target> <what it does>` lines; hooks: (repo, hook id) pairs; workflows: file name -> parsed workflow. | cmd_cicd · scripts/explain.py:94 | test_explain.py |
FIXTURE - a tiny fake runner that fills the run log until W1b-W1e produce real runs. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Effect class, line 40 | What the run log shows when a detector meets its incident (the engine's expected behaviour, per DAC). | module level · obs/fixture.py:52
module level · obs/fixture.py:51 (value) | no test names it |
FixtureRun class, line 73 | | FixtureRunner.__init__ · obs/fixture.py:100 | no test names it |
_counts function, line 78 | | FixtureRunner._effect · obs/fixture.py:179 | no test names it |
FixtureRunner class, line 90 | | Harness.build · twin/runner.py:97 | no test names it |
FixtureRunner.__init__ method, line 91 | | none found in the code | no test names it |
FixtureRunner.day_start method, line 102 | | FixtureRunner.run_day · obs/fixture.py:109
FixtureRunner._readers · obs/fixture.py:217
Harness.build · twin/runner.py:88 (by name)
ClinicalTrial._sites · world/domains/clinical.py:752 (by name)
ClinicalTrial._readings · world/domains/clinical.py:788 (by name)
ClinicalTrial._reading_truth · world/domains/clinical.py:853 (by name) and 5 more | no test names it |
FixtureRunner._pushed method, line 105 | | FixtureRunner.__init__ · obs/fixture.py:97 (value) | no test names it |
FixtureRunner.run_day method, line 108 | | Harness.build · twin/runner.py:101 (by name) | no test names it |
FixtureRunner._rerun method, line 130 | | FixtureRunner.run_day · obs/fixture.py:121 | no test names it |
FixtureRunner._dataset method, line 143 | | FixtureRunner.run_day · obs/fixture.py:118 | no test names it |
FixtureRunner._effect method, line 170 | | FixtureRunner._dataset · obs/fixture.py:155 | no test names it |
FixtureRunner._bookmarks method, line 189 | | FixtureRunner._dataset · obs/fixture.py:167 | no test names it |
FixtureRunner._system method, line 204 | | FixtureRunner._rerun · obs/fixture.py:140
FixtureRunner._dataset · obs/fixture.py:168 | no test names it |
FixtureRunner._readers method, line 216 | | FixtureRunner.run_day · obs/fixture.py:122 | no test names it |
Freshness against the manifest's targets (FRESH-01, OPS-01, G10). Pure functions, plus one reader of the store. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
FreshnessTarget class, line 31 | | target_from_view · obs/freshness.py:82 | test_obs_freshness.py |
FreshnessTarget.name method, line 41 | | none found in the code | no test names it |
FreshnessResult class, line 46 | | check · obs/freshness.py:135 | no test names it |
FreshnessResult.subject method, line 59 | | snoozed · obs/alerts.py:221 (value)
_fail · obs/alerts.py:259 (value)
freshness_tables · obs/ops.py:257 (value)
alert_tables · obs/ops.py:287 (value)
freshness_signals · obs/ops.py:356 (value)
unexplained_alerts · twin/runner.py:197 (value) | test_ingest_steps.py, test_obs_alerts.py and 3 more |
tier_for function, line 64 | | target_from_view · obs/freshness.py:82 | test_obs_freshness.py |
target_from_view function, line 73 | A dataset's targets from the store's merged view; None when the dataset declares no freshness target. | targets_from_store · obs/freshness.py:92 | test_obs_freshness.py |
targets_from_store function, line 87 | | none found in the code | test_obs_freshness.py |
state_of function, line 98 | | check · obs/freshness.py:136 | test_obs_freshness.py |
_hours function, line 106 | | check · obs/freshness.py:134 | no test names it |
last_success function, line 110 | | check · obs/freshness.py:130
next_due · obs/freshness.py:158 | test_obs_freshness.py |
_as_time function, line 119 | | check · obs/freshness.py:139 | no test names it |
check function, line 127 | Every target's dataset freshness, then each time-valued bookmark's lag behind its newest sibling. | none found in the code | test_azure_adapters.py, test_contract_rules.py and 30 more |
read_bookmarks function, line 150 | | none found in the code | no test names it |
next_due function, line 154 | When the next dataset would pass its error target if nothing loads before then: the one moment a lateness check is worth running (ADR-024 rule 4, no fixed interval). None when nothing can become late (never loaded datasets are already repor | none found in the code | test_obs_freshness.py |
Lineage before the engine's catalog (X7, LIN-01, B9): the engine's catalog records lineage from bronze on; the hops before it (source system -> mover -> landing -> bronze) are only in our file log. This turns file-log records into run event Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
events_from_files function, line 24 | Run events for the batches in `files`. `system_of` maps a source name to its system (erp -> erp_db), which names the input dataset's namespace; `at` is the event time (ISO, from the clock port). Duplicates skipped by the lander produce no e | none found in the code | test_obs_lineage.py |
Monitoring, pushed by runs (ADR-024 rule 4): nothing here runs on a fixed interval. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Notifier class, line 34 | Where an alert goes: a pager, a ticket queue, a digest. The route says which; the adapter delivers. | none found in the code | no test names it |
Notifier.send method, line 37 | | Monitor._evaluate · obs/monitor.py:81 (by name) | no test names it |
MemoryNotifier class, line 40 | | main · scripts/run_platform.py:424
FixtureRunner.__init__ · obs/fixture.py:95 | test_obs_dashboards.py, test_obs_notify.py |
MemoryNotifier.__init__ method, line 41 | | none found in the code | no test names it |
MemoryNotifier.send method, line 44 | | Monitor._evaluate · obs/monitor.py:81 (by name) | no test names it |
CheckOutput class, line 49 | | Monitor._evaluate · obs/monitor.py:84
Monitor.run_finished · obs/monitor.py:105 | no test names it |
_only function, line 55 | | Monitor.run_finished · obs/monitor.py:100 | no test names it |
Monitor class, line 63 | | main · scripts/run_platform.py:423
FixtureRunner.__init__ · obs/fixture.py:96 | test_obs_dashboards.py |
Monitor.__init__ method, line 64 | | none found in the code | no test names it |
Monitor._evaluate method, line 77 | | Monitor._lateness · obs/monitor.py:92
Monitor.run_finished · obs/monitor.py:103 | no test names it |
Monitor._lateness method, line 86 | | Monitor.run_finished · obs/monitor.py:104
Monitor.check_lateness · obs/monitor.py:108 | no test names it |
Monitor.run_finished method, line 96 | Pushed by the recorder as a run ends: the run's own datasets, detectors and checks, then lateness (a run that loads can end a lateness). | main.pushed · scripts/run_platform.py:408 (by name)
FixtureRunner._pushed · obs/fixture.py:106 (by name) | no test names it |
Monitor.check_lateness method, line 107 | | Monitor.wake · obs/monitor.py:114 | no test names it |
Monitor.wake method, line 110 | The scheduler's one wake-up: a lateness check only when one is due (`self.due`), else nothing runs. | write_ops_page · scripts/run_platform.py:131 (by name)
FixtureRunner.run_day · obs/fixture.py:124 (by name) | no test names it |
Monitor.copy_system method, line 116 | | FixtureRunner.run_day · obs/fixture.py:127 (by name) | test_obs_dashboards.py |
Monitor.tables method, line 119 | | none found in the code | test_genie_deploy.py, test_lakeformation.py and 39 more |
Platform-native screens and notifications, adopted, not rebuilt (ADR-024 rule 1, set a). Operations
The ops builder (G8, DPC-0057): run log + platform telemetry + freshness + alerts -> the compact ops tables. Pure. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
_secs function, line 93 | | runs_table · obs/ops.py:117
_dataset_row · obs/ops.py:140
step_durations_table · obs/ops.py:230 | no test names it |
check_kind function, line 97 | | none found in the code | no test names it |
_telemetry_jobs function, line 101 | Platform job runs no run-log run accounts for (all of them when no run log is connected). | runs_table · obs/ops.py:121
dataset_runs_table · obs/ops.py:155 | no test names it |
runs_table function, line 107 | | build · obs/ops.py:301 | no test names it |
_dataset_row function, line 136 | | dataset_runs_table · obs/ops.py:146 | no test names it |
dataset_runs_table function, line 144 | | build · obs/ops.py:295 | no test names it |
_performance function, line 159 | Attempts per dataset per day (a retry is any attempt after the first) and the time against the dataset's own median so far (a slow-down is SLOW_RATIO or more). Only what needs the run log: per dataset, not per job. | dataset_runs_table · obs/ops.py:156 | no test names it |
_count_by function, line 176 | | build · obs/ops.py:303 | no test names it |
steps_table function, line 184 | Every owner's step row (the one step format), with the owner and the owner's screen to go to. | build · obs/ops.py:305 | no test names it |
_fields function, line 201 | | restatements_table · obs/ops.py:211 | no test names it |
restatements_table function, line 205 | One row per restated window and measure (rows included): what finance asks when last month's number moved. | build · obs/ops.py:309 | test_ops_restatements.py |
step_durations_table function, line 220 | Time per step: from the dataset run's start (or the step before) to the step's record. | build · obs/ops.py:306 | no test names it |
ingestion_tables function, line 234 | | build · obs/ops.py:296 | no test names it |
freshness_tables function, line 255 | | build · obs/ops.py:297 | no test names it |
usage_daily function, line 269 | | build · obs/ops.py:312 | no test names it |
lineage_table function, line 281 | | build · obs/ops.py:313 | no test names it |
alert_tables function, line 286 | | build · obs/ops.py:298 | no test names it |
build function, line 293 | | none found in the code | test_semantic_governance.py, conftest.py and 17 more |
_maker function, line 322 | | run_signals · obs/ops.py:337
freshness_signals · obs/ops.py:355 | no test names it |
_maker.make function, line 325 | | run_signals · obs/ops.py:341
freshness_signals · obs/ops.py:356 | no test names it |
run_signals function, line 333 | A finished run's signals (pushed as it ends): every dataset it ran, failing or not, so recoveries resolve. Detectors and checks come only from the step rows (the one step format); details stay with the owner, and the alert carries the owner | none found in the code | no test names it |
freshness_signals function, line 354 | | none found in the code | no test names it |
The ops tables as a typed, portable file, so a realm can publish them where its dashboards read (B13). Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
type_of function, line 20 | | column_types · obs/ops_export.py:42 | no test names it |
column_types function, line 34 | Columns in first-seen order, each typed by its first non-empty value; integer widens to decimal if mixed. | export · obs/ops_export.py:66 | test_obs_dashboards.py, test_ops_export.py |
_carry function, line 47 | | export · obs/ops_export.py:68 | no test names it |
export function, line 59 | Declared columns win (so an empty table keeps its shape); a column the rows have beyond them is added, typed by its values. | module level · realms/databricks/engine/tasks/run_days.py:91
write_ops_page · scripts/run_platform.py:150
main · scripts/run_platform.py:737 | test_meta_loader.py, test_obs_trace.py and 1 more |
_back function, line 72 | | restore · obs/ops_export.py:84 | no test names it |
restore function, line 82 | | publish · adapters/spark_ops_tables.py:33 | test_ops_export.py |
obs/page.py
The local ops page (B10): one self-contained HTML file written after a local run, so a person can see what happened without a workspace. It reads only the ops tables (obs/ops.py, the same rows the ops metric views and dashboards read) plus Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
_cls function, line 31 | | _cell · obs/page.py:49 | no test names it |
_cell function, line 43 | | table · obs/page.py:60 | no test names it |
table function, line 53 | Rows as an HTML table; columns default to every key of the first row. More than `limit` rows are cut, and the cut is said. | render · obs/page.py:103 | test_lakeformation.py, test_semantic_governance.py and 84 more |
_tile function, line 65 | | render · obs/page.py:88 | no test names it |
_sum function, line 69 | | render · obs/page.py:91 | no test names it |
render function, line 73 | The page. `layer_checks`: {day, layer, equals_oracle}; `incidents`: {day, layer, incident, detector, dataset, verdict, evidence}; `hops`: {dataset, hop, n, p50, p95, p99, basis}. | write_ops_page · scripts/run_platform.py:141 | test_semantic_governance.py, test_obs_dashboards.py and 3 more |
Declarative pipelines report themselves (ADR-024 rule 5): the pipeline event log, behind one read-only port. Operations
The run recorder: one id from start to finish (G8, OPS-03, DPC-0057). Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
RecorderError class, line 40 | The recorder was used out of order (a dataset finished twice, a step after the dataset closed). | DatasetHandle._open · obs/recorder.py:69 | test_obs_recorder.py |
StepHandle class, line 45 | | DatasetHandle.step · obs/recorder.py:78 | no test names it |
StepHandle.warn method, line 50 | | FixtureRunner._effect · obs/fixture.py:186 (by name) | no test names it |
StepHandle.skip method, line 53 | | FixtureRunner._dataset · obs/fixture.py:165 (by name) | no test names it |
DatasetHandle class, line 57 | | RunHandle.dataset · obs/recorder.py:149 | no test names it |
DatasetHandle.__init__ method, line 58 | | none found in the code | no test names it |
DatasetHandle.store method, line 64 | | StoreGoldTables.__init__ · adapters/spark_gold.py:412 (value)
StoreGoldReader.__init__ · adapters/spark_gold.py:475 (value)
StoreSilverTables.__init__ · adapters/spark_silver.py:286 (value)
StoreBatchRegistry.__init__ · adapters/store_registry.py:18 (value)
_inputs · gold/harness.py:247 (value)
FixtureRunner.__init__ · obs/fixture.py:93 (value) and 4 more | gold_helpers.py, silver_helpers.py and 33 more |
DatasetHandle._open method, line 67 | | DatasetHandle._log · obs/recorder.py:72
DatasetHandle.error · obs/recorder.py:106
DatasetHandle.file · obs/recorder.py:111
DatasetHandle._finish · obs/recorder.py:117 | no test names it |
DatasetHandle._log method, line 71 | | DatasetHandle.step · obs/recorder.py:82
DatasetHandle.check · obs/recorder.py:95
DatasetHandle.detect · obs/recorder.py:100 | no test names it |
DatasetHandle.step method, line 77 | | MemoryMetaStore.log_step · adapters/memory_meta.py:198 (value)
PostgresMetaStore.log_step · adapters/postgres_meta.py:158 (value)
SparkRunLog.write · adapters/spark_delta.py:117 (value)
steps_table · obs/ops.py:188 (value)
step_durations_table · obs/ops.py:230 (value)
run_signals · obs/ops.py:343 (value) and 4 more | no test names it |
DatasetHandle.check method, line 89 | A quality rule, a reconciliation entry or a profile: its verdict as a step of this dataset run. | MemoryMetaStore.set_dq_rules · adapters/memory_meta.py:359 (value)
PostgresMetaStore.set_dq_rules · adapters/postgres_meta.py:241 (value)
SparkChecks.write · adapters/spark_delta.py:128 (value)
FixtureRunner._dataset · obs/fixture.py:159 (by name)
Monitor._lateness · obs/monitor.py:88 (by name) | test_azure_adapters.py, test_contract_rules.py and 30 more |
DatasetHandle.detect method, line 97 | A detector fired (DAC id, e.g. DEL-01). Handled is warned; blocked (the load stopped) is failed. | FixtureRunner._effect · obs/fixture.py:179 (by name) | silver_helpers.py, test_baseline_stage.py and 10 more |
DatasetHandle.error method, line 104 | An error the step handled (a contract mismatch held back, a retry that later passed): logged, not raised. | main · scripts/run_hop.py:108 (by name)
main · scripts/run_platform.py:372 (by name)
main · scripts/run_world.py:161 (by name)
FixtureRunner._effect · obs/fixture.py:177 (by name) | test_bundle.py, test_azure_adapters.py and 34 more |
DatasetHandle.file method, line 109 | | cmd_code · scripts/explain.py:107 (value)
FixtureRunner._dataset · obs/fixture.py:152 (by name)
FixtureRunner._effect · obs/fixture.py:174 (by name) | no test names it |
DatasetHandle._finish method, line 115 | | DatasetHandle.succeed · obs/recorder.py:126
DatasetHandle.fail · obs/recorder.py:129
DatasetHandle.skip · obs/recorder.py:132 | no test names it |
DatasetHandle.succeed method, line 124 | | FixtureRunner._rerun · obs/fixture.py:139 (by name)
FixtureRunner._dataset · obs/fixture.py:167 (by name)
RunHandle.dataset · obs/recorder.py:162 (by name) | test_end_to_end.py, test_ingest_world.py and 2 more |
DatasetHandle.fail method, line 128 | | RunHandle.dataset · obs/recorder.py:157 (by name)
RunRecorder.run · obs/recorder.py:200 (by name) | no test names it |
DatasetHandle.skip method, line 131 | | FixtureRunner._dataset · obs/fixture.py:165 (by name) | no test names it |
RunHandle class, line 135 | | RunRecorder.run · obs/recorder.py:192 | no test names it |
RunHandle.__init__ method, line 136 | | none found in the code | no test names it |
RunHandle.dataset method, line 144 | | main · realms/aws-databricks/meta_setup.py:88 (value)
module level · realms/databricks/declarative/silver.py:77 (value)
lakebase · realms/databricks/engine/tasks/probe.py:83 (value)
hop_delays · scripts/run_platform.py:161 (value)
main.housekeeping · scripts/run_platform.py:513 (value)
AzureLanding.read_bytes · adapters/azure_blob.py:102 (value) and 70 more | test_bundle.py, gold_helpers.py and 65 more |
RunHandle.job_tags method, line 164 | Tags for the job and compute this run uses: domain, data product and run id. Spend itself is the separate cost product's (ADR-024 rule 8); these tags are what lets it attribute spend to a run and a product. | FixtureRunner._system · obs/fixture.py:208 (by name) | test_obs_recorder.py |
RunHandle.lineage_run_id method, line 170 | The run id as lineage events carry it (UUID form of the same 128 bits). | none found in the code | test_obs_recorder.py |
RunHandle.log_fields method, line 174 | Fields for every structured log line written during the run. | none found in the code | test_obs_recorder.py |
RunRecorder class, line 179 | | FixtureRunner.__init__ · obs/fixture.py:97 | test_obs_recorder.py, test_obs_runlog.py and 1 more |
RunRecorder.__init__ method, line 180 | | none found in the code | no test names it |
RunRecorder._finished method, line 183 | | RunRecorder.run · obs/recorder.py:204 | no test names it |
RunRecorder.run method, line 188 | | none found in the code | no test names it |
The run log, read back for operations (DPC-0057): every run in a time window with its timestamps. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
RunRow class, line 21 | | StampingRunLog._add · adapters/memory_runlog.py:184
parse · adapters/postgres_runlog.py:41 | no test names it |
DatasetRunRow class, line 33 | | StampingRunLog._add · adapters/memory_runlog.py:190
parse · adapters/postgres_runlog.py:43 | test_obs_freshness.py |
StepRow class, line 48 | | StampingRunLog._add · adapters/memory_runlog.py:194
parse · adapters/postgres_runlog.py:46 | test_ops_restatements.py |
FileRow class, line 60 | | StampingRunLog._add · adapters/memory_runlog.py:197
parse · adapters/postgres_runlog.py:48 | no test names it |
ErrorRow class, line 74 | | StampingRunLog._add · adapters/memory_runlog.py:200
parse · adapters/postgres_runlog.py:50 | no test names it |
RunLogSnapshot class, line 85 | | StampingRunLog.snapshot · adapters/memory_runlog.py:170
NullRunLog.snapshot · adapters/null_runlog.py:16
parse · adapters/postgres_runlog.py:40
_only · obs/monitor.py:56
merge · obs/runlog.py:100 | test_obs_pipeline_events.py, test_ops_restatements.py |
RunLogReader class, line 93 | | none found in the code | no test names it |
RunLogReader.snapshot method, line 94 | Runs of the domain that started in [since, until), with everything recorded for them, in record order. | Monitor.run_finished · obs/monitor.py:99 (by name)
Monitor.check_lateness · obs/monitor.py:108 (by name)
Monitor.tables · obs/monitor.py:120 (by name) | test_meta_loader.py, test_obs_recorder.py and 10 more |
merge function, line 99 | | none found in the code | test_baseline_stage.py, test_meta_store.py and 7 more |
The one step format (ADR-024 rule 2): every owner writes one result row per check to the run log. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
kind_of function, line 33 | The owner kind of a step name, or None for a pipeline step (land, bronze, silver.<stage>, ingest.*). | check_kind · obs/ops.py:98
steps_table · obs/ops.py:188
step_durations_table · obs/ops.py:230
run_signals · obs/ops.py:343
owner_of · obs/steps.py:40 | test_gold_harness.py, test_hop_reports.py and 4 more |
owner_of function, line 39 | | steps_table · obs/ops.py:190
run_signals · obs/ops.py:344 | test_obs_steps.py |
severity function, line 44 | | steps_table · obs/ops.py:193 | test_contract_rules.py, test_explain.py and 17 more |
violations function, line 48 | What breaks the one step format in this row (empty when it keeps it). | none found in the code | test_gold_harness.py, test_obs_steps.py and 3 more |
The platform's success criteria and the scorecard a run computes against them (B20; CCA-P D4 4.1: define what good means first, one metric per dimension with a target, no blended score). Pure. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Criterion class, line 28 | | module level · obs/success.py:36 | no test names it |
Evidence class, line 60 | What one run saw. Empty fields mean the run did not look, never that all was well. | success_scorecard · scripts/run_platform.py:173
module level · obs/success.py:148 (value) | test_obs_success.py |
Score class, line 74 | | scorecard · obs/success.py:157 | no test names it |
_pct function, line 84 | | _all · obs/success.py:91 | no test names it |
_all function, line 88 | | _c1 · obs/success.py:97
_c2 · obs/success.py:101
_c3 · obs/success.py:105
_c4 · obs/success.py:109
_c5 · obs/success.py:118 | no test names it |
_c1 function, line 94 | | module level · obs/success.py:149 (value) | no test names it |
_c2 function, line 100 | | module level · obs/success.py:149 (value) | no test names it |
_c3 function, line 104 | | module level · obs/success.py:149 (value) | no test names it |
_c4 function, line 108 | | module level · obs/success.py:149 (value) | no test names it |
_c5 function, line 112 | | module level · obs/success.py:149 (value) | no test names it |
_c8 function, line 122 | | module level · obs/success.py:149 (value) | no test names it |
_c9 function, line 133 | | module level · obs/success.py:149 (value) | no test names it |
_c15 function, line 140 | | module level · obs/success.py:149 (value) | no test names it |
scorecard function, line 152 | | success_scorecard · scripts/run_platform.py:192 | test_obs_success.py |
rows function, line 165 | The scorecard as ops table rows (ops_success), for the dashboards. | none found in the code | no test names it |
The platform's own records, behind one port: job runs, lineage, and access (ADR-024: platform telemetry adopted). Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
JobRunRow class, line 26 | | DatabricksSystemTables.jobs · adapters/databricks_system.py:81
FixtureRunner._system · obs/fixture.py:209 | test_obs_dashboards.py, test_obs_system.py |
LineageRow class, line 37 | | DatabricksSystemTables.lineage · adapters/databricks_system.py:95
FixtureRunner._system · obs/fixture.py:213 | test_obs_dashboards.py, test_obs_system.py |
AccessRow class, line 46 | | DatabricksSystemTables.access · adapters/databricks_system.py:102
FixtureRunner._readers · obs/fixture.py:220 | test_obs_dashboards.py, test_obs_system.py |
SystemSnapshot class, line 54 | | read_all · obs/system.py:69
DailyCopy.__init__ · obs/system.py:86
DailyCopy.copy · obs/system.py:93
DailyCopy.on_demand · obs/system.py:100 | test_obs_pipeline_events.py |
SystemTables class, line 60 | | none found in the code | no test names it |
SystemTables.jobs method, line 61 | | read_all · obs/system.py:69 (by name) | no test names it |
SystemTables.lineage method, line 63 | | read_all · obs/system.py:69 (by name) | test_obs_dashboards.py, test_obs_lineage.py and 10 more |
SystemTables.access method, line 65 | | generate · realms/databricks/semantic_governance/generate.py:173 (value)
read_all · obs/system.py:69 (by name) | test_lakeformation.py, test_semantic_governance.py and 5 more |
read_all function, line 68 | | DailyCopy.copy · obs/system.py:92
DailyCopy.on_demand · obs/system.py:100 | test_obs_system.py |
_business_day function, line 72 | | DailyCopy.copy · obs/system.py:89 | no test names it |
DailyCopy class, line 76 | The telemetry, copied once per business day from a watermark (ADR-024 rule 4). | Monitor.__init__ · obs/monitor.py:69 | test_obs_system.py |
DailyCopy.__init__ method, line 83 | | none found in the code | no test names it |
DailyCopy.copy method, line 88 | | none found in the code | no test names it |
DailyCopy.on_demand method, line 98 | The copy plus what is newer than the watermark, read now for the one who asked (costs a query). | none found in the code | test_obs_system.py |
The telemetry port: one id for a run, everywhere (G8). The run id IS the trace id. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
Span class, line 25 | | TracedStore.__init__ · obs/traced_store.py:26 (value) | no test names it |
Span.trace_id method, line 27 | | OtelSpan.child · adapters/otel_tracer.py:42 (value)
RunHandle.__init__ · obs/recorder.py:138 (value)
RunRecorder.run · obs/recorder.py:191 (value)
LocalSpan.child · obs/trace.py:82 (value) | test_obs_recorder.py, test_obs_trace.py and 1 more |
Span.span_id method, line 30 | | RunHandle.log_fields · obs/recorder.py:176 (value)
LocalSpan.child · obs/trace.py:82 (value) | test_obs_trace.py |
Span.child method, line 32 | | DatasetHandle.step · obs/recorder.py:78 (by name)
RunHandle.dataset · obs/recorder.py:146 (by name)
TracedStore.start_dataset_run · obs/traced_store.py:42 (by name)
TracedStore.log_step · obs/traced_store.py:49 (by name) | test_obs_trace.py, test_mimic_runner.py and 1 more |
Span.set method, line 34 | | none found in the code | no test names it |
Span.end method, line 36 | | OtelSpan.end · adapters/otel_tracer.py:58 (by name)
_complete · gold/harness.py:279 (by name)
_fact · gold/harness.py:329 (by name)
DatasetHandle.step · obs/recorder.py:83 (by name)
DatasetHandle._finish · obs/recorder.py:122 (by name)
RunRecorder.run · obs/recorder.py:203 (by name) and 3 more | no test names it |
Tracer class, line 39 | | none found in the code | no test names it |
Tracer.start method, line 40 | A new root span: a new trace, so a new run id. | none found in the code | silver_helpers.py, test_gold_calendar.py and 15 more |
valid_trace_id function, line 45 | | new_trace_id · obs/trace.py:52
as_uuid · obs/trace.py:65 | test_obs_trace.py |
new_trace_id function, line 49 | | OtelTracer.start · adapters/otel_tracer.py:75
LocalTracer.start · obs/trace.py:101 | test_obs_trace.py |
new_span_id function, line 56 | | OtelSpan.child · adapters/otel_tracer.py:43
OtelTracer.start · adapters/otel_tracer.py:75
LocalSpan.child · obs/trace.py:82
LocalTracer.start · obs/trace.py:101 | no test names it |
as_uuid function, line 63 | The run id in the UUID form lineage events want for their run id (same 128 bits). | RunHandle.lineage_run_id · obs/recorder.py:172 | test_obs_recorder.py, test_obs_trace.py |
LocalSpan class, line 71 | | LocalSpan.child · obs/trace.py:82
LocalTracer.start · obs/trace.py:101 | no test names it |
LocalSpan.child method, line 81 | | DatasetHandle.step · obs/recorder.py:78 (by name)
RunHandle.dataset · obs/recorder.py:146 (by name)
TracedStore.start_dataset_run · obs/traced_store.py:42 (by name)
TracedStore.log_step · obs/traced_store.py:49 (by name) | test_obs_trace.py, test_mimic_runner.py and 1 more |
LocalSpan.set method, line 84 | | none found in the code | no test names it |
LocalSpan.end method, line 87 | | OtelSpan.end · adapters/otel_tracer.py:58 (by name)
_complete · gold/harness.py:279 (by name)
_fact · gold/harness.py:329 (by name)
DatasetHandle.step · obs/recorder.py:83 (by name)
DatasetHandle._finish · obs/recorder.py:122 (by name)
RunRecorder.run · obs/recorder.py:203 (by name) and 3 more | no test names it |
LocalTracer class, line 96 | | main · scripts/run_platform.py:393
Harness.build · twin/runner.py:97 | test_obs_recorder.py, test_obs_runlog.py and 3 more |
LocalTracer.__init__ method, line 97 | | none found in the code | no test names it |
LocalTracer.start method, line 100 | | none found in the code | silver_helpers.py, test_gold_calendar.py and 15 more |
Live wiring of observability (B10, ADR-024): a metadata store that traces every run as it is recorded and pushes "run finished" to its hooks. The engine runners already record through the store port; wrapping the store wires them to the tra Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
TracedStore class, line 23 | | main · scripts/run_platform.py:410 | test_obs_traced_store.py |
TracedStore.__init__ method, line 24 | | none found in the code | no test names it |
TracedStore.__getattr__ method, line 29 | | none found in the code | no test names it |
TracedStore.start_run method, line 32 | | StampingRunLog.start_run · adapters/memory_runlog.py:77 (by name)
run_gold · gold/harness.py:132 (by name)
run_housekeeping · housekeeping.py:127 (by name)
ingest · ingest/runner.py:126 (by name)
RunRecorder.run · obs/recorder.py:191 (by name)
build_silver · silver/runner.py:85 (by name) | gold_helpers.py, test_housekeeping.py and 7 more |
TracedStore.start_dataset_run method, line 37 | | StampingRunLog.start_dataset_run · adapters/memory_runlog.py:83 (by name)
run_gold · gold/harness.py:139 (by name)
_sweep · housekeeping.py:146 (by name)
_one · ingest/runner.py:149 (by name)
RunHandle.dataset · obs/recorder.py:148 (by name)
_one · silver/runner.py:123 (by name) and 1 more | gold_helpers.py, test_housekeeping.py and 7 more |
TracedStore.log_step method, line 45 | | StampingRunLog.log_step · adapters/memory_runlog.py:88 (by name)
_step · gold/harness.py:168 (by name)
_sweep_leased · housekeeping.py:167 (by name)
_profile_step · ingest/runner.py:72 (by name)
_detect_steps · ingest/runner.py:109 (by name)
_one_leased · ingest/runner.py:196 (by name) and 4 more | test_meta_store.py, test_obs_traced_store.py |
TracedStore.finish_dataset_run method, line 53 | | StampingRunLog.finish_dataset_run · adapters/memory_runlog.py:106 (by name)
_fail · gold/harness.py:174 (by name)
_dimension · gold/harness.py:189 (by name)
_fact · gold/harness.py:342 (by name)
_sweep_leased · housekeeping.py:177 (by name)
_one_leased · ingest/runner.py:176 (by name) and 4 more | gold_helpers.py, test_housekeeping.py and 5 more |
TracedStore.finish_run method, line 59 | | StampingRunLog.finish_run · adapters/memory_runlog.py:110 (by name)
run_gold · gold/harness.py:156 (by name)
run_housekeeping · housekeeping.py:134 (by name)
ingest · ingest/runner.py:135 (by name)
RunRecorder.run · obs/recorder.py:201 (by name)
build_silver · silver/runner.py:97 (by name) | gold_helpers.py, test_housekeeping.py and 4 more |
Certified ops definitions (ADR-024 rule 7): ops metric views over the compact ops tables. Product-free. Operations
| Definition | What it does (its own first sentence) | Called by | Named in tests |
|---|
ViewMeasure class, line 18 | | _vm · obs/views.py:38
module level · obs/views.py:31 (value) | no test names it |
OpsView class, line 26 | | module level · obs/views.py:42 | no test names it |
OpsView.measure method, line 33 | | known_answers_sql · realms/databricks/semantic_governance/render.py:144 (value)
_tile_query · realms/databricks/semantic_governance/render.py:167 (value)
_widget · realms/databricks/semantic_governance/render.py:180 (value)
dashboard_json · realms/databricks/semantic_governance/render.py:215 (value)
oracle_value · semantic/metrics.py:280 (value)
_explain · twin/gate.py:133 (value) and 2 more | test_semantic_governance.py, test_end_to_end.py and 7 more |
_vm function, line 37 | | module level · obs/views.py:44 | no test names it |
view_for function, line 87 | | query_sql · adapters/lakeview.py:53
problem · obs/views.py:94 | no test names it |
problem function, line 91 | Why an aggregate query cannot be answered by the certified view over its table (None when it can). | none found in the code | test_gold_plan.py, test_meta_loader.py and 1 more |
lint function, line 107 | Definition errors in VIEWS (an unknown table, a repeated name, a view per table at most). | none found in the code | no test names it |