cleandataengineering.aiData Platform, from the big picture to the code
You are here: Start here › Code map › Who calls what › Deployable bundles (realms)

Data PlatformZoom level 3

Who calls what: Deployable bundles (realms)

Every class, function and method in Deployable bundles (realms), with who calls it. Found by reading the code. A call written as object.method() is matched by the method's name, so a same-named method elsewhere can appear: those rows say by name.

135 definitions in 19 modules. Jump to a module: generate_bundle.py · integration.py · lakeformation.py · meta_setup.py · generate_bundle.py · test_realm.py · enable_anomaly_detection.py · generate.py · compare.py · silver.py · probe.py · semantic_layer.py · silver.py · deploy_genie.py · generate.py · render.py · checks.py · main.py · paths.py

realms/aws-databricks/generate_bundle.py

Generate the Databricks Asset Bundle for Databricks on AWS from the compiled manifests (M4: jobs are generated, never hand-written). none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
BundleError
class, line 55
check · realms/aws-databricks/generate_bundle.py:63test_bundle.py
check
function, line 59
The domain's name, after the checks that stop a generation: a safe name, a timezone, at least one dataset.platform_job · realms/aws-databricks/generate_bundle.py:101
proof_job · realms/aws-databricks/generate_bundle.py:118
generate · realms/aws-databricks/generate_bundle.py:185
test_azure_adapters.py, test_contract_rules.py and 30 more
common
function, line 75
Task parameters every V1 task takes.hop_task · realms/aws-databricks/generate_bundle.py:88
proof_job · realms/aws-databricks/generate_bundle.py:126
test_gold_engine.py, test_secrets.py and 4 more
hop_task
function, line 81
platform_job · realms/aws-databricks/generate_bundle.py:102no test names it
tags
function, line 96
platform_job · realms/aws-databricks/generate_bundle.py:107
proof_job · realms/aws-databricks/generate_bundle.py:132
no test names it
platform_job
function, line 100
generate · realms/aws-databricks/generate_bundle.py:186test_bundle.py
proof_job
function, line 117
generate · realms/aws-databricks/generate_bundle.py:187test_bundle.py
target
function, line 141
root · realms/aws-databricks/generate_bundle.py:168test_bundle.py, test_obs_dashboards.py and 16 more
root
function, line 150
generate · realms/aws-databricks/generate_bundle.py:182no test names it
dump
function, line 176
generate · realms/aws-databricks/generate_bundle.py:182no test names it
generate
function, line 180
Relative output path -> file text. Pure: reads the compiled JSON, writes nothing.main · realms/aws-databricks/generate_bundle.py:195test_bundle.py, test_semantic_governance.py and 1 more
main
function, line 191
module level · realms/aws-databricks/generate_bundle.py:213no test names it

realms/aws-databricks/integration.py

Integration test on a deployed environment (CI: after deploy dev, before the release gate). none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
proof_jobs
function, line 25
The proof job of every compiled domain, by its key in the generated bundle.main · realms/aws-databricks/integration.py:39no test names it
command
function, line 30
main · realms/aws-databricks/integration.py:41test_cli.py
main
function, line 34
module level · realms/aws-databricks/integration.py:46no test names it

realms/aws-databricks/lakeformation.py

Lake Formation grants from the same access plan as Unity Catalog (ADR-028 rule 4, B12): AWS engines (Redshift, Athena) read the Unity Catalog tables through a Glue federated catalog, and Lake Formation decides what each reader sees. Permiss none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
_q
function, line 29
terraform · realms/aws-databricks/lakeformation.py:67no test names it
_db
function, line 33
intents · realms/aws-databricks/lakeformation.py:48
terraform · realms/aws-databricks/lakeformation.py:91
no test names it
intents
function, line 37
What Lake Formation must hold: one row per (principal, database, table, permission, filter).terraform · realms/aws-databricks/lakeformation.py:63test_lakeformation.py
terraform
function, line 59
The Terraform for the intents, plus one data-cells filter per (group, table) that hides protected columns and one per row-filtered table and limited group.none found in the codetest_lakeformation.py
drift
function, line 98
missing: in the model, not in Lake Formation (a reader is blocked). extra: in Lake Formation, not in the model (a reader sees more than the model allows: the dangerous side).none found in the codetest_lakeformation.py, test_governance_catalog.py and 4 more

realms/aws-databricks/meta_setup.py

Install the metadata store (ADR-010) on a Lakebase instance and load a domain's manifest into it. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
cli
function, line 31
dsn · realms/aws-databricks/meta_setup.py:43
token · realms/aws-databricks/meta_setup.py:50
wake · realms/aws-databricks/meta_setup.py:59
no test names it
say
function, line 38
wake · realms/aws-databricks/meta_setup.py:65
main · realms/aws-databricks/meta_setup.py:80
no test names it
dsn
function, line 42
main · realms/aws-databricks/meta_setup.py:76no test names it
token
function, line 48
main · realms/aws-databricks/meta_setup.py:77test_bundle.py, test_semantic_governance.py and 18 more
wake
function, line 53
Lakebase moved to Autoscaling (projects, branches, endpoints): an instance made on 4 Nov is now project <instance> with endpoint production/primary, and an endpoint can be disabled (seen 13 Nov: connections refused with "The endpoint has bemain · realms/aws-databricks/meta_setup.py:75no test names it
main
function, line 68
module level · realms/aws-databricks/meta_setup.py:94no test names it

realms/azure-databricks/generate_bundle.py

Generate the Databricks Asset Bundle that deploys the platform's engine job to an Azure Databricks workspace. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
repoint
function, line 38
A path in the engine's bundle (relative to realms/databricks/engine), as seen from this bundle.generate · realms/azure-databricks/generate_bundle.py:62no test names it
target
function, line 45
generate · realms/azure-databricks/generate_bundle.py:65test_bundle.py, test_obs_dashboards.py and 16 more
generate
function, line 53
render · realms/azure-databricks/generate_bundle.py:76test_bundle.py, test_semantic_governance.py and 1 more
render
function, line 74
main · realms/azure-databricks/generate_bundle.py:83test_semantic_governance.py, test_obs_dashboards.py and 3 more
main
function, line 79
module level · realms/azure-databricks/generate_bundle.py:96no test names it

realms/azure-databricks/tests/test_realm.py

Offline checks of the Azure Databricks realm: names, layers, secrets, budget, isolation, pinned versions, bundle. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
files
function, line 34
test_no_hard_coded_ids_emails_or_secrets · realms/azure-databricks/tests/test_realm.py:180test_bundle.py, test_semantic_governance.py and 42 more
text
function, line 39
test_every_layer_has_a_container_and_an_external_location · realms/azure-databricks/tests/test_realm.py:131
test_every_catalog_layer_has_a_container_behind_it · realms/azure-databricks/tests/test_realm.py:141
test_unity_catalog_module_has_the_pieces_the_engine_job_needs · realms/azure-databricks/tests/test_realm.py:149
test_personal_data_has_its_own_account_with_a_purge · realms/azure-databricks/tests/test_realm.py:153
test_no_hard_coded_ids_emails_or_secrets · realms/azure-databricks/tests/test_realm.py:183
test_budget_module_has_an_alert · realms/azure-databricks/tests/test_realm.py:203
and 6 more
no test names it
clean
function, line 43
python-hcl2 keeps the quotes of strings and wraps expressions in ${...}; undo both.parse · realms/azure-databricks/tests/test_realm.py:59test_ingest_world.py, test_semantic_metrics.py and 8 more
parse
function, line 57
variables · realms/azure-databricks/tests/test_realm.py:64
example · realms/azure-databricks/tests/test_realm.py:71
backend · realms/azure-databricks/tests/test_realm.py:75
test_ingest_formats.py, test_ingest_steps.py and 19 more
variables
function, line 62
prefix_rule · realms/azure-databricks/tests/test_realm.py:79
test_default_layers_are_the_platforms_layers · realms/azure-databricks/tests/test_realm.py:125
test_every_layer_has_a_container_and_an_external_location · realms/azure-databricks/tests/test_realm.py:130
test_every_catalog_layer_has_a_container_behind_it · realms/azure-databricks/tests/test_realm.py:139
test_personal_data_has_its_own_account_with_a_purge · realms/azure-databricks/tests/test_realm.py:155
test_owner_values_have_no_default_in_tracked_files · realms/azure-databricks/tests/test_realm.py:191
test_bundle.py, test_adf_generator.py and 2 more
example
function, line 70
test_example_names_are_legal · realms/azure-databricks/tests/test_realm.py:105
test_owner_values_have_no_default_in_tracked_files · realms/azure-databricks/tests/test_realm.py:194
test_budget_is_small_and_wired · realms/azure-databricks/tests/test_realm.py:208
test_dev_and_prod_have_their_own_state_and_settings · realms/azure-databricks/tests/test_realm.py:214
test_gold_harness.py, test_ingest_pipelines.py and 33 more
backend
function, line 74
test_example_names_are_legal · realms/azure-databricks/tests/test_realm.py:109
test_dev_and_prod_have_their_own_state_and_settings · realms/azure-databricks/tests/test_realm.py:214
test_backend_is_azure_and_driven_by_a_file · realms/azure-databricks/tests/test_realm.py:264
no test names it
prefix_rule
function, line 78
test_prefix_rule_matches_what_azure_allows · realms/azure-databricks/tests/test_realm.py:95no test names it
test_prefix_rule_matches_what_azure_allows
function, line 93
The rule in variables.tf accepts exactly the prefixes whose every derived name is legal in Azure.none found in the codeno test names it
test_example_names_are_legal
function, line 104
none found in the codeno test names it
test_secret_references_map_to_key_vault_names
function, line 118
none found in the codeno test names it
test_default_layers_are_the_platforms_layers
function, line 124
none found in the codeno test names it
test_every_layer_has_a_container_and_an_external_location
function, line 129
none found in the codeno test names it
test_every_catalog_layer_has_a_container_behind_it
function, line 138
none found in the codeno test names it
test_unity_catalog_module_has_the_pieces_the_engine_job_needs
function, line 148
none found in the codeno test names it
test_personal_data_has_its_own_account_with_a_purge
function, line 152
none found in the codeno test names it
test_no_hard_coded_ids_emails_or_secrets
function, line 178
none found in the codeno test names it
test_owner_values_have_no_default_in_tracked_files
function, line 190
none found in the codeno test names it
test_budget_module_has_an_alert
function, line 202
none found in the codeno test names it
test_budget_is_small_and_wired
function, line 207
none found in the codeno test names it
test_dev_and_prod_have_their_own_state_and_settings
function, line 213
none found in the codeno test names it
test_every_resource_name_carries_the_environment
function, line 232
none found in the codeno test names it
test_unity_catalog_objects_are_isolated_to_their_workspace
function, line 241
none found in the codeno test names it
test_connector_reaches_only_its_own_environments_storage
function, line 245
none found in the codeno test names it
test_provider_is_pinned_exactly_at_a_release_older_than_two_weeks
function, line 256
none found in the codeno test names it
test_backend_is_azure_and_driven_by_a_file
function, line 262
none found in the codeno test names it
test_committed_bundle_is_up_to_date
function, line 268
none found in the codetest_bundle.py
test_bundle_targets_pick_the_environment_and_no_host
function, line 273
none found in the codeno test names it
test_bundle_has_the_engine_job_and_its_files_exist
function, line 279
none found in the codeno test names it
test_bundle_file_has_no_credentials
function, line 291
none found in the codeno test names it

realms/databricks/dashboards/enable_anomaly_detection.py

Turn on the platform's anomaly detection (freshness and completeness, limits learned per table) on every schema of the silver and gold catalogs: the adopted half of "freshness" (obs/native.py `anomaly_detection`). Idempotent: a schema alrea Operations page and dashboards

DefinitionWhat it does (its own first sentence)Called byNamed in tests
cli
function, line 18
main · realms/databricks/dashboards/enable_anomaly_detection.py:32no test names it
main
function, line 26
module level · realms/databricks/dashboards/enable_anomaly_detection.py:49no test names it

realms/databricks/dashboards/generate.py

Write (or check) the ops dashboards, the ops metric views and the adopted native set (ADR-024). Operations page and dashboards

DefinitionWhat it does (its own first sentence)Called byNamed in tests
files
function, line 29
stale_files · realms/databricks/dashboards/generate.py:41
main · realms/databricks/dashboards/generate.py:53
test_bundle.py, test_semantic_governance.py and 42 more
stale_files
function, line 39
Generated files no definition produces any more (a dashboard or view removed).main · realms/databricks/dashboards/generate.py:60no test names it
main
function, line 46
module level · realms/databricks/dashboards/generate.py:72no test names it

realms/databricks/declarative/compare.py

Compare the declarative silver (schema declarative_proof) with the job-built silver, dataset by dataset: current live rows (history) or all rows (append), version counts, and rows that differ on the business columns. Declarative engine

DefinitionWhat it does (its own first sentence)Called byNamed in tests
q
function, line 7
module level · realms/databricks/declarative/compare.py:15no test names it

realms/databricks/declarative/silver.py

The declarative silver engine on Databricks (ADR-035): one generic definition for every dataset whose silver_engine is `declarative` (or the datasets named in dp.only, for proofs). Declarative engine

realms/databricks/engine/tasks/probe.py

What Databricks serverless allows the engine (13 Nov probe). Each check prints one line: ok or FAIL with the reason. Throwaway schema, dropped at the end. Orchestration, end to end

realms/databricks/engine/tasks/semantic_layer.py

After a proof run: create the business metric views (the semantic generator's committed SQL for this environment) and ask each certified known answer of the DEPLOYED views, the way a dashboard tile or a Genie answer would. Orchestration, end to end

DefinitionWhat it does (its own first sentence)Called byNamed in tests
_statements
function, line 18
create_views · realms/databricks/engine/tasks/semantic_layer.py:25
check · realms/databricks/engine/tasks/semantic_layer.py:36
no test names it
create_views
function, line 22
none found in the codeno test names it
_same
function, line 30
check · realms/databricks/engine/tasks/semantic_layer.py:46no test names it
check
function, line 34
One row per certified known answer: the deployed view's number, gold's (Python) and the world's.none found in the codetest_azure_adapters.py, test_contract_rules.py and 30 more

realms/databricks/pipelines/silver.py

Silver, as a Lakeflow Declarative Pipeline: maximum use of what Databricks offers, driven by the metadata DB. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
valid
function, line 30
none found in the codeazure_fakes.py, gold_helpers.py and 14 more
quarantine
function, line 35
none found in the codesilver_helpers.py, test_azure_adapters.py and 10 more

realms/databricks/semantic_governance/deploy_genie.py

Deploy the generated Genie space (B12, SERV-06): create it, or update the space with the same title. Semantic layer

DefinitionWhat it does (its own first sentence)Called byNamed in tests
payload
function, line 23
The create/update request body from the generated file (warehouse id filled in per environment).main · realms/databricks/semantic_governance/deploy_genie.py:51test_genie_deploy.py, test_governance_catalog.py and 6 more
existing
function, line 29
main · realms/databricks/semantic_governance/deploy_genie.py:52test_genie_deploy.py, azure_fakes.py and 8 more
cli
function, line 36
main · realms/databricks/semantic_governance/deploy_genie.py:52no test names it
main
function, line 43
module level · realms/databricks/semantic_governance/deploy_genie.py:66no test names it

realms/databricks/semantic_governance/generate.py

Generate the semantic layer, BI, catalog sync, access model and audit files for one domain's slice (cost belongs to the separate cost tool since 4 Nov 2026; the jobs keep the tag convention, governance.tables.COMPUTE_TAGS). Semantic layer

realms/databricks/semantic_governance/render.py

Databricks rendering of the kernel's semantic, BI, governance and access definitions (product names live here). Semantic layer

DefinitionWhat it does (its own first sentence)Called byNamed in tests
RenderError
class, line 33
ident · realms/databricks/semantic_governance/render.py:43
label · realms/databricks/semantic_governance/render.py:50
principal · realms/databricks/semantic_governance/render.py:60
catalog_action_sql · realms/databricks/semantic_governance/render.py:281
test_semantic_governance.py
ident
function, line 40
table · realms/databricks/semantic_governance/render.py:65
catalog_action_sql · realms/databricks/semantic_governance/render.py:270
_grant_sql · realms/databricks/semantic_governance/render.py:299
mask_function · realms/databricks/semantic_governance/render.py:310
_mask_sql · realms/databricks/semantic_governance/render.py:322
_row_filter_sql · realms/databricks/semantic_governance/render.py:327
and 1 more
test_semantic_governance.py, test_value_objects.py
label
function, line 47
A metric view's dimension or measure name ('OTIF %'), back-quoted._where · realms/databricks/semantic_governance/render.py:131
known_answers_sql · realms/databricks/semantic_governance/render.py:147
_tile_query · realms/databricks/semantic_governance/render.py:167
test_semantic_governance.py, test_semantic_metrics.py
lit
function, line 54
_where · realms/databricks/semantic_governance/render.py:131
known_answers_sql · realms/databricks/semantic_governance/render.py:144
_tags · realms/databricks/semantic_governance/render.py:264
catalog_action_sql · realms/databricks/semantic_governance/render.py:275
_members · realms/databricks/semantic_governance/render.py:306
_mask_sql · realms/databricks/semantic_governance/render.py:315
and 2 more
no test names it
principal
function, line 58
_grant_sql · realms/databricks/semantic_governance/render.py:296test_semantic_governance.py
table
function, line 64
metric_view_sql · realms/databricks/semantic_governance/render.py:117
known_answers_sql · realms/databricks/semantic_governance/render.py:142
_tile_query · realms/databricks/semantic_governance/render.py:165
catalog_action_sql · realms/databricks/semantic_governance/render.py:268
_grant_sql · realms/databricks/semantic_governance/render.py:302
_mask_sql · realms/databricks/semantic_governance/render.py:322
and 1 more
test_lakeformation.py, test_semantic_governance.py and 84 more
scalar
function, line 68
A YAML scalar: JSON strings are valid YAML, so quoting is never guessed.metric_view_yaml · realms/databricks/semantic_governance/render.py:90test_rest_api_connector.py
stable_id
function, line 73
genie_space_json · realms/databricks/semantic_governance/render.py:239no test names it
_dimension_comment
function, line 80
metric_view_yaml · realms/databricks/semantic_governance/render.py:103no test names it
metric_view_yaml
function, line 89
metric_view_sql · realms/databricks/semantic_governance/render.py:121no test names it
metric_view_sql
function, line 116
none found in the codeno test names it
_where
function, line 126
known_answers_sql · realms/databricks/semantic_governance/render.py:147no test names it
known_answers_sql
function, line 137
One row per certified known answer: expected, got from the metric view, and ok (ADR-022, SERV-06).none found in the codeno test names it
_slug
function, line 160
dashboard_json · realms/databricks/semantic_governance/render.py:215no test names it
_tile_query
function, line 164
dashboard_json · realms/databricks/semantic_governance/render.py:216no test names it
_widget
function, line 174
dashboard_json · realms/databricks/semantic_governance/render.py:222no test names it
dashboard_json
function, line 205
An AI/BI dashboard (.lvdash.json): a page per certified view, three tile kinds, every dataset a metric view query (FLOW-01: dashboards read metric views and gold only).none found in the codeno test names it
genie_space_json
function, line 232
The Genie space: the certified views as its only tables, instructions and sample questions from the metric files (SERV-06, DPC-0076). `space` is the API's serialized_space before serialising; the deployer adds the warehouse id of the environone found in the codeno test names it
_tags
function, line 263
catalog_action_sql · realms/databricks/semantic_governance/render.py:278no test names it
catalog_action_sql
function, line 267
catalog_sync_sql · realms/databricks/semantic_governance/render.py:285test_semantic_governance.py
catalog_sync_sql
function, line 284
none found in the codeno test names it
_grant_sql
function, line 295
access_sql · realms/databricks/semantic_governance/render.py:347no test names it
_members
function, line 305
_mask_sql · realms/databricks/semantic_governance/render.py:321
_row_filter_sql · realms/databricks/semantic_governance/render.py:328
no test names it
mask_function
function, line 309
_mask_sql · realms/databricks/semantic_governance/render.py:317no test names it
_mask_sql
function, line 313
access_sql · realms/databricks/semantic_governance/render.py:348no test names it
_row_filter_sql
function, line 326
access_sql · realms/databricks/semantic_governance/render.py:350no test names it
access_sql
function, line 339
none found in the codeno test names it
audit_sql
function, line 359
Reads of tables with protected columns and every grant or tag change (SEC-04, DPC-0078).none found in the codeno test names it

realms/databricks/tasks/checks.py

The day's checks on Databricks. Every result carries the run id, so the evidence view joins them to the run log. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
run_checks
function, line 20
none found in the codeno test names it
detectors
function, line 61
SIM-01: each planted incident is noticed on its day, and nothing fires on a clean day.run_checks · realms/databricks/tasks/checks.py:46test_obs_steps.py
metric_view
function, line 97
SERV-04: the business definition lives in one place, and it agrees with the gold table.run_checks · realms/databricks/tasks/checks.py:57test_semantic_governance.py, test_governance_catalog.py and 2 more

realms/databricks/tasks/main.py

Slice 1 job steps on Databricks. One entry file; the job calls it once per step. none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
args
function, line 19
module level · realms/databricks/tasks/main.py:31no test names it
step_reset
function, line 49
module level · realms/databricks/tasks/main.py:166 (value)no test names it
step_prepare
function, line 57
module level · realms/databricks/tasks/main.py:166 (value)no test names it
probe_platform
function, line 92
DE-057: which system tables this workspace exposes. Cost, audit and lineage depend on them.step_prepare · realms/databricks/tasks/main.py:63no test names it
step_land
function, line 110
module level · realms/databricks/tasks/main.py:166 (value)no test names it
step_finish
function, line 121
module level · realms/databricks/tasks/main.py:166 (value)no test names it
metric_view_sql
function, line 142
step_finish · realms/databricks/tasks/main.py:131no test names it

realms/databricks/v1/tasks/paths.py

Where the V1 jobs meet: the landing volume the feeder writes and ingestion reads, and the folders the proof driver and the platform tasks pass files through. One catalog per environment and layer (ADR-012). none yet

DefinitionWhat it does (its own first sentence)Called byNamed in tests
landing
function, line 5
none found in the codetest_azure_adapters.py, test_cloud_adapters.py and 13 more
hops
function, line 9
none found in the codeno test names it
sources
function, line 13
none found in the codetest_semantic_governance.py, test_end_to_end.py and 20 more
reports
function, line 17
none found in the codetest_azure_adapters.py, test_hop_reports.py and 6 more