Home › Learn › Databricks interview prep › dbt and ODCS

Databricks interview prep · data engineering

dbt and the Open Data Contract Standard

What problems dbt solves and where it stops, how it compares with declarative pipelines, and what the Open Data Contract Standard solves.

Maintenance commandsRow-level securityCompute choiceJoinsSkew and shuffleAQE, Photon, POCost and performance leversDeclarative and semanticAsset BundlesLakeflow ConnectUniFormVector Searchdbt and ODCSEgress and ingressMaintenance in picturesClustering and compactionMaintenance strategySlow writesBackup and recoveryTaggingQuestion bank

Defaults, limits and preview status change often. Check the current Databricks docs before quoting a number.

SELECT files in Gitref() builds thedependency graphTests and docs run withthe buildTables and views appearin the lakehouse

dbt is the T in ELT: tested, documented, version-controlled SQL models.

dbt

What problems it solves

dbt handles the transform step. Data is already loaded; dbt takes SELECT statements, works out their order, runs them on your engine, and adds tests, docs and version control. It does not extract, load or process data itself.

Problem before dbtWhat dbt does
Scattered scripts and procedures nobody can findEvery transformation is a SQL file in a repo, one model per file
Execution order hand-wired in a scheduler{{ ref('orders') }} makes the dependency graph, so order is inferred
Hard-coded table names differ between dev and prodTargets switch schema and database; same code everywhere
No review or history for SQLGit: pull requests, blame, rollback
Quality checked by handBuilt-in tests (unique, not null, accepted values, relationships) plus custom SQL tests
Nobody knows what a table meansDocs next to code and a generated site with a lineage graph
Copy-pasted SQLMacros (Jinja) and packages
Boilerplate to create, insert, merge, dropYou write only the SELECT; a materialization makes it a view, table, incremental model or snapshot
Full reloads of big tablesIncremental models (append, merge, insert overwrite)
Tracking changing dimensionsSnapshots implement SCD type 2
No CI for data codeRun and test only changed models and dependents on a pull request
Metrics defined differently in each dashboardSemantic layer (MetricFlow) defines them once

Does not solve: ingestion, streaming, non-SQL transformation (Python models exist in some adapters), orchestration of everything else, performance tuning, platform governance, or a bad data design.

On Databricks

dbt-databricks adapter

Compare

dbt versus Lakeflow Declarative Pipelines

dbtDeclarative pipelines
LanguageSQL with Jinja (Python models in some cases)SQL and Python
StrengthEngineering workflow for SQL models, tests, docs, portable across warehousesStreaming, incremental processing, expectations, CDC, managed compute
StreamingNot its focusFirst-class
PortabilityMany enginesDatabricks (core upstreamed to Spark)
TestingRich test framework and CI practiceExpectations inside the pipeline
Typical fitAnalytics engineering, gold models, multi-warehouse teamsIngestion-heavy medallion pipelines, streaming, CDC

Many teams use pipelines for ingestion and bronze to silver, and dbt for gold. Others standardize on one. Choose by team skills, streaming needs and portability.

ODCS

Open Data Contract Standard

ODCS is an open, vendor-neutral YAML specification for a data contract: an agreement between the team that produces a dataset and the teams that consume it. It is maintained by the Bitol project under the Linux Foundation and grew out of PayPal's template. Check the current spec version before quoting section names.

Most data breakage is communication failure between producers and consumers, not technology failure.

Without a contractWhat goes wrong
A source renames a column or changes a typeDownstream pipelines and dashboards break; you hear from angry users
A consumer assumes a column is never nullSilent wrong numbers
No owner namedIncidents take days to route
Freshness lives in someone's headA daily table arrives at noon
Column meaning unclear (currency, tax)Different teams compute different answers
Sensitive fields not markedPrivacy mistakes
Each consumer writes its own checksDuplicate, inconsistent rules
Each platform has its own metadata formatContracts cannot move between tools
SectionStates
FundamentalsName, version, status, domain, description, purpose
SchemaTables, columns, types, required, keys, descriptions, classification (such as PII), business terms
Data qualityUniqueness, completeness, ranges, row counts, custom SQL checks, with severity
Service levelsFreshness, latency, availability, retention
Team and rolesOwner, steward, contacts, access
ServersWhere the data lives
Support and pricingChannels, cost
Custom propertiesExtensions
apiVersion: v3.0.0
kind: DataContract
id: orders-contract
name: Orders
version: 1.2.0
status: active
domain: sales
schema:
  - name: orders
    properties:
      - name: order_id
        logicalType: string
        required: true
        unique: true
      - name: customer_email
        logicalType: string
        classification: PII
slaProperties:
  - property: freshness
    value: 24
    unit: h
Contract elementWhere enforced on Databricks
SchemaDelta schema and constraints, Unity Catalog tables
Quality rulesPipeline expectations, dbt tests, generated checks
Ownership, descriptions, tagsUnity Catalog comments and governed tags
AccessGrants, row filters, column masks
Freshness and SLAsJobs monitoring, data quality monitoring, alerts
Change managementCI in Git comparing contract versions

It does not enforce anything by itself; value comes from CI checks and runtime validation. It also does not fix organizational problems. dbt model contracts are narrower: they enforce column names and types at build time. Start with widely used, high-value datasets. See also data contracts.

Practice

Questions to rehearse

  1. What does ref() do and why is it more than a table name?
  2. Table, view, incremental and ephemeral models: when each?
  3. What can go wrong in an incremental model? (Late data, missing unique key causing duplicates, schema changes.)
  4. How do snapshots do SCD2?
  5. How would you run dbt in CI so a pull request builds only what changed?
  6. dbt versus Lakeflow Declarative Pipelines: how do you choose?
  7. What breaking changes would you detect in a data contract automatically?
  8. What happens when a producer violates a contract at 2am?
In an interview, say: dbt gives SQL transformations software-engineering discipline; ODCS gives producers and consumers one written, checkable agreement. Neither does ingestion or enforcement alone.

FAQ

Questions people ask

Does dbt process the data?

No. It sends SQL to your engine, which does the work.

Is ODCS a tool?

No. It is a file format; tools read it to generate and run checks.

Next

Where to go next

All Databricks interview topics · All data engineering · Databricks on this site