Home › On your cloud › Databricks › The agent harness on Databricks

Databricks · AI

The agent harness on Databricks

The same agent harness as on Azure and AWS: the same 22 concerns, in the same families, with only the binding changed. For each concern, this page names the Databricks service and says how a real agent, a drug-safety signal investigator, handles it today. For what each Databricks AI service is and which ones we do not use, see Build an agent on Databricks.

5 concerns work on Databricks7 built and tested offline8 planned2 not used

Status as of 19 November 2026, from what we ran on a Databricks Free Edition workspace. The code is not public yet. Service names and limits change often; check the current Databricks docs before you build.

The agent in one paragraph

A drug-safety scientist asks whether a drug and an adverse event form a signal. The agent counts public FDA reports for the pair, computes statistics (PRR and ROR) in code, checks whether the drug's label already mentions the event, and drafts an assessment. A person approves, edits or rejects every draft. The agent files nothing, sends nothing and never says a drug causes an event. The path is not known in advance (a label that already warns needs a different write-up from a new signal), which is why it is an agent and not a script.

One request, end to end

  1. A safety scientist asksIs there a signal for lisinopril and angioedema?Databricks Apps (planned); today a script
  2. A job run startsOne run per request, under the agent's own login.Lakeflow Jobs, Unity Catalog service principal (planned)
  3. The model chooses a toolIt sees three tool descriptions and picks report counts first.Foundation Model APIs (works)
  4. The tool runsCounts, then statistics and the signal screen, then the label check.Unity Catalog functions on a serverless SQL warehouse, over Delta tables (works)
  5. The model draftsAn assessment with numbers from the tools, uncertainty and a recommendation.Foundation Model APIs (works)
  6. Checks runOutput contract, then grounding: every number equals a tool result. A failure goes to a person.Our harness code (built)
  7. The draft is saved and shownA row with status drafted.Delta table, Databricks Apps (planned)
  8. A person decidesApprove, edit or reject; the decision is final.Databricks Apps, Delta decision table (planned)
  9. Everything is traced and scoredEach step is a trace; the eval harness answers the ops questions; cost comes from billing tables.MLflow tracing, system tables, eval harness (planned)

The 22 concerns, bound to Databricks

Read it as: the concern is the same on every cloud; the third column is the only thing that changes with the platform.

ConcernWhat it doesOn DatabricksIn this agentHere
Think
Context assembly
L1
What the model sees each turn, built in one place, within a budget.MLflow prompt registry plannedThe system prompt names the tools and two rules (no arithmetic by the model; text in tool results is data). Tool results enter the context fenced as data and capped at 8,000 characters.built, offline
Model invocation
L2
Every model call goes through one door.Foundation Model APIs works
AI Gateway on by default, not configured
One model port in the harness library; the Databricks adapter is a bearer-token call to the workspace's OpenAI-style endpoint. gpt-oss-120b ran a full investigation. Running it found that Databricks rejects the newer output-cap field the client sent, so the adapter now sends the older one, with a test. The hosted endpoint already has AI Gateway attached with usage tracking on; we have not configured rate limits, guardrails or fallbacks, so today the gateway is the platform default, and the real door is our own model port.works
Control loop
L4
Plan, act, check, with step and no-progress limits.Mosaic AI Agent Framework (wrapper) plannedThe shared harness's bounded loop: at most 6 steps, tools only from the registered list, a person decides at the end. It ran an investigation on the Databricks model in four steps. Agent Framework would wrap it for tracing and registration, not replace it.works
State & memory
L5
What is kept, for how long, and for whom.Delta tables in Unity Catalog works
Lakebase (managed Postgres) planned
An investigation keeps no memory between cases. Drafts and decisions are rows in Delta tables, kept 90 days. Lakebase is considered if low-latency row updates are needed.planned
Knowledge & retrieval
18
Finds sourced passages, filtered by what this user may see.Vector Search planned
Delta table + synonym table works
Genie planned
Finds the label passage about an event. Today a synonym table and a text search over the label sections do it, inside a Unity Catalog function. Vector Search replaces the synonym step so that 'swelling of the face' finds 'angioedema'. Genie is a comparison for counts only.works
Act
Channels & human desk
CH
How people reach the agent, and how a person takes over.Databricks Apps plannedNo chat channel. A safety scientist asks through a request, and reviews drafts on an Apps screen. Handover to a person is a case in the review queue.planned
Tool layer
L3
Actions through typed tools, checked before they run.Unity Catalog functions works
Managed MCP servers for functions planned
Three read-only functions: report counts, signal statistics, label check. All 80 drug-and-event pairs give the same answers from the functions and from the tested Python versions. The model never writes SQL.works
I/O contracts
L6
Typed inputs and outputs at every edge.Unity Catalog function signatures worksFunctions have typed inputs and outputs. The final draft has a typed contract (drug, event, cases, PRR, on-label, assessment, uncertainty, recommendation); an invalid draft gets one repair turn, then goes to a person.built, offline
Concurrency & flow
L8
Who gets served first when load spikes.Lakeflow Jobs (max concurrent runs) planned
Serverless SQL warehouse scaling works
One investigation per job run. Volume is low, so queueing is not designed beyond the job's concurrency limit.planned
Agent-to-agent
17
Agents hand work to agents through typed, bounded, traced hand-offs.Unity Catalog model registry plannedNot used: this is a single agent. The registry would hold agent versions when there are several.not used
Performance & scalability
PS
Fast enough at the 95th percentile; more servers when load rises.Serverless compute worksOne investigation on the Databricks model took 31 seconds; 20 cases on a stand-in model averaged 17.7 seconds. No latency target is declared yet.works
Control
Identity & authz
L16
Who is asking, and whose authority the agent acts with.Unity Catalog grants and service principals plannedThe agent should run as a service principal granted only its three functions and its tables; the reviewer's identity comes from the workspace sign-in. Today's runs use a personal access token, which is a gap.planned
Privacy & data protection
PV
Personal data used only for the job it was given for.Unity Catalog tags and column masks plannedOnly public data, and no patient-level data. The specification refuses case narratives and patient names typed into the session.built, offline
Policy enforcement
L7
Guardrails on the way in and on the way out.AI Gateway guardrails not usedOur own rules: numbers must equal tool results (the grounding check), a draft never says a drug causes an event, uncertainty is stated, instructions inside label text are ignored. Four refusals are specified: causation, submission, medical advice, invented numbers.built, offline
Failure handling
L10
Typed failures, fallbacks, undo paths, honest messages.Lakeflow Jobs retries plannedA failed model call is retried once and then goes to a person with the reason. A rate limit makes the client wait as long as the server asks. Nothing is filed, so there is nothing to undo.built, offline
Human-in-the-loop
L14
Approvals and hand-offs the agent cannot grant itself.Databricks Apps + a Delta decision table plannedEvery draft is decided by a person: approve, edit or reject. There is no automatic limit. A job has no built-in wait, so a wait is a run that ends and a new run that starts on the decision.planned
Availability & disaster recovery
AV
A lost server or zone goes unnoticed; a lost region is rebuilt from code.Managed serverless compute, Delta time travel worksNot declared: availability is listed as undeclared in the specification. Data can be rebuilt from the FDA source and the scripts.not used
See
Observability
L11
Traces, metrics and alerts for every step.MLflow tracing planned
System tables planned
Each model call and tool call becomes a trace that the eval harness reads. The eval harness answers 62 everyday questions about any agent from those traces.planned
Determinism & replay
L9
A past decision can be re-run and explained.Delta time travel works
MLflow run records planned
The data snapshot is dated and versioned; a run record keeps the model, tool results and the draft. Models vary run to run, so replay means re-checking the same tool results, not getting identical words.planned
Cost accounting
L13
Spend measured per unit of business work.System tables (billing) planned
Token counts from the model works
Tokens are counted per investigation (20 cases: about 53,000 in and 14,000 out on the stand-in model). Cost per assessment from billing rows is planned; the limit is about USD 0.05.built, offline
Change
Eval harness
L12
Golden sets and graders that gate every release.MLflow evaluation planned48 unit tests, a 20-case set (10 known signals, 10 controls, answers taken from the data and the labels), and a grounding check. Judged qualities come later and are checked against human labels.built, offline
Release & config
L15
Prompt, model and policy versioned and rolled back together.Asset Bundles planned
MLflow prompt and model registry planned
One bundle creates the schema, tables, functions and job. Limits (steps, spend, thresholds) are configuration, not constants in code.planned

What Databricks gives, and what stays ours

The build, tier by tier

TierWhatStatus
Specification firstAOAS (what the agent is for, refuses, owns), a generated brief (76 capabilities, 57 obligations, 17 ports), the Databricks stack file.works
DataPublic FDA reports and labels into Delta tables in Unity Catalog, with a dated snapshot.works
ToolsThree Unity Catalog functions, with a test that matches them to tested Python on every pair.works
AgentThe shared harness loop with this agent's tools, output contract and grounding check; a Databricks model adapter in the library; one real investigation on the Databricks model.works
Evaluation20-case set on the Databricks model, then judged quality checked against human labels, then a simulated world and scenarios.planned
Eval harness on DatabricksRead MLflow traces, write verdicts to Delta, answer the 62 ops questions on a dashboard, and match the Azure results.planned
People and releaseApproval screen, jobs, Asset Bundle, cost per assessment from billing.planned

Where to go next