Drug-safety signal investigator on Databricks

A pharmacovigilance agent built only on Databricks' own AI services, from public FDA data.

Starting · join earlyAI Engineering

4 of 11 tasks done

1 · Purpose

Drug-safety teams ask a recurring question: is there a signal that this drug causes this event? Answering it means counting adverse-event reports, comparing against other drugs, checking whether the label already warns about it, and judging whether the pattern is real or a reporting artefact. The path is not known in advance, which is what makes it a fit for an agent. This project builds that investigator end to end using Databricks' own AI offerings: Unity Catalog, Foundation Model APIs, Unity Catalog functions as tools, Mosaic AI Agent Framework and MLflow. It is also the worked example for the Databricks AI row of the platform page. A safety scientist approves every assessment; the agent files nothing. Data is public openFDA reports and drug labels.

For: Databricks and pharma data teams, architects choosing a lakehouse AI stack, people preparing for AI architect roles.

2 · Business benefits

Numbers by code, judgement by modelReport counts and disproportionality statistics are computed in code, never by the model.
Governed by designData, tools and permissions sit in Unity Catalog, so the agent sees only what it is allowed to.
Every step tracedMLflow records each tool call, so an assessment can be audited later.
Honest about the platformWhere a Databricks service is not available on a free workspace, the guide says so and shows the workaround.
Reusable patternThe same shape fits other safety, quality and compliance investigations.

3 · Status and how to join

Starting · join early. Early days: the shape is still being decided, so joiners now have the most say. Lead: Basant Choudhary.

Good first tasks this week:

  • Pick five drugs with a known label change for the test set
  • Review the signal logic in the plan

The code is not public yet; joiners get access.

4 · Requirements

What it must do

How well it must do it

5 · Architecture, very high level

Public safety dataFDA adverse-event reports anddrug labels.Governed tablesCleaned reports, counts andlabel text, with permissions.Statistics toolsCounts, disproportionality andduplicate checks.Label finderFinds the label passage about agiven event.AgentPlans the investigation anddrafts the assessment.Model serviceThe reasoning and embeddingmodels.Tracing and evaluationRecords each step and scoresresults against known signals.Reviewer deskWhere a safety scientistapproves or rejects.ingesttablesmodel call
Components and what each is responsible for, with no tool names: the tools come in the next section.

Also: Governed tables → Label finder: label text · Agent → Statistics tools: tool call · Agent → Label finder: tool call · Agent → Reviewer desk: draft assessment · Agent → Tracing and evaluation: trace

6 · Tech stack, per platform

ComponentDatabricksOpen stack
Public safety dataopenFDA API, reachable ✓—
Governed tablesDelta tables in Unity Catalog—
Statistics toolsUnity Catalog functions—
Label finderembeddings in Delta plus a function (Vector Search is not available on the free workspace)—
AgentMosaic AI Agent Frameworknot built
Model serviceFoundation Model APIs, gpt-oss-120b and bge-large available ✓—
Tracing and evaluationMLflow tracing and evaluation—
Reviewer deskDatabricks Apps, or a job that writes a review file—

Bold with ✓ = built and running today. Everything else is the planned or possible binding: the components stay the same, only the tools change.

7 · What needs to be done

Phase 0: check 1 of 3 done

TaskStatusSizeSkill
Check what Databricks AI services the free workspace allows (models, tables and Genie work; Vector Search, serving, warehouse start and model queries were refused on the day, cause unconfirmed)doingSDatabricks
Plan, mapping each need to a Databricks servicedoneSArchitecture
Design against the agent harness, assurance and evaluation standards (Databricks stack file added; profile written)doingMArchitecture

Phase 1: data 1 of 2 done

TaskStatusSizeSkill
Ingest 10 drugs and 8 events: 21 million reports and labels, as local filesdoneMData engineering
Load into Delta tables in Unity Catalog with data-quality checkstodoMDatabricks

Phase 2: tools and agent 2 of 4 done

TaskStatusSizeSkill
Tools as plain Python with tests: counts, PRR and ROR, label checkdoneMSQL, Python
Register the tools as Unity Catalog functionstodoMSQL, Python
Agent loop with output contract, grounding check and 48 tests; run on a stand-in modeldoneLAgents
Package with Agent Framework and trace in MLflowtodoLAgents

Phase 3: evaluation and guide 0 of 2 done

TaskStatusSizeSkill
Score on 10 known signals and 10 controlsdoingMAI testing
Write the guide, with the free-workspace limits statedtodoMTechnical writing

Size: S = an evening or two, M = about a week part-time, L = a few weeks.

8 · Roles

RoleHow manyHours / weekWhat they do
Databricks engineer13-5 hours/weekOwns the tables, functions and MLflow set-up.
Agent engineer13-5 hours/weekBuilds the agent and its evaluation.
Drug-safety reviewer11-2 hours/weekChecks the signal logic and the assessment wording.

9 · What you'll learn

Level-up questions

10 · Links