Clinical-trial data review agent
An agent that finds impossible or inconsistent patient data in a clinical trial and drafts the query sent to the site.
1 · Purpose
In a clinical trial, a data manager reads patient records looking for things that cannot be true: a visit before consent, a lab value out of range, a serious adverse event with no matching entry. Fixed rules catch many of these; the rest need someone to read across one patient's whole record and use judgement. This agent does that reading and drafts the question to send to the trial site. A person approves every query; the agent never sends one itself. It uses the public CDISC pilot study data, so no real patient or employer data is involved.
2 · Business benefits
3 · Status and how to join
Good first tasks this week:
- Review a sample of draft queries and say which you would send
- Propose one more planted problem and its expected finding
4 · Requirements
What it must do
- Reads one patient's visits, labs, adverse events and medications through typed read-only tools.
- Runs deterministic edit checks first and keeps their findings separate from the agent's.
- Drafts a plain-language query for each problem worth raising, with the evidence attached.
- Waits for a data manager to approve, edit or reject each draft.
- Scores itself against a planted trial of 24 known problems and 16 clean patients.
How well it must do it
- The model never writes SQL; it calls named tools over certified tables.
- Text inside patient records is treated as data, never as instructions.
- Every run is recorded so a past review can be replayed.
- Cost per patient is measured and reported with the results.
- No real patient data and no employer system or data is used.
5 · Architecture, very high level
6 · Tech stack, per platform
| Component | Open stack | Databricks | Azure |
|---|---|---|---|
| Trial system | OpenClinica Community 3.x, public CDISC pilot data (set up) | — | — |
| Data layer | SDTM-aligned tables, local files ✓ | Delta tables in Unity Catalog (designed, not built) | — |
| Edit checks | Python rules, 202 of 202 checks pass ✓ | — | — |
| Agent | Claude tool-use loop, in-process tools ✓ | tools as Unity Catalog functions (designed, not built) | not built |
| Model | Claude ✓ | — | — |
| Reviewer desk | files for the data manager to mark ✓ | — | — |
| Evaluation | planted trial, scorer, batch runner ✓ | MLflow evaluation (designed, not built) | — |
7 · What needs to be done
Version 1 5 of 5 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Planted trial: 24 problems of 12 kinds, 16 clean patients | done | M | Clinical data |
| Deterministic edit checks (202 of 202 pass) | done | M | Python |
| Agent loop with read tools, scored: recall 21 of 24 (87.5%), $0.21 per patient | done | M | Agents |
| Injection test: hidden instruction resisted 2 of 2 | done | S | Security |
| First reviewer labels: 5 of 5 drafts worth sending | done | S | Clinical data review |
Version 2 0 of 3 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Visit-window and age rules | todo | M | Clinical data |
| Study notes for study-wide blanks | todo | M | Agents |
| The worth-sending gate | todo | M | AI testing |
On a lakehouse 0 of 2 done
| Task | Status | Size | Skill |
|---|---|---|---|
| Ingest the pilot data into Delta tables with clinical-standard layers | todo | L | Data engineering |
| Write approved queries back to the trial system as discrepancy notes | todo | M | Integration |
8 · Roles
| Role | How many | Hours / week | What they do |
|---|---|---|---|
| Clinical data reviewer | 1 | 1-2 hours/week | Labels drafts, says which queries are worth sending. |
| Data engineer | 1 | 3-5 hours/week | Owns the data layer and the edit checks. |
| Agent engineer | 1 | 3-5 hours/week | Owns the agent loop, tools and evaluation. |
9 · What you'll learn
- Where an agent fits in clinical data management, and where plain rules are better.
- How to give a model typed tools over certified tables instead of raw SQL.
- How to build a planted-problem test set and measure recall and cost honestly.
- How to test for instructions hidden inside data.
Level-up questions
- Which of the 24 planted problems should a rule catch instead of the agent?
- What would you change so a reviewer's rejection teaches the agent?
- What breaks first when this runs on a real study with 100 times the patients?