Drug-Safety Agent on Databricks · Eval and gates
How we know it works
Anyone can show an agent working once. The harder question is how you know it keeps working and what it does when it is pushed. This agent is tested in three layers, from fast unit tests up to a gate that compares it with its specification.
| Layer | What it is | Today |
|---|---|---|
| Unit tests | Table-driven pytest: the router, the statistics, the tools, the loop with a scripted model, the store, the weekly screen and the desk over HTTP | 128 passing |
| Scenarios | Ten scripted conversations run against a simulated world with a model twin | 10 of 10 pass |
| Four gates | Behaviour, features, structure and harness, each read against the specification | Behaviour and Structure pass; Features 24 of 33; Harness 2 of 76 |
| Model evaluation | Twenty real pairs through a real model; deterministic checks only | 20 of 20 grounded; a small sample |
Why a failing verdict is on show. The gates are only worth anything if they can say no. Showing the current "fail" with its reasons is the point of the exercise, and each reason is a to-do with a name on it.