cleandataengineering.aiDrug-Safety Agent on Databricks, drawn simply
You are here: Eval and gates

Drug-Safety Agent on Databricks · Eval and gates

How we know it works

Anyone can show an agent working once. The harder question is how you know it keeps working and what it does when it is pushed. This agent is tested in three layers, from fast unit tests up to a gate that compares it with its specification.

LayerWhat it isToday
Unit testsTable-driven pytest: the router, the statistics, the tools, the loop with a scripted model, the store, the weekly screen and the desk over HTTP128 passing
ScenariosTen scripted conversations run against a simulated world with a model twin10 of 10 pass
Four gatesBehaviour, features, structure and harness, each read against the specificationBehaviour and Structure pass; Features 24 of 33; Harness 2 of 76
Model evaluationTwenty real pairs through a real model; deterministic checks only20 of 20 grounded; a small sample

Why a failing verdict is on show. The gates are only worth anything if they can say no. Showing the current "fail" with its reasons is the point of the exercise, and each reason is a to-do with a name on it.

Look inside