Drug-Safety Agent on Databricks · Checks
Why you can trust a number
A model can write fluent, confident text that is wrong. So nothing the model says reaches a person until deterministic code has looked at it. There are four places a check sits, in the order a request meets them.
| Where | Check | Fails how | Page |
|---|---|---|---|
| Before the model | Refusal and hand-off rules | A fixed reply; the model is never called | what it will not do |
| Around the loop | Step, cost and size limits | The loop stops; a person is told | The limits |
| On the draft's shape | Output contract: required fields, right types, one repair turn | One repair turn; then a hand-off | Output contract |
| On the draft's facts | Grounding: every number equals a tool result | Not shown as a finished draft; a hand-off with the mismatches | Grounding |
| On the data the model reads | Label text is data, never instructions | Standing instruction plus a planted-text scenario | Label text is data |
Why deterministic. A check that is itself a model is one more thing to trust. These are a few lines of code each. They give the same verdict every time, and each can be tested with a table of examples.
Look inside
The shape of the draftA draft must have the right fields in the right types, or it is sent back once.Grounding: every number equals a tool resultThe most important check: the draft's numbers and label verdict must match what the tools returned.Label text is data, not instructionsA drug label could contain the words 'ignore your rules'. The agent must treat them as text.