Drug-Safety Agent on Databricks · Tools · Statistics in code
Statistics in code, never in the model
Language models are unreliable at arithmetic and confident about it. So the model never computes a ratio. A tool does, and the grounding check later confirms the model copied the tool's number exactly.
Two copies, one answer. The same formulas exist twice: in Python (fast to test, runs anywhere) and in SQL inside the Unity Catalog function (runs next to the data). A parity script compares them for every one of the 80 drug-and-event pairs and reports how many differ. See the two-definitions question.
Undefined is not zero. If a count is zero so that a ratio cannot be calculated, the result is null, not a made-up number. A zero in a safety draft would look like reassurance.
The real code
Cut from the repository each time this site is built, so it always matches what runs. Each header names the file and the line it was cut from.
src/drugsafety/stats.py:32ROR: undefined (None) when it cannot be calculateddef ror(t: Table) -> float | None:
"""Reporting odds ratio: (a*d) / (b*c). None when undefined."""
if t.b == 0 or t.c == 0:
return None
return (t.a * t.d) / (t.b * t.c)src/drugsafety/stats.py:39Its 95% confidence interval, on the log scaledef ror_ci95(t: Table) -> tuple[float, float] | None:
"""95% confidence interval of the ROR (log scale). None when undefined or a == 0 or d == 0."""
if min(t.a, t.b, t.c, t.d) == 0:
return None
r = (t.a * t.d) / (t.b * t.c)
se = math.sqrt(1 / t.a + 1 / t.b + 1 / t.c + 1 / t.d)
return (math.exp(math.log(r) - 1.96 * se), math.exp(math.log(r) + 1.96 * se))src/drugsafety/tools.py:68From the four counts to the tool's resultdef stats_from_counts(drug: str, event: str, a: int, b: int, c: int, d: int) -> dict:
"""PRR, ROR with its 95% interval, and the screen verdict, from a 2x2 table. Arithmetic is code, never the model."""
t = Table(a, b, c, d)
ci = ror_ci95(t)
return {
"drug": drug,
"event": event,
"cases": t.a,
"prr": prr(t),
"ror": ror(t),
"ror_ci95": list(ci) if ci else None,
"screens_as_signal": is_signal(t),
"note": "Screening statistic from spontaneous reports. A reason to look, not proof of causation.",
}