cleandataengineering.aiDrug-Safety Agent on Databricks, drawn simply
You are here: Tools › Statistics in code

Drug-Safety Agent on Databricks · Tools · Statistics in code

Statistics in code, never in the model

Language models are unreliable at arithmetic and confident about it. So the model never computes a ratio. A tool does, and the grounding check later confirms the model copied the tool's number exactly.

Two copies, one answer. The same formulas exist twice: in Python (fast to test, runs anywhere) and in SQL inside the Unity Catalog function (runs next to the data). A parity script compares them for every one of the 80 drug-and-event pairs and reports how many differ. See the two-definitions question.

Undefined is not zero. If a count is zero so that a ratio cannot be calculated, the result is null, not a made-up number. A zero in a safety draft would look like reassurance.

The real code

Cut from the repository each time this site is built, so it always matches what runs. Each header names the file and the line it was cut from.

src/drugsafety/stats.py:32ROR: undefined (None) when it cannot be calculated
def ror(t: Table) -> float | None:
    """Reporting odds ratio: (a*d) / (b*c). None when undefined."""
    if t.b == 0 or t.c == 0:
        return None
    return (t.a * t.d) / (t.b * t.c)
src/drugsafety/stats.py:39Its 95% confidence interval, on the log scale
def ror_ci95(t: Table) -> tuple[float, float] | None:
    """95% confidence interval of the ROR (log scale). None when undefined or a == 0 or d == 0."""
    if min(t.a, t.b, t.c, t.d) == 0:
        return None
    r = (t.a * t.d) / (t.b * t.c)
    se = math.sqrt(1 / t.a + 1 / t.b + 1 / t.c + 1 / t.d)
    return (math.exp(math.log(r) - 1.96 * se), math.exp(math.log(r) + 1.96 * se))
src/drugsafety/tools.py:68From the four counts to the tool's result
def stats_from_counts(drug: str, event: str, a: int, b: int, c: int, d: int) -> dict:
    """PRR, ROR with its 95% interval, and the screen verdict, from a 2x2 table. Arithmetic is code, never the model."""
    t = Table(a, b, c, d)
    ci = ror_ci95(t)
    return {
        "drug": drug,
        "event": event,
        "cases": t.a,
        "prr": prr(t),
        "ror": ror(t),
        "ror_ci95": list(ci) if ci else None,
        "screens_as_signal": is_signal(t),
        "note": "Screening statistic from spontaneous reports. A reason to look, not proof of causation.",
    }