cleandataengineering.aiDrug-Safety Agent on Databricks, drawn simply
You are here: Business flow › The weekly screen

Drug-Safety Agent on Databricks · Business flow · The weekly screen

The channel in: the weekly screen

Most of the business value is not in answering one question. It is in a job that runs every week, looks at every pair, and hands a reviewer a short list. The analyst's morning becomes a queue, not a search.

Goes inEvery drug-and-event pair the data holds (80 here)
What happensCode works out which pairs screen as a signal and are not already on the label; each gets an investigation and a grounding check
Comes outDrafts in the reviewer queue, and hand-offs for anything that could not be finished

Who chooses. Plain code chooses which pairs are worth a draft: at least 3 reports, a PRR of at least 2, and the label does not already mention the event. In this data that is 10 of 80 pairs. No model decides what is worth a look.

Safe to run twice. A pair that already has a draft, or a decision, is skipped. A pair is never queued twice, and a decided pair is never redrafted.

A word on these particular results. These are screening results on a small public sample. A pair that screens is a reason to look, not a finding; that is why a person reviews each draft.

A real run

The screen was run against this data with gpt-oss-120b through Groq's free tier. It queued these drafts for a reviewer. The free tier limits tokens per minute, so the job paces itself; a case that hit the limit became a hand-off and was retried.

IdDrugEventReportsPRRDrafted by
ASM-0A9984ibuprofenANGIOEDEMA30305.3weekly-screen
ASM-5B38E0lisinoprilGASTROINTESTINAL HAEMORRHAGE33602.7weekly-screen
ASM-924484ibuprofenGASTROINTESTINAL HAEMORRHAGE27322.4weekly-screen
ASM-05961ElisinoprilLACTIC ACIDOSIS11962.5weekly-screen
ASM-CDFD2BamiodaroneGASTROINTESTINAL HAEMORRHAGE2395.3weekly-screen

The real code

Cut from the repository each time this site is built, so it always matches what runs. Each header names the file and the line it was cut from.

src/drugsafety/screen.py:29Which pairs are worth a draft: plain code
def candidates() -> list[tuple[str, str]]:
    """Pairs that screen as a signal and are not on the label, strongest first. No model involved."""
    out = []
    for drug, event in T._pairs():
        s = T.signal_stats(drug, event)
        if s["screens_as_signal"] and not T.label_mentions(drug, event)["on_label"]:
            out.append((s["prr"], drug, event))
    return [(d, e) for _, d, e in sorted(out, reverse=True)]
src/drugsafety/screen.py:39The weekly job: pick, skip what is known, draft, queue
async def run_screen(
    llm,
    store: AssessmentStore,
    *,
    limit: int | None = None,
    pause_s: float = 0.0,
    reviewer_id: str = "safety-team",
    source: str = "weekly-screen",
) -> ScreenReport:
    rep = ScreenReport(pairs_screened=len(T._pairs()))
    rep.candidates = candidates()[:limit]
    for drug, event in rep.candidates:
        aid = assessment_id(drug, event)
        try:
            store.get(aid)
            rep.already_known.append(aid)  # drafted or decided before: never queue the same pair twice
            continue
        except StoreError:
            pass
        if rep.queued or rep.escalated:
            await asyncio.sleep(pause_s)  # a free model tier limits tokens per minute; pace the batch instead of failing it
        reply = await handle(f"Investigate {drug} and {event}", reviewer_id=reviewer_id, llm=llm, store=store, source=source)
        rep.spend_usd += reply.draft.spend_usd if reply.draft else 0.0
        if reply.assessment_id:
            rep.queued.append(reply.assessment_id)
        elif reply.escalation_id:
            rep.escalated.append(reply.escalation_id)
    return rep