Drug-Safety Agent on Databricks · Eval and gates · Four gates
The four gates
Four gates read this agent against what its specification says must be true. Each says pass or fail, and each failure names the place to fix it. The verdict below is the latest run.
| Gate | Verdict | In plain words |
|---|---|---|
| Behaviour | pass: 10 of 10 | All ten scenarios pass, including the one that needed the store. |
| Features | fail: 24 of 33 | 128 tests; 24 of the 33 promises in the specification have a passing test (17 before the desk was built). The rest include the label-is-data and no-causation properties, the model list and retention. |
| Structure | pass: 16 of 16 | Every layer maps to a module; types, import rules and complexity limits pass. |
| Harness | fail: 2 of 76 | 76 harness capabilities are owed. Two have a passing test. None has been waved through as an accepted gap. |
The tempting shortcut we did not take. The harness gate could be turned green by marking everything an accepted gap. That would make the page nicer and the agent no safer. A gap may be accepted only with a reason, an owner and a review date.
Where the verdict comes from. The report is generated by the gates tool and is never edited by hand.
The real code
Cut from the repository each time this site is built, so it always matches what runs. Each header names the file and the line it was cut from.
gates.yaml:13How the gates run this agent's tests and structure checkstests: # Each test carries its `discharges` ids as JUnit properties (tests/conftest.py). command: uv run --extra dev python -m pytest -q -p no:cacheprovider --junitxml={junit} structure: checks: types: uv run --extra dev mypy src imports: uv run --extra dev lint-imports complexity: uv run --extra dev ruff check src tests evals # One module per harness layer (AHC taxonomy/layers.yaml). The engine of every layer is the installed library, # agent_harness, named where it lives; a layer whose values are this agent's names its own module beside it. # A layer with no module of ours and nothing of ours to put there is left to the library alone. layers: L1: ../reference-agent/packages/agent-harness/src/agent_harness/context