Agent Harness Tutorial · End to end · Zoom level 3
The release gate
The gate is a file (gates.yaml) that lists what must be true before a release. CI runs it on every push. Images are only published when everything is green.
The two release checks in evaluators.yaml, tool_selection (did it pick the right tool?) and must_include (did the reply say what it must?), need a pass rate of 1.0. One miss blocks the release.
Uncertain is not a pass. A result the gate can't decide goes to a person. And the line can only go up: a release may never be worse than the last one.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
gates.yaml:5gates.yaml: what must be true before a releaseapiVersion: gates/v0 agent: motor-claims-fnol aoas: ../clean-ai-engineering/drafts/examples/motor-claims-fnol.aoas.yaml profile: harness-profile.yaml # `agenttwin.Binding`: `(live, *, wrap, clock, model) -> Subject`. binding: evals.gate_binding:open_subject tests: # Each test carries its `discharges` ids as JUnit properties (tests/conftest.py). command: uv run python -m pytest -q -p no:cacheprovider --junitxml={junit} structure: checks: types: uv run mypy imports: uv run lint-imports complexity: uv run ruff check src tests evals # One module per harness layer (AHC taxonomy/layers.yaml). The engine of every # layer is the installed library, agent_harness — named where it lives, in the # reference agent's checkout this project depends on by path — and a layer # whose values are this agent's names its own module beside it. layers: L1: ../reference-agent/packages/agent-harness/src/agent_harness/context L2: ../reference-agent/packages/agent-harness/src/agent_harness/llm L3: ../reference-agent/packages/agent-harness/src/agent_harness/tools L4: - ../reference-agent/packages/agent-harness/src/agent_harness/loop - src/claims_fnol/router L5: ../reference-agent/packages/agent-harness/src/agent_harness/state L6: - ../reference-agent/packages/agent-harness/src/agent_harness/contracts - src/claims_fnol/contracts L7: - ../reference-agent/packages/agent-harness/src/agent_harness/policy
Tech choices (what we chose, Azure and open-source options): L15 · Release & config