Agent Harness Tutorial · Tech stack
L4 · Control loop
Plan, act, check, with step and no-progress limits. Family: Think · one turn, in order.
Why our choiceAgents that touch money need every step testable and checks at exact positions. A hand-written loop has the lowest lock-in and works with any model.
| Option | Where from | Verdict | Why |
|---|---|---|---|
| Hand-written loop | ours | ✓ chosen | about 400 lines, agent_harness |
| Budgets | ours | ✓ chosen | 12 steps, 60 s, $0.50 |
| Foundry Agent Service | Azure | ✗ dropped | the platform would own the loop |
| Microsoft Agent Framework | Azure | ✗ dropped | for .NET / Azure-first teams |
| Pydantic AI Agent | open source | ✗ dropped | OK for simple A1–A3 agents |
| LangGraph | open source | ✗ dropped | workflow-shaped agents only |
| Claude Agent SDK | open source | ✗ dropped | Claude-only, loop hidden |
| Anthropic tool runner | open source | · considered | small Claude-only agents |
How it works inside the harness: Core loop.
Compared: Who owns the control loop?
| Option | From | Verdict | Who owns the loop | Any model | Can tests reach every step | Lock-in | Observability | Best for |
|---|---|---|---|---|---|---|---|---|
| Plain Python (hand-written) | ours | chosen | us | yes | yes | none | our OTel spans | assured, transactional agents |
| Pydantic AI Agent | open source | model layer; A1–A3 | framework; we can step it | yes | yes if we step the loop | low: open source, any cloud | OTel built in | typed, testable Python agents |
| LangGraph | open source | workflow-shaped only | framework runs our graph | yes | yes, node by node | medium: graph model, LangSmith | full only with LangSmith | workflow agents with approvals |
| Claude Agent SDK | vendor | internal agents only | the SDK | Claude only | partly: the loop is hidden | high: Claude-only | partial traces | coding, ops, research agents |
| Anthropic tool runner (thin helper) | vendor | small Claude agents | us, with hooks | Claude only | yes: the steps are ours | medium: Claude-only | our OTel spans | small Claude-only agents |
| Microsoft Agent Framework | open source | .NET teams only | framework | yes | trajectory checks re-proved | medium: framework model | OTel, Azure-native | .NET or Azure-first teams |
| Foundry Agent Service (prompt agent) | Azure | rejected | Azure | Foundry models | no: loop and retries hidden | high: Azure-only service | Foundry tracing | low-code agents, not assured ones |
DecisionWe write the loop ourselves and use Pydantic AI only as the model layer, so budgets and checks sit at exact, testable positions. We would use LangGraph for workflow-shaped agents, Agent Framework for a .NET team, and full Pydantic AI for simple A1–A3 agents.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
for step in range(budgets.max_steps):
ended = await this.step(step)
if ended is not None:
return ended
return stopped(run_span, trace, TerminationReason.STEP_BUDGET_EXHAUSTED, PASS_ON)
@dataclass
class _Run:
"""One run's state, and the phases of a step as methods: each readable on its
own, the state they share named once here rather than threaded through locals."""
span: Span
trace: Trace
identity: Identity
llm: LLMClient
tools: ToolClient
registry: ToolRegistry
local_tools: dict[str, LocalTool]
run_id: RunId
system_prompt: str
budgets: Budgets
meter: Meter | None
oscillation_threshold: int
now: Callable[[], int]
"""The run's clock, injected like every other one here: a module that read
the wall clock itself would be untestable about time, which is the whole
subject of `fresh` (AHC L9, F-021)."""
fresh: freshness.Freshness
fan_out: int