Agent Harness Tutorial · Cost
Budgets, layer by layer
"Budget" means six different things in this harness, one for each size of thing that can run away: a single model call, a trip round the loop, a whole turn, a conversation, a minute of traffic, and a month of spending. Each is enforced in a different place, and each does something different when it is hit.
Why so many? Each runaway is only visible at one size. One answer that rambles is caught per call. A loop that asks for 30 tools is caught per trip. Fourteen calls that are really one stuck task are visible only per turn: that is why the step and cost limits live in the loop, not in the model client. A conversation going round in circles shows only across turns. A bill only shows across a month.
Three kinds of reaction. Some budgets trim and carry on (context, tool results, saved history). Some stop the turn with a named reason and an honest reply, usually "let me pass you to a colleague". The outermost ones only tell someone: the Azure budget never switches anything off.
| Budget | Scope | Limit | Enforced by | When it is hit | Kind |
|---|---|---|---|---|---|
| Output tokens | per call | 4,096 | sent to the model as max_tokens | the answer is cut; a cut answer is never sent (output_length_reached) | stops |
| Model call time | per call | 60 s | the HTTP client | Unavailable → wait and retry | retries |
| Tool result size | per call | 8,000 chars | GatedTools._bound() | the result is cut, with a "[truncated]" mark | trims |
| Context | per trip | 24,000 chars | assembled() | whole exchanges dropped from the middle | trims |
| Tool calls | per trip / turn | 8 / 24 | over_fan_out() | tool_call_budget_exhausted | stops |
| Trips | per turn | 12 | the loop | step_budget_exhausted | stops |
| Time | per turn | 60 s | _out_of_time() | deadline_reached | stops |
| Cost | per turn | $0.50 | the Meter | cost_ceiling_reached (checked only when the model wants more tools) | stops |
| Retries | per turn | 4 | unit_retries() | model_unavailable | stops |
| Same call again | per turn | 3rd time | the call's signature | oscillation_detected | stops |
| Turns | per conversation | 10 | Tier-2 rule turns-exceeded | handed to a person | hands off |
| Hand-offs | per conversation | 2 | escalations_raised | no third hand-off | stops |
| Saved history | per conversation | 32,000 chars | bounded() when saving | oldest text trimmed | trims |
| Gateway calls | per minute | 30 | APIM rate-limit | 429 → wait as told, retry | retries |
| Groq tokens | per minute | 8,000 (free tier) | Groq | 429 → wait, retry; in practice 3–5 loop turns a minute | retries |
| Content Safety | per month | 5,000 records (F0) | Azure | fails open: the verdict is "unavailable" and the call passes | lets through |
| Azure spend | per month | $10 | infra/modules/budget.bicep | an email to the owner; nothing stops | tells |
Two more levels are designed but not configured yet: a team's monthly budget in Cost Management with an alert at 80%, and a token limit per agent identity and customer tier at APIM (not available on the Consumption tier, F-39). The ceiling is checked between calls, because you can't un-spend a call: one call can overshoot, bounded by its 4,096 output tokens. At today's prices twelve trips cost about $0.05, so the $0.50 ceiling has never been reached.
Every per-turn limit sits in one frozen object, Budgets, and can be changed per environment without code (for example CLAIMS_MAX_STEPS). Each "stops" reason is written on the trace, so we can count how often each budget fires; see when the loop stops.
Want every detail? The long version is in the Series: budgets and limits.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
agent_harness/config/__init__.py:13Every per-turn budget, in one frozen objectclass Budgets(BaseModel):
"""Ceilings that can actually stop a call.
Two positions, deliberately. `max_output_tokens` is a per-call backstop at
P3; `max_steps` and `max_cost_usd` are per-task and enforced at P4, because
only the loop can see that fourteen calls are one runaway task rather than
fourteen tasks. AAC-0093 and AAC-0008 are different obligations for exactly
this reason.
"""
model_config = ConfigDict(frozen=True)
max_steps: int = 12
max_cost_usd: float = 0.50
max_output_tokens: int = 4096
max_tool_result_chars: int = 8000
"""Tool results are bounded before they enter context — AAC-0105."""
max_turn_seconds: int = 60
"""Wall clock for one turn (AHC-0096), checked before each step."""
max_tool_calls_per_step: int = 8
"""How many tool calls one step may plan (AHC-0097). A step budget counts
steps, so without this one step asking for fifty look-ups — or fifty
writes — spends one of the twelve and does fifty things."""
max_tool_calls_per_turn: int = 24
"""How many tool calls one turn may plan across all its steps (AHC-0097).
Reaching either bound stops the turn before the step's calls run, as
`tool_call_budget_exhausted` — never a silent cut to the first N (AHC-0059)."""
max_retries_per_unit: int = 4
"""AHC-0024: retries across the whole turn, every call's together (F-087)."""agent_harness/resilience/__init__.py:88The turn's shared retry allowance@contextmanager
def unit_retries(limit: int) -> Iterator[None]:
"""One unit of work's retry allowance, shared by every call inside it.
AHC-0024 and F-087: "a retry count exists per unit of work, not per call
site". Each call kept its own few attempts, so a twelve-step turn could
retry twenty-four times. Inside this scope every retry spends from one
budget; when it is gone a failure is raised instead of retried. Outside any
scope only the per-call bound applies.
"""
token = _UNIT.set([limit])
try:
yield
finally:
_UNIT.reset(token)agent_harness/loop/plan.py:82Tool calls per trip and per turndef over_fan_out(asked: int, so_far: int, budgets: Budgets) -> str | None:
"""`per_step` or `per_turn`: the bound a plan of `asked` calls would pass, after
`so_far` calls already planned this turn; `None` when it passes neither."""
if asked > budgets.max_tool_calls_per_step:
return "per_step"
if so_far + asked > budgets.max_tool_calls_per_turn:
return "per_turn"
return Noneinfra/policies/groq-api.xml:34Per minute, at the gateway <!-- Request rate per subscription key (the agent's): 30 calls a minute. -->
<rate-limit calls="30" renewal-period="60" remaining-calls-header-name="x-ratelimit-remaining" />infra/modules/budget.bicep:1Per month: the Azure budget// WHAT IT IS
// A cost budget on the resource group rg-claims-fnol-dev: $10 a month, with an
// email to the owner when actual spend reaches 50% and 80%, and when the month's
// forecast reaches 100%. A budget never stops anything; it only tells you.
//
// WHICH CONCERN IT SERVES
// Cost (architecture deck, slide 53; PLAN.html Tier 3, "a $10 budget alert
// before anything else"). The 12-month free offer is used up, so PostgreSQL is
// the one real cost; this is the tripwire if something else starts billing.
//
// EXPECTED DEV COST
// $0. Budgets are free (Cost Management).
//
// ORDER
// main.bicep makes every other module depend on this one, so ARM creates the
// budget first. Cost data reaches Cost Management with a delay of hours, so an
// alert is never instant.
targetScope = 'resourceGroup'
@description('Budget name.')
param name string = 'budget-claims-fnol-dev'
@description('Monthly amount in the billing currency (USD for this subscription).')
param amount int = 10
@description('Who gets the alert emails.')
param contactEmail string
@description('First day of the budget period, yyyy-MM-01T00:00:00Z (main.bicep: this month).')
param startDate string
resource budget 'Microsoft.Consumption/budgets@2023-05-01' = {
name: name