cleandataengineering.aiAgent Harness, drawn simply
You are here: Cost › Budgets

Agent Harness Tutorial · Cost

Budgets, layer by layer

"Budget" means six different things in this harness, one for each size of thing that can run away: a single model call, a trip round the loop, a whole turn, a conversation, a minute of traffic, and a month of spending. Each is enforced in a different place, and each does something different when it is hit.

Goes inA call, a trip, a turn, a conversation, a minute, a month
What happensEach layer counts its own thing and has its own limit, checked where that thing is visible
Comes outThe smallest possible reaction: cut, trim, retry, stop the turn, hand to a person, or email
PER MONTH · Azure$10 budget · email at 50%, 80% and a 100% forecast · never stops anything→ tells a personPER MINUTE · the gatewayAPIM 30 calls · Groq 8,000 tokens · Content Safety 5,000 a month→ wait and retryPER CONVERSATION10 turns → a person · at most 2 hand-offs · history saved ≤ 32,000 chars→ hand off / trimPER TURN12 trips · 60 s · $0.50 · 24 tool calls · 4 retries · same call 3×→ stop the turnPER TRIP8 tool calls · context ≤ 24,000 chars→ stop / trimPER CALL4,096 output tokens · 60 s timeout · tool result ≤ 8,000 chars→ stop / cut
Outside to inside: the outer layers are about money and capacity; the inner ones are about one runaway answer or loop.

Why so many? Each runaway is only visible at one size. One answer that rambles is caught per call. A loop that asks for 30 tools is caught per trip. Fourteen calls that are really one stuck task are visible only per turn: that is why the step and cost limits live in the loop, not in the model client. A conversation going round in circles shows only across turns. A bill only shows across a month.

Three kinds of reaction. Some budgets trim and carry on (context, tool results, saved history). Some stop the turn with a named reason and an honest reply, usually "let me pass you to a colleague". The outermost ones only tell someone: the Azure budget never switches anything off.

BudgetScopeLimitEnforced byWhen it is hitKind
Output tokensper call4,096sent to the model as max_tokensthe answer is cut; a cut answer is never sent (output_length_reached)stops
Model call timeper call60 sthe HTTP clientUnavailable → wait and retryretries
Tool result sizeper call8,000 charsGatedTools._bound()the result is cut, with a "[truncated]" marktrims
Contextper trip24,000 charsassembled()whole exchanges dropped from the middletrims
Tool callsper trip / turn8 / 24over_fan_out()tool_call_budget_exhaustedstops
Tripsper turn12the loopstep_budget_exhaustedstops
Timeper turn60 s_out_of_time()deadline_reachedstops
Costper turn$0.50the Metercost_ceiling_reached (checked only when the model wants more tools)stops
Retriesper turn4unit_retries()model_unavailablestops
Same call againper turn3rd timethe call's signatureoscillation_detectedstops
Turnsper conversation10Tier-2 rule turns-exceededhanded to a personhands off
Hand-offsper conversation2escalations_raisedno third hand-offstops
Saved historyper conversation32,000 charsbounded() when savingoldest text trimmedtrims
Gateway callsper minute30APIM rate-limit429 → wait as told, retryretries
Groq tokensper minute8,000 (free tier)Groq429 → wait, retry; in practice 3–5 loop turns a minuteretries
Content Safetyper month5,000 records (F0)Azurefails open: the verdict is "unavailable" and the call passeslets through
Azure spendper month$10infra/modules/budget.bicepan email to the owner; nothing stopstells

Two more levels are designed but not configured yet: a team's monthly budget in Cost Management with an alert at 80%, and a token limit per agent identity and customer tier at APIM (not available on the Consumption tier, F-39). The ceiling is checked between calls, because you can't un-spend a call: one call can overshoot, bounded by its 4,096 output tokens. At today's prices twelve trips cost about $0.05, so the $0.50 ceiling has never been reached.

Every per-turn limit sits in one frozen object, Budgets, and can be changed per environment without code (for example CLAIMS_MAX_STEPS). Each "stops" reason is written on the trace, so we can count how often each budget fires; see when the loop stops.

Want every detail? The long version is in the Series: budgets and limits.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

agent_harness/config/__init__.py:13Every per-turn budget, in one frozen object
class Budgets(BaseModel):
    """Ceilings that can actually stop a call.

    Two positions, deliberately. `max_output_tokens` is a per-call backstop at
    P3; `max_steps` and `max_cost_usd` are per-task and enforced at P4, because
    only the loop can see that fourteen calls are one runaway task rather than
    fourteen tasks. AAC-0093 and AAC-0008 are different obligations for exactly
    this reason.
    """

    model_config = ConfigDict(frozen=True)

    max_steps: int = 12
    max_cost_usd: float = 0.50
    max_output_tokens: int = 4096
    max_tool_result_chars: int = 8000
    """Tool results are bounded before they enter context — AAC-0105."""
    max_turn_seconds: int = 60
    """Wall clock for one turn (AHC-0096), checked before each step."""
    max_tool_calls_per_step: int = 8
    """How many tool calls one step may plan (AHC-0097). A step budget counts
    steps, so without this one step asking for fifty look-ups — or fifty
    writes — spends one of the twelve and does fifty things."""
    max_tool_calls_per_turn: int = 24
    """How many tool calls one turn may plan across all its steps (AHC-0097).
    Reaching either bound stops the turn before the step's calls run, as
    `tool_call_budget_exhausted` — never a silent cut to the first N (AHC-0059)."""
    max_retries_per_unit: int = 4
    """AHC-0024: retries across the whole turn, every call's together (F-087)."""
agent_harness/resilience/__init__.py:88The turn's shared retry allowance
@contextmanager
def unit_retries(limit: int) -> Iterator[None]:
    """One unit of work's retry allowance, shared by every call inside it.

    AHC-0024 and F-087: "a retry count exists per unit of work, not per call
    site". Each call kept its own few attempts, so a twelve-step turn could
    retry twenty-four times. Inside this scope every retry spends from one
    budget; when it is gone a failure is raised instead of retried. Outside any
    scope only the per-call bound applies.
    """
    token = _UNIT.set([limit])
    try:
        yield
    finally:
        _UNIT.reset(token)
agent_harness/loop/plan.py:82Tool calls per trip and per turn
def over_fan_out(asked: int, so_far: int, budgets: Budgets) -> str | None:
    """`per_step` or `per_turn`: the bound a plan of `asked` calls would pass, after
    `so_far` calls already planned this turn; `None` when it passes neither."""
    if asked > budgets.max_tool_calls_per_step:
        return "per_step"
    if so_far + asked > budgets.max_tool_calls_per_turn:
        return "per_turn"
    return None
infra/policies/groq-api.xml:34Per minute, at the gateway
    <!-- Request rate per subscription key (the agent's): 30 calls a minute. -->
    <rate-limit calls="30" renewal-period="60" remaining-calls-header-name="x-ratelimit-remaining" />
infra/modules/budget.bicep:1Per month: the Azure budget
// WHAT IT IS
//   A cost budget on the resource group rg-claims-fnol-dev: $10 a month, with an
//   email to the owner when actual spend reaches 50% and 80%, and when the month's
//   forecast reaches 100%. A budget never stops anything; it only tells you.
//
// WHICH CONCERN IT SERVES
//   Cost (architecture deck, slide 53; PLAN.html Tier 3, "a $10 budget alert
//   before anything else"). The 12-month free offer is used up, so PostgreSQL is
//   the one real cost; this is the tripwire if something else starts billing.
//
// EXPECTED DEV COST
//   $0. Budgets are free (Cost Management).
//
// ORDER
//   main.bicep makes every other module depend on this one, so ARM creates the
//   budget first. Cost data reaches Cost Management with a delay of hours, so an
//   alert is never instant.

targetScope = 'resourceGroup'

@description('Budget name.')
param name string = 'budget-claims-fnol-dev'

@description('Monthly amount in the billing currency (USD for this subscription).')
param amount int = 10

@description('Who gets the alert emails.')
param contactEmail string

@description('First day of the budget period, yyyy-MM-01T00:00:00Z (main.bicep: this month).')
param startDate string

resource budget 'Microsoft.Consumption/budgets@2023-05-01' = {
  name: name