Agent Harness Tutorial · Tech stack
L10 · Failure handling
Typed failures, fallbacks, undo paths, honest messages. Family: Control · around every turn.
Why our choiceA retry hidden inside an SDK is invisible to our retry budget, so retries and fallbacks live in one place we control.
| Option | Where from | Verdict | Why |
|---|---|---|---|
| ResilientLLM | ours | ✓ chosen | one retry budget per turn |
| Typed failures | ours | ✓ chosen | mapped from Pydantic AI |
| Ledger abandon-and-retry | ours | ✓ chosen | |
| APIM backend pools + breaker | Azure | ◐ planned | tier 10 |
| Container Apps health probes | Azure | ◐ planned | in the design |
| tenacity | open source | · considered | not used |
| Own resilience wrapper | open source | ✓ used | chosen |
How it works inside the harness: Layers around the call.
Compared: What retries and fails over when a call breaks?
| Option | From | Verdict | Where it acts | Knows our typed failures | Shares one retry budget | Status |
|---|---|---|---|---|---|---|
| ResilientLLM (own wrapper) | ours | chosen | inside the LLMClient port | yes | yes | built |
| tenacity | open source | alternative | any Python call | only what we tell it | if we wire it | listed open source option |
| Provider SDK retries | vendor | rejected | inside the openai SDK | no | no: invisible to us | off: max_retries=0 |
| Pydantic AI fallback model | open source | chosen | inside the model adapter | partly | yes, one budget | design: degradation path |
| APIM backend pools + breaker | Azure | planned | at the gateway | no | must share our budget | planned, Tier 10 |
| Health probes + Front Door | Azure | chosen | containers and regions | no | n/a | final (design) |
DecisionModel retries live in our ResilientLLM wrapper with one retry budget, so SDK retries are switched off to stay visible. APIM backend pools and the circuit breaker are planned for production failover across backends and regions.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
class ResilientLLM:
"""L2 · L10. Any model client, behind a declared degradation path.
Three controls, each answering a different condition — and the reason they
are one decorator is that they must agree about which condition this is:
- **A rate limit** (`ModelThrottled`) waits exactly the provider's
`retry-after`, shared across every caller, and is **not** a breaker
failure — being told to slow down is not evidence the provider is down
(AHC-0021).
- **A failure** (`ModelUnavailable`) is retried with capped, jittered backoff,
a bounded number of times, each retry a span (AHC-0024), and counts
towards the breaker.
- **An open breaker** fails fast without calling the provider at all, until a
single probe is allowed after the cooldown (AHC-0005).
Malformed output is none of these and is never retried: the same prompt to
the same model produced something unreadable, and a retry mostly buys a
second bill. When the attempts run out, the last provider failure is raised
typed, and the loop turns it into the customer's `Failed`.
"""
def __init__(
self,
inner: LLMClient,
*,
attempts: int = 3,
backoff: Backoff | None = None,
breaker: CircuitBreaker | None = None,
throttle: Throttle | None = None,
sleep: Callable[[float], Awaitable[None]] | None = None,
) -> None:
self.inner = inner
self.attempts = attempts