Learn › Agent Harness, the big picture › Stop 7 › What a harness must do
Stop 7 of 9 · Pages
Keep going: failures, speed and scale
What makes an agent dependable on a bad day. None of it is visible in a demo.
- 15 · Survive a crash
- 16 · Do each action only once
- 17 · Cope when something fails
- 18 · Be fast enough
- 19 · Serve many people at once
Survive a crash
Also called: Durable execution, checkpoints
What happens to a half-finished job when a server restarts?
Why
An agent's job can last minutes, or days when it waits for approval. Servers restart far more often than that.
Without it
A restart loses the work, or repeats a payment.
Think about
- Save after every step, not only at the end
- On restart, skip the steps that already finished
- How long a paused job may wait before it expires
In the big picture: Memory and state, in the Think family
Do each action only once
Also called: Idempotency
What if the same step is tried twice?
Why
After a time-out, nobody knows whether the first attempt worked. Trying again must not create a second claim or a second payment.
Without it
Two claims, two payments or two emails for one request.
Think about
- A unique key for every action
- Tools that recognise a repeat and return the first result
- Extra care for money and messages to customers
In the big picture: Tool hub, in the Act family
Cope when something fails
Also called: Time-outs, retries, fallbacks, circuit breakers
What does the agent do when a tool or the model is down?
Why
Every system an agent touches will fail sometimes. A good agent retries a little, then stops and says so.
Without it
The agent retries all night, or invents an answer.
Think about
- How long to wait before giving up on each call
- How many retries, and how far apart
- What the person is told, and who picks the work up
In the big picture: Model gateway and Human desk
Be fast enough
Also called: Latency, streaming, caching
How long will a person wait, and what do they see while waiting?
Why
Each turn is a model call, and model calls are slow next to ordinary code. Five turns can feel long.
Without it
People give up before the answer arrives.
Think about
- A time budget for the whole answer and for each turn
- Show words as they arrive, and say what step is running
- A smaller model for easy steps, and reuse of repeated text
In the big picture: Model gateway, in the Think family
Serve many people at once
Also called: Scale, concurrency
Does it still work with a thousand conversations at the same time?
Why
If a copy of the agent keeps a conversation in its own memory, that conversation is lost when the copy goes away.
Without it
It works in the demo and fails on the first busy day.
Think about
- All state in the shared store, none inside the agent
- Limits from model providers are shared by all copies
- A storm of requests should queue up and not fall over
In the big picture: Memory and state, and the front door