Learn › Agent Harness, the big picture › Stop 7 › What a harness must do

Stop 7 of 9 · Pages

Keep going: failures, speed and scale

What makes an agent dependable on a bad day. None of it is visible in a demo.

Requirement 15 of 33

Survive a crash

Also called: Durable execution, checkpoints

What happens to a half-finished job when a server restarts?

Turn 3 of 5a claim halfregisteredA crashor a restartResume at turn 3not from thebeginning
Save after every step, so that no step is lost.

Why

An agent's job can last minutes, or days when it waits for approval. Servers restart far more often than that.

Without it

A restart loses the work, or repeats a payment.

Think about

  • Save after every step, not only at the end
  • On restart, skip the steps that already finished
  • How long a paused job may wait before it expires

In the big picture: Memory and state, in the Think family

Requirement 16 of 33

Do each action only once

Also called: Idempotency

What if the same step is tried twice?

"Create the claim"sent, no replyTry againwas the first onereceived?Still one claimthe second tryis recognised
Retries are safe only when a repeat changes nothing.

Why

After a time-out, nobody knows whether the first attempt worked. Trying again must not create a second claim or a second payment.

Without it

Two claims, two payments or two emails for one request.

Think about

  • A unique key for every action
  • Tools that recognise a repeat and return the first result
  • Extra care for money and messages to customers

In the big picture: Tool hub, in the Act family

Requirement 17 of 33

Cope when something fails

Also called: Time-outs, retries, fallbacks, circuit breakers

What does the agent do when a tool or the model is down?

A tool is downor very slowWait, retry, stopa few times,then give upHand over honestlywith everythingtried so far
Failing well is a feature.

Why

Every system an agent touches will fail sometimes. A good agent retries a little, then stops and says so.

Without it

The agent retries all night, or invents an answer.

Think about

  • How long to wait before giving up on each call
  • How many retries, and how far apart
  • What the person is told, and who picks the work up

In the big picture: Model gateway and Human desk

Requirement 18 of 33

Be fast enough

Also called: Latency, streaming, caching

How long will a person wait, and what do they see while waiting?

A questionFirst words quicklyshow progress whilethe loop runsThe full answera little later
Several turns take several seconds. Design for the wait.

Why

Each turn is a model call, and model calls are slow next to ordinary code. Five turns can feel long.

Without it

People give up before the answer arrives.

Think about

  • A time budget for the whole answer and for each turn
  • Show words as they arrive, and say what step is running
  • A smaller model for easy steps, and reuse of repeated text

In the big picture: Model gateway, in the Think family

Requirement 19 of 33

Serve many people at once

Also called: Scale, concurrency

Does it still work with a thousand conversations at the same time?

A thousand chatsat the same timeMany copiesof the agent, noneholding memory itselfOne shared storefor all state
Keep nothing in the agent itself, and it can be copied freely.

Why

If a copy of the agent keeps a conversation in its own memory, that conversation is lost when the copy goes away.

Without it

It works in the demo and fails on the first busy day.

Think about

  • All state in the shared store, none inside the agent
  • Limits from model providers are shared by all copies
  • A storm of requests should queue up and not fall over

In the big picture: Memory and state, and the front door

← ActSee →