Learn › Agent Harness, the big picture › Stop 7

Stop 7 of 9 · Pages

What a harness must do

Before the big picture: everything a harness has to do, one need at a time. 33 requirements in seven groups, one small picture each.

How to read every picture.Left: what comes in. Lavender, in the middle: the requirement. Right: what comes out. A pink cross on an arrow means "stopped here".
ON THE PATH OF EVERY REQUESTOpen: Receive the requestReceivethe request5 requirementsopen ›Open: Think: the loop and the modelThink: the loopand the model4 requirementsopen ›Open: Act: tools, checks, peopleAct: tools,checks, people5 requirementsopen ›AROUND EVERY REQUESTOpen: Keep goingKeep goingfailures, speed, scale5 requirementsopen ›Open: SeeSeelogs, traces, cost5 requirementsopen ›Open: Prove and changeProve and changeevals, tests, release5 requirementsopen ›Open: ProtectProtectsecrets, data, the law4 requirementsopen ›
Seven groups. The top row is what an agent does. The bottom row is how well and how safely it does it.

The top row follows a request: it is received, the agent thinks, and the agent acts. These are the functional requirements, the things an agent does. The bottom row applies to every request at once: keep going, see, prove and change, and protect. These are the non-functional requirements, how well and how safely it does them.

Receive the request →Think →Act →Keep going →See →Prove and change →Protect →

All 33 requirements

RequirementAlso calledThe question an engineer asksGroup
1A front door where requests arriveAPI server, gateway, channel adaptersWhere does a message arrive, whatever channel it came from?Receive the request
2Know who is askingAuthenticationIs this person really who they say they are?Receive the request
3Know what they are allowed to doAuthorization, permissionsWhat may this person see and change?Receive the request
4Check what comes inInput guardrailsIs this request safe to hand to the model?Receive the request
5A fair share for everyoneRate limits, quotas, priorities, a queueWhat happens when everyone asks at once?Receive the request
6Run the loopOrchestration, the agent loopWhat drives the turns, and when does it stop?Think
7Call the model, reliablyModel gatewayHow does every agent reach a model, and what if the model is slow or down?Think
8Give the model the right contextContext building, retrieval, promptsWhat exactly does the model get to see on each turn?Think
9RememberMemory and stateWhat does it remember within a conversation, and across them?Think
10Use toolsTool hub, tool calling, MCPHow does the agent do anything in the real systems?Act
11Treat what tools return as dataTool-result guardrails, fencingWhat if a document the agent reads contains an instruction?Act
12Check before actingAction guardrails, pre-tool checks, policyShould this particular action be allowed to run?Act
13Ask a personHuman approval, human in the loopWhich actions need a person to say yes?Act
14Check what goes outOutput guardrailsIs this reply safe and true enough to send?Act
15Survive a crashDurable execution, checkpointsWhat happens to a half-finished job when a server restarts?Keep going
16Do each action only onceIdempotencyWhat if the same step is tried twice?Keep going
17Cope when something failsTime-outs, retries, fallbacks, circuit breakersWhat does the agent do when a tool or the model is down?Keep going
18Be fast enoughLatency, streaming, cachingHow long will a person wait, and what do they see while waiting?Keep going
19Serve many people at onceScale, concurrencyDoes it still work with a thousand conversations at the same time?Keep going
20LogsLoggingWhat happened, and when?See
21TracesTracing, spansWhat was the whole journey of this one request?See
22Numbers and alertsMetrics, dashboards, alertingIs it healthy right now, and who is told when it is not?See
23Know and cap the costCost metering, budgetsWhat does one conversation cost, and what stops it running away?See
24An audit trailAudit logWho did what, why, and with whose approval?See
25EvalsEvaluations, golden sets, gradersIs it any good, and how would we know?Prove and change
26Attack it before others doRed teaming, adversarial testingDoes the harness hold when someone tries to break it?Prove and change
27Release safelyVersioning, staged rollout, rollbackHow does a change reach users without hurting them?Prove and change
28Change settings without changing codeConfigurationCan we change a limit without a new release?Prove and change
29Learn from every missFeedback loopHow does a mistake in production make the agent better?Prove and change
30Keep secrets away from the modelSecrets managementWhere do passwords and keys live?Protect
31Protect personal dataMasking, data minimisation, retentionHow much personal data does the model really need?Protect
32Keep customers apartIsolation, multi-tenancyCan one customer's data, memory or limits leak into another's?Protect
33Name an owner and a risk levelGovernanceWho answers for this agent, and how much freedom does it have?Protect

Guardrails come in four kinds

"We need guardrails" is four different requirements, because there are four different places to stand.

Where it standsNameWhat it catchesGroup
On the way inInput guardrailsHidden orders, personal data, off-topic requestsReceive the request
On what the agent readsTool-result guardrailsInstructions hidden in documents, emails and recordsAct
Before an actionAction guardrailsPayments, deletions and anything outside the person's rightsAct
On the way outOutput guardrailsWrong claims, other people's data, promises the company cannot keepAct

The usual names for the non-functional requirements

If your team uses the standard words, this is where each one is covered.

Not every agent needs all of them.A small internal helper needs a slice. An agent that moves money needs nearly everything. The risk level of the job decides, and that is the last requirement on the list.

Next: stop 8 puts all of these into one picture. Every box in it will be a requirement you have already met.