Learn › Agent Harness, the big picture › Stop 7 › What a harness must do

Stop 7 of 9 · Pages

See: logs, traces, numbers and cost

An agent makes decisions nobody wrote down in advance. So everything it does has to be written down as it happens.

Requirement 20 of 33

Logs

Also called: Logging

What happened, and when?

Every stepof every partA log linetime, who, what,the resultAn engineercan search it later
A log is one line about one event.

Why

When something goes wrong at 11:02 pm, a log is how you find out what each part was doing at 11:02 pm.

Without it

"It failed" is all anyone can say.

Think about

  • The same fields in every line, so that lines can be searched
  • One identifier for a conversation, on every line about it
  • No passwords and no personal data in the logs

In the big picture: Observability, in the See family

Requirement 21 of 33

Traces

Also called: Tracing, spans

What was the whole journey of this one request?

One requestA traceevery turn, tool andcheck, in order, timedSee the wrong turnand how longeach step took
Logs are separate lines. A trace joins them into one story.

Why

An agent takes a different path each time. A trace shows the path it took this time: which turns, which tools, which checks, and how long each took.

Without it

Nobody can explain why the agent did what it did.

Think about

  • A step in the trace for every model call, tool run and check
  • What to record on each: the input, the output, tokens, the verdict
  • How long traces are kept, and who may read them

In the big picture: Observability, in the See family

Requirement 22 of 33

Numbers and alerts

Also called: Metrics, dashboards, alerting

Is it healthy right now, and who is told when it is not?

Thousands of tracesNumbers over timeerrors, speed,refusals, hand-oversAn alertwhen a numbermoves
One trace tells a story. A thousand traces tell you a trend.

Why

Nobody reads every trace. Counting them shows whether the agent is getting slower, failing more or being attacked more.

Without it

Problems are found by customers.

Think about

  • The few numbers that matter: success, speed, cost, refusals
  • The level at which each one wakes somebody up
  • Who that somebody is

In the big picture: Observability, in the See family

Requirement 23 of 33

Know and cap the cost

Also called: Cost metering, budgets

What does one conversation cost, and what stops it running away?

Every model calluses tokensA meter and a budgetfor each conversation,each agent, each teamWarn or stopat the cap
Every turn costs money, and each turn costs more than the last.

Why

Stop 4 showed the request growing on every turn. Cost follows it, and a loop with no limit has no upper cost.

Without it

The first sign of trouble is the invoice.

Think about

  • A meter on every model call, added up for each conversation
  • A budget for a conversation, an agent and a team
  • What happens at the cap: a warning, a cheaper model or a stop

In the big picture: Cost control, in the See family

Requirement 24 of 33

An audit trail

Also called: Audit log

Who did what, why, and with whose approval?

An actiona payout, a changeA permanent recordwho asked, what ran,who approvedThe auditoror the regulator
Logs are for engineers. The audit trail is for everyone else.

Why

When a customer disputes a decision, or a regulator asks, the business must show what the agent did and on whose authority.

Without it

The company cannot show that it followed its own rules.

Think about

  • Which actions must be recorded: money, data, approvals, refusals
  • A record that cannot be changed afterwards
  • How long the law says it must be kept

In the big picture: Governance, the rail on the right

← Keep goingProve and change →