Agent Harness Tutorial · End to end · Zoom level 3
What every span carries (OpenTelemetry)
Every span the agent emits is declared in a span contract: its name, the fields it must carry, and the ones it may carry. A test fails if code emits a span that isn't declared. These tables are read from the contract itself each time the site is built.
Goes inA step of the turn, such as one model call
What happensOpen a span with the contract's required fields; record errors; redact anything personal
Comes outOne row in App Insights (AppDependencies), with these fields as Properties
All 36 declared spans
| Span | Must carry | May carry | Declared by |
|---|---|---|---|
agent.turn | agent.run.id, session.id, user.id, agent.synthetic, agent.captured | agent.config.fingerprint, agent.resolution, agent.context.stored_chars, agent.turn.result, agent.turn.rule_id, agent.reply.redacted, agent.input, agent.reply, agent.enabled | harness |
agent.run | agent.run.id, agent.tenant | agent.termination.reason, agent.cost.usd, agent.cost.call_usd, agent.model.malformed, agent.tool_calls.bound, agent.policy.blocked_by | harness |
http.chat | — | agent.tenant, user.id, session.id, http.status_code, http.refusal_detail, agent.result, agent.synthetic | harness |
agent.step | agent.step, agent.run.id | agent.context.chars, agent.context.exchanges, agent.context.trimmed | harness |
agent.escalation.raise | agent.escalation.id, agent.escalation.tier, agent.escalation.rule_id | agent.escalation.rules_version | harness |
agent.escalation.lapse | agent.escalation.id, agent.escalation.rule_id | agent.escalation.waited_s | harness |
agent.escalation.wait | agent.escalation.id | agent.escalation.waited_s | harness |
agent.escalation.queue | user.id | agent.escalation.depth | harness |
agent.escalation.refused | — | http.refusal_detail | harness |
agent.channel.webhook | agent.channel.outcome | — | harness |
agent.channel.opening | agent.channel.outcome | — | harness |
agent.channel.turn | agent.channel.outcome | — | harness |
agent.portal.refused | — | http.refusal_detail | harness |
agent.escalation.resolve | agent.escalation.id, agent.escalation.rule_id, agent.escalation.outcome | agent.escalation.waited_s | harness |
gen_ai.chat | gen_ai.provider.name | gen_ai.system, gen_ai.operation.name, gen_ai.request.model, gen_ai.request.temperature, gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, agent.resolution, agent.cassette.match, agent.gateway.x-content-safety, prompt, response | harness |
agent.tool | gen_ai.tool.name, agent.tool.side_effect, agent.idempotency.key | agent.tool.replayed, agent.tool.truncated, agent.tool.outcome, agent.tool.arguments, agent.tool.result | harness |
agent.tool.local | gen_ai.tool.name, agent.tool.side_effect, agent.tool.outcome | agent.tool.arguments, agent.tool.result | harness |
agent.feedback | session.id, user.id, agent.feedback | — | harness |
agent.tools.list | agent.tools.count, agent.tools.rejected | — | harness |
agent.policy | agent.policy.position | agent.policy.blocked_by, agent.policy.errored | harness |
agent.approval.queue | user.id | agent.approval.depth | harness |
agent.approval.request | agent.approval.id, agent.approval.action | — | harness |
agent.approval.decide | agent.approval.id, agent.approval.granted | — | harness |
agent.flow.fanout | agent.flow.count | agent.flow.peak | harness |
agent.flow.throttled | agent.flow.delay_s | — | harness |
agent.freshness.refresh | agent.freshness.rows | — | harness |
agent.freshness.resume | agent.freshness.rows | — | harness |
agent.breaker | agent.breaker.state | — | harness |
agent.llm.retry | agent.retry.attempt, agent.retry.reason | — | harness |
agent.route | agent.route.kind, agent.route.reason, agent.router.rules_version | — | claims agent |
agent.direct | agent.handler | — | claims agent |
agent.opening | agent.tenant, agent.opening.claims, agent.opening.in_flight | — | claims agent |
agent.approval.assess | agent.approval.id, agent.payout.automatic_limit_inr | — | claims agent |
agent.approval.resume | agent.approval.id | — | claims agent |
agent.approval.carry_out | agent.approval.id | — | claims agent |
agent.promise.unbacked | agent.promise.kind, agent.promise.rules_version | — | claims agent |
Every field, and what it means
| Field | Meaning (from the code's own docstring) |
|---|---|
agent.captured | This turn's words were kept. Without the marker, a rule that found no contradiction in a turn whose reply was never recorded would count as a pass. |
agent.config.fingerprint | — |
agent.context.chars | How full the assembled transcript was on this call. AAC-0103 says context growth is tracked across releases, and a committed baseline does that between releases. This does it *per call*, which is the half that tells you whether a threshold is anywhere near right — a system that never trims has headroom it is not using, and one that trims constantly is losing information silently. |
agent.context.exchanges | — |
agent.context.stored_chars | What the checkpoint actually holds, which is a different question from what the model was sent and was for a long time nobody's. |
agent.context.trimmed | Units dropped to make the call fit. The number that decides whether the rest of the context work is justified or premature. |
agent.cost.call_usd | — |
agent.cost.usd | — |
agent.enabled | The kill switch's state for this turn (`entrypoint.switch`, A13); absent where no switch is wired. |
agent.escalation.id | The join key across raise, wait and lapse. Without one on every span, queue wait time is not computable from the trace. |
agent.escalation.rule_id | Which rule fired. Sliced by this, the outcome of an escalation answers the question that tunes the rule set — AAC-0020's over-refusal rate, in the shape this agent actually has. |
agent.escalation.tier | — |
agent.feedback | up · down, from the customer, against a conversation (AHC-0112). |
agent.freshness.rows | How many rows were read again because an irreversible action would otherwise have run on an old belief — AHC-0107. Worth a number rather than a flag: a rate that climbs says the world is moving faster than the conversation, which is a fact about the business and not about the agent. |
agent.freshness.uncheckable | Set when a belief was stale and the surface offered no way to read it again. Said out loud rather than passed over, because the alternative is a run that proceeded on an old value and left nothing to say so. |
agent.idempotency.key | — |
agent.input | — |
agent.iteration | — |
agent.model.malformed | How many provider responses could not be parsed this run — AHC-0001. An attribute rather than a log line: a parse failure rate is a number that moves when a model is swapped, and one that only exists in logs is one nobody plots. |
agent.reply | — |
agent.reply.redacted | Whether redaction would change the reply — personal data or a secret in what the customer was about to be shown (AAC-0006). On every turn, not only captured ones, because it is the one content check that must not be sampled. |
agent.resolution | mock | replay | real | shadow. A verdict is not interpretable without it. |
agent.route.kind | — |
agent.route.reason | AAC-0100 — the serving route is recorded, with its reason and its cost. |
agent.run.id | — |
agent.step | — |
agent.synthetic | The turn came from a declared synthetic customer — the canary (AHC-0113). Every rate excludes these. |
agent.tenant | AAC-0104 — spend is attributable to tenant, feature and route. Tenant here, feature is the handler or intent, route is ROUTE_KIND above. |
agent.termination.reason | — |
agent.tool.arguments | — |
agent.tool.outcome | ok · refused · error · replayed. `refused` is the far system answering no — `allowed: false` — which is a correct answer and not an error. |
agent.tool.result | — |
agent.tool.side_effect | — |
agent.tool_calls.bound | `per_step` or `per_turn`: which of AHC-0097's two bounds stopped the turn, set only when one did. The termination says a bound was reached; this says which, because the two are raised for different reasons. |
agent.turn.result | completed · refused · escalated · needs_approval · failed — how the turn ended. |
agent.turn.rule_id | The rule that refused or escalated, where one did. |
gen_ai.operation.name | — |
gen_ai.provider.name | — |
gen_ai.request.model | — |
gen_ai.response.model | — |
gen_ai.system | Deprecated. Renamed to `gen_ai.provider.name` in semantic-conventions v1.37.0, and the whole `gen_ai.*` namespace has since moved to its own repository. Both are emitted for one release cycle, because a backend built against the current spec no longer matches the old name — a dashboard grouping by `gen_ai.system` goes dark the moment the libraries around it update. Drop this constant, the two call sites in `llm`, the one in `cassette`, and its entry in the contract together. |
gen_ai.tool.name | — |
gen_ai.usage.input_tokens | — |
gen_ai.usage.output_tokens | — |
session.id | — |
user.id | — |
No words by default. agent.input and agent.reply are only filled when the turn's words are captured; on Azure agent.captured is false, and anything captured goes through redact() first and is capped at 4,000 characters.