cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Eval › The trace › Span fields

Agent Harness Tutorial · End to end · Zoom level 3

What every span carries (OpenTelemetry)

Every span the agent emits is declared in a span contract: its name, the fields it must carry, and the ones it may carry. A test fails if code emits a span that isn't declared. These tables are read from the contract itself each time the site is built.

Goes inA step of the turn, such as one model call
What happensOpen a span with the contract's required fields; record errors; redact anything personal
Comes outOne row in App Insights (AppDependencies), with these fields as Properties
http.chatagent.turnagent.run.id, session.id, user.id, agent.syntheticagent.routeagent.route.kind, agent.route.reason, agent.router.rules_versionagent.runagent.run.id, agent.tenantagent.tools.listagent.tools.count, agent.tools.rejectedagent.stepagent.step, agent.run.idgen_ai.chatgen_ai.provider.nameagent.policyagent.policy.positionagent.toolgen_ai.tool.name, agent.tool.side_effect, agent.idempotency.keyagent.escalation.raiseagent.escalation.id, agent.escalation.tier, agent.escalation.rule_idagent.approval.requestagent.approval.id, agent.approval.action
Indented spans happen inside the span above. Full lists below.

All 36 declared spans

SpanMust carryMay carryDeclared by
agent.turnagent.run.id, session.id, user.id, agent.synthetic, agent.capturedagent.config.fingerprint, agent.resolution, agent.context.stored_chars, agent.turn.result, agent.turn.rule_id, agent.reply.redacted, agent.input, agent.reply, agent.enabledharness
agent.runagent.run.id, agent.tenantagent.termination.reason, agent.cost.usd, agent.cost.call_usd, agent.model.malformed, agent.tool_calls.bound, agent.policy.blocked_byharness
http.chat—agent.tenant, user.id, session.id, http.status_code, http.refusal_detail, agent.result, agent.syntheticharness
agent.stepagent.step, agent.run.idagent.context.chars, agent.context.exchanges, agent.context.trimmedharness
agent.escalation.raiseagent.escalation.id, agent.escalation.tier, agent.escalation.rule_idagent.escalation.rules_versionharness
agent.escalation.lapseagent.escalation.id, agent.escalation.rule_idagent.escalation.waited_sharness
agent.escalation.waitagent.escalation.idagent.escalation.waited_sharness
agent.escalation.queueuser.idagent.escalation.depthharness
agent.escalation.refused—http.refusal_detailharness
agent.channel.webhookagent.channel.outcome—harness
agent.channel.openingagent.channel.outcome—harness
agent.channel.turnagent.channel.outcome—harness
agent.portal.refused—http.refusal_detailharness
agent.escalation.resolveagent.escalation.id, agent.escalation.rule_id, agent.escalation.outcomeagent.escalation.waited_sharness
gen_ai.chatgen_ai.provider.namegen_ai.system, gen_ai.operation.name, gen_ai.request.model, gen_ai.request.temperature, gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, agent.resolution, agent.cassette.match, agent.gateway.x-content-safety, prompt, responseharness
agent.toolgen_ai.tool.name, agent.tool.side_effect, agent.idempotency.keyagent.tool.replayed, agent.tool.truncated, agent.tool.outcome, agent.tool.arguments, agent.tool.resultharness
agent.tool.localgen_ai.tool.name, agent.tool.side_effect, agent.tool.outcomeagent.tool.arguments, agent.tool.resultharness
agent.feedbacksession.id, user.id, agent.feedback—harness
agent.tools.listagent.tools.count, agent.tools.rejected—harness
agent.policyagent.policy.positionagent.policy.blocked_by, agent.policy.erroredharness
agent.approval.queueuser.idagent.approval.depthharness
agent.approval.requestagent.approval.id, agent.approval.action—harness
agent.approval.decideagent.approval.id, agent.approval.granted—harness
agent.flow.fanoutagent.flow.countagent.flow.peakharness
agent.flow.throttledagent.flow.delay_s—harness
agent.freshness.refreshagent.freshness.rows—harness
agent.freshness.resumeagent.freshness.rows—harness
agent.breakeragent.breaker.state—harness
agent.llm.retryagent.retry.attempt, agent.retry.reason—harness
agent.routeagent.route.kind, agent.route.reason, agent.router.rules_version—claims agent
agent.directagent.handler—claims agent
agent.openingagent.tenant, agent.opening.claims, agent.opening.in_flight—claims agent
agent.approval.assessagent.approval.id, agent.payout.automatic_limit_inr—claims agent
agent.approval.resumeagent.approval.id—claims agent
agent.approval.carry_outagent.approval.id—claims agent
agent.promise.unbackedagent.promise.kind, agent.promise.rules_version—claims agent

Every field, and what it means

FieldMeaning (from the code's own docstring)
agent.capturedThis turn's words were kept. Without the marker, a rule that found no contradiction in a turn whose reply was never recorded would count as a pass.
agent.config.fingerprint—
agent.context.charsHow full the assembled transcript was on this call. AAC-0103 says context growth is tracked across releases, and a committed baseline does that between releases. This does it *per call*, which is the half that tells you whether a threshold is anywhere near right — a system that never trims has headroom it is not using, and one that trims constantly is losing information silently.
agent.context.exchanges—
agent.context.stored_charsWhat the checkpoint actually holds, which is a different question from what the model was sent and was for a long time nobody's.
agent.context.trimmedUnits dropped to make the call fit. The number that decides whether the rest of the context work is justified or premature.
agent.cost.call_usd—
agent.cost.usd—
agent.enabledThe kill switch's state for this turn (`entrypoint.switch`, A13); absent where no switch is wired.
agent.escalation.idThe join key across raise, wait and lapse. Without one on every span, queue wait time is not computable from the trace.
agent.escalation.rule_idWhich rule fired. Sliced by this, the outcome of an escalation answers the question that tunes the rule set — AAC-0020's over-refusal rate, in the shape this agent actually has.
agent.escalation.tier—
agent.feedbackup · down, from the customer, against a conversation (AHC-0112).
agent.freshness.rowsHow many rows were read again because an irreversible action would otherwise have run on an old belief — AHC-0107. Worth a number rather than a flag: a rate that climbs says the world is moving faster than the conversation, which is a fact about the business and not about the agent.
agent.freshness.uncheckableSet when a belief was stale and the surface offered no way to read it again. Said out loud rather than passed over, because the alternative is a run that proceeded on an old value and left nothing to say so.
agent.idempotency.key—
agent.input—
agent.iteration—
agent.model.malformedHow many provider responses could not be parsed this run — AHC-0001. An attribute rather than a log line: a parse failure rate is a number that moves when a model is swapped, and one that only exists in logs is one nobody plots.
agent.reply—
agent.reply.redactedWhether redaction would change the reply — personal data or a secret in what the customer was about to be shown (AAC-0006). On every turn, not only captured ones, because it is the one content check that must not be sampled.
agent.resolutionmock | replay | real | shadow. A verdict is not interpretable without it.
agent.route.kind—
agent.route.reasonAAC-0100 — the serving route is recorded, with its reason and its cost.
agent.run.id—
agent.step—
agent.syntheticThe turn came from a declared synthetic customer — the canary (AHC-0113). Every rate excludes these.
agent.tenantAAC-0104 — spend is attributable to tenant, feature and route. Tenant here, feature is the handler or intent, route is ROUTE_KIND above.
agent.termination.reason—
agent.tool.arguments—
agent.tool.outcomeok · refused · error · replayed. `refused` is the far system answering no — `allowed: false` — which is a correct answer and not an error.
agent.tool.result—
agent.tool.side_effect—
agent.tool_calls.bound`per_step` or `per_turn`: which of AHC-0097's two bounds stopped the turn, set only when one did. The termination says a bound was reached; this says which, because the two are raised for different reasons.
agent.turn.resultcompleted · refused · escalated · needs_approval · failed — how the turn ended.
agent.turn.rule_idThe rule that refused or escalated, where one did.
gen_ai.operation.name—
gen_ai.provider.name—
gen_ai.request.model—
gen_ai.response.model—
gen_ai.systemDeprecated. Renamed to `gen_ai.provider.name` in semantic-conventions v1.37.0, and the whole `gen_ai.*` namespace has since moved to its own repository. Both are emitted for one release cycle, because a backend built against the current spec no longer matches the old name — a dashboard grouping by `gen_ai.system` goes dark the moment the libraries around it update. Drop this constant, the two call sites in `llm`, the one in `cassette`, and its entry in the contract together.
gen_ai.tool.name—
gen_ai.usage.input_tokens—
gen_ai.usage.output_tokens—
session.id—
user.id—

No words by default. agent.input and agent.reply are only filled when the turn's words are captured; on Azure agent.captured is false, and anything captured goes through redact() first and is capped at 4,000 characters.