Home › Key concepts › Tracing an AI system
Key concept · Walk (intermediate) · the same on every platform
Tracing an AI system
Record every model call, tool call, token and cost for each turn.
What it is
A trace for one user turn shows each step: context built, model call (with tokens and cost), tool requested, check result, tool output, final answer. Traces are your debugger, your audit log and the raw material for evaluation.
Why it matters
When an agent does something odd, the trace says why. It also shows where the money goes.
An example
A trace shows a turn cost $0.004 and that 80% of tokens were an old document repeated in the context.
The common mistake
Logging only the final answer.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Tracingsee every model call, tool call, token and cent | CloudWatch + X-Ray (OpenTelemetry) | App Insights (OpenTelemetry) | Cloud Trace (OpenTelemetry) | MLflow tracing | Phoenix or Langfuse |
Related ideas
Where it shows up
Part of the Walk level on these paths: AI Engineering (safety and proof).