One user request may become a retrieval, three model calls and two tool invocations.
A trace ties all of that to a single identifier so the whole request can be read as one story.
Each step becomes a span recording its input, output, duration, tokens and cost.
Without traces, a slow or wrong answer is a mystery with no way to find which step caused it.
Monitoring tells you something is wrong; tracing is how you find out what.
A single request to an AI system is rarely a single operation.
It may rewrite a query, search an index, rerank the results, call a model, then call a tool and call the model again.
When the answer is wrong or slow, the useful question is which of those steps was responsible.
Aggregate monitoring cannot answer it, because averages have already discarded the individual request.
Tracing assigns every request an identifier and records each step as a span beneath it.
Spans nest, so a model call inside a retrieval step appears as a child of it.
Each span records timing, inputs, outputs, token counts, cost and any error raised.
Read top to bottom, a trace is the complete story of one request.
That is what makes latency debugging tractable: the slow span is visible immediately.
It is equally valuable for quality, since a wrong answer usually has a wrong retrieval behind it.
Seeing the exact passages that were supplied is often enough to diagnose the failure outright.
The practice comes from distributed systems, and OpenTelemetry is the common standard for it.
AI-specific tools add token and cost accounting and store prompts and completions in full.
Sampling keeps the volume manageable, but errors and slow requests should always be kept.
Storing prompts means storing user input, so retention and redaction are part of setting this up.
One question on this card. Every option explains itself once you pick, right or wrong.
22 concepts in this module, then a graded test with every answer explained.