OpenTelemetry GenAI

In plain words

Three companies calling the same shortlist by three different names is a reporting nightmare. Agent monitoring had the same mess until OpenTelemetry fixed the naming. Now every agent run is an invoke_agent span, every tool call is a child span under it, and the labels all start with gen_ai. And the prompt text itself stays out of the trace, because traces are readable by everyone.

How it flows

A tiny example

with span("invoke_agent refund_bot", kind="INTERNAL"):
    set_attr("gen_ai.provider.name", "anthropic")
    set_attr("gen_ai.request.model", "toy-model-1")
    with span("chat"):
        set_attr("gen_ai.input.ref", stash(prompt))
        reply = call_model(prompt)
    with span("tool_call lookup_order"):
        row = lookup_order(8891)

Notice the span carries only a reference ID for the prompt, never the prompt text itself.

What you will learn

• Why every agent trace should use the same names, no matter which tool built it.
• The three kinds of spans in the OpenTelemetry GenAI standard: model, agent, tool.
• The main gen_ai.* attributes and what each one answers.
• Why prompts should not sit inside your traces, and what to store instead.

The problem, simply

Think about a placement drive at your college. Three companies come. TCS calls the shortlist "Round 1 cleared". Infosys calls it "Level A". Amazon calls it "OA qualified". Same thing, three names.

Now the placement cell has to make one report for all three. Somebody must translate every column by hand. Painful, and mistakes creep in.

Agent monitoring had exactly this problem. One tool called an agent run agent.start, another run_chain, another agent_loop. So if you switched frameworks, every dashboard you built broke.

OpenTelemetry is the open standard that most software already uses to record what happened inside a system. In 2024 a working group inside it, called the GenAI SIG, sat down and fixed the naming for AI agents. One schema. Every vendor targets it. Your dashboard keeps working.

The idea

Spans and traces, in one line

A span is one recorded step: a name, a start time, an end time, and some labelled fields called attributes. A trace is all the spans of one request, arranged parent-child like a tree.

So when Priya asks "what is my refund status", the whole answer is one trace, and every model call and tool call inside it is a span.

The three span categories

The GenAI standard says there are exactly three kinds of spans you will emit.

• Model / client spans. One raw call to the model. Named by the operation, like chat. Emitted by the provider's own library.
• Agent spans. create_agent when the agent object is built, and invoke_agent when it actually runs.
• Tool spans. One per tool call. Each one is a child of the agent span that triggered it.

IMP

Note: The parent-child link is the whole point. An orphan tool span floating alone tells you nothing about which agent run it belonged to.

Naming the agent span

The agent run span is named invoke_agent. If you gave the agent a name, the span becomes invoke_agent <that name> — for example invoke_agent refund_bot.

Every span also has a kind, saying whether the work happened here or elsewhere.

• CLIENT — you called an agent service running on someone else's machine, over the network.
• INTERNAL — the agent ran inside your own process. A LangChain, CrewAI or a hand-written ReAct loop is INTERNAL.

LangChain and CrewAI are popular agent frameworks. ReAct is the basic think-then-act loop from a 2022 research paper. All three run in your process, so all three give INTERNAL spans.

The attributes that matter

Attributes are just labelled fields on a span. All the AI ones start with gen_ai..

• gen_ai.provider.name — who served the model: anthropic, openai, aws.bedrock, google.vertex.
• gen_ai.request.model — the model ID you asked for.
• gen_ai.response.model — the model that actually answered. It can differ, because routing happens.
• gen_ai.agent.name — which agent this is.
• gen_ai.operation.name — chat, completion, invoke_agent, tool_call.
• gen_ai.data_source.id — for retrieval, which document store or corpus was searched.

IMPRemember: without gen_ai.provider.name on your spans, a dashboard comparing two providers is simply impossible to build later.

A worked example

Suppose Rahul builds a refund agent for a Swiggy-style app. Ananya types "where is my ₹240 refund".

One trace comes out. At the top, invoke_agent refund_bot, kind INTERNAL. Under it, a chat span with provider anthropic. Under that, a tool_call span for lookup_order that took 180 ms. Then one more chat span to write the final reply.

Two weeks later refunds feel slow. Rahul groups his dashboard by span name and sees lookup_order has gone from 180 ms to 3 seconds. The model was never the problem. He found that in thirty seconds, because the names were standard.

Content capture: the careful part

Here is the trick most people get wrong. By default, the standard says instrumentation should not record the actual prompts and replies. You have to switch it on deliberately, using attributes like gen_ai.input.messages and gen_ai.output.messages.

Why off by default? A prompt contains whatever the user typed. Ananya's phone number. Rahul's UPI reference. Traces are readable by the whole ops team, and that is not where customer data should live.

The recommended production pattern is exactly that: keep the real text in a proper store like S3 or your log system, and put only a reference ID on the span. Whoever is debugging looks up that ID, if they have permission.

One environment variable

Most of these conventions are still experimental, so names can change between versions. To pin the current set, set this before your app starts:

OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

Warning

Warning: Skip this and a backend upgrade can rename your attributes. Your dashboards go blank and nobody knows why.

Datadog from version 1.37 maps these attributes into its own AI view directly. Grafana, Honeycomb and Jaeger read the raw attributes, so you build the panels yourself.

Build it

A tiny tracer in plain Python. No libraries, no real model calls. It emits the span shapes the standard describes.

import time, uuid

SPANS = []            # all finished spans
STORE = {}            # "external" content store, prompts live here
_stack = []           # current parent chain

class Span:
    def __init__(self, name, kind, attrs):
        self.name, self.kind, self.attrs = name, kind, attrs
        self.parent = _stack[-1].name if _stack else None
        self.depth = len(_stack)

    def __enter__(self):
        self.t0 = time.time()
        _stack.append(self)
        return self

    def __exit__(self, *e):
        self.ms = round((time.time() - self.t0) * 1000, 1)
        _stack.pop()
        SPANS.append(self)

def span(name, kind="INTERNAL", **attrs):
    return Span(name, kind, attrs)

def stash(text):
    """Keep prompt text outside the trace; return only a pointer id."""
    ref = uuid.uuid4().hex[:8]
    STORE[ref] = text
    return ref

def fake_model(prompt):          # stands in for a real LLM call
    return "Refund credited on 2 Sep." if "order" in prompt else "call lookup_order"

def lookup_order(order_id):
    time.sleep(0.01)
    return {"order": order_id, "refund": "done"}

# --- one agent run -------------------------------------------------
with span("create_agent", **{"gen_ai.agent.name": "refund_bot"}):
    pass

with span("invoke_agent refund_bot", **{"gen_ai.agent.name": "refund_bot",
                                        "gen_ai.operation.name": "invoke_agent"}):
    q = "where is my refund for order 8891"
    with span("chat", **{"gen_ai.operation.name": "chat",
                         "gen_ai.provider.name": "anthropic",
                         "gen_ai.request.model": "toy-model-1",
                         "gen_ai.input.ref": stash(q)}):
        plan = fake_model(q)

    with span("tool_call lookup_order", **{"gen_ai.operation.name": "tool_call",
                                           "gen_ai.data_source.id": "orders_db"}):
        row = lookup_order(8891)

    with span("chat", **{"gen_ai.operation.name": "chat",
                         "gen_ai.provider.name": "anthropic",
                         "gen_ai.request.model": "toy-model-1",
                         "gen_ai.output.ref": stash(fake_model(str(row)))}):
        pass

for s in sorted(SPANS, key=lambda s: s.t0):   # start order = tree order
    print("  " * s.depth + f"{s.name}  [{s.kind}] {s.ms}ms  {s.attrs}")
print("\nExternal store (not in the trace):")
for k, v in STORE.items():
    print(f"  {k} -> {v}")

Look at three things. The indentation shows the parent-child tree, with the chat and tool spans sitting under invoke_agent. The attributes carry only gen_ai.input.ref and gen_ai.output.ref, never the text. And the text shows up only at the bottom, in the separate store.

Where you will see this

• Coding agents like Claude Code and Cursor, where traces show which tool call ate the time.
• Customer-support bots at any scale, where one slow tool quietly ruins the response time.
• Retrieval systems, where gen_ai.data_source.id tells you which document store answered.
• Food-delivery and travel assistants that chain several tools per user question.
• Any company running two model providers at once and needing one dashboard for both.

Common mistakes

• Dumping full prompts into spans. Traces are widely readable, so you just leaked phone numbers and API keys to everyone with dashboard access.
• Forgetting gen_ai.provider.name. Six months later you want to compare providers on cost and latency, and the data is not there.
• Losing the parent link. Tool spans float alone, so you can never say which agent run was slow.
• Skipping the stability opt-in. An upgrade renames attributes and every panel you built goes empty.
• Inventing your own span names anyway. The benefit is that names are shared. Custom names put you back to translating columns by hand.

If they ask in an interview

Q: What are the three span categories in the GenAI conventions?

A: Model or client spans for raw LLM calls, agent spans which are create_agent and invoke_agent, and tool spans, one per tool invocation. Tool spans are children of the agent span, so the trace forms a tree of the whole run.

Q: When is an invoke_agent span CLIENT and when is it INTERNAL?

A: CLIENT when you are calling a remote agent service over the network, like a hosted assistants or Bedrock agents API. INTERNAL when the agent loop runs inside your own process, like LangChain, CrewAI or a ReAct loop you wrote yourself.

Q: Should traces contain the prompt text?

A: By default no. The standard says instrumentation should not capture inputs and outputs unless you opt in. In production the recommended pattern is to store the content in an external store and put only a reference ID on the span, so sensitive data does not sit in traces the whole ops team can read.

Try these

• Write a 40-line tracer of your own around a simple two-tool agent loop. Print the span tree with indentation and check every tool span has the agent span as parent.
• Take that tracer and move the prompts into a small SQLite table. Keep only the row ID on the span. Confirm no prose is left in the printed spans.
• Add gen_ai.data_source.id to a retrieval step over two different local folders of text files, then group your printed spans by that attribute and compare timings.
• Print a small summary table from your spans alone: for each gen_ai.request.model, how many tool calls errored. That is the kind of question standard attributes are meant to answer.

Words, simply

Quick recap

• One shared schema means your dashboards survive a change of framework or provider.
• Three span kinds only: model, agent, tool, and the tool spans hang under the agent span.
• Keep prompt text out of traces; store it elsewhere and put just the reference ID on the span.