Skip to content

Commit 051f271

Browse files
committed
Make OpenTelemetry tracing the single default middleware
The SDK shipped two server-side OpenTelemetry span mechanisms: a dispatch-tier `otel_middleware` wired on by default (span `MCP handle <method>`) and a context-tier `OpenTelemetryMiddleware` the docs told users to append (span `<method>`, plus GenAI attributes). Appending the documented one produced a second, nested span, and the "you must append it" instruction was wrong since a span was already emitted by default. Collapse to one tier: remove `otel_middleware` (and its `__all__` export), default `dispatch_middleware` to empty, and seed `Server.middleware` with `OpenTelemetryMiddleware()` so tracing is on by default through the richer context-tier span. Drop it from the list to opt out. Docs get a dedicated OpenTelemetry page; the middleware page now points to it.
1 parent 5b2713d commit 051f271

12 files changed

Lines changed: 209 additions & 220 deletions

File tree

‎docs/advanced/middleware.md‎

Lines changed: 8 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -79,33 +79,12 @@ In increasing order of how much you should hesitate:
7979
an elicitation) while handling `initialize` therefore **deadlocks the connection**: the
8080
response you are waiting for can never be read. Fire-and-forget notifications are fine.
8181

82-
## `OpenTelemetryMiddleware`
83-
84-
The SDK ships one middleware: `OpenTelemetryMiddleware`. Construct it and append it
85-
(`server.middleware.append(OpenTelemetryMiddleware())`), exactly the line you already wrote
86-
for `log_timing`.
87-
88-
Every inbound message becomes a `SERVER` span named after the method and its target, so a
89-
`tools/call` for `search_books` is the span `tools/call search_books`.
90-
91-
* Every span carries `mcp.method.name` and `mcp.protocol.version`; a request's span also
92-
carries its JSON-RPC request id (a notification has none).
93-
* A `tools/call` span gets OpenTelemetry's GenAI semantic conventions,
94-
`gen_ai.operation.name` (`"execute_tool"`) and `gen_ai.tool.name`, so a tracing UI groups
95-
your tool calls the way it groups any other agent's. A `prompts/get` span gets
96-
`gen_ai.prompt.name`. The list methods carry no `gen_ai.*` keys.
97-
* A handler that raises sets the span's status to error. So does a tool result with
98-
`is_error=True`.
99-
100-
!!! tip
101-
The SDK depends only on `opentelemetry-api`. With no exporter installed those spans are
102-
no-ops, so appending this middleware costs you nothing. Install `opentelemetry-sdk` plus an
103-
exporter and everything lights up, with no server change.
104-
105-
The import is the catch. The class lives at `from mcp.server._otel import OpenTelemetryMiddleware`
106-
today, and the leading underscore is not an accident: it is the same provisional flag this whole
107-
page opened with. The SDK has not given it a public spelling yet, so the import path is the one
108-
line here you should expect to change.
82+
## The one middleware that ships on by default
83+
84+
The SDK ships exactly one middleware, and it is already on your server's list: the one that
85+
emits an OpenTelemetry span for every message. You don't append it, and most of the time you
86+
don't think about it. It is a no-op until you install an exporter, and it has its own page:
87+
**OpenTelemetry**.
10988

11089
!!! info
11190
If you have written ASGI middleware, you already know this shape. Starlette's
@@ -121,9 +100,8 @@ line here you should expect to change.
121100
unknown methods) and runs outermost-first.
122101
* `ctx.request_id is None` is how you tell a notification from a request.
123102
* Raise instead of calling `call_next` to refuse one message; the connection survives.
124-
* `OpenTelemetryMiddleware` turns each message into a span (with GenAI attributes on tool
125-
calls and prompt gets) for the price of one `append`, and costs nothing until you install
126-
an exporter.
103+
* The SDK's own OpenTelemetry tracing is a middleware too, already on the list. See
104+
**OpenTelemetry**.
127105
* The whole surface is provisional. Observe with it; don't build on it.
128106

129107
That is everything that wraps a request. **Authorization** is what decides whether the request

‎docs/advanced/opentelemetry.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# OpenTelemetry
2+
3+
Your server is already traced. You don't have to add anything.
4+
5+
Every server you create emits an [OpenTelemetry](https://opentelemetry.io/) span for every
6+
message it handles. You didn't write that, and you don't import it. It is there the moment you
7+
call `MCPServer(...)`.
8+
9+
```python title="server.py"
10+
--8<-- "docs_src/opentelemetry/tutorial001.py"
11+
```
12+
13+
That is a complete, traced server. Call `search_books` and a span is created for it. The same is
14+
true for the low-level `Server`: the tracing lives on both.
15+
16+
## What you get
17+
18+
Every inbound message becomes a `SERVER` span named after the method and its target. So a
19+
`tools/call` for `search_books` is the span `tools/call search_books`, and a bare `tools/list`
20+
is just `tools/list`.
21+
22+
Each span carries a few attributes:
23+
24+
* `mcp.method.name` and `mcp.protocol.version`, on every span.
25+
* `jsonrpc.request.id`, on a request (a notification has none).
26+
* A handler that raises sets the span status to error. So does a tool result with `is_error=True`.
27+
28+
And because tracing a tool call is such a common thing to want, `tools/call` spans speak
29+
OpenTelemetry's [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/):
30+
31+
* `gen_ai.operation.name`, set to `"execute_tool"`.
32+
* `gen_ai.tool.name`, set to the tool being called.
33+
34+
A `prompts/get` span gets `gen_ai.prompt.name` in the same spirit. The list methods carry no
35+
`gen_ai.*` keys, because there is nothing to name.
36+
37+
!!! tip
38+
Those GenAI attributes are the reason a tracing UI groups your tool calls the way it groups
39+
any other agent's. You get that grouping for free, with no extra code.
40+
41+
## It costs nothing until you want it
42+
43+
Here is the part that makes "on by default" a comfortable default.
44+
45+
The SDK depends only on `opentelemetry-api`, the lightweight half of OpenTelemetry. With no SDK
46+
and no exporter installed, creating a span is a no-op. So the spans your server is emitting right
47+
now cost you almost nothing, and nobody is collecting them.
48+
49+
The day you want to *see* them, you install the other half and point it somewhere:
50+
51+
```console
52+
uv add opentelemetry-sdk opentelemetry-exporter-otlp
53+
```
54+
55+
Configure an exporter the usual OpenTelemetry way, and every span the SDK has been quietly
56+
creating lights up. Your server code does not change. Not one line.
57+
58+
!!! info
59+
[Pydantic Logfire](https://logfire.pydantic.dev/) is one such backend, and it does the
60+
configuration for you: `pip install logfire`, `logfire.configure()`, and your MCP spans show
61+
up in the live view. It is built on OpenTelemetry, so anything below applies to it too.
62+
63+
## Traces that cross the wire
64+
65+
A trace is most useful when it follows a request from the client into the server, in one
66+
connected picture.
67+
68+
When the client and the server both run the SDK, that connection is automatic. The client injects
69+
the [W3C trace context](https://www.w3.org/TR/trace-context/) into the request, and the server
70+
reads it back out, so the server span nests under the client span in the same trace. This is
71+
[SEP-414](https://github.com/modelcontextprotocol/modelcontextprotocol), and you get it without
72+
asking.
73+
74+
If the inbound message carries no trace context, for example a request from a client that is not
75+
the SDK, the server span simply parents to whatever span is already current on the server, rather
76+
than starting a brand-new orphan trace.
77+
78+
## Turning it off
79+
80+
Tracing is a middleware, the first one on your server's list. If you really want a server that
81+
emits no spans, take it off:
82+
83+
```python
84+
from mcp.server._otel import OpenTelemetryMiddleware
85+
86+
mcp._lowlevel_server.middleware[:] = [
87+
m for m in mcp._lowlevel_server.middleware if not isinstance(m, OpenTelemetryMiddleware)
88+
]
89+
```
90+
91+
!!! warning
92+
That import has a leading underscore, and that is on purpose. The class is provisional, the
93+
same way [`Server.middleware`](middleware.md) is provisional, so the import path is something
94+
you should expect to change. You almost never need this: with no exporter installed the spans
95+
are free, so the usual answer is to leave them on and not install an exporter.
96+
97+
## Recap
98+
99+
* Every `MCPServer` and every low-level `Server` emits one `SERVER` span per inbound message, out
100+
of the box. You write nothing.
101+
* Spans carry `mcp.method.name` and `mcp.protocol.version`; `tools/call` and `prompts/get` also
102+
carry GenAI attributes so your tool calls group like any other agent's.
103+
* It costs nothing until you install an OpenTelemetry SDK and an exporter, and then it lights up
104+
with no change to your server.
105+
* Client-to-server trace context propagates automatically when both sides run the SDK.
106+
107+
Next, the thing that decides whether a request runs at all: **Authorization**.

‎docs/tutorial/logging.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,8 +64,8 @@ went to standard error: the terminal, not the wire.
6464

6565
!!! info
6666
If what you actually want is *tracing* (every request, how long it took, whether it failed), you
67-
don't want log lines, you want spans. The SDK ships an `OpenTelemetryMiddleware` for exactly that.
68-
See **Middleware**.
67+
don't want log lines, you want spans. Your server already emits them: the SDK traces every
68+
message with OpenTelemetry out of the box. See **OpenTelemetry**.
6969

7070
## Recap
7171

‎docs_src/opentelemetry/__init__.py‎

Whitespace-only changes.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
from mcp.server import MCPServer
2+
3+
mcp = MCPServer("Bookshop")
4+
5+
6+
@mcp.tool()
7+
def search_books(query: str) -> str:
8+
"""Search the catalog by title or author."""
9+
return f"Found 3 books matching {query!r}."

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ nav:
4242
- The low-level Server: advanced/low-level-server.md
4343
- Pagination: advanced/pagination.md
4444
- Middleware: advanced/middleware.md
45+
- OpenTelemetry: advanced/opentelemetry.md
4546
- Authorization: advanced/authorization.md
4647
- OAuth clients: advanced/oauth-clients.md
4748
- Session groups: advanced/session-groups.md

‎src/mcp/server/lowlevel/server.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ async def main():
5252
from starlette.routing import Mount, Route
5353
from typing_extensions import TypeVar
5454

55+
from mcp.server._otel import OpenTelemetryMiddleware
5556
from mcp.server.auth.middleware.auth_context import AuthContextMiddleware
5657
from mcp.server.auth.middleware.bearer_auth import BearerAuthBackend, RequireAuthMiddleware
5758
from mcp.server.auth.provider import OAuthAuthorizationServerProvider, TokenVerifier
@@ -231,10 +232,13 @@ def __init__(
231232
# Context-tier middleware: wraps every inbound request (including
232233
# `initialize`, lookup, validation, handler) with
233234
# `(ctx, call_next)`. Applied in `ServerRunner._on_request`.
235+
# `OpenTelemetryMiddleware` ships on by default so every server emits a
236+
# SERVER span per message; it is a no-op until an OTel exporter is
237+
# installed. Drop it from this list to opt out.
234238
# TODO(L54): provisional - signature and semantics change with the
235239
# Context/middleware rework (covariant `Context[L]`, outbound seam) before
236240
# v2 final.
237-
self.middleware: list[ServerMiddleware[LifespanResultT]] = []
241+
self.middleware: list[ServerMiddleware[LifespanResultT]] = [OpenTelemetryMiddleware()]
238242
logger.debug("Initializing server %r", name)
239243

240244
_spec_requests: list[tuple[str, type[BaseModel], RequestHandler[LifespanResultT, Any] | None]] = [

‎src/mcp/server/runner.py‎

Lines changed: 4 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -37,15 +37,13 @@
3737
)
3838
from mcp_types import methods as _methods
3939
from mcp_types.version import HANDSHAKE_PROTOCOL_VERSIONS, LATEST_HANDSHAKE_VERSION, LATEST_MODERN_VERSION
40-
from opentelemetry.trace import SpanKind, StatusCode
4140
from pydantic import BaseModel, ValidationError
4241
from typing_extensions import TypeVar
4342

4443
from mcp.server.connection import Connection
4544
from mcp.server.context import CallNext, HandlerResult, ServerMiddleware, ServerRequestContext
4645
from mcp.server.models import InitializationOptions
4746
from mcp.server.session import ServerSession
48-
from mcp.shared._otel import extract_trace_context, otel_span
4947
from mcp.shared._stream_protocols import ReadStream, WriteStream
5048
from mcp.shared.dispatcher import DispatchContext, Dispatcher, DispatchMiddleware, OnNotify, OnRequest
5149
from mcp.shared.exceptions import MCPError
@@ -62,7 +60,6 @@
6260
"ServerRunner",
6361
"aclose_shielded",
6462
"modern_on_request",
65-
"otel_middleware",
6663
"serve_connection",
6764
"serve_loop",
6865
"serve_one",
@@ -91,58 +88,6 @@ def _extract_meta(params: Mapping[str, Any] | None) -> RequestParamsMeta | None:
9188
return None
9289

9390

94-
def otel_middleware(call_next: OnRequest) -> OnRequest:
95-
"""Dispatch-tier middleware that wraps each request in an OpenTelemetry span.
96-
97-
Mirrors the span shape of the existing `Server._handle_request`: span name
98-
`"MCP handle <method> [<target>]"`, `mcp.method.name` attribute, W3C
99-
trace context extracted from `params._meta` (SEP-414), and an ERROR
100-
status if the handler raises.
101-
"""
102-
103-
async def wrapped(
104-
dctx: DispatchContext[TransportContext], method: str, params: Mapping[str, Any] | None
105-
) -> dict[str, Any]:
106-
target: str | None
107-
match params:
108-
case {"name": str() as target}:
109-
pass
110-
case _:
111-
target = None
112-
parent: Any | None
113-
match params:
114-
case {"_meta": {**meta}}:
115-
parent = extract_trace_context(meta)
116-
case _:
117-
parent = None
118-
span_name = f"MCP handle {method}{f' {target}' if target else ''}"
119-
# `otel_middleware` wraps `on_request` only, so `request_id` is always set.
120-
attributes = {"mcp.method.name": method, "jsonrpc.request.id": str(dctx.request_id)}
121-
with otel_span(
122-
span_name,
123-
kind=SpanKind.SERVER,
124-
attributes=attributes,
125-
context=parent,
126-
record_exception=False,
127-
set_status_on_exception=False,
128-
) as span:
129-
try:
130-
return await call_next(dctx, method, params)
131-
except MCPError as e:
132-
span.set_status(StatusCode.ERROR, e.error.message)
133-
raise
134-
except ValidationError:
135-
# Mirror the sanitized wire response; pydantic messages carry client input.
136-
span.set_status(StatusCode.ERROR, "Invalid request parameters")
137-
raise
138-
except Exception as e:
139-
span.record_exception(e)
140-
span.set_status(StatusCode.ERROR, str(e))
141-
raise
142-
143-
return wrapped
144-
145-
14691
def _dump_result(result: Any) -> dict[str, Any]:
14792
if result is None:
14893
return {}
@@ -196,7 +141,10 @@ class ServerRunner(Generic[LifespanT]):
196141
_: KW_ONLY
197142
init_options: InitializationOptions | None = None
198143
"""`InitializeResult` payload. Defaults to `server.create_initialization_options()`."""
199-
dispatch_middleware: Sequence[DispatchMiddleware] = (otel_middleware,)
144+
dispatch_middleware: Sequence[DispatchMiddleware] = ()
145+
"""Raw dispatch-tier wrappers `(dctx, method, params) -> dict`, applied outermost-first
146+
around `_on_request`. Empty by default; OpenTelemetry tracing lives at the context tier
147+
(`OpenTelemetryMiddleware`, seeded into `Server.middleware`)."""
200148

201149
@cached_property
202150
def on_request(self) -> OnRequest:
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
"""`docs/advanced/opentelemetry.md`: every claim the page makes, proved against the real SDK."""
2+
3+
import pytest
4+
from logfire.testing import CaptureLogfire
5+
6+
from docs_src.opentelemetry import tutorial001
7+
from mcp import Client
8+
9+
# See test_index.py for why this is a per-module mark and not a conftest hook.
10+
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]
11+
12+
13+
async def test_a_plain_server_is_traced_with_no_extra_code(capfire: CaptureLogfire) -> None:
14+
"""tutorial001: calling a tool emits a `tools/call` SERVER span, though the example adds no middleware."""
15+
async with Client(tutorial001.mcp) as client:
16+
await client.call_tool("search_books", {"query": "dune"})
17+
18+
spans = {s["name"]: s for s in capfire.exporter.exported_spans_as_dict()}
19+
assert "tools/call search_books" in spans
20+
21+
attributes = spans["tools/call search_books"]["attributes"]
22+
assert attributes["mcp.method.name"] == "tools/call"
23+
assert attributes["gen_ai.operation.name"] == "execute_tool"
24+
assert attributes["gen_ai.tool.name"] == "search_books"
25+
26+
27+
async def test_client_and_server_share_one_trace(capfire: CaptureLogfire) -> None:
28+
"""When both sides run the SDK, the client and server spans land in one trace (SEP-414)."""
29+
async with Client(tutorial001.mcp, mode="legacy") as client:
30+
await client.call_tool("search_books", {"query": "dune"})
31+
32+
spans = {s["name"]: s for s in capfire.exporter.exported_spans_as_dict()}
33+
client_span = spans["MCP send tools/call search_books"]
34+
server_span = spans["tools/call search_books"]
35+
assert server_span["context"]["trace_id"] == client_span["context"]["trace_id"]

0 commit comments

Comments
 (0)