Document the client response cache

Author: maxisbey

docs/advanced/caching.md

@@ -37,19 +37,63 @@ One caveat on paginated lists: the protocol requires the **same `cacheScope` on
 
 ## What the client sees
 
-On the client, the hints arrive as plain fields on every cacheable result — `ttl_ms` and `cache_scope`, already parsed:
+On a 2026-07-28 session, `Client` honors the hints for you: it has a built-in response cache, on by default. A result that arrives carrying a `ttlMs` is stored, and an identical call within that TTL is served from the cache — no round trip. A result that carries *no* hint is not cached: hint-less results get `CacheConfig.default_ttl_ms`, which defaults to `0` (immediately stale), so a server that declares nothing sees exactly the call-for-call traffic it always did.
 
-```python title="client.py" hl_lines="15"
+```python title="client.py" hl_lines="28 30 33"
 --8<-- "docs_src/caching/tutorial003.py"
 ```
 
-The SDK parses; it does not (yet) act. There is no built-in response cache: calling `list_tools()` twice makes two round trips, whatever the TTL said. The spec makes honoring optional — a client that ignores the hints entirely is fully conformant — so until the SDK grows a response cache, the supported path is to read the fields and do your own bookkeeping:
+Four calls, three fetches. The second call found a fresh entry and never reached the server; advancing the (injected) clock past the TTL made the third fetch again; the fourth said `cache_mode="refresh"`. That kwarg exists on the five caching verbs — `list_tools`, `list_prompts`, `list_resources`, `list_resource_templates`, `read_resource`:
 
-* **Freshness** is `now < t_received + ttl_ms / 1000`: record the clock when the response arrives, and treat the result as reusable until the TTL runs out. `ttl_ms == 0` means *immediately stale* — don't reuse it at all.
-* **Scope is a sharing rule, not a suggestion.** A `"private"` result may be reused only within the same authorization context — same access token, same cache. Never put `"private"` results in a cache shared across users.
-* **Notifications beat TTL.** If the server sends `list_changed` while your copy is still fresh, the copy is stale now — re-fetch.
+* `"use"` (the default) serves a fresh entry if there is one, and stores the fetch if not.
+* `"refresh"` never serves: it fetches and stores the result, replacing whatever was cached.
+* `"bypass"` makes the round trip without touching the cache at all — no read, no write.
 
-Against an **older server** (pre-2026 protocol), the fields are simply absent from the wire, and the models show their conservative defaults: `ttl_ms == 0`, `cache_scope == "private"` — stale and unshared, the right assumption for a server that declared nothing. If you need to distinguish "the server said 0" from "the server said nothing", check `"ttl_ms" in result.model_fields_set`: it's only set when the field actually arrived.
+To turn caching off entirely, construct with `Client(server, cache=False)`: every call is a round trip again, and `cache_mode`, while still accepted, does nothing.
+
+Scope is honored automatically too — `"private"` entries are keyed to the cache's *partition* (below); `"public"` ones may opt into wider sharing — and **notifications beat TTL**: a `list_changed` notification evicts the matching cached listing, and `resources/updated` evicts the cached read for its URI, however fresh they were.
+
+### Configuring it: `CacheConfig`
+
+```python
+from mcp.client import CacheConfig
+
+client = Client("https://api.example.com/mcp", cache=CacheConfig(default_ttl_ms=5_000))
+```
+
+* `store` — where entries live. The default is a fresh in-memory store per client; pass your own `ResponseCacheStore` implementation (Redis-backed, say) to share a cache across clients or processes. A custom store **requires** an explicit `partition`.
+* `partition` — the authorization-context label that keeps one principal's `"private"` entries from being served to another within a shared store.
+* `target_id` — explicit server identity, for custom transports and in-process servers (below).
+* `default_ttl_ms` — TTL applied to results that carry no `ttlMs` hint. The default `0` leaves hint-less results uncached.
+* `share_public` — serve server-asserted-`"public"` entries across partitions (below). Off by default.
+* `clock` — the wall-clock source, epoch seconds. Inject one, as the example above does, and expiry tests need no sleeping.
+
+!!! warning "Partition = verified principal"
+    Derive `partition` from a **verified credential** — a validated token's subject, for example. Never from request-supplied data, and never from the server URL (server identity is a separate key axis). The SDK is a library with no authentication of its own: whoever constructs the `CacheConfig` — the deployment, not the tenant — is the trust anchor. A multi-tenant gateway mints one `CacheConfig` per authenticated principal.
+
+    The partition is also fixed for the `Client`'s lifetime. If the connection's authorization context changes mid-session — a re-authentication as a different principal, say — the cache does not follow; construct a new `Client` for the new principal.
+
+Cache keys also carry the **server's identity**: the URL string you dialed, with any `user:pass@` userinfo stripped and otherwise byte-exact. No case folding, no query reordering, no trailing-slash cleanup — under-normalizing only costs sharing, while over-normalizing could merge two tenants (`?tenant=a` vs `?tenant=b`), so superficially different URLs simply don't share entries. When there is no URL — an in-process server, or a `Transport` instance — the client gets a random per-instance identity instead; set `CacheConfig.target_id` to name the server (with a custom store this is required, and construction says so). The identity is sha256-hashed before it enters key material, so a URL carrying secrets in its query string never appears in store keys — don't log the pre-hash form yourself, either.
+
+!!! warning "`share_public` trusts the server, fleet-wide"
+    By default even `"public"` entries stay within their partition. `share_public=True` serves entries the server marked `cacheScope: "public"` to **every** partition using the store — trusting the server's classification on behalf of all of them. A server that stamps `"public"` on per-tenant data (by bug or by malice) then leaks one tenant's response to the others. The flag is deliberately constructor-level only: the per-call `cache_mode` can narrow caching, but nothing per-call can widen sharing.
+
+### What the cache never does
+
+* **Session-tier calls bypass it.** `client.session.list_tools()` and friends always make the round trip; the cache lives on the `Client` verbs.
+* **`server/discover` stays out of it.** The discover result is delivered once, at connect, and never enters the response cache — even when it carries a `ttlMs`. If you persist one yourself to skip the reconnect probe ([`prior_discover`](../client/protocol-versions.md#reconnecting-with-prior_discover)), its freshness is your bookkeeping: `DiscoverResult` carries `ttl_ms` and `cache_scope`, already parsed, for exactly that purpose.
+* **Continuation pages are never cached.** Only cursor-less calls participate. A continuation page rejected for an expired cursor does *evict* the cached listing — the listing changed under it.
+* **Multi-round-trip reads are never cached.** A `read_resource` seeded with `input_responses`/`request_state`, or one that resolves through input rounds, never enters the cache (a spec MUST).
+* **Notification eviction needs notifications.** Eviction is only as good as the transport's delivery — the modern in-process path (`Client(server)` with the default `mode="auto"`) does not deliver standalone notifications today.
+* **No stale-if-error.** An expired entry is never served because the refetch failed; the error propagates.
+* **No coalescing.** Two concurrent identical calls are two fetches.
+* On a **shared persistent store**, a session that negotiated a different protocol era than the entry's writer may be served the writer's entry until TTL or eviction — accepted, and bounded by the cache's 24-hour TTL cap.
+
+### Reading the hints yourself
+
+The hints are also plain fields on every cacheable result — `result.ttl_ms` and `result.cache_scope`, already parsed — if you want to layer your own bookkeeping on top of (or instead of) the built-in cache.
+
+Against an **older server** (pre-2026 protocol), the fields are simply absent from the wire, and the models show their conservative defaults: `ttl_ms == 0`, `cache_scope == "private"` — stale and unshared, the right assumption for a server that declared nothing. The cache treats a legacy session the same way: hints are never consulted there (whatever keys appear on the wire), only `default_ttl_ms` applies, and its default of `0` caches nothing — a pre-2026 connection behaves exactly as it did before the cache existed. If you need to distinguish "the server said 0" from "the server said nothing", check `"ttl_ms" in result.model_fields_set`: it's only set when the field actually arrived.
 
 ## Older clients
 
@@ -61,4 +105,5 @@ Clients on pre-2026 protocol versions never see either field — the SDK strips
 * `cache_hints={method: CacheHint(...)}` at construction (both `MCPServer` and `Server`) sets server-wide values per method.
 * A handler that sets the fields on its result overrides the map, per field.
 * `"public"` is a promise that the result is identical for every caller. It is not access control.
-* Clients read the hints as `result.ttl_ms` / `result.cache_scope` and own the caching decision themselves — the SDK has no built-in response cache yet.
+* `Client` honors the hints automatically: its response cache is on by default, serves fresh entries instead of refetching, and caches nothing for servers (or sessions) that provide no hints.
+* Per call, `cache_mode="refresh"` refetches and `"bypass"` skips the cache; `cache=False` at construction turns it off entirely.

docs/migration.md

@@ -427,6 +427,10 @@ On `ClientSession`, `call_tool` / `get_prompt` / `read_resource` still return th
 
 For protocol 2026-07-28 over Streamable HTTP, a tool's input-schema property may carry an `x-mcp-header` annotation. When a tool the client has listed is called, each annotated argument is mirrored into an `Mcp-Param-<name>` request header (string verbatim, integer as decimal, boolean as `true`/`false`, base64-sentinel-wrapped when not header-safe; `null`/absent arguments are omitted). The argument is also left in the request body. `list_tools` caches a tool's annotations, so list a tool before calling it to enable mirroring; a tool the client never listed emits no `Mcp-Param-*` headers. Other transports ignore the annotation.
 
+### `Client` verbs may serve cached responses ([SEP-2549](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2549))
+
+On protocol 2026-07-28, servers attach caching hints (`ttlMs`, `cacheScope`) to the cacheable results, and `Client` now honors them: `list_tools`, `list_prompts`, `list_resources`, `list_resource_templates`, and `read_resource` may serve a cached response instead of making a round trip, for as long as the server's `ttlMs` says the result is fresh. Servers that send no hints — including every pre-2026 server — see identical call-for-call behavior, because hint-less results are not cached. Pass `Client(..., cache=False)` to disable the cache and restore v1 behavior exactly; per-call control (`cache_mode`) and configuration (`CacheConfig`) are described in [Caching hints](advanced/caching.md).
+
 ### Server extensions API ([SEP-2133](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2133))
 
 `MCPServer` now accepts opt-in extensions that bundle MCP behaviour behind a

docs_src/caching/tutorial003.py

@@ -1,15 +1,34 @@
+from typing import Any
+
+from mcp_types import ListToolsResult, PaginatedRequestParams, Tool
+
 from mcp import Client
-from mcp.server import CacheHint, MCPServer
+from mcp.client import CacheConfig
+from mcp.server import CacheHint, Server, ServerRequestContext
+
+fetches = 0
+now = 1_000_000.0
+
 
-mcp = MCPServer("Weather", cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")})
+async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams | None) -> ListToolsResult:
+    global fetches
+    fetches += 1
+    return ListToolsResult(tools=[Tool(name="forecast", input_schema={"type": "object"})])
 
 
-@mcp.tool()
-def forecast(city: str) -> str:
-    return f"Sunny in {city}"
+server = Server(
+    "Weather",
+    on_list_tools=list_tools,
+    cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")},
+)
 
 
 async def main() -> None:
-    async with Client(mcp) as client:
-        tools = await client.list_tools()
-        print(f"{len(tools.tools)} tools, fresh for {tools.ttl_ms / 1000:.0f}s, scope={tools.cache_scope}")
+    global now
+    async with Client(server, cache=CacheConfig(clock=lambda: now)) as client:
+        await client.list_tools()  # fetch 1
+        await client.list_tools()  # fresh for 60s: served from the cache
+        now += 60.0
+        await client.list_tools()  # the TTL ran out: fetch 2
+        await client.list_tools(cache_mode="refresh")  # skip the cache read: fetch 3
+        print(f"4 calls, {fetches} fetches")

src/mcp/client/caching.py

@@ -137,6 +137,11 @@ class CacheConfig:
     authentication of its own: whoever constructs the `CacheConfig` - the
     deployment, not the tenant - is the trust anchor. Multi-tenant gateways
     mint one `CacheConfig` per authenticated principal.
+
+    The partition is fixed for the `Client`'s lifetime: if the connection's
+    authorization context changes mid-session (a re-authentication as a
+    different principal), the cache does not follow - construct a new
+    `Client` for the new principal.
     """
 
     target_id: str | None = None

tests/client/test_client_caching.py

@@ -6,6 +6,7 @@
 The coordinator's own behavior is covered in `test_caching.py`.
 """
 
+import hashlib
 import json
 import time
 from collections.abc import AsyncIterator
@@ -148,6 +149,16 @@ def test_userinfo_variants_of_a_server_url_share_one_cache_identity() -> None:
     assert _private_arm(bare) == _private_arm(with_password) == _private_arm(with_token)
 
 
+def test_the_server_url_is_sha256_hashed_before_it_enters_key_material() -> None:
+    """The arm carries sha256(url-sans-userinfo), not the URL itself, so a secret
+    in the query string never appears in store keys. SDK-defined; pins the docs'
+    secrets-never-in-keys claim — raw-URL key material would fail here."""
+    client = Client("https://user:pass@example.com/mcp?api_key=SECRET")
+
+    arm_id = hashlib.sha256(b"https://example.com/mcp?api_key=SECRET").hexdigest()
+    assert _private_arm(client) == json.dumps(["private", arm_id, ""])
+
+
 def test_urls_differing_only_in_query_have_distinct_cache_identities() -> None:
     """URL identity is byte-exact outside userinfo — `?tenant=a` and `?tenant=b`
     must never share entries (over-normalization would merge tenants). SDK-defined."""
@@ -966,6 +977,56 @@ async def on_message(message: IncomingMessage) -> None:
     assert reads == [uri, uri]
 
 
+async def test_the_modern_in_process_path_drops_the_eviction_notification() -> None:
+    """Pins the documented transport gap: the default in-process connection
+    (mode="auto", DirectDispatcher) does not deliver standalone server notifications,
+    so a tools/list_changed emitted mid-call never reaches the cache - the warm entry
+    survives and the next `list_tools` is still served from it. Delivery on this path
+    would happen inline within the awaited `call_tool`, so asserting after it returns
+    is race-free. If this test starts failing, the path gained delivery: flip the
+    `docs/advanced/caching.md` eviction caveat and the legacy-mode notification tests."""
+    fetches: list[str | None] = []
+
+    async def list_tools(ctx: ServerRequestContext, params: types.PaginatedRequestParams | None) -> ListToolsResult:
+        fetches.append(params.cursor if params is not None else None)
+        return ListToolsResult(tools=[Tool(name="touch", input_schema={"type": "object"})])
+
+    async def call_tool(ctx: ServerRequestContext, params: types.CallToolRequestParams) -> CallToolResult:
+        assert params.name == "touch"
+        await ctx.session.send_tool_list_changed()
+        return CallToolResult(content=[TextContent(text="ok")])
+
+    server = Server(
+        "notify",
+        on_list_tools=list_tools,
+        on_call_tool=call_tool,
+        cache_hints={"tools/list": CacheHint(ttl_ms=60_000)},
+    )
+
+    async with Client(server, cache=CacheConfig(clock=_ManualClock())) as client:
+        await client.list_tools()
+        await client.call_tool("touch", {})
+        await client.list_tools()  # still served from the warm entry: no eviction arrived
+
+    assert fetches == [None]
+
+
+async def test_a_discover_result_never_enters_the_response_cache() -> None:
+    """SDK ruling (documented): the response cache covers the five `Client` verbs
+    only. The connect-time server/discover result is never stored, even when it
+    carries a `ttlMs` hint - a persisted `prior_discover`'s freshness is the user's
+    bookkeeping (`DiscoverResult` carries the parsed hints for it)."""
+    server = Server("hinted", cache_hints={"server/discover": CacheHint(ttl_ms=60_000)})
+
+    async with Client(server, cache=CacheConfig(clock=_ManualClock())) as client:
+        discover = client.session.discover_result
+        assert discover is not None
+        assert discover.ttl_ms == 60_000  # the hint arrived with the probe result...
+        store = _coordinator(client)._store
+        assert isinstance(store, InMemoryResponseCacheStore)
+        assert store._entries == {}  # ...and nothing entered the cache
+
+
 # --- The inbound ttlMs clamp (parse seam) ---