[OPIK-8252] Answer 401 invalid_token on an expired OAuth token so hosts refresh - #182
Merged
Merged
Conversation
Hosted-connector users lost the Opik MCP connector about an hour after
connecting: the access token expired, opik-backend answered 401 on the data
call, and opik-mcp wrapped that into a tool error inside an HTTP 200. The MCP
host never saw a 401 on the MCP request, so it never ran the refresh_token
grant it was perfectly able to run, and Claude kept showing a "connected"
connector that could not make a single call.
opik-mcp is the OAuth resource server here, and the MCP authorization spec
(2026-07-28, Token Handling) is explicit: validate the access token, answer
HTTP 401 for an invalid or expired one. So BearerAuthMiddleware now validates
every opik_mcp_at_-prefixed bearer on the MCP path against opik-backend's
introspection endpoint before forwarding, and answers
401 WWW-Authenticate: Bearer realm="opik-mcp", error="invalid_token",
error_description="...", resource_metadata="..."
when the backend says the token is dead. That is the RFC 6750 signal hosts
key their refresh on. Introspection is a three-way outcome now (valid /
invalid / unknown) instead of "identity or None": a network error or 5xx is
"unknown" and fails open, so a backend hiccup degrades to "forward as before"
rather than a mass logout. The session-creating request still makes exactly
one call, which both validates and feeds the instructions blob and analytics.
API-key bearers are untouched. A token whose bound resource differs from
OPIK_MCP_RESOURCE_URI is logged at warning level (strict RFC 8707 rejection
is a follow-up, once production confirms the two agree).
No cache yet — that is the next commit — so every request round-trips.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQuKKW95kFTeHAW3jVyTj
… bearer When opik-backend answered 401 to a data call the tool error said "Check OPIK_API_KEY and OPIK_WORKSPACE", and the read tool wrapped it as "Permission denied ... the current workspace may not have access". Under the hosted OAuth connector neither is true: the access token expired. Users went hunting for API keys, and the model told them to reconnect the connector — for a recovery the client performs on its own the moment it sees the 401 that the previous commit made the retry run into. One hint, one source of truth (auth_context.oauth_token_expired_hint), keyed on the bearer this request is forwarding: an OAuth token gets "The Opik access token is expired or revoked. Retry this call — the MCP client refreshes the token on the next request."; an API key keeps the old wording. The read tool now reserves "permission denied" for 403, where it is true. The write envelope (BackendError) appends the same hint to its 401. This still matters after validation: a token that dies inside the validation-cache window reaches the backend once, and that one error has to point the model at a retry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQuKKW95kFTeHAW3jVyTj
…ires_at Validating every request must not double the load on opik-backend. A "valid" introspection answer is now remembered per token digest in the existing bounded credential store and trusted for OPIK_MCP_OAUTH_VALIDATION_CACHE_TTL_S (default 30 s); inside that window the backend is not asked again, after it the next request re-validates. Only positive answers are cached, so a freshly refreshed token works on first use. Two things shrink the window in which a token that died mid-cache is still forwarded. When a data call comes back 401 under an OAuth bearer the entry is dropped on the spot (auth_context.note_backend_401, called from the one place each rendering layer already handles that 401), so the very next MCP request gets the invalid_token 401 that triggers the host's refresh — now, not after the TTL. And when the backend reports expires_at (opik-backend gains that field in a companion PR; older releases simply don't send it) the entry is capped at that instant minus a 10 s skew margin, and a request landing after the expiry is answered 401 straight from the cache. A second handshake on a cached token (host reconnect) reads the identity remembered at first validation, so the instructions blob still names the workspace without a second round-trip. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQuKKW95kFTeHAW3jVyTj
…ty wrapper Addresses the two-axis review of the previous three commits. Layering. auth_context had grown an import of credential_identity and a note_backend_401() that evicted a process-wide cache from inside a message helper — and, through BackendError.build, from inside an error envelope. The hint is pure again and stays in auth_context; the eviction now lives beside the HTTP call it belongs to (opik_client.note_backend_401, called from _raise_for_status for reads and from the write finalizer for writes). Spec gap. A 200 from introspection whose JSON was not an object was treated as "valid" and cached for the full TTL. A malformed answer is no answer: it is "unknown" now, fails open, and caches nothing. Covered at the HTTP seam alongside the 5xx and network cases. Smaller review points: resolve_oauth_identity, a one-line delegate with no production caller, is gone and its tests speak to introspect_oauth_token directly; the cache entry is a named tuple instead of an anonymous pair; the cache verdict uses the same "invalid" word as the introspection status so the middleware switches on one vocabulary; the ~70 lines of OAuth policy inline in dispatch() are a _validate_oauth_bearer method; the audience warning compares the resource exactly as configured (a trailing-slash difference is precisely what a strict check would trip on, so the warning must show it); stale comments that still described handshake-only introspection are brought up to date; and the pre-existing asymmetry in what the three credential stores key on is written down next to them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQuKKW95kFTeHAW3jVyTj
…the cache TTL The transport section still said opik-mcp performs no local credential validation. Since OPIK-8252 that is true only of API keys; OAuth access tokens are validated against opik-backend on every request and answered with an invalid_token 401 when dead. Adds the new OPIK_MCP_OAUTH_VALIDATION_CACHE_TTL_S setting to the environment table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2 tasks
An "unknown" introspection outcome forwards the request UNVALIDATED. That was logged at DEBUG, so at the default INFO level a resource server that cannot reach its introspection endpoint looked identical to one whose tokens simply expire inside the cache window — exactly the two cases we could not tell apart on dev when an expired token got a tool error twice in a row instead of the invalid_token 401. Each fail-open path now logs one WARNING naming the endpoint and the cause (status, exception class, or body type); the "invalid" path stays quiet, that one is routine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…not the handshake's Found on dev and staging: after the token expired, the host did refresh, yet every tool call kept failing with the backend's 401 until the session was re-initialised. A tool runs in the MCP session task, which the SDK forks from the `initialize` request, so `inbound_authorization` inside a tool still held the handshake-time bearer: the middleware validated the NEW token, the outbound client forwarded the OLD, dead one. Wrap the lowlevel CallToolRequest handler (same in-place swap as install_tools_listed_emitter) to re-bind `inbound_authorization` and `inbound_workspace` from the Starlette request the SDK attaches to its request context. Each message is handled in its own task, so the set/reset is isolated. stdio has no request and is left untouched. Regression test drives initialize with token A and tools/call with token B over the real ASGI app and asserts the data call carries B (it carried A before). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alexkuzmik
approved these changes
Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
Hosted-connector users lost the Opik MCP connector about an hour after connecting: the OAuth access token expired, opik-backend answered 401 on the data call, and opik-mcp wrapped that into a tool error inside an HTTP 200. The MCP host never saw a 401 on the MCP request, so it never ran the
refresh_tokengrant it holds a valid refresh token for, and Claude kept showing a "connected" connector that could not make a single call. Reconnecting in account settings did not help an open conversation.opik-mcp is the OAuth resource server here, and the MCP authorization spec (2026-07-28, Token Handling) is explicit: validate the access token on every request, answer HTTP 401 for an invalid or expired one.
BearerAuthMiddlewarevalidates everyopik_mcp_at_…bearer on the MCP path against opik-backend'sPOST /opik/auth-oauthbefore forwarding. A definite 401 from introspection becomes401+WWW-Authenticate: Bearer realm="opik-mcp", error="invalid_token", error_description="…", resource_metadata="…"(RFC 6750 §3.1, the code hosts key their refresh on). Missing/malformedAuthorizationkeeps its current shape.valid/invalid/unknown). Network error, timeout, 5xx or a malformed body areunknownand fail open, so a backend hiccup degrades to "forward as before", never to a mass logout. Each fail-open is logged once at WARNING with the cause, so an unreachable introspection endpoint is distinguishable from ordinary expiry at the default log level.OPIK_MCP_OAUTH_VALIDATION_CACHE_TTL_S(default 30 s), capped at the backend'sexpires_atminus a 10 s skew margin when reported. Negative answers are never cached. A 401 from a data call evicts the entry so the next MCP request re-validates immediately.tools/call, not the handshake's. Tools run in the MCP session task, which the SDK forks frominitialize, so the inbound-auth ContextVars inside a tool held handshake-time values. Harmless while a token never changed mid-session; fatal once the 401 above makes hosts refresh: the middleware validated the new token and the outbound client forwarded the old one, so every call after a refresh met the backend's 401.install_request_auth_rebindingre-binds the vars from the SDK's per-request context on eachtools/call; stdio is untouched. Found on staging in the end-to-end test, reproduced bytests/test_oauth_token_rotation.py.resourcevsOPIK_MCP_RESOURCE_URImismatch is logged at warning level only. Strict RFC 8707 audience rejection is a follow-up once production confirms the two agree.Companion PR comet-ml/opik#8153 adds
expires_atto the introspection response (additive); this PR reads it when present and falls back to the TTL otherwise. Either can deploy first.Follow-up, not in this PR:
inbound_mcp_session_idis frozen in the session task the same way, so after a refresh the analytics session pairing keyed on the new header misses. Telemetry only.Change checklist
Issues
Testing
uv run pytest -q→ 1222 passed.uv run ruff check src tests,uv run ruff format --check .,uv run mypyclean.tests/test_oauth_token_validation.pydrives the real ASGI app over HTTP with opik-backend mocked by respx at the introspection and data endpoints: expired token → 401 with theinvalid_tokenchallenge; 5xx / connection error / non-JSON 200 / non-object 200 fail open; API-key bearer never introspected; handshake makes one call and names the workspace; cache honours TTL, is evicted on upstream 401, is capped byexpires_atand answers 401 from cache after expiry; OAuth-mode vs API-key 401 wording; 403 keeps the permission wording.tests/test_oauth_token_rotation.py:initializewith token A,tools/callwith token B → the data call carries B. Failed before the rebinding commit with A on the wire.staging.dev.comet.com, backend withexpires_at): live OAuth handshake names the workspace,list(project)works, dead token →401 invalid_token, noAuthorization→ challenge withouterror=. After the access token expired, the build without the rebinding fix reproduced the user-reported failure on retry; the fixed image (sha-13f3bd4c902b) is the one to deploy for the post-expiry check.grant_type=refresh_tokenin opik-backend logs after the first hour of a Claude.ai session, and no tool error on the following call.Documentation
README transport section now states the two-bearer contract and documents
OPIK_MCP_OAUTH_VALIDATION_CACHE_TTL_S.🤖 Generated with Claude Code