diff --git a/CHANGELOG.md b/CHANGELOG.md index f593dc322..b1374dbf1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,24 @@ Releases follow [Semantic Versioning](https://semver.org/). ### Features +- **mcp:** schema-deferred direct mode — a new `direct_tool_response_mode` key (`full` | `deferred`, + **default `full`, so this is opt-in and changes nothing until you turn it on**). In `deferred`, + the direct enumeration surface (`/mcp/all`, and `/mcp` under `routing_mode: "direct"`) lists every + tool with its name, description, annotations and a compact Spec-085 signature, advertising + `inputSchema` as exactly `{"type":"object"}` and dropping `outputSchema`; agents recover a full + schema on demand via `describe_tool`, and a call whose arguments do not fit is rejected + **before** it reaches the upstream, with the schema attached so the agent can self-correct. + Hot-reloadable — flipping it rebuilds the listing and notifies connected clients, no restart. + **Measured token saving is ~30%** (29.7% on the 45-tool reference corpus, 34.8% on a 527-tool + snapshot) — materially less than the ~88% originally projected, because names, descriptions and + annotations, not schemas, dominate the payload. (spec 102, [#971](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/971)) +- **settings:** both serialization modes are now reachable from the UI — Settings → General → + "Detail in tool-search results" (`tool_response_mode`) and "Detail in Direct-mode listings" + (`direct_tool_response_mode`), in the Web UI and the macOS tray. They previously had four ways to + set them (config file, env var, serve flag, REST) and no UI at all, so a tray-only user could not + reach `direct_tool_response_mode`. Neither carries a restart badge — both hot-reload, unlike + `routing_mode` above them. + - **mcp:** `describe_tool` check mode — an optional `check: true` returns one availability verdict per id (up to 50) from the spec-098 preflight evaluator instead of schemas, so an agent can gate a multi-step plan without leaving the MCP session. Optional `filters` @@ -31,6 +49,24 @@ Releases follow [Semantic Versioning](https://semver.org/). ### Bug Fixes +- **mcp/describe_tool:** on the direct surface, the returned definition named the tool + `:` and recommended a `call_with` intent variant — neither of which that surface + can be asked for, so an agent following the response verbatim got "tool not found". It now + echoes the registered `__` name and omits `call_with` (the permission signal + remains in `annotations`, which it was derived from). Both id spellings still work as input. + ([#1083](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1083)) +- **api:** a config value the validator rejects now returns **400** with the offending field in a + structured `validation_errors` payload, instead of 500. 500 is reserved for genuine failures to + persist a valid config. ([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084)) +- **api:** `/api/v1/status` embedded a status snapshot whose `upstream_stats` and `tools_indexed` + were never refreshed — reporting `"Connecting"` and `0` for the life of a process whose servers + were long ready, while the sibling top-level field was correct. Both now come from one live + source, on the REST poll and on the SSE stream. + ([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084)) +- **logging:** ~97 log calls in the REST API passed key/value pairs to a sugared logger method that + concatenates them, producing runs like `patcherrorconfiguration...` instead of structured fields. + ([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084)) + - **homebrew:** One-line install in docs + guard tap job against pre-release tags (#486) ([#486](https://github.com/smart-mcp-proxy/mcpproxy-go/pull/486)) ([`1098701`](https://github.com/smart-mcp-proxy/mcpproxy-go/commit/109870116fe17aa1ce7ecfd603962c1d3de21ba0)) ### CI/Build diff --git a/ROADMAP.md b/ROADMAP.md index 999d6943b..422ec88a7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -726,7 +726,7 @@ graph LR
🟢 Deferred-schema serialization for the direct tools/list surface (spec 102) — Done · P1 -> Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema (~88% smaller listing), with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% PROJECTION ABOVE: 29.7% on the frozen reference corpus and 34.8% on a 527-tool snapshot, against SC-001's >=70% target, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 is knowingly unmet and awaits a maintainer decision (restate per corpus shape, or re-target at a schema-heavy corpus) — see specs/102-schema-deferred/tasks.md T035a. Unblocks token-bench — and that measured shortfall is now the first thing token-bench has to explain. +> Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema, with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082; #1083/#1084 fixed in #1086. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% ORIGINALLY PROJECTED: 29.7% on the frozen 45-tool reference corpus and 34.8% on a 527-tool snapshot, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 was RESTATED per corpus shape (maintainer decision 2026-08-29): now >=25% on the 45-tool corpus and >=30% at fleet scale, both asserted in internal/server/mcp_routing_deferred_tokens_test.go, with the original 70% kept as an upper tripwire. Unblocks token-bench — and that measured shortfall is the first thing token-bench has to explain. Spec: [102-schema-deferred](./specs/102-schema-deferred/) · PR: #1063 diff --git a/internal/httpapi/config_apply_error_status_test.go b/internal/httpapi/config_apply_error_status_test.go new file mode 100644 index 000000000..e2e6b4609 --- /dev/null +++ b/internal/httpapi/config_apply_error_status_test.go @@ -0,0 +1,102 @@ +package httpapi + +import ( + "bytes" + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "go.uber.org/zap" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" + runtime "github.com/smart-mcp-proxy/mcpproxy-go/internal/runtime" +) + +// #1084: PATCH /api/v1/config answered 500 for a value the config validator +// rejected — e.g. {"direct_tool_response_mode":"bogus"}. The behaviour was safe +// (the file was left untouched) but the status class was wrong, and the status +// class is what tells a client whether retrying could ever help. +type applyErrController struct { + baseController + result *runtime.ConfigApplyResult + err error +} + +func (m *applyErrController) GetCurrentConfig() any { return &config.Config{APIKey: "k"} } +func (m *applyErrController) GetConfig() (*config.Config, error) { + return &config.Config{Listen: "127.0.0.1:8080"}, nil +} +func (m *applyErrController) GetConfigPath() string { return "/tmp/mcp_config.json" } +func (m *applyErrController) ApplyConfig(*config.Config, string) (*runtime.ConfigApplyResult, error) { + return m.result, m.err +} + +func patchConfigWith(t *testing.T, ctrl *applyErrController, body map[string]any) *httptest.ResponseRecorder { + t.Helper() + srv := NewServer(ctrl, zap.NewNop().Sugar(), nil) + raw, err := json.Marshal(body) + require.NoError(t, err) + req := httptest.NewRequest(http.MethodPatch, "/api/v1/config", bytes.NewReader(raw)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-API-Key", "k") + w := httptest.NewRecorder() + srv.ServeHTTP(w, req) + return w +} + +// A rejected value is the operator's mistake: 400, with the offending field +// named in a structured payload rather than only inside the prose message. +func TestPatchConfig_RejectedValueIs400WithValidationErrors(t *testing.T) { + w := patchConfigWith(t, &applyErrController{ + result: &runtime.ConfigApplyResult{ + Success: false, + ValidationErrors: []config.ValidationError{ + {Field: "direct_tool_response_mode", Message: `invalid direct tool response mode: bogus (must be "full" or "deferred")`}, + }, + }, + err: errors.New(`configuration validation failed: direct_tool_response_mode: invalid direct tool response mode: bogus`), + }, map[string]any{"direct_tool_response_mode": "bogus"}) + + require.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String()) + + var envelope map[string]interface{} + require.NoError(t, json.Unmarshal(w.Body.Bytes(), &envelope)) + assert.Equal(t, false, envelope["success"]) + + data, ok := envelope["data"].(map[string]interface{}) + require.True(t, ok, "the 400 must carry a structured payload, not just prose: %s", w.Body.String()) + errs, ok := data["validation_errors"].([]interface{}) + require.True(t, ok, "validation_errors must be present") + require.Len(t, errs, 1) + + first, ok := errs[0].(map[string]interface{}) + require.True(t, ok) + assert.Equal(t, "direct_tool_response_mode", first["field"], + "a client must be able to point at the offending field without parsing the message") +} + +// A failure with no validation errors is a genuine server fault and must stay +// in the 500 class — otherwise the fix would mislabel real outages as bad input. +func TestPatchConfig_PersistFailureStays500(t *testing.T) { + w := patchConfigWith(t, &applyErrController{ + result: &runtime.ConfigApplyResult{Success: false}, + err: errors.New("failed to write config file: disk full"), + }, map[string]any{"tools_limit": 20}) + + require.Equal(t, http.StatusInternalServerError, w.Code, "body=%s", w.Body.String()) + assert.NotContains(t, w.Body.String(), "validation_errors") +} + +// A nil result alongside an error must not panic the classifier. +func TestPatchConfig_NilResultWithErrorStays500(t *testing.T) { + w := patchConfigWith(t, &applyErrController{ + result: nil, + err: errors.New("controller exploded"), + }, map[string]any{"tools_limit": 20}) + + require.Equal(t, http.StatusInternalServerError, w.Code, "body=%s", w.Body.String()) +} diff --git a/internal/httpapi/server.go b/internal/httpapi/server.go index 9ca385487..a26a668a5 100644 --- a/internal/httpapi/server.go +++ b/internal/httpapi/server.go @@ -453,7 +453,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { // These connections are authenticated via OS-level permissions (UID/SID matching) source := transport.GetConnectionSource(r.Context()) if source == transport.ConnectionSourceTray { - s.logger.Debug("Tray connection - skipping API key validation", + s.logger.Debugw("Tray connection - skipping API key validation", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr), zap.String("source", string(source))) @@ -481,7 +481,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { // SECURITY: API key is REQUIRED for all TCP connections to REST API // Empty API key is not allowed - this prevents accidental exposure if cfg.APIKey == "" { - s.logger.Warn("TCP connection rejected - API key not configured", + s.logger.Warnw("TCP connection rejected - API key not configured", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr)) s.writeError(w, r, http.StatusUnauthorized, "API key authentication required but not configured. Please set MCPPROXY_API_KEY or configure api_key in config file.") @@ -491,7 +491,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { // Extract token from request token := ExtractToken(r) if token == "" { - s.logger.Warn("TCP connection with missing API key", + s.logger.Warnw("TCP connection with missing API key", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr)) s.writeError(w, r, http.StatusUnauthorized, "Invalid or missing API key") @@ -506,7 +506,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { // Check if the token matches the global API key (admin) if token == cfg.APIKey { - s.logger.Debug("TCP connection with valid API key", + s.logger.Debugw("TCP connection with valid API key", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr)) ctx := auth.WithAuthContext(r.Context(), auth.AdminContext()) @@ -515,7 +515,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { } // Token doesn't match anything - s.logger.Warn("TCP connection with invalid API key", + s.logger.Warnw("TCP connection with invalid API key", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr)) s.writeError(w, r, http.StatusUnauthorized, "Invalid or missing API key") @@ -526,7 +526,7 @@ func (s *Server) apiKeyAuthMiddleware() func(http.Handler) http.Handler { // handleAgentTokenAuth validates an agent token and sets the appropriate AuthContext. func (s *Server) handleAgentTokenAuth(w http.ResponseWriter, r *http.Request, next http.Handler, token string) { if s.tokenStore == nil || s.dataDir == "" { - s.logger.Warn("Agent token presented but token store not configured", + s.logger.Warnw("Agent token presented but token store not configured", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr)) s.writeError(w, r, http.StatusUnauthorized, "Agent tokens are not configured on this server") @@ -535,14 +535,14 @@ func (s *Server) handleAgentTokenAuth(w http.ResponseWriter, r *http.Request, ne hmacKey, err := auth.GetOrCreateHMACKey(s.dataDir) if err != nil { - s.logger.Error("Failed to get HMAC key for agent token validation", zap.Error(err)) + s.logger.Errorw("Failed to get HMAC key for agent token validation", zap.Error(err)) s.writeError(w, r, http.StatusInternalServerError, "Internal server error") return } agentToken, err := s.tokenStore.ValidateAgentToken(token, hmacKey) if err != nil { - s.logger.Warn("Agent token validation failed", + s.logger.Warnw("Agent token validation failed", zap.String("path", r.URL.Path), zap.String("remote_addr", r.RemoteAddr), zap.String("error", err.Error())) @@ -553,7 +553,7 @@ func (s *Server) handleAgentTokenAuth(w http.ResponseWriter, r *http.Request, ne // Update last-used timestamp in background go func() { if updateErr := s.tokenStore.UpdateAgentTokenLastUsed(agentToken.Name); updateErr != nil { - s.logger.Warn("Failed to update agent token last-used timestamp", + s.logger.Warnw("Failed to update agent token last-used timestamp", zap.String("name", agentToken.Name), zap.Error(updateErr)) } @@ -567,7 +567,7 @@ func (s *Server) handleAgentTokenAuth(w http.ResponseWriter, r *http.Request, ne authCtx := agentToken.AuthContext() ctx := auth.WithAuthContext(r.Context(), authCtx) - s.logger.Debug("Agent token authenticated", + s.logger.Debugw("Agent token authenticated", zap.String("agent_name", agentToken.Name), zap.String("token_prefix", agentToken.TokenPrefix), zap.String("path", r.URL.Path), @@ -995,7 +995,7 @@ func (s *Server) setupRoutes() { // Note: Swagger UI is mounted directly on the main mux (not via HTTP API server) // See internal/server/server.go for swagger handler registration - s.logger.Debug("HTTP API routes setup completed", + s.logger.Debugw("HTTP API routes setup completed", "api_routes", "/api/v1/*", "sse_route", "/events", "health_routes", "/healthz,/readyz,/livez,/ready") @@ -1078,7 +1078,7 @@ func (s *Server) writeJSON(w http.ResponseWriter, status int, data interface{}) w.Header().Set("Content-Type", "application/json") w.WriteHeader(status) if err := json.NewEncoder(w).Encode(data); err != nil { - s.logger.Error("Failed to encode JSON response", "error", err) + s.logger.Errorw("Failed to encode JSON response", "error", err) } } @@ -1131,12 +1131,16 @@ func (s *Server) handleGetStatus(w http.ResponseWriter, _ *http.Request) { autostartDataDir = cfg.DataDir } + // One traversal, used for both the top-level field and the nested snapshot, + // so the two cannot disagree and the O(servers) walk happens once (#1084). + liveUpstreamStats := s.controller.GetUpstreamStats() + response := map[string]interface{}{ "running": s.controller.IsRunning(), "edition": editionValue, "listen_addr": s.controller.GetListenAddress(), - "upstream_stats": s.controller.GetUpstreamStats(), - "status": s.controller.GetStatus(), + "upstream_stats": liveUpstreamStats, + "status": withLiveUpstreamStats(s.controller.GetStatus(), liveUpstreamStats), "routing_mode": routingMode, "timestamp": time.Now().Unix(), // Unix seconds at which this core process started, so a UI can render a @@ -1404,7 +1408,7 @@ func (s *Server) handleGetServers(w http.ResponseWriter, r *http.Request) { }).ListServers(r.Context()) if err != nil { - s.logger.Error("Failed to list servers via management service", "error", err) + s.logger.Errorw("Failed to list servers via management service", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get servers") return } @@ -1452,7 +1456,7 @@ func (s *Server) handleGetServers(w http.ResponseWriter, r *http.Request) { // Fallback to legacy path if management service not available genericServers, err := s.controller.GetAllServers() if err != nil { - s.logger.Error("Failed to get servers", "error", err) + s.logger.Errorw("Failed to get servers", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get servers") return } @@ -1861,12 +1865,12 @@ func (s *Server) handleAddServer(w http.ResponseWriter, r *http.Request) { s.writeError(w, r, http.StatusConflict, err.Error()) return } - logger.Error("Failed to add server", "server", req.Name, "error", err) + logger.Errorw("Failed to add server", "server", req.Name, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to add server: %v", err)) return } - logger.Info("Server added successfully", "server", req.Name, "quarantined", quarantined) + logger.Infow("Server added successfully", "server", req.Name, "quarantined", quarantined) s.writeSuccess(w, contracts.ServerActionResponse{ Server: req.Name, Action: "add", @@ -1904,12 +1908,12 @@ func (s *Server) handleRemoveServer(w http.ResponseWriter, r *http.Request) { s.writeError(w, r, http.StatusNotFound, err.Error()) return } - logger.Error("Failed to remove server", "server", serverID, "error", err) + logger.Errorw("Failed to remove server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to remove server: %v", err)) return } - logger.Info("Server removed successfully", "server", serverID) + logger.Infow("Server removed successfully", "server", serverID) s.writeSuccess(w, contracts.ServerActionResponse{ Server: serverID, Action: "remove", @@ -2147,12 +2151,12 @@ func (s *Server) handlePatchServer(w http.ResponseWriter, r *http.Request) { s.writeError(w, r, http.StatusNotFound, err.Error()) return } - logger.Error("Failed to update server", "server", serverName, "error", err) + logger.Errorw("Failed to update server", "server", serverName, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to update server: %v", err)) return } - logger.Info("Server updated successfully", "server", serverName) + logger.Infow("Server updated successfully", "server", serverName) s.writeSuccess(w, map[string]interface{}{ "message": fmt.Sprintf("Server '%s' updated successfully", serverName), "restart_required": true, @@ -2269,7 +2273,7 @@ func (s *Server) handleConvertConfigToSecret(w http.ResponseWriter, r *http.Requ // state so it's traceable. ref := secret.Ref{Type: secretTypeKeyring, Name: req.SecretName} if err := resolver.Store(ctx, ref, value); err != nil { - s.logger.Error("config-to-secret: keyring store failed", + s.logger.Errorw("config-to-secret: keyring store failed", "server", serverName, "scope", req.Scope, "key", req.Key, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("failed to store secret: %v", err)) return @@ -2301,7 +2305,7 @@ func (s *Server) handleConvertConfigToSecret(w http.ResponseWriter, r *http.Requ updates.ReconnectOnUse = sc.ReconnectOnUse if err := s.controller.UpdateServer(ctx, serverName, updates); err != nil { - s.logger.Error("config-to-secret: keyring write succeeded but config update failed; secret is stored but not referenced", + s.logger.Errorw("config-to-secret: keyring write succeeded but config update failed; secret is stored but not referenced", "server", serverName, "scope", req.Scope, "key", req.Key, "secret_name", req.SecretName, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("secret stored as %q but config update failed: %v", req.SecretName, err)) return @@ -2309,7 +2313,7 @@ func (s *Server) handleConvertConfigToSecret(w http.ResponseWriter, r *http.Requ // Wake any servers that depend on the new secret. Best-effort. if err := s.controller.NotifySecretsChanged(ctx, "store", req.SecretName); err != nil { - s.logger.Warn("config-to-secret: failed to notify runtime of secret change", + s.logger.Warnw("config-to-secret: failed to notify runtime of secret change", "name", req.SecretName, "error", err) } @@ -2347,7 +2351,7 @@ func (s *Server) handleEnableServer(w http.ResponseWriter, r *http.Request) { }).EnableServer(r.Context(), serverID, true) if err != nil { - s.logger.Error("Failed to enable server via management service", "server", serverID, "error", err) + s.logger.Errorw("Failed to enable server via management service", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to enable server: %v", err)) return } @@ -2365,15 +2369,15 @@ func (s *Server) handleEnableServer(w http.ResponseWriter, r *http.Request) { // Fallback to legacy async path async, err := s.toggleServerAsync(serverID, true) if err != nil { - s.logger.Error("Failed to enable server", "server", serverID, "error", err) + s.logger.Errorw("Failed to enable server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to enable server: %v", err)) return } if async { - s.logger.Debug("Server enable dispatched asynchronously", "server", serverID) + s.logger.Debugw("Server enable dispatched asynchronously", "server", serverID) } else { - s.logger.Debug("Server enable completed synchronously", "server", serverID) + s.logger.Debugw("Server enable completed synchronously", "server", serverID) } response := contracts.ServerActionResponse{ @@ -2414,7 +2418,7 @@ func (s *Server) handleDisableServer(w http.ResponseWriter, r *http.Request) { }).EnableServer(r.Context(), serverID, false) if err != nil { - s.logger.Error("Failed to disable server via management service", "server", serverID, "error", err) + s.logger.Errorw("Failed to disable server via management service", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to disable server: %v", err)) return } @@ -2432,15 +2436,15 @@ func (s *Server) handleDisableServer(w http.ResponseWriter, r *http.Request) { // Fallback to legacy async path async, err := s.toggleServerAsync(serverID, false) if err != nil { - s.logger.Error("Failed to disable server", "server", serverID, "error", err) + s.logger.Errorw("Failed to disable server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to disable server: %v", err)) return } if async { - s.logger.Debug("Server disable dispatched asynchronously", "server", serverID) + s.logger.Debugw("Server disable dispatched asynchronously", "server", serverID) } else { - s.logger.Debug("Server disable completed synchronously", "server", serverID) + s.logger.Debugw("Server disable completed synchronously", "server", serverID) } response := contracts.ServerActionResponse{ @@ -2469,7 +2473,7 @@ func (s *Server) handleForceReconnectServers(w http.ResponseWriter, r *http.Requ reason := r.URL.Query().Get("reason") if err := s.controller.ForceReconnectAllServers(reason); err != nil { - s.logger.Error("Failed to trigger force reconnect for servers", + s.logger.Errorw("Failed to trigger force reconnect for servers", "reason", reason, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to reconnect servers: %v", err)) @@ -2509,7 +2513,7 @@ func (s *Server) handleRestartAll(w http.ResponseWriter, r *http.Request) { result, err := mgmtSvc.RestartAll(r.Context()) if err != nil { - s.logger.Error("RestartAll operation failed", "error", err) + s.logger.Errorw("RestartAll operation failed", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to restart all servers: %v", err)) return } @@ -2541,7 +2545,7 @@ func (s *Server) handleEnableAll(w http.ResponseWriter, r *http.Request) { result, err := mgmtSvc.EnableAll(r.Context()) if err != nil { - s.logger.Error("EnableAll operation failed", "error", err) + s.logger.Errorw("EnableAll operation failed", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to enable all servers: %v", err)) return } @@ -2573,7 +2577,7 @@ func (s *Server) handleDisableAll(w http.ResponseWriter, r *http.Request) { result, err := mgmtSvc.DisableAll(r.Context()) if err != nil { - s.logger.Error("DisableAll operation failed", "error", err) + s.logger.Errorw("DisableAll operation failed", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to disable all servers: %v", err)) return } @@ -2618,7 +2622,7 @@ func (s *Server) handleRestartServer(w http.ResponseWriter, r *http.Request) { if isOAuthError { // OAuth required is not a failure - restart succeeded but OAuth is needed - s.logger.Info("Server restart completed, OAuth login required", + s.logger.Infow("Server restart completed, OAuth login required", "server", serverID, "error", errStr) @@ -2633,7 +2637,7 @@ func (s *Server) handleRestartServer(w http.ResponseWriter, r *http.Request) { } // Non-OAuth error - treat as failure - s.logger.Error("Failed to restart server via management service", "server", serverID, "error", err) + s.logger.Errorw("Failed to restart server via management service", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to restart server: %v", err)) return } @@ -2667,7 +2671,7 @@ func (s *Server) handleRestartServer(w http.ResponseWriter, r *http.Request) { if isOAuthError { // OAuth required is not a failure - restart succeeded but OAuth is needed - s.logger.Info("Server restart completed, OAuth login required", + s.logger.Infow("Server restart completed, OAuth login required", "server", serverID, "error", errStr) @@ -2682,17 +2686,17 @@ func (s *Server) handleRestartServer(w http.ResponseWriter, r *http.Request) { } // Non-OAuth error - treat as failure - s.logger.Error("Failed to restart server", "server", serverID, "error", err) + s.logger.Errorw("Failed to restart server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to restart server: %v", err)) return } - s.logger.Debug("Server restart completed synchronously", "server", serverID) + s.logger.Debugw("Server restart completed synchronously", "server", serverID) case <-time.After(35 * time.Second): // Longer timeout for restart (30s connect timeout + 5s buffer) - s.logger.Debug("Server restart executing asynchronously", "server", serverID) + s.logger.Debugw("Server restart executing asynchronously", "server", serverID) go func() { if err := <-done; err != nil { - s.logger.Error("Asynchronous server restart failed", "server", serverID, "error", err) + s.logger.Errorw("Asynchronous server restart failed", "server", serverID, "error", err) } }() } @@ -2733,10 +2737,10 @@ func (s *Server) handleDiscoverServerTools(w http.ResponseWriter, r *http.Reques return } - s.logger.Info("Manual tool discovery triggered via API", "server", serverID) + s.logger.Infow("Manual tool discovery triggered via API", "server", serverID) if err := s.controller.DiscoverServerTools(r.Context(), serverID); err != nil { - s.logger.Error("Failed to discover tools for server", "server", serverID, "error", err) + s.logger.Errorw("Failed to discover tools for server", "server", serverID, "error", err) if strings.Contains(err.Error(), "not found") { s.writeError(w, r, http.StatusNotFound, fmt.Sprintf("Server not found: %s", serverID)) @@ -2780,10 +2784,10 @@ func (s *Server) handleRefreshServer(w http.ResponseWriter, r *http.Request) { return } - s.logger.Info("Manual tool refresh triggered via API", "server", serverID) + s.logger.Infow("Manual tool refresh triggered via API", "server", serverID) if err := s.controller.DiscoverServerTools(r.Context(), serverID); err != nil { - s.logger.Error("Failed to refresh tools for server", "server", serverID, "error", err) + s.logger.Errorw("Failed to refresh tools for server", "server", serverID, "error", err) if strings.Contains(err.Error(), "not found") { s.writeError(w, r, http.StatusNotFound, fmt.Sprintf("Server not found: %s", serverID)) @@ -2815,7 +2819,7 @@ func (s *Server) toggleServerAsync(serverID string, enabled bool) (bool, error) case <-time.After(asyncToggleTimeout): go func() { if err := <-errCh; err != nil { - s.logger.Error("Asynchronous server toggle failed", "server", serverID, "enabled", enabled, "error", err) + s.logger.Errorw("Asynchronous server toggle failed", "server", serverID, "enabled", enabled, "error", err) } }() return true, nil @@ -2855,7 +2859,7 @@ func (s *Server) handleServerLogin(w http.ResponseWriter, r *http.Request) { result, err := mgmtSvc.TriggerOAuthLoginQuick(r.Context(), serverID) if err != nil { - s.logger.Error("Failed to trigger OAuth login", "server", serverID, "error", err) + s.logger.Errorw("Failed to trigger OAuth login", "server", serverID, "error", err) // Spec 020: Check for structured OAuth errors and return them directly var oauthFlowErr *contracts.OAuthFlowError @@ -2865,7 +2869,7 @@ func (s *Server) handleServerLogin(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusBadRequest) if encErr := json.NewEncoder(w).Encode(oauthFlowErr); encErr != nil { - s.logger.Error("Failed to encode OAuth flow error response", "error", encErr) + s.logger.Errorw("Failed to encode OAuth flow error response", "error", encErr) } return } @@ -2875,7 +2879,7 @@ func (s *Server) handleServerLogin(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") w.WriteHeader(http.StatusBadRequest) if encErr := json.NewEncoder(w).Encode(oauthValidationErr); encErr != nil { - s.logger.Error("Failed to encode OAuth validation error response", "error", encErr) + s.logger.Errorw("Failed to encode OAuth validation error response", "error", encErr) } return } @@ -2959,7 +2963,7 @@ func (s *Server) handleServerLogout(w http.ResponseWriter, r *http.Request) { } if err := mgmtSvc.TriggerOAuthLogout(r.Context(), serverID); err != nil { - s.logger.Error("Failed to trigger OAuth logout", "server", serverID, "error", err) + s.logger.Errorw("Failed to trigger OAuth logout", "server", serverID, "error", err) // Map errors to HTTP status codes if strings.Contains(err.Error(), "management disabled") || strings.Contains(err.Error(), "read-only") { @@ -3005,7 +3009,7 @@ func (s *Server) handleQuarantineServer(w http.ResponseWriter, r *http.Request) } if err := s.controller.QuarantineServer(serverID, true); err != nil { - s.logger.Error("Failed to quarantine server", "server", serverID, "error", err) + s.logger.Errorw("Failed to quarantine server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to quarantine server: %v", err)) return } @@ -3041,7 +3045,7 @@ func (s *Server) handleUnquarantineServer(w http.ResponseWriter, r *http.Request } if err := s.controller.QuarantineServer(serverID, false); err != nil { - s.logger.Error("Failed to unquarantine server", "server", serverID, "error", err) + s.logger.Errorw("Failed to unquarantine server", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to unquarantine server: %v", err)) return } @@ -3087,7 +3091,7 @@ func (s *Server) handleGetServerTools(w http.ResponseWriter, r *http.Request) { tools, err := mgmtSvc.GetServerTools(r.Context(), serverID) if err != nil { - s.logger.Error("Failed to get server tools", "server", serverID, "error", err) + s.logger.Errorw("Failed to get server tools", "server", serverID, "error", err) // Map errors to HTTP status codes (T018) if strings.Contains(err.Error(), "not found") { @@ -3164,7 +3168,7 @@ func (s *Server) enrichServerTools(serverID string, tools []map[string]interface } } if firstErr != nil { - s.logger.Debug("Tool approval enrichment partial", "server", serverID, "enriched", enrichedCount, "total", len(typedTools), "error", firstErr) + s.logger.Debugw("Tool approval enrichment partial", "server", serverID, "enriched", enrichedCount, "total", len(typedTools), "error", firstErr) } return typedTools } @@ -3194,7 +3198,7 @@ func (s *Server) handleGetGlobalTools(w http.ResponseWriter, r *http.Request) { // here must not fail the whole page — usage columns just stay zero. usage, usageErr := s.controller.AggregateToolUsage(time.Now().Add(-globalToolsUsageWindow)) if usageErr != nil { - s.logger.Warn("Global tools: usage aggregation failed, continuing without usage", "error", usageErr) + s.logger.Warnw("Global tools: usage aggregation failed, continuing without usage", "error", usageErr) usage = map[string]storage.ToolUsageStat{} } @@ -3246,7 +3250,7 @@ func (s *Server) handleGetGlobalTools(w http.ResponseWriter, r *http.Request) { // we could gather, flag the rest as partial. resp.Partial = true resp.FailedServers = append(resp.FailedServers, name) - s.logger.Debug("Global tools: server tools fetch failed", "server", name, "error", terr) + s.logger.Debugw("Global tools: server tools fetch failed", "server", name, "error", terr) continue } @@ -3313,7 +3317,7 @@ func (s *Server) handleGetServerLogs(w http.ResponseWriter, r *http.Request) { logEntries, err := s.controller.GetServerLogs(serverID, tail) if err != nil { - s.logger.Error("Failed to get server logs", "server", serverID, "error", err) + s.logger.Errorw("Failed to get server logs", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to get logs: %v", err)) return } @@ -3357,7 +3361,7 @@ func (s *Server) handleSearchTools(w http.ResponseWriter, r *http.Request) { results, err := s.controller.SearchTools(query, limit) if err != nil { - s.logger.Error("Failed to search tools", "query", query, "error", err) + s.logger.Errorw("Failed to search tools", "query", query, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Search failed: %v", err)) return } @@ -3440,7 +3444,7 @@ func (s *Server) handleSSEEvents(w http.ResponseWriter, r *http.Request) { defer s.controller.UnsubscribeEvents(eventsCh) } - s.logger.Debug("SSE connection established", + s.logger.Debugw("SSE connection established", "status_channel_nil", statusCh == nil, "events_channel_nil", eventsCh == nil) @@ -3449,18 +3453,19 @@ func (s *Server) handleSSEEvents(w http.ResponseWriter, r *http.Request) { defer heartbeat.Stop() // Send initial status + initialLiveStats := s.controller.GetUpstreamStats() initialStatus := map[string]interface{}{ "running": s.controller.IsRunning(), "listen_addr": s.controller.GetListenAddress(), - "upstream_stats": s.controller.GetUpstreamStats(), - "status": s.controller.GetStatus(), + "upstream_stats": initialLiveStats, + "status": withLiveUpstreamStats(s.controller.GetStatus(), initialLiveStats), "timestamp": time.Now().Unix(), "started_at": processStart.Unix(), } - s.logger.Debug("Sending initial SSE status event", "data", initialStatus) + s.logger.Debugw("Sending initial SSE status event", "data", initialStatus) if err := s.writeSSEEvent(w, flusher, canFlush, "status", initialStatus); err != nil { - s.logger.Error("Failed to write initial SSE event", "error", err) + s.logger.Errorw("Failed to write initial SSE event", "error", err) return } s.logger.Debug("Initial SSE status event sent successfully") @@ -3476,7 +3481,7 @@ func (s *Server) handleSSEEvents(w http.ResponseWriter, r *http.Request) { "timestamp": time.Now().Unix(), } if err := s.writeSSEEvent(w, flusher, canFlush, "ping", pingData); err != nil { - s.logger.Error("Failed to write SSE heartbeat", "error", err) + s.logger.Errorw("Failed to write SSE heartbeat", "error", err) return } case status, ok := <-statusCh: @@ -3484,17 +3489,21 @@ func (s *Server) handleSSEEvents(w http.ResponseWriter, r *http.Request) { return } + // The channel snapshot is a point-in-time capture that never + // passes through GetStatus(), so without this the stream's nested + // stats stay stale for the life of the connection (#1084). + eventLiveStats := s.controller.GetUpstreamStats() response := map[string]interface{}{ "running": s.controller.IsRunning(), "listen_addr": s.controller.GetListenAddress(), - "upstream_stats": s.controller.GetUpstreamStats(), - "status": status, + "upstream_stats": eventLiveStats, + "status": withLiveUpstreamStats(status, eventLiveStats), "timestamp": time.Now().Unix(), "started_at": processStart.Unix(), } if err := s.writeSSEEvent(w, flusher, canFlush, "status", response); err != nil { - s.logger.Error("Failed to write SSE event", "error", err) + s.logger.Errorw("Failed to write SSE event", "error", err) return } case evt, ok := <-eventsCh: @@ -3509,7 +3518,7 @@ func (s *Server) handleSSEEvents(w http.ResponseWriter, r *http.Request) { } if err := s.writeSSEEvent(w, flusher, canFlush, string(evt.Type), eventPayload); err != nil { - s.logger.Error("Failed to write runtime SSE event", "error", err) + s.logger.Errorw("Failed to write runtime SSE event", "error", err) return } } @@ -3607,7 +3616,7 @@ func (s *Server) handleGetSecretRefs(w http.ResponseWriter, r *http.Request) { // Get all secret references from available providers refs, err := resolver.ListAll(ctx) if err != nil { - s.logger.Error("Failed to list secret references", "error", err) + s.logger.Errorw("Failed to list secret references", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to list secret references") return } @@ -3686,7 +3695,7 @@ func (s *Server) handleGetConfigSecrets(w http.ResponseWriter, r *http.Request) // Extract config-referenced secrets and environment variables configSecrets, err := resolver.ExtractConfigSecrets(ctx, cfg) if err != nil { - s.logger.Error("Failed to extract config secrets", "error", err) + s.logger.Errorw("Failed to extract config secrets", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to extract config secrets") return } @@ -3762,7 +3771,7 @@ func (s *Server) handleSetSecret(w http.ResponseWriter, r *http.Request) { err := resolver.Store(ctx, ref, request.Value) if err != nil { - s.logger.Error("Failed to store secret", "name", request.Name, "error", err) + s.logger.Errorw("Failed to store secret", "name", request.Name, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to store secret: %v", err)) return } @@ -3770,7 +3779,7 @@ func (s *Server) handleSetSecret(w http.ResponseWriter, r *http.Request) { // Notify runtime that secrets changed (this will restart affected servers) if runtime := s.controller; runtime != nil { if err := runtime.NotifySecretsChanged(ctx, "store", request.Name); err != nil { - s.logger.Warn("Failed to notify runtime of secret change", + s.logger.Warnw("Failed to notify runtime of secret change", "name", request.Name, "error", err) } @@ -3839,7 +3848,7 @@ func (s *Server) handleDeleteSecret(w http.ResponseWriter, r *http.Request) { err := resolver.Delete(ctx, ref) if err != nil { - s.logger.Error("Failed to delete secret", "name", name, "error", err) + s.logger.Errorw("Failed to delete secret", "name", name, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to delete secret: %v", err)) return } @@ -3847,7 +3856,7 @@ func (s *Server) handleDeleteSecret(w http.ResponseWriter, r *http.Request) { // Notify runtime that secrets changed (this will restart affected servers) if runtime := s.controller; runtime != nil { if err := runtime.NotifySecretsChanged(ctx, "delete", name); err != nil { - s.logger.Warn("Failed to notify runtime of secret deletion", + s.logger.Warnw("Failed to notify runtime of secret deletion", "name", name, "error", err) } @@ -3886,7 +3895,7 @@ func (s *Server) handleGetDiagnostics(w http.ResponseWriter, r *http.Request) { }).Doctor(r.Context()) if err != nil { - s.logger.Error("Failed to get diagnostics via management service", "error", err) + s.logger.Errorw("Failed to get diagnostics via management service", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get diagnostics") return } @@ -3901,7 +3910,7 @@ func (s *Server) handleGetDiagnostics(w http.ResponseWriter, r *http.Request) { // Fallback to legacy path if management service not available genericServers, err := s.controller.GetAllServers() if err != nil { - s.logger.Error("Failed to get servers for diagnostics", "error", err) + s.logger.Errorw("Failed to get servers for diagnostics", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get servers") return } @@ -4019,7 +4028,7 @@ func (s *Server) handleGetTokenStats(w http.ResponseWriter, r *http.Request) { tokenStats, err := s.controller.GetTokenSavings() if err != nil { - s.logger.Error("Failed to calculate token savings", "error", err) + s.logger.Errorw("Failed to calculate token savings", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to calculate token savings: %v", err)) return } @@ -4157,7 +4166,7 @@ func (s *Server) handleGetToolCalls(w http.ResponseWriter, r *http.Request) { } if err != nil { - s.logger.Error("Failed to get tool calls", "error", err, "session_id", sessionID) + s.logger.Errorw("Failed to get tool calls", "error", err, "session_id", sessionID) s.writeError(w, r, http.StatusInternalServerError, "Failed to get tool calls") return } @@ -4201,7 +4210,7 @@ func (s *Server) handleGetToolCallDetail(w http.ResponseWriter, r *http.Request) // Get tool call by ID toolCall, err := s.controller.GetToolCallByID(id) if err != nil { - s.logger.Error("Failed to get tool call detail", "id", id, "error", err) + s.logger.Errorw("Failed to get tool call detail", "id", id, "error", err) s.writeError(w, r, http.StatusNotFound, "Tool call not found") return } @@ -4255,7 +4264,7 @@ func (s *Server) handleGetServerToolCalls(w http.ResponseWriter, r *http.Request // Get server tool calls toolCalls, err := s.controller.GetServerToolCalls(serverID, limit) if err != nil { - s.logger.Error("Failed to get server tool calls", "server", serverID, "error", err) + s.logger.Errorw("Failed to get server tool calls", "server", serverID, "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get server tool calls") return } @@ -4365,7 +4374,7 @@ func (s *Server) handleReplayToolCall(w http.ResponseWriter, r *http.Request) { s.writeError(w, r, http.StatusTooManyRequests, limitErr.UserMessage()) return } - s.logger.Error("Failed to replay tool call", "id", id, "error", err) + s.logger.Errorw("Failed to replay tool call", "id", id, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to replay tool call: %v", err)) return } @@ -4406,7 +4415,7 @@ func (s *Server) handleGetConfig(w http.ResponseWriter, r *http.Request) { cfg, err := s.controller.GetConfig() if err != nil { - s.logger.Error("Failed to get configuration", "error", err) + s.logger.Errorw("Failed to get configuration", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get configuration") return } @@ -4454,7 +4463,7 @@ func (s *Server) handleValidateConfig(w http.ResponseWriter, r *http.Request) { // Perform validation validationErrors, err := s.controller.ValidateConfig(&cfg) if err != nil { - s.logger.Error("Failed to validate configuration", "error", err) + s.logger.Errorw("Failed to validate configuration", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Validation failed: %v", err)) return } @@ -4500,8 +4509,7 @@ func (s *Server) handleApplyConfig(w http.ResponseWriter, r *http.Request) { // Apply configuration result, err := s.controller.ApplyConfig(&cfg, cfgPath) if err != nil { - s.logger.Error("Failed to apply configuration", "error", err) - s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to apply configuration: %v", err)) + s.writeApplyConfigError(w, r, "Failed to apply configuration", result, err) return } @@ -4552,7 +4560,7 @@ func (s *Server) handlePatchDockerIsolation(w http.ResponseWriter, r *http.Reque // that logic here. cfg, err := s.controller.GetConfig() if err != nil { - s.logger.Error("Failed to get configuration for docker-isolation patch", "error", err) + s.logger.Errorw("Failed to get configuration for docker-isolation patch", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to read configuration") return } @@ -4569,8 +4577,7 @@ func (s *Server) handlePatchDockerIsolation(w http.ResponseWriter, r *http.Reque cfgPath := s.controller.GetConfigPath() result, err := s.controller.ApplyConfig(cfg, cfgPath) if err != nil { - s.logger.Error("Failed to apply docker-isolation toggle", "error", err) - s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to apply configuration: %v", err)) + s.writeApplyConfigError(w, r, "Failed to apply docker-isolation toggle", result, err) return } @@ -4621,7 +4628,7 @@ func (s *Server) handlePatchConfig(w http.ResponseWriter, r *http.Request) { // untouched fields, including masked secrets, are preserved verbatim. cfg, err := s.controller.GetConfig() if err != nil { - s.logger.Error("Failed to get configuration for patch", "error", err) + s.logger.Errorw("Failed to get configuration for patch", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to read configuration") return } @@ -4634,13 +4641,13 @@ func (s *Server) handlePatchConfig(w http.ResponseWriter, r *http.Request) { // deep-merge the patch onto without enumerating every field. baseBytes, err := json.Marshal(cfg) if err != nil { - s.logger.Error("Failed to marshal live configuration", "error", err) + s.logger.Errorw("Failed to marshal live configuration", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to read configuration") return } var baseMap map[string]interface{} if err := json.Unmarshal(baseBytes, &baseMap); err != nil { - s.logger.Error("Failed to unmarshal live configuration", "error", err) + s.logger.Errorw("Failed to unmarshal live configuration", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to read configuration") return } @@ -4649,7 +4656,7 @@ func (s *Server) handlePatchConfig(w http.ResponseWriter, r *http.Request) { mergedBytes, err := json.Marshal(baseMap) if err != nil { - s.logger.Error("Failed to marshal merged configuration", "error", err) + s.logger.Errorw("Failed to marshal merged configuration", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to build configuration") return } @@ -4661,8 +4668,7 @@ func (s *Server) handlePatchConfig(w http.ResponseWriter, r *http.Request) { result, err := s.controller.ApplyConfig(&merged, s.controller.GetConfigPath()) if err != nil { - s.logger.Error("Failed to apply configuration patch", "error", err) - s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to apply configuration: %v", err)) + s.writeApplyConfigError(w, r, "Failed to apply configuration patch", result, err) return } @@ -4677,6 +4683,75 @@ func (s *Server) handlePatchConfig(w http.ResponseWriter, r *http.Request) { s.writeSuccess(w, response) } +// writeApplyConfigError reports an ApplyConfig failure with the right status +// class (#1084). +// +// ApplyConfig fails for two very different reasons: the operator sent a value +// the config rejects, or the server could not persist/commit a valid one. Both +// used to come back as 500, so `{"direct_tool_response_mode":"bogus"}` — a +// plain enum typo — was reported as a server fault, which is also what tells a +// client whether retrying could ever help. The structured ValidationErrors on +// the result are what distinguish them; when they are present this is a 400 and +// the payload carries them so the caller can point at the offending field. +// +// Returns the status it wrote, for the callers that log. +func (s *Server) writeApplyConfigError(w http.ResponseWriter, r *http.Request, msg string, result *internalRuntime.ConfigApplyResult, err error) { + if result != nil && len(result.ValidationErrors) > 0 { + // A rejected value is the operator's, not the server's: log it at warn + // and answer 400. The structured errors ride in `data` so a client can + // point at the offending field instead of scraping the message. + s.logger.Warnw(msg, "error", err) + requestID := reqcontext.GetRequestID(r.Context()) + s.writeJSON(w, http.StatusBadRequest, contracts.APIResponse{ + Success: false, + Error: fmt.Sprintf("%s: %v", msg, err), + RequestID: requestID, + Data: map[string]interface{}{ + "validation_errors": contracts.ConvertValidationErrors(result.ValidationErrors), + }, + }) + return + } + s.logger.Errorw(msg, "error", err) + s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("%s: %v", msg, err)) +} + +// withLiveUpstreamStats returns a copy of a status snapshot whose nested +// upstream_stats / tools_indexed carry the LIVE values passed in (#1084). +// +// The snapshot comes from the last PUBLISHED status event, and nothing +// refreshes its embedded stats once connection settles: they sat at +// "Connecting"/0 for the whole life of a process whose servers were long Ready, +// while the sibling top-level upstream_stats — computed live from the +// supervisor's StateView — was correct. Two fields describing one thing +// disagreed and only one ever converged. +// +// Applied here rather than inside Server.GetStatus() for two reasons: every +// emission site (the REST poll and both SSE paths) already computes the live +// stats once for its top-level field, so this reuses that value instead of +// repeating an O(servers) traversal per response; and the SSE stream forwards a +// snapshot straight off the channel, which never passes through GetStatus() at +// all. +// +// Copies rather than mutates: the SSE value is a map owned by the publisher, +// and refreshing it in place would edit something another goroutine may hold. +// Keys are preserved rather than dropped — clients read data.status.*. +func withLiveUpstreamStats(status interface{}, live map[string]interface{}) interface{} { + snapshot, ok := status.(map[string]interface{}) + if !ok || live == nil { + return status + } + refreshed := make(map[string]interface{}, len(snapshot)) + for k, v := range snapshot { + refreshed[k] = v + } + refreshed["upstream_stats"] = live + if totalTools, ok := live["total_tools"].(int); ok { + refreshed["tools_indexed"] = totalTools + } + return refreshed +} + // deepMergeJSON recursively merges patch into base. When both base[k] and // patch[k] are JSON objects (map[string]interface{}), they are merged // recursively; otherwise patch[k] overwrites base[k] (arrays and scalars @@ -4752,7 +4827,7 @@ func (s *Server) handleCallTool(w http.ResponseWriter, r *http.Request) { s.writeError(w, r, http.StatusTooManyRequests, limitErr.UserMessage()) return } - s.logger.Error("Failed to call tool", "tool", request.ToolName, "error", err) + s.logger.Errorw("Failed to call tool", "tool", request.ToolName, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to call tool: %v", err)) return } @@ -4775,7 +4850,7 @@ func (s *Server) handleCallTool(w http.ResponseWriter, r *http.Request) { func (s *Server) handleListRegistries(w http.ResponseWriter, r *http.Request) { registries, err := s.controller.ListRegistries() if err != nil { - s.logger.Error("Failed to list registries", "error", err) + s.logger.Errorw("Failed to list registries", "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to list registries: %v", err)) return } @@ -4785,7 +4860,7 @@ func (s *Server) handleListRegistries(w http.ResponseWriter, r *http.Request) { for i, reg := range registries { regMap, ok := reg.(map[string]interface{}) if !ok { - s.logger.Warn("Invalid registry type", "registry", reg) + s.logger.Warnw("Invalid registry type", "registry", reg) continue } @@ -4875,7 +4950,7 @@ func (s *Server) handleSearchRegistryServers(w http.ResponseWriter, r *http.Requ }) return } - s.logger.Error("Failed to search registry servers", "registry", registryID, "error", err) + s.logger.Errorw("Failed to search registry servers", "registry", registryID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to search servers: %v", err)) return } @@ -4885,7 +4960,7 @@ func (s *Server) handleSearchRegistryServers(w http.ResponseWriter, r *http.Requ for i, srv := range servers { srvMap, ok := srv.(map[string]interface{}) if !ok { - s.logger.Warn("Invalid server type", "server", srv) + s.logger.Warnw("Invalid server type", "server", srv) continue } @@ -5003,7 +5078,7 @@ func (s *Server) handleAddFromRegistry(w http.ResponseWriter, r *http.Request) { if err != nil { status := registryAddErrorStatus(rerr.Code) if status >= http.StatusInternalServerError { - logger.Error("Add from registry failed", "registry", registryID, "server", serverID, "error", err) + logger.Errorw("Add from registry failed", "registry", registryID, "server", serverID, "error", err) } s.writeRegistryAddError(w, r, status, rerr) return @@ -5053,7 +5128,7 @@ func (s *Server) handleAddRegistrySource(w http.ResponseWriter, r *http.Request) if err != nil { status := registryAddErrorStatus(rerr.Code) if status >= http.StatusInternalServerError { - logger.Error("Add registry source failed", "url", req.URL, "error", err) + logger.Errorw("Add registry source failed", "url", req.URL, "error", err) } s.writeRegistryAddError(w, r, status, rerr) return @@ -5098,7 +5173,7 @@ func (s *Server) handleRemoveRegistrySource(w http.ResponseWriter, r *http.Reque if err != nil { status := registryAddErrorStatus(rerr.Code) if status >= http.StatusInternalServerError { - logger.Error("Remove registry source failed", "id", registryID, "error", err) + logger.Errorw("Remove registry source failed", "id", registryID, "error", err) } s.writeRegistryAddError(w, r, status, rerr) return @@ -5151,7 +5226,7 @@ func (s *Server) handleEditRegistrySource(w http.ResponseWriter, r *http.Request if err != nil { status := registryAddErrorStatus(rerr.Code) if status >= http.StatusInternalServerError { - logger.Error("Edit registry source failed", "id", registryID, "error", err) + logger.Errorw("Edit registry source failed", "id", registryID, "error", err) } s.writeRegistryAddError(w, r, status, rerr) return @@ -5189,7 +5264,7 @@ func (s *Server) handleRefreshRegistryCache(w http.ResponseWriter, r *http.Reque cleared, err := s.controller.RefreshRegistryCache(registryID) if err != nil { - s.logger.Error("Failed to refresh registry cache", "registry", registryID, "error", err) + s.logger.Errorw("Failed to refresh registry cache", "registry", registryID, "error", err) s.writeError(w, r, http.StatusInternalServerError, fmt.Sprintf("Failed to refresh registry cache: %v", err)) return } @@ -5308,7 +5383,7 @@ func (s *Server) handleGetSessions(w http.ResponseWriter, r *http.Request) { // Get recent sessions from controller sessions, total, err := s.controller.GetRecentSessions(limit, status) if err != nil { - s.logger.Error("Failed to get sessions", "error", err) + s.logger.Errorw("Failed to get sessions", "error", err) s.writeError(w, r, http.StatusInternalServerError, "Failed to get sessions") return } @@ -5360,7 +5435,7 @@ func (s *Server) handleGetSessionDetail(w http.ResponseWriter, r *http.Request) // Get session by ID session, err := s.controller.GetSessionByID(id) if err != nil { - s.logger.Error("Failed to get session detail", "id", id, "error", err) + s.logger.Errorw("Failed to get session detail", "id", id, "error", err) s.writeError(w, r, http.StatusNotFound, "Session not found") return } diff --git a/internal/httpapi/status_live_stats_test.go b/internal/httpapi/status_live_stats_test.go new file mode 100644 index 000000000..fb5f763a6 --- /dev/null +++ b/internal/httpapi/status_live_stats_test.go @@ -0,0 +1,85 @@ +package httpapi + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// #1084: /api/v1/status embedded a status snapshot whose upstream_stats and +// tools_indexed were a point-in-time capture nothing refreshed. They sat at +// "Connecting"/0 for the life of a process whose servers were long Ready, while +// the sibling top-level upstream_stats was correct. +func staleSnapshot() map[string]interface{} { + return map[string]interface{}{ + "running": true, + "phase": "Ready", + "listen_addr": "127.0.0.1:8080", + "upstream_stats": map[string]interface{}{ + "servers": map[string]interface{}{"everything": map[string]interface{}{"state": "Connecting"}}, + "total_tools": 0, + }, + "tools_indexed": 0, + } +} + +func liveStats() map[string]interface{} { + return map[string]interface{}{ + "servers": map[string]interface{}{"everything": map[string]interface{}{"state": "Ready"}}, + "total_tools": 13, + } +} + +func TestWithLiveUpstreamStats_RefreshesTheStaleNestedFields(t *testing.T) { + out, ok := withLiveUpstreamStats(staleSnapshot(), liveStats()).(map[string]interface{}) + require.True(t, ok) + + assert.Equal(t, 13, out["tools_indexed"], "tools_indexed must track the live count") + stats, ok := out["upstream_stats"].(map[string]interface{}) + require.True(t, ok) + assert.Equal(t, 13, stats["total_tools"]) + + servers := stats["servers"].(map[string]interface{}) + everything := servers["everything"].(map[string]interface{}) + assert.Equal(t, "Ready", everything["state"], + "the nested per-server state must not remain stuck at Connecting") +} + +// Unrelated keys survive: clients read data.status.* and the fix must refresh, +// not replace, the snapshot. +func TestWithLiveUpstreamStats_PreservesEverythingElse(t *testing.T) { + out := withLiveUpstreamStats(staleSnapshot(), liveStats()).(map[string]interface{}) + assert.Equal(t, true, out["running"]) + assert.Equal(t, "Ready", out["phase"]) + assert.Equal(t, "127.0.0.1:8080", out["listen_addr"]) +} + +// It must COPY. The SSE path passes a map owned by the publisher, and editing +// it in place would mutate something another goroutine may still hold. +func TestWithLiveUpstreamStats_DoesNotMutateItsInput(t *testing.T) { + in := staleSnapshot() + withLiveUpstreamStats(in, liveStats()) + + assert.Equal(t, 0, in["tools_indexed"], "the caller's snapshot must be untouched") + original := in["upstream_stats"].(map[string]interface{}) + assert.Equal(t, 0, original["total_tools"]) +} + +// Degenerate inputs must pass through rather than panic or blank the payload. +func TestWithLiveUpstreamStats_PassesThroughWhenItCannotRefresh(t *testing.T) { + assert.Nil(t, withLiveUpstreamStats(nil, liveStats())) + + // A non-map status (a future typed snapshot) is returned untouched. + assert.Equal(t, "opaque", withLiveUpstreamStats("opaque", liveStats())) + + // No live stats to apply: keep what we had rather than emit nothing. + unchanged := withLiveUpstreamStats(staleSnapshot(), nil).(map[string]interface{}) + assert.Equal(t, 0, unchanged["tools_indexed"]) + + // total_tools absent or wrongly typed leaves tools_indexed alone rather + // than zeroing it. + partial := withLiveUpstreamStats(staleSnapshot(), map[string]interface{}{"servers": map[string]interface{}{}}).(map[string]interface{}) + assert.Equal(t, 0, partial["tools_indexed"]) + assert.NotNil(t, partial["upstream_stats"]) +} diff --git a/internal/runtime/apply_config_validation_error_test.go b/internal/runtime/apply_config_validation_error_test.go new file mode 100644 index 000000000..034718796 --- /dev/null +++ b/internal/runtime/apply_config_validation_error_test.go @@ -0,0 +1,58 @@ +package runtime + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/smart-mcp-proxy/mcpproxy-go/internal/config" +) + +// #1084: a config value the validator rejects is the OPERATOR's mistake, not a +// server fault. ApplyConfig returned the detail only inside the error string, +// so callers could not tell "you sent a bad value" from "we failed to persist a +// good one" without string-matching — and the REST handlers reported every +// apply failure as 500, including a plain enum typo. +// +// The structured errors now ride on the RESULT, which is what lets a caller +// classify without parsing prose. +func TestApplyConfig_ValidationFailureCarriesStructuredErrors(t *testing.T) { + r := &Runtime{} + + cfg := &config.Config{ + Listen: "127.0.0.1:8080", + DataDir: t.TempDir(), + TLS: &config.TLSConfig{}, + DirectToolResponseMode: "bogus", + } + + result, err := r.ApplyConfig(cfg, "") + require.Error(t, err, "an invalid enum value must fail the apply") + require.NotNil(t, result, "the result must be returned even on failure — it is what classifies the error") + + assert.False(t, result.Success) + require.NotEmpty(t, result.ValidationErrors, + "without these the caller cannot distinguish a bad value from a server fault") + + var found bool + for _, ve := range result.ValidationErrors { + if ve.Field == "direct_tool_response_mode" { + found = true + assert.Contains(t, ve.Message, "bogus", "the message must name the rejected value") + } + } + assert.True(t, found, "the offending field must be named: %+v", result.ValidationErrors) +} + +// The nil-config guard is a genuine caller/programming fault, not operator +// input: it must NOT be dressed up as a validation error, or the handler would +// answer 400 for a server-side bug. +func TestApplyConfig_NilConfigIsNotAValidationError(t *testing.T) { + r := &Runtime{} + result, err := r.ApplyConfig(nil, "") + require.Error(t, err) + require.NotNil(t, result) + assert.Empty(t, result.ValidationErrors, + "a nil config is a server-side fault and must stay in the 500 class") +} diff --git a/internal/runtime/runtime.go b/internal/runtime/runtime.go index 8f47e7050..6ed610139 100644 --- a/internal/runtime/runtime.go +++ b/internal/runtime/runtime.go @@ -1498,8 +1498,14 @@ func (r *Runtime) ApplyConfig(newCfg *config.Config, cfgPath string) (*ConfigApp validationErrors := newCfg.ValidateDetailed() if len(validationErrors) > 0 { r.mu.Unlock() // Unlock before returning + // Carry the structured errors back on the RESULT, not only inside the + // error string (#1084). A caller cannot tell "the operator sent a bad + // value" from "the server failed to persist" by string-matching, so the + // REST handlers used to report every apply failure as 500 — including + // a plainly invalid enum value, which is a 400. return &ConfigApplyResult{ - Success: false, + Success: false, + ValidationErrors: validationErrors, }, fmt.Errorf("configuration validation failed: %v", validationErrors[0].Error()) } diff --git a/internal/server/mcp_describe_direct_test.go b/internal/server/mcp_describe_direct_test.go index 402fb75f4..e05649d1c 100644 --- a/internal/server/mcp_describe_direct_test.go +++ b/internal/server/mcp_describe_direct_test.go @@ -164,7 +164,96 @@ func TestDescribeDirect_BothIDFormsResolveIdentically(t *testing.T) { resp := callDescribeDirect(t, p, ctx, []interface{}{"we__ird__do_thing"}) require.Len(t, resp.Definitions, 1) assert.Equal(t, "we__ird", resp.Definitions[0]["server"]) - assert.Equal(t, "we__ird:do_thing", resp.Definitions[0]["name"]) + // #1083: the name echoed back is the DIRECT surface's registered display + // name, not the canonical id. This assertion previously pinned + // "we__ird:do_thing" — the canonical form, which this surface cannot be + // asked to call. + assert.Equal(t, "we__ird__do_thing", resp.Definitions[0]["name"]) +} + +// #1083: every identifier a direct-surface definition hands back must be one +// this surface can actually be asked for. +// +// Deferred mode tells agents to recover schemas through describe_tool, so this +// is the live discovery path, not a curiosity. Before the fix the response +// named the tool ":" (the surface registers "__") +// and recommended `call_with: "call_tool_read"` (this surface exposes no intent +// variants at all) — both produced "tool not found" when followed verbatim. +func TestDescribeDirect_ReturnedIdentifiersAreCallableOnThisSurface(t *testing.T) { + p := newDirectDescribeProxy(t) + ctx := context.Background() + + registered := map[string]bool{} + for _, name := range p.loadDirectCatalog().DisplayNames() { + registered[name] = true + } + require.NotEmpty(t, registered, "fixture must register direct tools") + + for _, id := range []string{"github__read_file", "github__delete_repo", "we__ird__do_thing"} { + t.Run(id, func(t *testing.T) { + resp := callDescribeDirect(t, p, ctx, []interface{}{id}) + require.Len(t, resp.Definitions, 1) + def := resp.Definitions[0] + + name, _ := def["name"].(string) + assert.True(t, registered[name], + "describe_tool returned name %q, which is not a tool this surface registers (registered: %v)", + name, registered) + + // `call_with` names call_tool_read/write/destructive — dispatch + // variants the direct surface does not expose. Dropped rather than + // retargeted; `annotations` still carries the permission signal it + // was derived from. + _, hasCallWith := def["call_with"] + assert.False(t, hasCallWith, + "call_with names an intent variant the direct surface does not expose") + assert.Contains(t, def, "annotations", + "dropping call_with must not lose the permission signal") + }) + } + + // Asking by the canonical form still WORKS as input — only the echoed + // identifier changes. + viaCanonical := callDescribeDirect(t, p, ctx, []interface{}{"github:read_file"}) + require.Len(t, viaCanonical.Definitions, 1) + assert.Equal(t, "github__read_file", viaCanonical.Definitions[0]["name"], + "both input forms must echo back the callable name") +} + +// The indexed surface is untouched: there `:` IS the callable id +// and `call_with` names a real dispatch variant, so both must survive. The +// resolver signals "not the direct surface" with an empty display name, and the +// seam is a no-op on it. +func TestDescribeIndexed_KeepsCanonicalNameAndCallWith(t *testing.T) { + entry := map[string]interface{}{ + "name": "github:delete_repo", + "call_with": string(contracts.ToolVariantDestructive), + } + applyDirectSurfaceIdentity(entry, "") + + assert.Equal(t, "github:delete_repo", entry["name"], + "the indexed surface's canonical name must survive untouched") + assert.Equal(t, string(contracts.ToolVariantDestructive), entry["call_with"], + "call_with names a real dispatch variant on the indexed surface") +} + +// ...and the indexed branch of the resolver is what supplies that empty name, +// so the no-op above is actually the path taken rather than a hypothetical. +func TestResolveDescribeDefinition_IndexedSurfaceReportsNoDisplayName(t *testing.T) { + p := newDirectDescribeProxy(t) + + // Unresolvable on the indexed surface (the fixture is not indexed) — the + // display name must still be empty, never a direct-surface leak. + _, _, displayName, idErr := p.resolveDescribeDefinition( + context.Background(), describeSurfaceIndexed, "github:read_file") + require.NotNil(t, idErr, "the fixture is deliberately not indexed") + assert.Empty(t, displayName, "the indexed surface must never report a direct display name") + + // The direct surface does report one. + _, _, directName, directErr := p.resolveDescribeDefinition( + context.Background(), describeSurfaceDirect, "github:read_file") + require.Nil(t, directErr) + assert.Equal(t, "github__read_file", directName) } // T043: the permission-tier gate. A read-scoped token cannot LIST a destructive @@ -182,7 +271,7 @@ func TestDescribeDirect_PermissionTierGate(t *testing.T) { resp := callDescribeDirect(t, p, readOnly, []interface{}{"github__delete_repo", "github__read_file"}) require.Len(t, resp.Definitions, 1, "only the read-tier tool may describe") - assert.Equal(t, "github:read_file", resp.Definitions[0]["name"]) + assert.Equal(t, "github__read_file", resp.Definitions[0]["name"]) byID := describeErrorsByID(resp) require.Contains(t, byID, "github__delete_repo") @@ -227,7 +316,7 @@ func TestDescribeDirect_PendingToolStillDescribesFromSnapshot(t *testing.T) { require.Empty(t, resp.Errors, "a listed tool is never undescribable (SC-007)") require.Len(t, resp.Definitions, 1) - assert.Equal(t, "github:read_file", resp.Definitions[0]["name"]) + assert.Equal(t, "github__read_file", resp.Definitions[0]["name"]) assert.Equal(t, "Read a file", resp.Definitions[0]["description"], "the definition comes from the catalog snapshot, not the index") } @@ -276,13 +365,15 @@ func TestDescribeDirect_OutputSchemaPresentOnlyWhenDeclared(t *testing.T) { byName[def["name"].(string)] = def } - withSchema := byName["github:create_issue"] + // Keyed by the DISPLAY name each definition echoes back (#1083). + withSchema := byName["github__create_issue"] require.Contains(t, withSchema, "output_schema", "a declaring tool must carry output_schema") raw, err := json.Marshal(withSchema["output_schema"]) require.NoError(t, err) assert.JSONEq(t, `{"type":"object","properties":{"number":{"type":"integer"}}}`, string(raw)) - assert.NotContains(t, byName["github:read_file"], "output_schema", + require.Contains(t, byName, "github__read_file", "the non-declaring tool must still be in the batch") + assert.NotContains(t, byName["github__read_file"], "output_schema", "a tool with no output schema must not carry an empty one") } @@ -325,14 +416,25 @@ func TestDescribeDirect_PendingDestructiveKeepsItsRealAnnotations(t *testing.T) require.Len(t, resp.Definitions, 1) def := resp.Definitions[0] - // contracts.DeriveCallWith answers the VARIANT name, which is what an agent - // calls; the task text's "destructive" is the tier it names. - assert.Equal(t, string(contracts.ToolVariantDestructive), def["call_with"], - "the safety hint must come from the catalog snapshot, not from an absent StateView entry") + + // #1083 dropped `call_with` from this surface — it named an intent variant + // the direct surface does not expose. The D10 property it guarded is + // unchanged and still asserted here, at its actual source: the annotations + // must come from the CATALOG SNAPSHOT, not from the absent StateView entry + // that would silently downgrade a pending destructive tool to read. + _, hasCallWith := def["call_with"] + assert.False(t, hasCallWith, "call_with is not emitted on the direct surface") annotations, ok := def["annotations"].(map[string]interface{}) require.True(t, ok, "the definition must carry the upstream annotations") - assert.Equal(t, true, annotations["destructiveHint"]) + assert.Equal(t, true, annotations["destructiveHint"], + "the safety signal must come from the catalog snapshot, not from an absent StateView entry") + + // And the derived tier is still recoverable from what IS emitted: + // DeriveCallWith is a pure function of these annotations, so dropping the + // derived field loses no information a consumer cannot recompute. + assert.Equal(t, string(contracts.ToolVariantDestructive), + contracts.DeriveCallWith(&config.ToolAnnotations{DestructiveHint: boolPtr(true)})) } // The override must not leak onto the retrieve path: full-mode entries keep @@ -476,7 +578,7 @@ func TestDescribeDirect_ServerRemovedBetweenListAndDescribe(t *testing.T) { []interface{}{"we__ird__do_thing", "github__read_file"}) require.Len(t, resp.Definitions, 1, "the surviving id must still answer — the batch does not fail") - assert.Equal(t, "github:read_file", resp.Definitions[0]["name"]) + assert.Equal(t, "github__read_file", resp.Definitions[0]["name"], "#1083: the echoed name is the callable one") byID := describeErrorsByID(resp) require.Contains(t, byID, "we__ird__do_thing") @@ -511,14 +613,21 @@ func TestDescribeDirect_DisplayAndCanonicalNamespaceOverlap(t *testing.T) { byName[def["name"].(string)] = def } + // Definitions are now keyed by the DISPLAY name each one echoes (#1083), + // and the two display names stay distinct — so this still discriminates + // between the two tools, which is the whole point of the test. (Before + // #1083 the keys were the two CANONICAL ids, "x:y:z" and "x__y:z".) + require.Len(t, byName, 2, "the two tools must remain separable by echoed name") + // "x__y:z" resolves as a DISPLAY name only — to server "x". - require.Contains(t, byName, "x:y:z") - assert.Equal(t, "x", byName["x:y:z"]["server"]) + require.Contains(t, byName, "x__y:z") + assert.Equal(t, "x", byName["x__y:z"]["server"]) // The other tool is reachable by its own display name, and its canonical - // form is withheld because it cannot name one of the two. - require.Contains(t, byName, "x__y:z") - assert.Equal(t, "x__y", byName["x__y:z"]["server"]) + // form — which is the string "x__y:z", i.e. the FIRST tool's display name — + // is withheld because it cannot unambiguously name one of the two. + require.Contains(t, byName, "x__y__z") + assert.Equal(t, "x__y", byName["x__y__z"]["server"]) cat := p.loadDirectCatalog() _, resolvesCanonically := cat.LookupCanonical("x__y:z") diff --git a/internal/server/mcp_describe_tool.go b/internal/server/mcp_describe_tool.go index 7395ea9bd..d67ef6015 100644 --- a/internal/server/mcp_describe_tool.go +++ b/internal/server/mcp_describe_tool.go @@ -115,11 +115,16 @@ func buildDescribeToolTool() mcp.Tool { // The two surfaces answer from different authorities and with different id // grammars; everything downstream — the shared builder, the score deletion, the // additive output_schema — is common. +// +// The fourth return is the DIRECT surface's registered display name +// ("__"), empty on the indexed surface. It is what the direct +// surface can actually be asked to call, so the assembly seam swaps it over the +// canonical ":" the shared builder emits (#1083). func (p *MCPProxyServer) resolveDescribeDefinition( ctx context.Context, surface describeSurface, id string, -) (*config.ToolMetadata, toolEntryOpts, map[string]interface{}) { +) (*config.ToolMetadata, toolEntryOpts, string, map[string]interface{}) { if surface == describeSurfaceDirect { entry, ok := p.resolveDirectDescribeID(ctx, id) if !ok { @@ -129,18 +134,18 @@ func (p *MCPProxyServer) resolveDescribeDefinition( if corrected, ok := p.suggestDirectToolID(ctx, id); ok { remediation = fmt.Sprintf("Tool not found. Tool ids are case-sensitive — did you mean '%s'?", corrected) } - return nil, toolEntryOpts{}, describeToolIDError(id, describeErrNotFound, remediation) + return nil, toolEntryOpts{}, "", describeToolIDError(id, describeErrNotFound, remediation) } // The catalog entry carries the UPSTREAM annotations. Letting the // builder fall back to its StateView lookup would silently downgrade a // listed-but-pending destructive tool to call_with "read", because the // StateView does not carry it (D10). - return entry.toolMetadata(), toolEntryOpts{annotationsOverride: entry.Annotations}, nil + return entry.toolMetadata(), toolEntryOpts{annotationsOverride: entry.Annotations}, entry.DisplayName, nil } serverName, toolName, ok := splitServerTool(id) if !ok { - return nil, toolEntryOpts{}, describeToolIDError(id, describeErrNotFound, describeMalformedIDRemediation) + return nil, toolEntryOpts{}, "", describeToolIDError(id, describeErrNotFound, describeMalformedIDRemediation) } visible, reason := p.toolVisibleToSession(ctx, serverName, toolName) @@ -151,15 +156,38 @@ func (p *MCPProxyServer) resolveDescribeDefinition( remediation = fmt.Sprintf("Tool not found. Tool ids are case-sensitive — did you mean '%s'?", canonical) } } - return nil, toolEntryOpts{}, describeToolIDError(id, code, remediation) + return nil, toolEntryOpts{}, "", describeToolIDError(id, code, remediation) } meta := p.lookupIndexedTool(serverName, toolName) if meta == nil { // Disappeared between the visibility check and the lookup. - return nil, toolEntryOpts{}, describeToolIDError(id, describeErrNotFound, describeNotFoundRemediation) + return nil, toolEntryOpts{}, "", describeToolIDError(id, describeErrNotFound, describeNotFoundRemediation) + } + return meta, toolEntryOpts{}, "", nil +} + +// applyDirectSurfaceIdentity rewrites a definition so every identifier in it is +// one the DIRECT surface can actually be asked for (#1083). +// +// The shared builder emits the canonical ":" name and a `call_with` +// naming one of the call_tool_read/write/destructive intent variants. Both are +// right on the indexed surface and wrong on this one: the direct surface +// registers "__" and exposes no intent variants at all, so an +// agent that followed either field verbatim got "tool not found" for both. +// Deferred mode makes that a live path rather than a curiosity — recovering a +// schema through describe_tool is exactly what it tells agents to do. +// +// `call_with` is dropped rather than retargeted: it answers "which variant do I +// dispatch through", a question this surface does not have. The permission +// signal it carried is not lost — `annotations` travels on the same definition +// and is the source `call_with` was derived from. +func applyDirectSurfaceIdentity(entry map[string]interface{}, displayName string) { + if displayName == "" { + return } - return meta, toolEntryOpts{}, nil + entry["name"] = displayName + delete(entry, "call_with") } // applyDescribeOutputSchema attaches the tool's declared output schema to a @@ -312,7 +340,7 @@ func (p *MCPProxyServer) handleDescribeToolOnSurface(ctx context.Context, reques definitions := make([]map[string]interface{}, 0, len(ids)) idErrors := make([]map[string]interface{}, 0) for _, id := range ids { - meta, opts, idErr := p.resolveDescribeDefinition(ctx, surface, id) + meta, opts, directDisplayName, idErr := p.resolveDescribeDefinition(ctx, surface, id) if idErr != nil { idErrors = append(idErrors, idErr) continue @@ -330,6 +358,12 @@ func (p *MCPProxyServer) handleDescribeToolOnSurface(ctx context.Context, reques // every tool that declares one (R2), which FR-010 forbids. applyDescribeOutputSchema(entry, meta.OutputSchemaJSON) + // #1083: on the direct surface the shared builder's canonical name and + // intent-variant `call_with` are both uncallable. Applied at the same + // assembly seam as output_schema, for the same reason — doing it inside + // the shared builder would change indexed-surface bytes. + applyDirectSurfaceIdentity(entry, directDisplayName) + definitions = append(definitions, entry) } diff --git a/internal/server/mcp_routing_deferred_tokens_test.go b/internal/server/mcp_routing_deferred_tokens_test.go index 338b43bbc..b8109bb36 100644 --- a/internal/server/mcp_routing_deferred_tokens_test.go +++ b/internal/server/mcp_routing_deferred_tokens_test.go @@ -2,6 +2,7 @@ package server import ( "encoding/json" + "fmt" "os" "testing" @@ -22,60 +23,51 @@ import ( // stays T075/T076. The encoder is cl100k_base — the same one the spec-083 // profiler pins, and the same one TestDescribeTool_DefinitionTokenBudget uses. // -// ── SC-001 IS NOT REACHABLE ON THIS CORPUS, AND THAT IS A SPEC-NUMBERS DEFECT ── +// ── SC-001 WAS RESTATED PER CORPUS SHAPE (maintainer decision, 2026-08-29) ── // -// SC-001 declares "≥70% smaller in tokens than full mode" on the frozen 45-tool -// reference corpus. Measured here (cl100k_base, real renderer): +// As originally written SC-001 asked for ≥70% reduction on the frozen 45-tool +// corpus and ≥85% on a ~100-tool fleet. T035a measured: // -// full = 6116 tokens (inputSchema 2604, annotations 1125, -// description 1784, name 286, outputSchema 0) -// deferred = 4301 tokens -// reduction = 29.7% +// corpus_v2.tools.json 45 tools 6116 → 4301 29.7% (asked ≥70%) +// livemcptool_snapshot 527 tools 99918 → 65138 34.8% (asked ≥85%) // -// The shortfall is arithmetic, not an implementation defect. Deferral can only -// remove the upstream inputSchema — 2604 of 6116 tokens, 42.6% of the payload. -// Everything else is either required by FR-004 (the untouched description, -// unchanged annotations, the name) or is the appended signature FR-004 mandates. -// So even a hypothetical listing that deleted BOTH the schema and the signature -// would reach only 38.9%: the ceiling on this corpus is below SC-001's floor. +// The ceiling on corpus_v2 is 38.9% — what deleting BOTH the schema and the +// signature would yield — so no implementation of this design could reach 70% +// on this corpus shape. The original number assumed schemas dominate the +// payload (spec-083 profiling put them at ~77%); at these shapes names, +// descriptions and annotations dominate, and FR-004 forbids touching those. // -// SC-001's threshold was calibrated on issue #971's fleet shape (~300 tokens per -// tool, ~30K per 100 tools), where the schema is assumed to be the dominant -// term. corpus_v2 is schema-light and description-heavy (~136 tokens/tool), and -// mcp-go's unconditional annotations block alone is a fixed ~25 tokens per entry -// that no serialization change can touch. +// SC-001 now asks ≥25% on corpus_v2 and ≥30% on the 527-tool snapshot — roughly +// 15% relative below each measured value, as regression headroom rather than a +// second projection. Both bounds are asserted here, so this file IS the SC-001 +// gate rather than a stand-in for one. // -// SC-001's second clause does not hold either. Measured the same way over the -// 527-tool livemcptool snapshot (specs/083-discovery-profiler/datasets/ -// livemcptool_snapshot/tools.json, ~190 tokens/tool — the closest thing this -// repo has to the "~100-tool fleet"): full = 99918, deferred = 65138, -// reduction = 34.8%, against SC-001's projected ≥85%. So the criterion is not -// merely mis-fitted to the reference corpus; the ≥70%/≥85% numbers assume the -// upstream schema is a far larger share of a tools/list payload than it is in -// either dataset available here. -// -// Truncating descriptions to close the gap is not available: FR-004 requires the -// existing description untouched. -// -// The gate asserted below is therefore a REGRESSION FLOOR on the implementation, -// not a restatement of SC-001. Resolving SC-001 itself — restating its threshold -// per corpus shape, or re-targeting the criterion at a schema-heavy corpus — is -// escalated to T076, where the criterion is formally re-measured and recorded. +// The 70% tripwire is retained: clearing it would mean the corpus premise +// behind the restatement has changed, and the criterion must be re-derived +// rather than left in place. + const ( // deferredCorpusPath is the frozen 45-tool reference corpus WITH schemas. // corpus_v1 (spec 065) carries no schemas and cannot measure deferral. deferredCorpusPath = "../../specs/083-discovery-profiler/datasets/corpus_v2.tools.json" - // deferredCorpusReductionFloor is the regression floor for the measured - // payload reduction (measured: 29.7%). Set just under it so an - // implementation regression — a schema leaking back into a deferred entry, a - // signature ballooning — fails, while corpus-neutral churn does not. + // deferredLargeCorpusPath is the 527-tool LiveMCPBench snapshot — the + // second half of the revised SC-001, and the only in-repo corpus large + // enough to stand in for a real fleet. + deferredLargeCorpusPath = "../../specs/083-discovery-profiler/datasets/livemcptool_snapshot/tools.json" + + // deferredCorpusReductionFloor is revised SC-001's reference-corpus bound + // (measured 29.7%, ceiling 38.9%). deferredCorpusReductionFloor = 0.25 - // deferredCorpusSC001Target is SC-001's declared threshold, kept here as the - // documented record of what the spec asks for versus what this corpus can - // deliver. Deliberately NOT asserted — see the defect note above. - deferredCorpusSC001Target = 0.70 + // deferredLargeCorpusReductionFloor is revised SC-001's fleet-scale bound + // (measured 34.8%). + deferredLargeCorpusReductionFloor = 0.30 + + // deferredCorpusSC001Tripwire is the ORIGINAL threshold, retained as an + // upper guard. Clearing it means the corpus premise behind the restatement + // no longer holds and SC-001 must be re-derived — not silently enjoyed. + deferredCorpusSC001Tripwire = 0.70 // deferredCorpusNonLossyFloor is SC-002 as written: ≥80% of corpus tools are // callable one-shot from the deferred listing, i.e. their signature is not @@ -144,6 +136,47 @@ func renderCorpusTokens(t *testing.T, mode string, tools []*config.ToolMetadata) return total } +// loadDeferredLargeCorpus reads the 527-tool LiveMCPBench snapshot. Its rows +// use `inputSchema` where corpus_v2 uses `schema`, and carry no tool_id, so it +// needs its own decoder rather than a shared one. +func loadDeferredLargeCorpus(t *testing.T) []*config.ToolMetadata { + t.Helper() + + raw, err := os.ReadFile(deferredLargeCorpusPath) + require.NoError(t, err, "the LiveMCPBench snapshot must be present") + + var corpus struct { + ToolCount int `json:"tool_count"` + Tools []struct { + Server string `json:"server"` + Tool string `json:"tool"` + Description string `json:"description"` + InputSchema json.RawMessage `json:"inputSchema"` + } `json:"tools"` + } + require.NoError(t, json.Unmarshal(raw, &corpus)) + require.Len(t, corpus.Tools, 527, "SC-001's fleet-scale bound is stated against the 527-tool snapshot") + + out := make([]*config.ToolMetadata, 0, len(corpus.Tools)) + for i, tool := range corpus.Tools { + params := string(tool.InputSchema) + if params == "" || params == "null" { + params = `{"type":"object"}` + } + out = append(out, &config.ToolMetadata{ + ServerName: tool.Server, + Name: tool.Tool, + Description: tool.Description, + ParamsJSON: params, + // Distinct per row: the signature cache is keyed by hash, so a + // shared one would collapse 527 tools onto a single signature and + // silently understate the deferred payload. + Hash: fmt.Sprintf("large-corpus-%d", i), + }) + } + return out +} + func TestDeferredDirect_TokenReduction_Corpus45(t *testing.T) { tools := loadDeferredCorpus(t) @@ -153,19 +186,40 @@ func TestDeferredDirect_TokenReduction_Corpus45(t *testing.T) { require.Positive(t, full) reduction := 1 - float64(deferred)/float64(full) - t.Logf("SC-001 measurement (corpus_v2, 45 tools, cl100k_base): full=%d deferred=%d reduction=%.1f%% (spec target %.0f%%)", - full, deferred, reduction*100, deferredCorpusSC001Target*100) + t.Logf("SC-001 (corpus_v2, 45 tools, cl100k_base): full=%d deferred=%d reduction=%.1f%% (revised bound %.0f%%)", + full, deferred, reduction*100, deferredCorpusReductionFloor*100) assert.GreaterOrEqualf(t, reduction, deferredCorpusReductionFloor, - "deferred rendering regressed: %.1f%% reduction is below the %.0f%% floor", + "revised SC-001: %.1f%% reduction is below the %.0f%% reference-corpus bound", reduction*100, deferredCorpusReductionFloor*100) - // A guard on the guard: if a future change ever DOES clear SC-001 on this - // corpus, the defect note above has gone stale and must be revisited rather - // than silently left in place. - assert.Lessf(t, reduction, deferredCorpusSC001Target, - "reduction now clears SC-001's %.0f%% on this corpus — update the spec-defect note and promote this to a real gate", - deferredCorpusSC001Target*100) + assert.Lessf(t, reduction, deferredCorpusSC001Tripwire, + "reduction now clears the original %.0f%% on this corpus — the shape premise behind the SC-001 restatement no longer holds; re-derive the criterion instead of leaving it", + deferredCorpusSC001Tripwire*100) +} + +// The fleet-scale half of revised SC-001. corpus_v2 is 45 tools; a real +// deployment is hundreds, and the reduction is a function of corpus shape, so +// asserting only the small corpus would leave half the criterion unenforced. +func TestDeferredDirect_TokenReduction_LargeCorpus(t *testing.T) { + tools := loadDeferredLargeCorpus(t) + + full := renderCorpusTokens(t, config.DirectToolResponseModeFull, tools) + deferred := renderCorpusTokens(t, config.DirectToolResponseModeDeferred, tools) + + require.Positive(t, full) + reduction := 1 - float64(deferred)/float64(full) + + t.Logf("SC-001 (livemcptool snapshot, %d tools, cl100k_base): full=%d deferred=%d reduction=%.1f%% (revised bound %.0f%%)", + len(tools), full, deferred, reduction*100, deferredLargeCorpusReductionFloor*100) + + assert.GreaterOrEqualf(t, reduction, deferredLargeCorpusReductionFloor, + "revised SC-001: %.1f%% reduction is below the %.0f%% fleet-scale bound", + reduction*100, deferredLargeCorpusReductionFloor*100) + + assert.Lessf(t, reduction, deferredCorpusSC001Tripwire, + "reduction now clears the original %.0f%% at fleet scale — re-derive SC-001 rather than leaving the restatement in place", + deferredCorpusSC001Tripwire*100) } func TestDeferredDirect_NonLossyShare_Corpus45(t *testing.T) { diff --git a/roadmap.yaml b/roadmap.yaml index 7e8dd93fa..68ae8e008 100644 --- a/roadmap.yaml +++ b/roadmap.yaml @@ -385,7 +385,7 @@ epics: spec: specs/102-schema-deferred pr: "#1063" depends_on: [] - note: "Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema (~88% smaller listing), with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% PROJECTION ABOVE: 29.7% on the frozen reference corpus and 34.8% on a 527-tool snapshot, against SC-001's >=70% target, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 is knowingly unmet and awaits a maintainer decision (restate per corpus shape, or re-target at a schema-heavy corpus) — see specs/102-schema-deferred/tasks.md T035a. Unblocks token-bench — and that measured shortfall is now the first thing token-bench has to explain." + note: "Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema, with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082; #1083/#1084 fixed in #1086. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% ORIGINALLY PROJECTED: 29.7% on the frozen 45-tool reference corpus and 34.8% on a 527-tool snapshot, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 was RESTATED per corpus shape (maintainer decision 2026-08-29): now >=25% on the 45-tool corpus and >=30% at fleet scale, both asserted in internal/server/mcp_routing_deferred_tokens_test.go, with the original 70% kept as an upper tripwire. Unblocks token-bench — and that measured shortfall is the first thing token-bench has to explain." - id: token-bench title: "Token-efficiency benchmark: measured savings, published results" diff --git a/specs/102-schema-deferred/plan.md b/specs/102-schema-deferred/plan.md index 3cfe679c3..cc3ea5fb1 100644 --- a/specs/102-schema-deferred/plan.md +++ b/specs/102-schema-deferred/plan.md @@ -124,7 +124,7 @@ Technical approach, grounded in the code: **Target Platform**: Linux/macOS/Windows core binary, personal + server editions — pure `internal/`, edition-agnostic (no `//go:build server` code touched; server-edition build verified). **Project Type**: Single Go project (core server). **Performance Goals**: Constitution I unaffected — serialization + registration-time work only; signatures come from the index-warmed cache via a pure read (`Peek`), no per-request compilation (FR-005); validation is memoized per tool hash and cheaper than the upstream round trip it prevents. -**Constraints**: FR-015 byte-stability of full-mode direct listings; FR-010 existing goldens pass unregenerated except the two enumerated FR-009 regens; FR-008 identical tool-set membership across modes; FR-013b fail-open validation; SC-001 ≥70% token reduction on the 45-tool corpus (**this constraint was NOT met — measured 29.7%, ceiling 38.9%; see spec.md SC-001**). +**Constraints**: FR-015 byte-stability of full-mode direct listings; FR-010 existing goldens pass unregenerated except the two enumerated FR-009 regens; FR-008 identical tool-set membership across modes; FR-013b fail-open validation; SC-001 token reduction (**restated 2026-08-29**: ≥25% on the 45-tool corpus and ≥30% at fleet scale, after the original ≥70%/≥85% proved unreachable — measured 29.7%/34.8%, ceiling 38.9%; see spec.md SC-001). **Scale/Scope**: ~6 packages touched: `internal/server` (core of the change), `internal/config`, `internal/runtime` (one clause), `internal/toolsig` (one method), `cmd/mcpproxy` (one flag), `bench/arms` (one measurement arm for the SC-001/SC-002 gates); plus docs and generated OpenAPI artifacts. ## Constitution Check diff --git a/specs/102-schema-deferred/spec.md b/specs/102-schema-deferred/spec.md index 1178263d7..d3b84cc10 100644 --- a/specs/102-schema-deferred/spec.md +++ b/specs/102-schema-deferred/spec.md @@ -2,7 +2,7 @@ **Feature Branch**: `102-schema-deferred` **Created**: 2026-08-24 -**Status**: Draft — SHIPPED 2026-08-29 (all 89 tasks, PR #1063; settings UI in #1082). **SC-001 is measured and UNMET** (29.7% / 34.8% vs a >=70% target, 38.9% ceiling) and awaits a maintainer decision; see the SC-001 note under Measurable Outcomes. +**Status**: Draft — SHIPPED 2026-08-29 (all 89 tasks, PR #1063; settings UI in #1082). SC-001 was measured as unmet against its original ≥70%/≥85% thresholds and has been **restated per corpus shape** (maintainer decision, 2026-08-29) — see Measurable Outcomes. **Input**: User description: "schema_deferred routing (issue #971): direct-mode tools/list keeps ALL tool names + descriptions but defers inputSchema; agents fetch full schemas on demand via describe_tool. Built as a thin composition over the Spec 085 compact-router machinery (compact signatures, describe_tool, pre-dispatch validation + self-healing errors) per the maintainer's accepted direction (issue #971 comment, 2026-08-13)." **Related**: #971 (accepted-direction, 2026-08-13). Spec 085 (compact router — the reused machinery: signature cache, `describe_tool`, pre-dispatch validation, self-healing invalid-params errors). Specs 098/099 (preflight evaluator + `describe_tool` `check` mode — the full describe_tool contract travels wherever the tool is exposed). Industry precedent: Atlassian mcp-compressor "Low Compression" (94 tools: 17,600 → 3,900 tokens), Alibaba Qoder two-phase discovery, MCP client best-practices Catalog → Inspect → Execute. @@ -19,11 +19,11 @@ Per the maintainer's accepted direction, this is **not a new routing mode**: `ro ### User Story 1 - Deferred enumeration: see everything, pay for schemas only when needed (Priority: P1) -An agent connects to the direct surface of a proxy with deferred serialization enabled. `tools/list` returns every visible upstream tool — same `serverName__toolName` names, same annotations — but each entry carries the description plus its precompiled compact signature (`*` = required, `~` = lossy) instead of a full `inputSchema`; the declared input schema is a minimal permissive object. For a 100-tool fleet the listing was projected to drop from ~30K to roughly 3.5–5K tokens; **the shipped measurement contradicts that projection** — a 527-tool snapshot went 99,918 → 65,138 tokens (~34.8%), because names, descriptions and annotations, not schemas, carry most of the payload (see the SC-001 note). For flat (non-lossy) tools the agent can still call directly from the signature with zero extra round trips, which held. +An agent connects to the direct surface of a proxy with deferred serialization enabled. `tools/list` returns every visible upstream tool — same `serverName__toolName` names, same annotations — but each entry carries the description plus its precompiled compact signature (`*` = required, `~` = lossy) instead of a full `inputSchema`; the declared input schema is a minimal permissive object. For a 100-tool fleet the listing was originally projected to drop from ~30K to roughly 3.5–5K tokens; **the shipped measurement contradicts that projection** — a 527-tool snapshot went 99,918 → 65,138 tokens (34.8%), because names, descriptions and annotations, not schemas, carry most of the payload. SC-001 has been restated accordingly (see Measurable Outcomes). For flat (non-lossy) tools the agent can still call directly from the signature with zero extra round trips, which held. **Why this priority**: This is the headline gap #971 names — full visibility at deferred-schema cost — and the entire token win of the feature. -**Independent Test**: Enable deferral on a fixture proxy with multiple upstream servers; fetch `tools/list` on the direct surface and assert: every tool present, no entry carries upstream schema properties, every entry's description ends with its compact signature, and total payload tokens are reduced versus full mode on the frozen 45-tool reference corpus. (The ≥70% figure this test originally named was not met — measured 29.7%; the shipped gate in `bench/arms` asserts a 25% floor plus an upper guard, per T076.) +**Independent Test**: Enable deferral on a fixture proxy with multiple upstream servers; fetch `tools/list` on the direct surface and assert: every tool present, no entry carries upstream schema properties, every entry's description ends with its compact signature, and total payload tokens are reduced by **≥25%** versus full mode on the frozen 45-tool reference corpus (revised SC-001; measured 29.7%). **Acceptance Scenarios**: @@ -180,22 +180,35 @@ An operator flips deferred serialization on (or off) in config with the proxy ru ### Measurable Outcomes -- **SC-001** (**NOT MET — measured, escalated, awaiting maintainer decision**): As written, this - required a ≥70% token reduction on the frozen 45-tool reference corpus and ≥85% on a ~100-tool - fleet. +- **SC-001** (**REVISED 2026-08-29 — maintainer decision; original thresholds were unreachable**): + Deferred direct-surface `tools/list` payload is **≥25% smaller in tokens** than full mode on the + frozen 45-tool reference corpus, and **≥30% smaller** on the 527-tool LiveMCPBench snapshot, both + measured by the spec-083 profiler pipeline with its pinned `cl100k_base` encoder. Both corpora are + in-repo and both bounds are asserted (`internal/server/mcp_routing_deferred_tokens_test.go`). - > **Measured outcome (2026-08-29, T035a).** 29.7% on the reference corpus and 34.8% on a 527-tool - > snapshot (99,918 → 65,138 tokens). The **arithmetic ceiling is 38.9%** — that is what deleting - > *both* the schema and the signature would yield — so no implementation of this design can reach - > 70% on this corpus shape. The target rested on Spec 083 profiling attributing ~77% of the payload - > to schemas; at the corpus shapes actually measured, names, descriptions and annotations dominate. + > **Why the number changed.** As originally written SC-001 asked for ≥70% on the reference corpus + > and ≥85% on a ~100-tool fleet. Measured (T035a): > - > This is recorded rather than quietly restated: the feature works and delivers a real ~30% saving, - > but the criterion as written is unreachable. **Two options for the maintainer:** restate the - > threshold per corpus shape, or re-target SC-001 at a schema-heavy corpus where the original - > premise holds. Until then the shipped gate (`bench/arms`, T076) asserts a 25% floor plus an upper - > guard that fails loudly if anything ever does clear 70%. See tasks.md T035a and the Phase 3 - > checkpoint for the full measurement table. + > | corpus | tools | full | deferred | reduction | originally asked | + > |---|---|---|---|---|---| + > | `corpus_v2.tools.json` (frozen reference) | 45 | 6116 | 4301 | **29.7%** | ≥70% | + > | `livemcptool_snapshot/tools.json` | 527 | 99918 | 65138 | **34.8%** | ≥85% | + > + > The **arithmetic ceiling on the reference corpus is 38.9%** — what deleting *both* the schema and + > the signature would yield — so no implementation of this design could have reached 70% on this + > corpus shape. The original target rested on Spec 083 profiling attributing ~77% of the payload to + > schemas; at the shapes actually measured, names, descriptions and annotations dominate, and FR-004 + > forbids touching those. + > + > **The revised thresholds sit ~15% relative below each measured value** (29.7 → 25, 34.8 → 30), + > which is regression headroom for corpus-neutral churn rather than a second projection. They are + > deliberately stated per corpus, because the reduction is a function of corpus shape, not of the + > implementation. + > + > A **70% tripwire is retained and asserted**: if either corpus ever clears the original figure, the + > premise behind this restatement has changed and the criterion must be re-derived rather than + > silently left in place. A schema-heavy corpus would legitimately behave that way. + - **SC-002**: ≥80% of corpus tools (the non-lossy share; Spec 085 lossy-rate gate <20%) are callable one-shot from the deferred listing — zero `describe_tool` round trips — verified against the corpus signature set. - **SC-003**: An agent whose guessed arguments fail validation completes the call in exactly one retry using only the error-embedded schema (no additional discovery calls), demonstrated in E2E. - **SC-004**: With deferral off, automated comparison shows direct-surface responses byte-identical to pre-feature behavior modulo the single enumerated surface delta (FR-010); the existing tool-surface goldens pass **unregenerated** (the delta lands on a surface they do not cover), and the new direct-surface built-in gate pins that delta. diff --git a/specs/102-schema-deferred/tasks.md b/specs/102-schema-deferred/tasks.md index f39638609..fd9b9050a 100644 --- a/specs/102-schema-deferred/tasks.md +++ b/specs/102-schema-deferred/tasks.md @@ -104,7 +104,19 @@ So T004 gates Phase 1; T001–T003 gate their own dependents and are otherwise c **Checkpoint**: US1 is independently demonstrable — flip the config, fetch `tools/list`, observe the token drop with the full tool set intact. -> ### ⚠️ Escalation from T035a — SC-001's thresholds are not reachable +> ### ✅ RESOLVED 2026-08-29 — SC-001 restated per corpus shape (was: escalation from T035a) +> +> **Maintainer decision:** adapt the threshold. SC-001 now asks **≥25%** on the frozen +> 45-tool corpus and **≥30%** on the 527-tool snapshot — roughly 15% relative below each +> measured value, as regression headroom rather than a second projection. Both bounds are +> now ASSERTED in `internal/server/mcp_routing_deferred_tokens_test.go` +> (`TestDeferredDirect_TokenReduction_Corpus45` / `_LargeCorpus`), so that file is the +> SC-001 gate rather than a stand-in. The original 70% is retained as an upper tripwire: +> clearing it means the corpus premise changed and the criterion must be re-derived. +> +> The original escalation is kept below, because it is the evidence the decision rests on. +> +> ### ⚠️ Escalation from T035a — SC-001's original thresholds were not reachable > > T035a ran the gates and SC-002 passes (39/45 = 86.7% one-shot callable, floor > 80%). **SC-001 does not, and cannot on either dataset in this repo.** Measured