Skip to content

Commit 71e04e2

Browse files
authored
fix: direct-surface describe ids (#1083), config-apply errors (#1084); restate SC-001 (#1086)
* fix: direct-surface describe ids (#1083), config-apply errors (#1084); restate SC-001 Closes #1083. Closes #1084. #1083 — describe_tool on the DIRECT surface returned identifiers that surface cannot be asked for: the canonical "<server>:<tool>" (it registers "<server>__<tool>") and a `call_with` naming call_tool_read/write/destructive (intent variants the direct surface does not expose). Following either verbatim gave "tool not found". Deferred mode makes this the live discovery path, since it tells agents to recover schemas through describe_tool. applyDirectSurfaceIdentity sits at the same assembly seam as output_schema — putting it in the shared builder would change indexed-surface bytes. It swaps the registered display name in and drops `call_with`. Dropped rather than retargeted: it answers "which variant do I dispatch through", a question this surface does not have. No signal is lost — DeriveCallWith is a pure function of `annotations`, which travels on the same definition, so a consumer can recompute the tier. The D10 safety test is re-pointed at that source rather than deleted. Both id spellings still WORK as input; only the echo changes. Verified live: the identifier describe_tool returns now executes. #1084 — three defects: 1. A value the validator rejects returned 500. ApplyConfig discarded its structured errors, so handlers could not tell "you sent a bad value" from "we failed to persist a good one" without string-matching. The errors now ride on the result and writeApplyConfigError classifies: 400 with the offending field in a structured payload, 500 reserved for real faults. 2. internal/httpapi/server.go called SugaredLogger.Error(msg, "key", val), which CONCATENATES — hence "patcherrorconfiguration...". 97 call sites converted to the *w variants, including ones passing zap.Field values to the sugared API (broken a second way). Argument-parity checked per call. 3. /api/v1/status embedded a status snapshot whose upstream_stats and tools_indexed nothing refreshed — stuck at "Connecting"/0 for the life of a process whose servers were long Ready, disagreeing with the correct top-level field. Cross-model review caught two defects in my first attempt: refreshing inside Server.GetStatus() made every poll walk all servers TWICE, and the SSE stream forwards a snapshot that never passes through GetStatus() at all, so it stayed stale. Now a withLiveUpstreamStats helper at the API layer, applied at all three emission sites, each reusing the traversal it already does for its top-level field. It copies rather than mutates — the SSE map is owned by the publisher. SC-001 restated per corpus shape (maintainer decision). The original ≥70%/≥85% was unreachable: measured 29.7% (45-tool corpus) and 34.8% (527-tool snapshot) against an arithmetic ceiling of 38.9%, because the target assumed schemas dominate the payload and names/descriptions/annotations actually do. Now ≥25% and ≥30% — roughly 15% relative headroom below each measured value, as a regression floor rather than a second projection. Both bounds are asserted, so that test file IS the SC-001 gate; a new fleet-scale case independently reproduces T035a's numbers from its own loader. The original 70% is retained as an upper tripwire: clearing it means the corpus premise changed. * docs(changelog): record spec 102, the settings UI, and these fixes The Unreleased section is hand-written, and the flagship feature of the next release was missing from it entirely: schema-deferred direct mode shipped in #1063 with no changelog entry, and the settings UI in #1082 likewise. The feature entry states the two things a reader most needs and would otherwise get wrong: that it is OPT-IN (default `full`, unchanged behaviour), and that the measured saving is ~30% — not the ~88% the roadmap projected until today. Any launch copy drafted from the old number would be wrong. Also records #1083 and #1084 under Bug Fixes.
1 parent cb97dd1 commit 71e04e2

14 files changed

Lines changed: 790 additions & 206 deletions

‎CHANGELOG.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,24 @@ Releases follow [Semantic Versioning](https://semver.org/).
2323

2424
### Features
2525

26+
- **mcp:** schema-deferred direct mode — a new `direct_tool_response_mode` key (`full` | `deferred`,
27+
**default `full`, so this is opt-in and changes nothing until you turn it on**). In `deferred`,
28+
the direct enumeration surface (`/mcp/all`, and `/mcp` under `routing_mode: "direct"`) lists every
29+
tool with its name, description, annotations and a compact Spec-085 signature, advertising
30+
`inputSchema` as exactly `{"type":"object"}` and dropping `outputSchema`; agents recover a full
31+
schema on demand via `describe_tool`, and a call whose arguments do not fit is rejected
32+
**before** it reaches the upstream, with the schema attached so the agent can self-correct.
33+
Hot-reloadable — flipping it rebuilds the listing and notifies connected clients, no restart.
34+
**Measured token saving is ~30%** (29.7% on the 45-tool reference corpus, 34.8% on a 527-tool
35+
snapshot) — materially less than the ~88% originally projected, because names, descriptions and
36+
annotations, not schemas, dominate the payload. (spec 102, [#971](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/971))
37+
- **settings:** both serialization modes are now reachable from the UI — Settings → General →
38+
"Detail in tool-search results" (`tool_response_mode`) and "Detail in Direct-mode listings"
39+
(`direct_tool_response_mode`), in the Web UI and the macOS tray. They previously had four ways to
40+
set them (config file, env var, serve flag, REST) and no UI at all, so a tray-only user could not
41+
reach `direct_tool_response_mode`. Neither carries a restart badge — both hot-reload, unlike
42+
`routing_mode` above them.
43+
2644
- **mcp:** `describe_tool` check mode — an optional `check: true` returns one availability
2745
verdict per id (up to 50) from the spec-098 preflight evaluator instead of schemas, so an
2846
agent can gate a multi-step plan without leaving the MCP session. Optional `filters`
@@ -31,6 +49,24 @@ Releases follow [Semantic Versioning](https://semver.org/).
3149

3250
### Bug Fixes
3351

52+
- **mcp/describe_tool:** on the direct surface, the returned definition named the tool
53+
`<server>:<tool>` and recommended a `call_with` intent variant — neither of which that surface
54+
can be asked for, so an agent following the response verbatim got "tool not found". It now
55+
echoes the registered `<server>__<tool>` name and omits `call_with` (the permission signal
56+
remains in `annotations`, which it was derived from). Both id spellings still work as input.
57+
([#1083](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1083))
58+
- **api:** a config value the validator rejects now returns **400** with the offending field in a
59+
structured `validation_errors` payload, instead of 500. 500 is reserved for genuine failures to
60+
persist a valid config. ([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084))
61+
- **api:** `/api/v1/status` embedded a status snapshot whose `upstream_stats` and `tools_indexed`
62+
were never refreshed — reporting `"Connecting"` and `0` for the life of a process whose servers
63+
were long ready, while the sibling top-level field was correct. Both now come from one live
64+
source, on the REST poll and on the SSE stream.
65+
([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084))
66+
- **logging:** ~97 log calls in the REST API passed key/value pairs to a sugared logger method that
67+
concatenates them, producing runs like `patcherrorconfiguration...` instead of structured fields.
68+
([#1084](https://github.com/smart-mcp-proxy/mcpproxy-go/issues/1084))
69+
3470
- **homebrew:** One-line install in docs + guard tap job against pre-release tags (#486) ([#486](https://github.com/smart-mcp-proxy/mcpproxy-go/pull/486)) ([`1098701`](https://github.com/smart-mcp-proxy/mcpproxy-go/commit/109870116fe17aa1ce7ecfd603962c1d3de21ba0))
3571

3672
### CI/Build

‎ROADMAP.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -726,7 +726,7 @@ graph LR
726726
<details>
727727
<summary>🟢 Deferred-schema serialization for the direct tools/list surface (spec 102) — Done · P1</summary>
728728

729-
> Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema (~88% smaller listing), with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% PROJECTION ABOVE: 29.7% on the frozen reference corpus and 34.8% on a 527-tool snapshot, against SC-001's >=70% target, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 is knowingly unmet and awaits a maintainer decision (restate per corpus shape, or re-target at a schema-heavy corpus) — see specs/102-schema-deferred/tasks.md T035a. Unblocks token-bench — and that measured shortfall is now the first thing token-bench has to explain.
729+
> Direct mode enumerates every upstream tool but always ships full inputSchema (~30K tokens for a 100-tool fleet; Spec 083 profiling put ~77% of the payload in schemas agents rarely read). Deferred serialization keeps every tool name, description and annotation and appends the Spec 085 compact signature instead of the schema, with describe_tool on the direct surface to recover it and the shipped pre-dispatch validation turning a wrong guess into one self-healing retry. Not a new routing_mode — a serialization mode of the direct surface, on the same tool_response_mode axis that already governs retrieve_tools. Spec merged in #1035 (issue #971, maintainer-accepted direction). COMPLETE: all 89 tasks shipped in #1063; the settings UI for both serialization axes followed in #1082; #1083/#1084 fixed in #1086. MEASURED SAVINGS FELL WELL SHORT OF THE ~88% ORIGINALLY PROJECTED: 29.7% on the frozen 45-tool reference corpus and 34.8% on a 527-tool snapshot, with 38.9% the arithmetic ceiling even if both the schema and the signature were deleted. The projection assumed schemas dominate the payload; names, descriptions and annotations turn out to carry most of it. SC-001 was RESTATED per corpus shape (maintainer decision 2026-08-29): now >=25% on the 45-tool corpus and >=30% at fleet scale, both asserted in internal/server/mcp_routing_deferred_tokens_test.go, with the original 70% kept as an upper tripwire. Unblocks token-bench — and that measured shortfall is the first thing token-bench has to explain.
730730

731731
Spec: [102-schema-deferred](./specs/102-schema-deferred/) · PR: #1063
732732

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
package httpapi
2+
3+
import (
4+
"bytes"
5+
"encoding/json"
6+
"errors"
7+
"net/http"
8+
"net/http/httptest"
9+
"testing"
10+
11+
"github.com/stretchr/testify/assert"
12+
"github.com/stretchr/testify/require"
13+
"go.uber.org/zap"
14+
15+
"github.com/smart-mcp-proxy/mcpproxy-go/internal/config"
16+
runtime "github.com/smart-mcp-proxy/mcpproxy-go/internal/runtime"
17+
)
18+
19+
// #1084: PATCH /api/v1/config answered 500 for a value the config validator
20+
// rejected — e.g. {"direct_tool_response_mode":"bogus"}. The behaviour was safe
21+
// (the file was left untouched) but the status class was wrong, and the status
22+
// class is what tells a client whether retrying could ever help.
23+
type applyErrController struct {
24+
baseController
25+
result *runtime.ConfigApplyResult
26+
err error
27+
}
28+
29+
func (m *applyErrController) GetCurrentConfig() any { return &config.Config{APIKey: "k"} }
30+
func (m *applyErrController) GetConfig() (*config.Config, error) {
31+
return &config.Config{Listen: "127.0.0.1:8080"}, nil
32+
}
33+
func (m *applyErrController) GetConfigPath() string { return "/tmp/mcp_config.json" }
34+
func (m *applyErrController) ApplyConfig(*config.Config, string) (*runtime.ConfigApplyResult, error) {
35+
return m.result, m.err
36+
}
37+
38+
func patchConfigWith(t *testing.T, ctrl *applyErrController, body map[string]any) *httptest.ResponseRecorder {
39+
t.Helper()
40+
srv := NewServer(ctrl, zap.NewNop().Sugar(), nil)
41+
raw, err := json.Marshal(body)
42+
require.NoError(t, err)
43+
req := httptest.NewRequest(http.MethodPatch, "/api/v1/config", bytes.NewReader(raw))
44+
req.Header.Set("Content-Type", "application/json")
45+
req.Header.Set("X-API-Key", "k")
46+
w := httptest.NewRecorder()
47+
srv.ServeHTTP(w, req)
48+
return w
49+
}
50+
51+
// A rejected value is the operator's mistake: 400, with the offending field
52+
// named in a structured payload rather than only inside the prose message.
53+
func TestPatchConfig_RejectedValueIs400WithValidationErrors(t *testing.T) {
54+
w := patchConfigWith(t, &applyErrController{
55+
result: &runtime.ConfigApplyResult{
56+
Success: false,
57+
ValidationErrors: []config.ValidationError{
58+
{Field: "direct_tool_response_mode", Message: `invalid direct tool response mode: bogus (must be "full" or "deferred")`},
59+
},
60+
},
61+
err: errors.New(`configuration validation failed: direct_tool_response_mode: invalid direct tool response mode: bogus`),
62+
}, map[string]any{"direct_tool_response_mode": "bogus"})
63+
64+
require.Equal(t, http.StatusBadRequest, w.Code, "body=%s", w.Body.String())
65+
66+
var envelope map[string]interface{}
67+
require.NoError(t, json.Unmarshal(w.Body.Bytes(), &envelope))
68+
assert.Equal(t, false, envelope["success"])
69+
70+
data, ok := envelope["data"].(map[string]interface{})
71+
require.True(t, ok, "the 400 must carry a structured payload, not just prose: %s", w.Body.String())
72+
errs, ok := data["validation_errors"].([]interface{})
73+
require.True(t, ok, "validation_errors must be present")
74+
require.Len(t, errs, 1)
75+
76+
first, ok := errs[0].(map[string]interface{})
77+
require.True(t, ok)
78+
assert.Equal(t, "direct_tool_response_mode", first["field"],
79+
"a client must be able to point at the offending field without parsing the message")
80+
}
81+
82+
// A failure with no validation errors is a genuine server fault and must stay
83+
// in the 500 class — otherwise the fix would mislabel real outages as bad input.
84+
func TestPatchConfig_PersistFailureStays500(t *testing.T) {
85+
w := patchConfigWith(t, &applyErrController{
86+
result: &runtime.ConfigApplyResult{Success: false},
87+
err: errors.New("failed to write config file: disk full"),
88+
}, map[string]any{"tools_limit": 20})
89+
90+
require.Equal(t, http.StatusInternalServerError, w.Code, "body=%s", w.Body.String())
91+
assert.NotContains(t, w.Body.String(), "validation_errors")
92+
}
93+
94+
// A nil result alongside an error must not panic the classifier.
95+
func TestPatchConfig_NilResultWithErrorStays500(t *testing.T) {
96+
w := patchConfigWith(t, &applyErrController{
97+
result: nil,
98+
err: errors.New("controller exploded"),
99+
}, map[string]any{"tools_limit": 20})
100+
101+
require.Equal(t, http.StatusInternalServerError, w.Code, "body=%s", w.Body.String())
102+
}

0 commit comments

Comments
 (0)