From 4af156a8e6e4ce465813ac6e17aab22a20659d37 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sat, 22 Aug 2026 07:25:14 +0300 Subject: [PATCH] fix(tray): compact the tray menu and stop rowing code_execution wrappers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The status-bar menu had grown long enough to scroll, which is the opposite of a glance, and the glance itself spent a row on an internal primitive. Menu length (3 rows back): - Drop the grey, disabled "N quarantined server(s)" line — Needs Attention already carries that fact, and actionably. - Drop "Report an Issue…" — the tracker is still reachable through the GitHub link in the About panel's credits. - Move "Add Server..." into the Servers submenu as its first item (Cmd+N still matches; performKeyEquivalent recurses into submenus). It stays at the top level only when there are no servers and therefore no submenu to hold it, so adding the first server is never unreachable. code_execution is an internal primitive, not a call the user made. Since its sub-calls became tool_call records of their own (#1016), they show the REAL work — "jira:get_issue ×3" — where the wrapper row said only "code_execution", naming no server and no tool. A SUCCESSFUL wrapper now rows in neither the tray glance nor the shared usage timeline; a FAILED one still does, through rule 3's failure branch, because a script that died before dispatch has no children to speak for it. Kept in lockstep: GlanceSelection.glanceInternalTools (Swift), AppState.countsTowardUsageTimeline (follows the same set), and runtime.glanceInternalTools (Go). usageAdmissionVersion 3 → 4 so snapshots counted under the old rule are rebuilt instead of carried forward. The Web UI activity list still shows code_execution parents — it is the drill-down home for a script — so the parity test now models the glance as the list filter plus the glance's own narrowing rather than equating the two. Co-Authored-By: Claude Fable 5 --- internal/runtime/usage_aggregate.go | 50 ++++--- internal/runtime/usage_catchup_test.go | 2 +- internal/runtime/usage_glance_parity_test.go | 85 +++++++++-- .../macos/MCPProxy/MCPProxy/MCPProxyApp.swift | 59 +++++--- .../Menu/Glance/GlanceSelection.swift | 26 +++- .../MCPProxy/MCPProxy/State/AppState.swift | 5 +- .../GlanceHeaderConsistencyTests.swift | 8 +- .../MCPProxyTests/GlanceSelectionTests.swift | 59 ++++++-- .../MCPProxyTests/MenuStructureTests.swift | 140 ++++++++++++++++++ .../contracts/usage-endpoint.md | 2 +- 10 files changed, 361 insertions(+), 75 deletions(-) create mode 100644 native/macos/MCPProxy/MCPProxyTests/MenuStructureTests.swift diff --git a/internal/runtime/usage_aggregate.go b/internal/runtime/usage_aggregate.go index 39db6337f..9ebee1634 100644 --- a/internal/runtime/usage_aggregate.go +++ b/internal/runtime/usage_aggregate.go @@ -150,10 +150,11 @@ type UsageAggregate struct { // usageAdmissionVersion identifies the CURRENT admission rule for the // aggregate. Bump it whenever Apply changes which records are counted (2: // internal tool calls and blocked policy decisions joined the timeline; 3: -// management built-ins excluded again per GlanceSelection rule 1), so -// pre-change snapshots are rebuilt instead of carried forward with hours -// counted under the old rule. -const usageAdmissionVersion = 3 +// management built-ins excluded again per GlanceSelection rule 1; 4: successful +// code_execution wrappers excluded — their sub-calls are tool_call records that +// count on their own), so pre-change snapshots are rebuilt instead of carried +// forward with hours counted under the old rule. +const usageAdmissionVersion = 4 func newUsageAggregate() *UsageAggregate { return &UsageAggregate{ @@ -197,12 +198,14 @@ func (a *UsageAggregate) tool(server, toolName string) *ToolUsage { // glance row-for-row, or the bars under the list disagree with the list // above them — the single most confusing thing a dashboard can do. // -// The glance shows tool_calls, the internal tool calls that are mcpproxy's own -// work (retrieve_tools, describe_tool, code_execution, upstream_servers …) and -// blocked policy decisions, so all three enter the timeline. Internal -// call_tool_* records are the one exclusion: they mirror a direct dispatch that -// ALREADY emitted its own tool_call record with the same request id, so -// counting them too would double every direct call. +// The glance shows tool_calls, some of the internal tool calls that are +// mcpproxy's own work (retrieve_tools, describe_tool — plus any internal call +// that FAILED) and blocked policy decisions, so all three enter the timeline. +// Internal call_tool_* records are excluded: they mirror a direct dispatch that +// ALREADY emitted its own tool_call record with the same request id, so counting +// them too would double every direct call. Successful code_execution records are +// excluded for the same double-counting reason one level up — see +// applyInternalToolCall. func (a *UsageAggregate) Apply(rec *storage.ActivityRecord) { if rec == nil || rec.ToolName == "" { return @@ -295,11 +298,14 @@ func (a *UsageAggregate) applyBlocked(rec *storage.ActivityRecord) { // handleCallToolVariant, which emits both) — counting both would double it. // // SUCCESSFUL internal calls count only for the built-ins the glance actually -// rows (GlanceSelection rule 3: retrieve_tools, describe_tool, -// code_execution): a successful upstream_servers or quarantine_security call -// is management chatter the glance hides, and counting it would put bars over -// hours with no rows. Failures count regardless — any internal failure is a -// glance row. +// rows (GlanceSelection rule 3: retrieve_tools, describe_tool): a successful +// upstream_servers or quarantine_security call is management chatter the glance +// hides, and counting it would put bars over hours with no rows. A successful +// code_execution is out for a different reason — the sub-calls its script made +// are tool_call records that already count, so the wrapper would double them +// under a name no upstream owns. Failures count regardless — any internal +// failure is a glance row, and a script that died before dispatching anything +// has nothing else to represent it. func (a *UsageAggregate) applyInternalToolCall(rec *storage.ActivityRecord) { if strings.HasPrefix(rec.ToolName, storage.InternalCallToolPrefix) { return @@ -319,13 +325,19 @@ func (a *UsageAggregate) applyInternalToolCall(rec *storage.ActivityRecord) { // glanceInternalTools are the built-ins whose SUCCESSFUL calls appear as rows // in the tray glance (GlanceSelection.swift rule 3) and therefore in the -// timeline; glanceManagementBuiltins are excluded from the glance entirely -// (rule 1), success or failure. Keep both in lockstep with the Swift sets and -// with AppState.countsTowardUsageTimeline. +// timeline: retrieve_tools and describe_tool. code_execution is NOT one of them +// — it is an internal primitive, and the upstream calls its scripts make are +// tool_call records that count on their own, so counting the wrapper too would +// bar a script twice while naming no tool. A FAILED code_execution still counts, +// through the failure branch in applyInternalToolCall: it has no sub-calls to +// speak for it. +// +// glanceManagementBuiltins are excluded from the glance entirely (rule 1), +// success or failure. Keep both in lockstep with the Swift sets and with +// AppState.countsTowardUsageTimeline. var glanceInternalTools = map[string]bool{ "retrieve_tools": true, "describe_tool": true, - "code_execution": true, } var glanceManagementBuiltins = map[string]bool{ diff --git a/internal/runtime/usage_catchup_test.go b/internal/runtime/usage_catchup_test.go index 367c00d04..6ceb7a563 100644 --- a/internal/runtime/usage_catchup_test.go +++ b/internal/runtime/usage_catchup_test.go @@ -132,7 +132,7 @@ func TestActivityService_SnapshotLoad_RebuildsWhenTheAdmissionRuleChanged(t *tes // The store holds an internal call the OLD rule never counted. require.NoError(t, mgr.SaveActivity(&storage.ActivityRecord{ - Type: storage.ActivityTypeInternalToolCall, ToolName: "code_execution", + Type: storage.ActivityTypeInternalToolCall, ToolName: "retrieve_tools", Status: storage.ActivityStatusSuccess, Timestamp: base, })) diff --git a/internal/runtime/usage_glance_parity_test.go b/internal/runtime/usage_glance_parity_test.go index c2575f477..bb3107c46 100644 --- a/internal/runtime/usage_glance_parity_test.go +++ b/internal/runtime/usage_glance_parity_test.go @@ -10,24 +10,52 @@ import ( "github.com/smart-mcp-proxy/mcpproxy-go/internal/storage" ) -// glanceRows applies the SAME admission rule the activity list uses by default -// (storage.DefaultActivityFilter) so the two populations can be compared -// directly rather than by eye. +// glanceRows counts the rows the TRAY GLANCE renders for these records, so the +// timeline can be compared against them directly rather than by eye. +// +// It is two stages, because the glance is NOT the activity list: +// +// stage 1 — storage.DefaultActivityFilter, the list's own default admission. +// It is what the Web UI shows, and it deliberately keeps the code_execution +// parent row: the list is that script's drill-down home. +// stage 2 — the glance's extra narrowing (GlanceSelection rules 1 and 3). The +// tray has five rows, so it drops what something else already represents: a +// successful code_execution (its sub-calls are tool_call records of their +// own), successful management chatter, and sheds that never ran. +// +// Stage 2 is spelled out literally instead of reading glanceInternalTools / +// glanceManagementBuiltins, so this stays an independent oracle rather than a +// restatement of the code under test. func glanceRows(records []*storage.ActivityRecord) int { filter := storage.DefaultActivityFilter() + rowsOnSuccess := map[string]bool{"retrieve_tools": true, "describe_tool": true} + management := map[string]bool{"upstream_servers": true, "quarantine_security": true} + n := 0 for _, rec := range records { - if filter.Matches(rec) { - n++ + if !filter.Matches(rec) { + continue + } + if rec.Type == storage.ActivityTypeInternalToolCall { + if management[rec.ToolName] { + continue // rule 1: never a row, whatever the status + } + if rec.Status == storage.ActivityStatusSuccess && !rowsOnSuccess[rec.ToolName] { + continue // rule 3: on success, only the discovery built-ins row + } } + if rec.Type == storage.ActivityTypeToolCall && rec.Status == storage.ActivityStatusRejected { + continue // spec 093: a shed call never executed + } + n++ } return n } // The bars under the Recent list have to count the rows in it. They used to -// count tool_calls only, so an agent whose traffic was mostly retrieve_tools -// and code_execution — which is most agents — looked at a list of work under an -// almost empty histogram. +// count tool_calls only, so an agent whose traffic was mostly retrieve_tools — +// which is most agents — looked at a list of work under an almost empty +// histogram. func TestUsageAggregate_TimelineMatchesTheGlancePopulation(t *testing.T) { ts := time.Date(2026, 6, 1, 10, 0, 0, 0, time.UTC) records := []*storage.ActivityRecord{ @@ -37,7 +65,10 @@ func TestUsageAggregate_TimelineMatchesTheGlancePopulation(t *testing.T) { // mcpproxy's own built-ins: visible in the glance, and until now // invisible in the histogram. {Type: storage.ActivityTypeInternalToolCall, ToolName: "retrieve_tools", Status: storage.ActivityStatusSuccess, DurationMs: 5, Timestamp: ts}, + // A script that ran: the glance shows the calls it MADE, not the + // wrapper, so this record is a row in neither list nor histogram. {Type: storage.ActivityTypeInternalToolCall, ToolName: "code_execution", Status: storage.ActivityStatusSuccess, DurationMs: 400, Timestamp: ts}, + {Type: storage.ActivityTypeToolCall, ServerName: "jira", ToolName: "get_issue", Status: storage.ActivityStatusSuccess, DurationMs: 30, Timestamp: ts, ParentID: "exec-1"}, {Type: storage.ActivityTypeInternalToolCall, ToolName: "describe_tool", Status: storage.ActivityStatusError, DurationMs: 2, Timestamp: ts}, // A policy block: a red row in the glance. {Type: storage.ActivityTypePolicyDecision, ServerName: "evil", ToolName: "exfil", Status: storage.ActivityStatusBlocked, Timestamp: ts}, @@ -52,10 +83,42 @@ func TestUsageAggregate_TimelineMatchesTheGlancePopulation(t *testing.T) { require.Len(t, timeline, 1) assert.EqualValues(t, glanceRows(records), timeline[0].Calls, "every row the user sees must be a call in the histogram") + assert.EqualValues(t, 6, timeline[0].Calls, + "two dispatches, the script's sub-call, retrieve_tools, the failed "+ + "describe_tool and the block — but not the code_execution wrapper") assert.EqualValues(t, 3, timeline[0].Errors, "the failed dispatch, the failed built-in and the block are all failures") } +// code_execution is an internal PRIMITIVE, not a call the user made: a script +// that ran is represented by the upstream calls it issued, which are tool_call +// records that count on their own. Counting the wrapper too would put a bar +// under a name no upstream owns and double the script. +// +// A FAILED one is the exception, and the reason is that it has no children: a +// script that died of a syntax error dispatched nothing, so its own record is +// the only trace of the attempt anywhere. +func TestUsageAggregate_CodeExecutionWrapperCountsOnlyWhenItFailed(t *testing.T) { + ts := time.Date(2026, 6, 1, 10, 0, 0, 0, time.UTC) + + agg := newUsageAggregate() + agg.Apply(&storage.ActivityRecord{ + Type: storage.ActivityTypeInternalToolCall, ToolName: "code_execution", + Status: storage.ActivityStatusSuccess, DurationMs: 400, Timestamp: ts, + }) + assert.Empty(t, agg.Timeline(), "a successful script bars through its sub-calls, not through itself") + assert.Empty(t, agg.Tools, "and it is not an upstream tool either") + + agg.Apply(&storage.ActivityRecord{ + Type: storage.ActivityTypeInternalToolCall, ToolName: "code_execution", + Status: storage.ActivityStatusError, DurationMs: 3, Timestamp: ts, + }) + timeline := agg.Timeline() + require.Len(t, timeline, 1) + assert.EqualValues(t, 1, timeline[0].Calls, "a script that never dispatched has only its own row") + assert.EqualValues(t, 1, timeline[0].Errors) +} + // The direct dispatch path emits BOTH a tool_call record and a call_tool_* // internal record for one call. The glance hides the internal one; so must the // histogram, or every direct call is counted twice. @@ -97,9 +160,9 @@ func TestUsageAggregate_BuiltinsStayOutOfThePerToolRollup(t *testing.T) { } // Successful management built-ins (upstream_servers, quarantine_security, …) -// are not glance rows — GlanceSelection rule 3 admits only retrieve_tools, -// describe_tool and code_execution plus any internal failure — so a config -// sweep must not paint bars over an hour with no visible rows. +// are not glance rows — GlanceSelection rule 3 admits only retrieve_tools and +// describe_tool plus any internal failure — so a config sweep must not paint +// bars over an hour with no visible rows. func TestUsageAggregate_SuccessfulManagementBuiltinsStayOutOfTheTimeline(t *testing.T) { ts := time.Date(2026, 6, 1, 10, 0, 0, 0, time.UTC) agg := newUsageAggregate() diff --git a/native/macos/MCPProxy/MCPProxy/MCPProxyApp.swift b/native/macos/MCPProxy/MCPProxy/MCPProxyApp.swift index ddbe0bac7..ff9470b81 100644 --- a/native/macos/MCPProxy/MCPProxy/MCPProxyApp.swift +++ b/native/macos/MCPProxy/MCPProxy/MCPProxyApp.swift @@ -659,6 +659,19 @@ final class AppController: NSObject, NSApplicationDelegate, NSWindowDelegate, NS connectClientForm.dismiss() } + /// The "Add Server..." item, built in one place because it is now rendered + /// from two: normally as the first entry of the Servers submenu, and at the + /// top level only when there are no servers and therefore no submenu. + /// + /// `target` is set explicitly — an item in a submenu with a nil target would + /// fall back to the responder chain, and the tray has none while the menu is + /// open, so the click would silently do nothing. + func makeAddServerItem() -> NSMenuItem { + let item = NSMenuItem(title: "Add Server...", action: #selector(showAddServer), keyEquivalent: "n") + item.target = self + return item + } + @objc private func showAddServer() { showMainWindow() // First switch to the Servers tab so ServersView is mounted and @@ -1070,23 +1083,26 @@ final class AppController: NSObject, NSApplicationDelegate, NSWindowDelegate, NS menu.addItem(.separator()) } - // Quarantine - if appState.quarantinedToolsCount > 0 { - let quarantineItem = NSMenuItem( - title: "\(appState.quarantinedToolsCount) quarantined server(s)", - action: nil, keyEquivalent: "") - quarantineItem.isEnabled = false - quarantineItem.image = NSImage(systemSymbolName: "shield.lefthalf.filled", - accessibilityDescription: "quarantine") - menu.addItem(quarantineItem) - menu.addItem(.separator()) - } + // No standalone quarantine line: "N quarantined server(s)" was a + // disabled, grey, non-actionable row, and the same fact is already + // carried — actionably — by the Needs Attention submenu above. The menu + // had grown long enough to scroll, and an informational row that cannot + // be clicked is the first thing to spend. // Servers — as a SUBMENU (not flat list) if !appState.servers.isEmpty { let serversMenuItem = NSMenuItem(title: "Servers (\(appState.servers.count))", action: nil, keyEquivalent: "") let serversSubmenu = NSMenu() + // "Add Server..." lives HERE, not at the top level: adding a server + // is a servers-menu action, and hoisting it out cost a permanent + // top-level row in a menu that had started to scroll. The Cmd+N key + // equivalent still works from a submenu — AppKit walks the whole menu + // tree when matching key equivalents — and still displays beside the + // title once the submenu is open. + serversSubmenu.addItem(makeAddServerItem()) + serversSubmenu.addItem(.separator()) + for server in appState.servers { let item = NSMenuItem(title: server.name, action: nil, keyEquivalent: "") @@ -1200,9 +1216,14 @@ final class AppController: NSObject, NSApplicationDelegate, NSWindowDelegate, NS } // Actions - let addServer = NSMenuItem(title: "Add Server...", action: #selector(showAddServer), keyEquivalent: "n") - addServer.target = self - menu.addItem(addServer) + // + // "Add Server..." normally lives inside the Servers submenu. The one + // exception is a proxy with no servers at all: there is no Servers + // submenu to hold it then, and the very user who most needs to add one + // would be left without a way to do it from the tray. + if appState.servers.isEmpty { + menu.addItem(makeAddServerItem()) + } // Spec 091 FR-001: the native connect journey, beside Add Server. The // item is built by its router so the item and its routing are tested @@ -1334,9 +1355,9 @@ final class AppController: NSObject, NSApplicationDelegate, NSWindowDelegate, NS docsItem.target = self menu.addItem(docsItem) - let issueItem = NSMenuItem(title: "Report an Issue…", action: #selector(reportIssue), keyEquivalent: "") - issueItem.target = self - menu.addItem(issueItem) + // No "Report an Issue…" item: the menu had grown long enough to scroll + // on a normal display, and the tracker is still reachable through the + // GitHub link in the About panel's credits block. let aboutItem = NSMenuItem(title: "About MCPProxy", action: #selector(showAboutPanel), keyEquivalent: "") aboutItem.target = self @@ -1643,10 +1664,6 @@ final class AppController: NSObject, NSApplicationDelegate, NSWindowDelegate, NS NSWorkspace.shared.open(ProjectLinks.docs) } - @objc private func reportIssue() { - NSWorkspace.shared.open(ProjectLinks.issues) - } - /// Shows the standard About panel (app name + version) with a credits block /// that links back to the homepage, source, and docs — so "About MCPProxy" /// carries the way back to the project, not just a version string. diff --git a/native/macos/MCPProxy/MCPProxy/Menu/Glance/GlanceSelection.swift b/native/macos/MCPProxy/MCPProxy/Menu/Glance/GlanceSelection.swift index 35bcf2472..e583f7002 100644 --- a/native/macos/MCPProxy/MCPProxy/Menu/Glance/GlanceSelection.swift +++ b/native/macos/MCPProxy/MCPProxy/Menu/Glance/GlanceSelection.swift @@ -118,8 +118,17 @@ enum GlanceSelection { /// Proxy administration built-ins. Never shown, whatever their status. static let managementBuiltIns: Set = ["upstream_servers", "quarantine_security"] - /// Discovery/execution built-ins that are worth a row even on success. - static let glanceInternalTools: Set = ["retrieve_tools", "code_execution", "describe_tool"] + /// Discovery built-ins that are worth a row even on success. + /// + /// `code_execution` is deliberately NOT here. It is an internal primitive — + /// a wrapper around the upstream calls a script makes — and since those + /// sub-calls became `tool_call` records of their own they speak for it far + /// better than it did: the glance now shows the REAL work ("jira_get_issue + /// ×3") instead of one opaque "code_execution" row that named no server and + /// no tool. A FAILED code_execution still rows, through rule 3's failure + /// branch below — a script that died of a syntax error has no children to + /// speak for it, so its own record is the only trace there is. + static let glanceInternalTools: Set = ["retrieve_tools", "describe_tool"] /// How many rows each list shows. static let rowLimit = 5 @@ -148,8 +157,10 @@ enum GlanceSelection { return entry.outcomeClass == .blocked } - // Rule 3 — discovery/execution built-ins, plus any internal failure - // (a wrapper that died before dispatch has no upstream record). + // Rule 3 — discovery built-ins, plus any internal failure (a wrapper + // that died before dispatch has no upstream record). A SUCCESSFUL + // code_execution falls through both clauses on purpose: its sub-calls + // are the rows now. if entry.type == "internal_tool_call" { return glanceInternalTools.contains(tool) || entry.status != "success" } @@ -178,9 +189,10 @@ enum GlanceSelection { /// per sub-call — the counter suffix in `mintCorrelationIDAt` is what makes /// it unique — and names the script through `parent_id`, which carries the /// parent's request id rather than a shared one. No child shares an id with - /// another child or with the parent, so a multi-tool script renders as the - /// script's own row plus one row per sub-call (the transparency the feature - /// exists for) instead of collapsing to one. + /// another child or with the parent, so a multi-tool script renders as one + /// row per sub-call (the transparency the feature exists for) instead of + /// collapsing to one. The successful wrapper itself never reaches this step + /// — rule 3 dropped it — so those rows are the script's whole story. /// /// Note the legacy `ToolCallRecord` written beside it still carries /// `RequestID: u.executionID` ("Use execution ID as request ID to link diff --git a/native/macos/MCPProxy/MCPProxy/State/AppState.swift b/native/macos/MCPProxy/MCPProxy/State/AppState.swift index bb0ab5b2d..1f6d7b40e 100644 --- a/native/macos/MCPProxy/MCPProxy/State/AppState.swift +++ b/native/macos/MCPProxy/MCPProxy/State/AppState.swift @@ -464,7 +464,10 @@ final class AppState: ObservableObject { // Mirrors GlanceSelection: management built-ins never row (rule 1), // call_tool_* echoes never do (their dispatch emitted its own // tool_call record), and other internal calls row on success only - // for the discovery/execution set, on failure always (rule 3). + // for the discovery set, on failure always (rule 3). A successful + // code_execution is therefore out of both the glance and the bars — + // its sub-calls are tool_call records that already count — while a + // failed one is in. guard !name.hasPrefix("call_tool_"), !GlanceSelection.managementBuiltIns.contains(name) else { return false } return entry.status != "success" diff --git a/native/macos/MCPProxy/MCPProxyTests/GlanceHeaderConsistencyTests.swift b/native/macos/MCPProxy/MCPProxyTests/GlanceHeaderConsistencyTests.swift index 332cf5823..cff04e504 100644 --- a/native/macos/MCPProxy/MCPProxyTests/GlanceHeaderConsistencyTests.swift +++ b/native/macos/MCPProxy/MCPProxyTests/GlanceHeaderConsistencyTests.swift @@ -195,8 +195,14 @@ final class GlanceHeaderConsistencyTests: XCTestCase { /// mirror a dispatch whose own `tool_call` record is also on the stream), /// and a `tool_call` shed by the concurrency limiter never executed. func testLiveIncrementMirrorsTheAggregatesTimelineRule() { + XCTAssertFalse(AppState.countsTowardUsageTimeline( + Self.typedEntry(type: "internal_tool_call", tool: "code_execution", status: "success")), + "a script that ran bars through its sub-calls; the wrapper would double it") + XCTAssertTrue(AppState.countsTowardUsageTimeline( + Self.typedEntry(type: "internal_tool_call", tool: "code_execution", status: "error")), + "a script that died before dispatch has no sub-calls to bar for it") XCTAssertTrue(AppState.countsTowardUsageTimeline( - Self.typedEntry(type: "internal_tool_call", tool: "code_execution", status: "success"))) + Self.typedEntry(type: "internal_tool_call", tool: "describe_tool", status: "success"))) XCTAssertTrue(AppState.countsTowardUsageTimeline( Self.typedEntry(type: "internal_tool_call", tool: "retrieve_tools", status: "error"))) XCTAssertFalse(AppState.countsTowardUsageTimeline( diff --git a/native/macos/MCPProxy/MCPProxyTests/GlanceSelectionTests.swift b/native/macos/MCPProxy/MCPProxyTests/GlanceSelectionTests.swift index 9d201f09b..5329783a9 100644 --- a/native/macos/MCPProxy/MCPProxyTests/GlanceSelectionTests.swift +++ b/native/macos/MCPProxy/MCPProxyTests/GlanceSelectionTests.swift @@ -12,13 +12,32 @@ final class GlanceSelectionTests: XCTestCase { // MARK: - Rule 3 - func testDiscoveryAndExecutionBuiltInsAreIncluded() { - for tool in ["retrieve_tools", "code_execution", "describe_tool"] { + func testDiscoveryBuiltInsAreIncluded() { + for tool in ["retrieve_tools", "describe_tool"] { let entry = Self.entry(id: tool, type: "internal_tool_call", tool: tool) XCTAssertTrue(GlanceSelection.qualifies(entry), "\(tool) should qualify") } } + /// `code_execution` is an internal primitive, not a call the user made. A + /// script that RAN is represented by the upstream calls it issued — those + /// are `tool_call` rows of their own — so the wrapper row said only + /// "code_execution", naming no server and no tool, while taking one of five + /// rows away from the work it hid. + func testASuccessfulCodeExecutionWrapperIsExcluded() { + let entry = Self.entry(id: "x", type: "internal_tool_call", tool: "code_execution") + XCTAssertFalse(GlanceSelection.qualifies(entry), + "a script that ran is shown through its sub-calls, not through its wrapper") + } + + /// The exception, through rule 3's failure branch: a script that died of a + /// syntax error dispatched nothing, so it has no children to speak for it + /// and its own record is the only trace of the attempt. + func testAFailedCodeExecutionWrapperIsIncluded() { + let entry = Self.entry(id: "x", type: "internal_tool_call", tool: "code_execution", status: "error") + XCTAssertTrue(GlanceSelection.qualifies(entry)) + } + func testSuccessfulCallToolWrapperIsExcluded() { let entry = Self.entry(id: "w", type: "internal_tool_call", tool: "call_tool_write") XCTAssertFalse(GlanceSelection.qualifies(entry)) @@ -115,12 +134,13 @@ final class GlanceSelectionTests: XCTestCase { // MARK: - code_execution sub-calls (issue C) - /// A `code_execution` script and the tools it called are separate rows, and - /// that is the point: the sub-calls are the transparency the parent row - /// cannot give. Rule 4 does not fold them together because every child gets - /// a request id of its own — it names the script through `parent_id`, which - /// carries the PARENT's request id, not a shared one. - func testCodeExecutionSubCallsEachKeepTheirOwnRow() { + /// The tools a `code_execution` script called each keep a row, and the + /// successful wrapper keeps none — that is the whole transparency story: the + /// glance shows the REAL work instead of one opaque row that named neither + /// server nor tool. Rule 4 does not fold the children together because every + /// child gets a request id of its own — it names the script through + /// `parent_id`, which carries the PARENT's request id, not a shared one. + func testCodeExecutionSubCallsEachKeepTheirOwnRowAndTheWrapperKeepsNone() { let parentCallID = "1770000000-sess-code_execution" let rows = GlanceSelection.activityRows(from: [ Self.entry(id: "c2", type: "tool_call", server: "jira", tool: "get_issue", @@ -131,14 +151,27 @@ final class GlanceSelectionTests: XCTestCase { requestId: parentCallID) ]) - XCTAssertEqual(rows.count, 3, "a script's sub-calls must not collapse into its own row") - XCTAssertEqual(rows.map(\.newest.id), ["c2", "c1", "p"]) - XCTAssertEqual(rows.map(\.count), [1, 1, 1]) - XCTAssertEqual(rows.last?.newest.parentId, nil, "the parent row has no parent of its own") - XCTAssertEqual(Set(rows.dropLast().compactMap(\.newest.parentId)), [parentCallID], + XCTAssertEqual(rows.count, 2, "a script's sub-calls must not collapse, and its wrapper must not row") + XCTAssertEqual(rows.map(\.newest.id), ["c2", "c1"]) + XCTAssertEqual(rows.map(\.count), [1, 1]) + XCTAssertEqual(Set(rows.compactMap(\.newest.parentId)), [parentCallID], "each child names the parent's request id, which is how a row navigates up") } + /// A script that FAILED keeps its row beside whatever it managed to + /// dispatch: the failure is the one thing its sub-calls cannot report. + func testAFailedCodeExecutionRowsBesideItsSubCalls() { + let parentCallID = "1770000000-sess-code_execution" + let rows = GlanceSelection.activityRows(from: [ + Self.entry(id: "p", type: "internal_tool_call", tool: "code_execution", + status: "error", requestId: parentCallID), + Self.entry(id: "c1", type: "tool_call", server: "github", tool: "create_issue", + requestId: "child-1", parentId: parentCallID) + ]) + + XCTAssertEqual(rows.map(\.newest.id), ["p", "c1"]) + } + /// Two sub-calls to the SAME tool are still one run, like any other pair of /// consecutive identical calls — grouping does not know about parentage. func testConsecutiveSubCallsToOneToolStillGroup() { diff --git a/native/macos/MCPProxy/MCPProxyTests/MenuStructureTests.swift b/native/macos/MCPProxy/MCPProxyTests/MenuStructureTests.swift new file mode 100644 index 000000000..12133622c --- /dev/null +++ b/native/macos/MCPProxy/MCPProxyTests/MenuStructureTests.swift @@ -0,0 +1,140 @@ +// MenuStructureTests.swift +// MCPProxyTests +// +// The top-level tray menu had grown long enough to scroll on a normal display, +// and a scrolling status-bar menu is the opposite of a glance. Three rows were +// spent to fix it, and each removal is a claim this file pins: +// +// - "N quarantined server(s)" was grey, disabled and unclickable, and the +// Needs Attention submenu already carries the same fact ACTIONABLY. +// - "Report an Issue…" left with it; the tracker is still reachable through +// the GitHub link in the About panel. +// - "Add Server..." moved into the Servers submenu, where it belongs — +// except when there are no servers and therefore no submenu to hold it. +// +// Driven through the real `rebuildMenu()` via the same seams +// `SupersedeMenuTests` uses, so a reorder that quietly reinstates a row fails +// here rather than shipping. + +import XCTest +import AppKit +@testable import MCPProxy + +@MainActor +final class MenuStructureTests: XCTestCase { + + private final class TestMenuHost: TrayMenuHost { + var menu: NSMenu? + } + + private func makeController(servers: [ServerStatus] = []) -> (AppController, TestMenuHost) { + let host = TestMenuHost() + let controller = AppController( + glanceDataSource: CountingGlanceDataSource(), menuHost: host + ) + controller.appState.coreState = .connected + controller.appState.servers = servers + return (controller, host) + } + + private func topLevelTitles(_ host: TestMenuHost) -> [String] { + (host.menu?.items ?? []).map(\.title) + } + + private func serversSubmenu(_ host: TestMenuHost) throws -> NSMenu { + let parent = try XCTUnwrap( + (host.menu?.items ?? []).first { $0.title.hasPrefix("Servers (") }, + "a proxy with servers must offer the Servers submenu") + return try XCTUnwrap(parent.submenu) + } + + // MARK: - Rows that are gone + + func testTheGreyQuarantineCountLineIsGone() { + let (controller, host) = makeController(servers: [Self.server(name: "github")]) + controller.appState.quarantinedToolsCount = 3 + controller.rebuildMenu() + + XCTAssertFalse(topLevelTitles(host).contains { $0.contains("quarantined server") }, + "the count is Needs Attention's job, and this row could not be clicked") + } + + func testReportAnIssueIsGone() { + let (controller, host) = makeController(servers: [Self.server(name: "github")]) + controller.rebuildMenu() + + XCTAssertFalse(topLevelTitles(host).contains { $0.contains("Report an Issue") }) + XCTAssertTrue(topLevelTitles(host).contains("Documentation"), + "the other project links stay — this was a length cut, not a retreat") + XCTAssertTrue(topLevelTitles(host).contains("About MCPProxy")) + } + + // MARK: - Add Server moved + + func testAddServerIsTheFirstItemOfTheServersSubmenu() throws { + let (controller, host) = makeController(servers: [ + Self.server(name: "github"), Self.server(name: "jira") + ]) + controller.rebuildMenu() + + XCTAssertFalse(topLevelTitles(host).contains { $0.hasPrefix("Add Server") }, + "it no longer costs a permanent top-level row") + + let submenu = try serversSubmenu(host) + let first = try XCTUnwrap(submenu.items.first) + XCTAssertEqual(first.title, "Add Server...") + XCTAssertTrue(submenu.items.count > 1 && submenu.items[1].isSeparatorItem, + "a separator keeps the action off the server list") + XCTAssertEqual(submenu.items.map(\.title).filter { !$0.isEmpty }, + ["Add Server...", "github", "jira"]) + } + + /// A submenu item with a nil target falls back to a responder chain the tray + /// does not have while its menu is open, so the click would silently do + /// nothing — the exact failure this move could have introduced. + func testAddServerStaysWiredAndKeepsItsKeyEquivalent() throws { + let (controller, host) = makeController(servers: [Self.server(name: "github")]) + controller.rebuildMenu() + + let addServer = try XCTUnwrap(try serversSubmenu(host).items.first) + XCTAssertNotNil(addServer.action) + XCTAssertTrue(addServer.target === controller) + XCTAssertEqual(addServer.keyEquivalent, "n") + XCTAssertEqual(addServer.keyEquivalentModifierMask, .command) + } + + /// The one case that keeps it at the top level: with no servers there is no + /// Servers submenu, and the user who most needs to add one would otherwise + /// have no way to do it from the tray. + func testAddServerStaysAtTheTopLevelWhenThereAreNoServers() throws { + let (controller, host) = makeController(servers: []) + controller.rebuildMenu() + + XCTAssertFalse(topLevelTitles(host).contains { $0.hasPrefix("Servers (") }) + let addServer = try XCTUnwrap( + (host.menu?.items ?? []).first { $0.title.hasPrefix("Add Server") }, + "adding the first server must never be unreachable") + XCTAssertNotNil(addServer.action) + XCTAssertTrue(addServer.target === controller) + } + + // MARK: - Helpers + + /// Built through the canonical Codable path so the fixture survives future + /// field additions on `ServerStatus`. + private static func server(name: String) -> ServerStatus { + let json = """ + { + "id": "\(name)", + "name": "\(name)", + "protocol": "http", + "enabled": true, + "connected": true, + "quarantined": false, + "tool_count": 4 + } + """.data(using: .utf8)! + // swiftlint:disable:next force_try + return try! JSONDecoder().decode(ServerStatus.self, from: json) + } +} diff --git a/specs/069-observability-usage-graphs/contracts/usage-endpoint.md b/specs/069-observability-usage-graphs/contracts/usage-endpoint.md index 29fe8ce05..3782f116b 100644 --- a/specs/069-observability-usage-graphs/contracts/usage-endpoint.md +++ b/specs/069-observability-usage-graphs/contracts/usage-endpoint.md @@ -55,7 +55,7 @@ Auth: `X-API-Key` (REST default). - `token_source: "bytes"` labels the size-based proxy (FR-006); FR-010 will switch this to `"estimated_tokens"`. - `tokens_saved*` echoed from existing `ServerTokenMetrics` (FR-007 / SC-008). - `avg_*` computed over `sized_calls` only (records with `0` bytes excluded); `null`/omitted when `sized_calls == 0`. -- `blocked` counts policy-prevented attempts (persisted as blocked `policy_decision` records, not executed tool_calls). A blocked attempt never ran, so it is **not** included in `calls` and contributes no latency/bytes — it only increments `blocked` and `last_used`. The timeline mirrors the tray glance row-for-row: it counts executed tool calls, blocked attempts (as calls and errors), and mcpproxy's own internal calls (`retrieve_tools`, `describe_tool`, `code_execution`; plus any failed non-management built-in), and excludes `call_tool_*` echo records, management built-ins (`upstream_servers`, `quarantine_security`), and concurrency-rejected sheds. *(Amended by PR #1016 — the timeline previously tracked executed upstream tool_calls only, which left the histogram empty under retrieve_tools/code_execution-heavy traffic; the glance↔timeline parity is pinned by usage_glance_parity_test.go with the Go and Swift admission sets kept in lockstep.)* +- `blocked` counts policy-prevented attempts (persisted as blocked `policy_decision` records, not executed tool_calls). A blocked attempt never ran, so it is **not** included in `calls` and contributes no latency/bytes — it only increments `blocked` and `last_used`. The timeline mirrors the tray glance row-for-row: it counts executed tool calls, blocked attempts (as calls and errors), and mcpproxy's own internal calls (`retrieve_tools`, `describe_tool`; plus any failed non-management built-in, `code_execution` included), and excludes `call_tool_*` echo records, management built-ins (`upstream_servers`, `quarantine_security`), successful `code_execution` wrappers, and concurrency-rejected sheds. A successful `code_execution` is represented by its sub-calls — they are `tool_call` records that count on their own — so counting the wrapper too would double the script under a name no upstream owns; a **failed** one has no sub-calls to speak for it and still counts. *(Amended by PR #1016 — the timeline previously tracked executed upstream tool_calls only, which left the histogram empty under retrieve_tools/code_execution-heavy traffic. Amended again by `fix/tray-glance-compact` — successful `code_execution` wrappers left the set once their sub-calls became rows of their own; `usageAdmissionVersion` went to 4 so pre-change snapshots rebuild. The glance↔timeline parity is pinned by usage_glance_parity_test.go with the Go and Swift admission sets kept in lockstep.)* - `other` present only when the tool list was truncated to `top`. - Empty log → `tools: []`, `timeline: []`, `tokens_saved` from metrics (or 0) — never an error (FR-009 / SC-007).