Skip to content

[Feature]: Add schema_deferred routing mode — keep tool names & descriptions, strip inputSchema #971

Closed

Description

@JeffC0628

Problem

retrieve_tools mode has a blind-search problem: the agent sees only meta-tools and has no idea what upstream tools exist until it guesses a query. direct mode dumps all schemas upfront, eating 30K+ tokens for ~100 tools.

There's no middle ground: let the agent see all tool names + descriptions, but defer inputSchema until needed.

Proposal

Add routing_mode: "schema_deferred":

  • tools/list returns all upstream tools with name + description, but inputSchema set to {}
  • Agent calls the existing describe_tool(name) to get the full schema when it's ready to use a specific tool
  • No new execution primitives needed — describe_tool and call_tool_* already exist

Config:

{
  "routing_mode": "schema_deferred"
}

That's it. One mode, one behavior change.

Why

  • Agent gets full visibility into available tools (no blind search)
  • Context cost drops from ~30K to ~3.5K tokens for 100 tools (~88% reduction)
  • One extra describe_tool round-trip per tool call — acceptable trade-off

Proposed solution

Industry precedent

Alternatives considered

No response

Which edition?

Personal

Pre-submission checklist

  • I searched existing issues and discussions for a similar request

Activity

  1. Dumbris commented on Aug 13, 2026

    @Dumbris
    Member

    Thanks @JeffC0628 — this is a well-researched proposal, and the Catalog → Inspect → Execute framing (and the Atlassian/Qoder precedents) matches exactly where we've been heading. Good news: most of the machinery you're describing already shipped in Spec 085 ("compact router"), so schema_deferred can be built as a thin composition rather than a new pipeline.

    What exists today

    1. describe_tool is already a built-in: batch of up to 5 tool ids → full inputSchema + untruncated description, with the same visibility rules as search (quarantine/profile/disabled tools never leak).

    2. tool_response_mode: "compact" already does schema-deferral — but on the retrieve_tools surface:

    {
      "routing_mode": "retrieve_tools",   // default
      "tool_response_mode": "compact"     // hot-reloadable, no restart
    }

    In compact mode each result is id + one-line signature (sig, * = required, ~ = lossy/nested) + first-sentence description — no inputSchema. Measured −52.6% on our 45-tool reference corpus and −92% on a live 907-tool deployment. There's also a per-call override: retrieve_tools(query, detail: "full" | "compact"). And if an agent guesses args wrong anyway, the proxy validates before dispatch and returns the failing tool's full schema inside the error, so the worst case is one self-healing retry.

    1. Full-enumeration surface: routing_mode: "direct" (also always available at /mcp/all regardless of config) exposes every upstream tool as serverName__toolName — but today it always ships full schemas, which is the ~30K-token problem you're pointing at.

    What's genuinely new in your proposal is applying the deferral to the enumeration (tools/list) surface. That's a real gap — retrieve_tools requires a query, so there's no cheap "show me everything" view.

    Proposed direction (reusing the 085 machinery rather than paralleling it):

    • Make direct mode honor the schema-deferral axis: routing_mode: "direct" + deferred serialization → tools/list returns every tool with name, description plus its precompiled compact signature appended, and a minimal permissive inputSchema ({"type":"object"} rather than literal {} — some strict clients drop args not covered by the declared schema).
    • Expose describe_tool on that surface (it's currently retrieve_tools-mode-only by design; lifting that was already flagged as follow-up in 085).
    • Extend the pre-dispatch validation + schema-in-error self-healing to direct-mode calls.

    Two nice side effects of embedding the compact sig instead of stripping to bare name+description: flat tools stay one-shot callable with zero extra round-trips (the ~ lossy marker tells the agent when a describe_tool call is actually needed), and the signatures are already compiled at index time, so there's no per-request cost.

    Token math should land right around your estimate — our profiling showed ~77% of tool payloads is raw inputSchema.

    Marking this as accepted-direction; we'll spec it as a small follow-up to 085. Thanks again for the pointers to the Atlassian and Qoder writeups — genuinely useful prior art.

  2. 15 remaining items

  3. Dumbris commented on Sep 2, 2026

    @Dumbris
    Member

    Shipped. This landed as spec 102 (schema_deferred / deferred-schema serialization for the direct tools/list surface) in #1063, with the settings surface in #1082 and the header switcher in #1133.

    tools/list on the direct surface now returns names and descriptions with the schema deferred, and describe_tool fetches the full schema on demand — including on the direct surface, accepting both server:tool and server__tool ids. Configurable per the routing/schema-detail switcher in Settings.

    Measured savings are being published as part of spec 103 (token-bench). Closing — please reopen if the shipped behaviour differs from what you had in mind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions