Skip to content

perf(upstream): list tools from servers concurrently during discovery - #1814

Merged
Dumbris merged 2 commits into
smart-mcp-proxy:mainfrom
poolski:perf/parallel-tool-discovery
Nov 2, 2026
Merged

Dumbris merged 2 commits into
smart-mcp-proxy:mainfrom
poolski:perf/parallel-tool-discovery

Conversation

@poolski

@poolski poolski commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

A discovery sweep listed each connected server's tools one after another, so one slow or large upstream delayed every other server's tools/list by up to its own timeout.

The sweep now lists up to 8 servers at once (discoveryListConcurrency). Results are recorded in the original order with the same logging, markSwept and capture handling. A server's concurrent tools/list calls were already coalesced into one request, and each capture carries its own process-global sequence, so the order in which lists finish does not change how their results are applied.

TestDiscoverToolsListsServersConcurrently uses three upstreams whose tools/list waits for another list to be in flight and requires the sweep to overlap them.

Testing

  • go test -race ./internal/upstream/...
  • scripts/test-api-e2e.sh (70/70)
  • golangci-lint with .github/.golangci.yml, bare and --build-tags server: no new issues

A discovery sweep listed each connected server's tools one after
another, so one slow or large upstream delayed every other server's
tools/list by up to its own timeout. The sweep now lists up to 8
servers at once (discoveryListConcurrency) and records the results in
the same order and with the same logging as before. A server's
concurrent tools/list calls were already coalesced into one request,
and each capture carries its own sequence, so the order in which the
lists finish does not affect how their results are applied.
@poolski
poolski marked this pull request as ready for review November 2, 2026 09:36
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Register 12 servers whose tools/list holds until 8 are in flight, then assert the
peak never exceeds discoveryListConcurrency; removing the semaphore now fails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Dumbris
Dumbris enabled auto-merge (squash) November 2, 2026 17:45
@Dumbris

Dumbris commented Nov 2, 2026

Copy link
Copy Markdown
Member

Thanks — clean change, cross-model review (gpt-6.1-sol) came back clean too. The one gap was that nothing asserted the upper bound (removing the semaphore left every test green), so I pushed TestDiscoverToolsBoundsConcurrentLists (12 servers, asserts peak ≤ discoveryListConcurrency; verified it fails without the semaphore). Auto-merge is armed.

@Dumbris
Dumbris merged commit 764a1bf into smart-mcp-proxy:main Nov 2, 2026
45 of 46 checks passed
@Dumbris

Dumbris commented Nov 2, 2026

Copy link
Copy Markdown
Member

Merged. Thank you, @poolski! Discovery sweeps used to wait on each upstream's tools/list in turn, so one slow server delayed all of them by up to its own timeout; listing up to eight at once makes a fleet of upstreams index in roughly the time of the slowest one.

Contributions like this are what keep MCPProxy moving, and they matter a lot to the project.

One small ask, only if you feel like it: a short public note about your change (a tweet, a LinkedIn post, a line in a community you are part of) helps more people discover MCPProxy. No pressure at all, and I would be glad to repost it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants