Skip to content

chore(roadmap): add token-bench and tool-graph epics - #1047

Merged
github-actions[bot] merged 1 commit into
mainfrom
chore/roadmap-token-bench-tool-graph
Aug 25, 2026
Merged

github-actions[bot] merged 1 commit into
mainfrom
chore/roadmap-token-bench-tool-graph

Conversation

@Dumbris

@Dumbris Dumbris commented Aug 25, 2026

Copy link
Copy Markdown
Member

Stacks on #1041 (chore/roadmap-truth-sync) — merge this after #1041. Branched off that PR's head, so until #1041 lands GitHub shows its commit in this diff too. Only roadmap.yaml + ROADMAP.md change here.

Adds two backlog epics:

  • token-bench (P1, depends on schema-deferred from chore(roadmap): sync statuses — QA-gate T2 done, mcp-2026 unblocked, specs 101/102 tracked #1041) — measure the real token cost of every routing/savings mode combination (baseline, compact signatures 085, deferred schemas 102, optimistic calling via self-healing pre-dispatch validation, code_execution 096, stored scripts 097) on replayed real sessions and public suites, then publish results + methodology on mcpproxy.app/blog. Tasks: replay harness → public suites → blog post, plus a heartbeat v10 per-tool_response_mode token counter for real-world cohort validation.
  • tool-graph (P2) — experimental, flag-gated (experimental.tool_graph, off by default) local-only co-occurrence graph mined from the activity log: related_tools hints, session-aware rank boost in retrieve_tools, workflow mining into suggested stored scripts (097 synergy), and usage-chain analytics.

The standalone discovery-eval-harness epic is folded into token-bench-harness: the file has no superseded/tombstone convention, so the epic is kept as a pointer with a SUPERSEDED — folded into token-bench-harness … do not build standalone note and depends_on: [token-bench-harness]. That keeps the id resolvable as a depends_on target and leaves the eventual archive sweep unchanged.

tool-graph-analytics expresses its cross-epic dependency the way the schema supports — depends_on: [tool-graph-core, analytics-default-landing], a task-level edge into the analytics-dashboard epic, which the overview graph renders as analytics_dashboard --> tool_graph.

python3 scripts/gen-roadmap.py --check passes; ROADMAP.md regenerated in the same commit.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: d4fa1c1
Status: ✅  Deploy successful!
Preview URL: https://c2306fe2.mcpproxy-docs.pages.dev
Branch Preview URL: https://chore-roadmap-token-bench-to.mcpproxy-docs.pages.dev

View logs

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: chore/roadmap-token-bench-tool-graph

Available Artifacts

  • archive-darwin-amd64 (29 MB)
  • archive-darwin-arm64 (26 MB)
  • archive-linux-amd64 (17 MB)
  • archive-linux-arm64 (15 MB)
  • archive-windows-amd64 (29 MB)
  • archive-windows-arm64 (25 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (23 MB)
  • installer-dmg-darwin-arm64 (20 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 32816870910 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved (Model B): Paperclip review verdicts = ACCEPT and qa-gate green at this head SHA. Arming auto-merge; GitHub merges when all required checks pass.

@github-actions
github-actions Bot enabled auto-merge (squash) August 25, 2026 11:03
Two new backlog epics:

- token-bench (P1): measure real token cost across every routing/savings
  mode combo (baseline, compact 085, deferred schemas 102, optimistic
  calling, code_execution 096, stored scripts 097) on replayed sessions
  and public suites, then publish the numbers on mcpproxy.app/blog.
  Absorbs the standalone discovery-eval-harness epic, which is now marked
  SUPERSEDED and points at token-bench-harness.
- tool-graph (P2): experimental, flag-gated local co-occurrence graph
  mined from the activity log — next-tool hints, session-aware ranking,
  workflow mining and usage-chain analytics.

Stacks on #1041 (chore/roadmap-truth-sync); token-bench depends on the
schema-deferred epic added there. Merge after #1041.

ROADMAP.md regenerated; gen-roadmap.py --check passes.
@Dumbris
Dumbris force-pushed the chore/roadmap-token-bench-tool-graph branch from 126cc05 to d4fa1c1 Compare August 25, 2026 11:20
@github-actions
github-actions Bot merged commit cfffac2 into main Aug 25, 2026
36 checks passed

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved (Model B): Paperclip review verdicts = ACCEPT and qa-gate green at this head SHA. Arming auto-merge; GitHub merges when all required checks pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants