Skip to content

fix(usage): use nearest-rank for latency percentiles - #1764

Merged
Dumbris merged 1 commit into
smart-mcp-proxy:mainfrom
rudycelekli:fix/usage-percentile-nearest-rank-20261023-01
Oct 26, 2026
Merged

Dumbris merged 1 commit into
smart-mcp-proxy:mainfrom
rudycelekli:fix/usage-percentile-nearest-rank-20261023-01

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Bug

ToolUsage.Percentile computes the target rank as int64(float64(total) * p), which truncates. The nearest-rank method needs ceil. For small call counts the p95/p99 lands one rank too low, so the slowest call never shows up in the tail percentile. The Usage view (GET /api/v1/activity/usage, httpapi/activity.go) reports exactly these values as the tool's p95.

Trigger

One tool receives 9 calls at ~30 ms and 1 call at 3000 ms.

  • Expected p95: nearest rank ceil(9.5) = 10, which is the 3000 ms call, so the bucket bound is ≤ 5000 ms.
  • Actual: rank int(9.5) = 9, so p95 is ≤ 50 ms. The slow call is invisible.

Fix

Use math.Ceil for the target rank. The existing target < 1 clamp is kept, and large totals are unaffected, because the existing 100-call tests still pass.

Test

TestToolUsagePercentile_SmallTotalsUseNearestRank covers the 9-fast + 1-slow case above. It asserts p95 = 5000 (the bucket that holds 3000 ms) and p50 = 50.

Commands run

  • CGO_ENABLED=0 go test -count=1 ./internal/runtime: ok
  • CGO_ENABLED=0 go test -count=1 -run 'Usage|Percentile' ./internal/httpapi: ok
  • With the fix reverted (git stash of usage_aggregate.go), the new test fails with p95 = 50 ms, want 5000.
  • gofmt -l on the edited files: no output

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Dumbris Dumbris left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified: the PR test fails on main for the reason described and passes on this branch merged with main; race tests for the touched packages and CI lint (bare and server tags) are clean. Also built and tested together with the rest of this batch, independently reviewed, and cross-reviewed with a second model.

@Dumbris
Dumbris merged commit 206cb09 into smart-mcp-proxy:main Oct 26, 2026
46 checks passed
@Dumbris

Dumbris commented Oct 26, 2026

Copy link
Copy Markdown
Member

Merged. Thank you, @rudycelekli! P95/p99 in the Usage view use nearest-rank, so a lone slow call is no longer hidden.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants