Repository navigation
feat(ai-gateway): MCP rate limiting support - #7658
tomek-labuk wants to merge 7 commits into
Conversation
✅ Deploy Preview for kongdeveloper ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
tomek-labuk
added
product:ai-gateway
size: S
ed4bc52 to
8ba10a9
Compare
8ba10a9 to
36ab3d8
Compare
|
✅ KSAI Review: Finished Run report (federated) · 1 paid run · $0.2171 total
Reviewing cost $0.2171 |
![kong-ksai[bot]](https://avatars.githubusercontent.com/in/4495376?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Guaris
left a comment
Guaris
left a comment
There was a problem hiding this comment.
Wording here is a bit dense.
Unrelated to your changes in this PR, this doc is getting massive. Can we consider splitting it up into sections after the release?
| - **Protect an expensive tool**: limit a single tool, such as `search_flights`, to a small number of calls per second, without slowing down the rest of the server. | ||
| - **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. |
There was a problem hiding this comment.
These two cases feel the same to me.
There was a problem hiding this comment.
Yes, true. Let me trim it
|
|
||
| - **Protect an expensive tool**: limit a single tool, such as `search_flights`, to a small number of calls per second, without slowing down the rest of the server. | ||
| - **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. | ||
| - **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool. |
There was a problem hiding this comment.
I'm not sure what this one means at all.
There was a problem hiding this comment.
Actually. After a second thought: this is an overkill. Let me delete this section, these are covered across other docs.
@Guaris - true. I'll create a ticket for the non-release Nov milestone to do it. |
Co-authored-by: Angel <Guaris@users.noreply.github.com>
Co-authored-by: Angel <Guaris@users.noreply.github.com>
e60006f to
a442e35
Compare
https://deploy-preview-7658--kongdeveloper.netlify.app/ai-gateway/entities/ai-mcp-server/