From 2079a93c5cf18cfb6fc5cc2e6be9cdfc4613c2a7 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Wed, 21 Oct 2026 14:33:22 +0200 Subject: [PATCH 1/7] bottstrapped notes --- app/_ai_gateway_entities/ai-mcp-server.md | 59 ++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 3e107afd637..75ad173e853 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -200,7 +200,8 @@ rows: - use_case: "Secure MCP endpoints with AI Consumer identity or OAuth tokens" example: "Reference an [AI Auth Strategy](/ai-gateway/entities/ai-auth-strategy/) (`key-auth` or `openid-connect`) in [`access.auth_strategies`](#schema-aigateway-mcpserver-access). Advertise [protected resource metadata](#protected-resource-metadata) through `access.metadata` so MCP clients can discover the authorization server." - use_case: "Rate limiting" - example: "Use [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy to control MCP request volume per AI Consumer or AI Consumer Group." + example: "Set [`rate_limiting`](#rate-limiting) to cap requests per MCP method or tool. + Use [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy to control MCP request volume per AI Consumer or AI Consumer Group." - use_case: "Track all MCP traffic and ACL decisions." example: "Enable request and response logging through [AI logging Policies](/ai-gateway/policies/?category=logging) and audit trails." - use_case: "Protect sensitive data in tool calls" @@ -552,6 +553,62 @@ config: - kongair-internal-root-ca ``` +## Rate limiting {% new_in 2.3 %} + +The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field caps how many requests the server accepts for a specific MCP method or [tool](#tools). +For example, you can allow `tools/list` frequently but restrict an expensive `search_flights` tool to a few calls per second. + +The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes. +Other [server modes](#server-modes) reject it. + +Set a list of `limits`. +Each entry in the list has the following fields: + +{% table %} +columns: + - title: Field + key: field + - title: Description + key: description +rows: + - field: "`method`" + description: "The MCP method the limit applies to, for example `tools/call`. Set either `method` or `tool`, not both." + - field: "`tool`" + description: "The name of the tool the limit applies to, for example `search_flights`. Set either `method` or `tool`, not both." + - field: "`limit`" + description: "The number of requests allowed per window. Must be between 1 and 2147483646." + - field: "`window_size`" + description: "The window size in seconds. Must be `1` (second), `60` (minute), `3600` (hour), or `86400` (day)." +{% endtable %} + +Each limit must set exactly one of `method` or `tool`. +Each combination of target and `window_size` can appear only once. +You can set several windows for the same target, for example a per-hour and a per-day limit on one tool. +If either rule is broken, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. + +Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently. + +In the following example, the AI MCP Server allows 100 `tools/call` requests per minute. +It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day: + +```yaml +rate_limiting: + limits: + - method: tools/call + limit: 100 + window_size: 60 + - tool: search_flights + limit: 10 + window_size: 1 + - tool: search_flights + limit: 1000 + window_size: 86400 +``` + +{:.info} +> `rate_limiting` limits requests to the AI MCP Server as a whole. +> To limit requests per AI Consumer or AI Consumer Group, attach a [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy. + ## ACL tool control When exposing MCP servers through {{site.ai_gateway}}, you may need granular control over which authenticated [AI Consumers](/ai-gateway/entities/ai-consumer/) can discover and invoke specific tools. The MCP Server's ACL feature lets you define access rules at both the default level (which applies to all tools) and per-tool level (for fine-grained exceptions). From bb570332d2aca5179b4f60058ac6145b919b67b2 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Wed, 21 Oct 2026 15:10:02 +0200 Subject: [PATCH 2/7] Updates --- app/_ai_gateway_entities/ai-mcp-server.md | 42 ++++++++--------------- 1 file changed, 14 insertions(+), 28 deletions(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 75ad173e853..7e22be87179 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -553,42 +553,28 @@ config: - kongair-internal-root-ca ``` -## Rate limiting {% new_in 2.3 %} +## Rate limiting -The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field caps how many requests the server accepts for a specific MCP method or [tool](#tools). -For example, you can allow `tools/list` frequently but restrict an expensive `search_flights` tool to a few calls per second. +Some MCP operations cost more than others. +A single `tools/call` to a flight search tool can hit a metered upstream API, while `tools/list` is cheap. +The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field lets you cap requests for a specific MCP method or [tool](#tools), so an expensive tool or a runaway client can't exhaust your upstream quota. -The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes. -Other [server modes](#server-modes) reject it. +Use `rate_limiting` in the following cases: -Set a list of `limits`. -Each entry in the list has the following fields: +- **Protect an expensive tool**: limit a single tool, such as `search_flights`, to a small number of calls per second, without slowing down the rest of the server. +- **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. +- **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool. -{% table %} -columns: - - title: Field - key: field - - title: Description - key: description -rows: - - field: "`method`" - description: "The MCP method the limit applies to, for example `tools/call`. Set either `method` or `tool`, not both." - - field: "`tool`" - description: "The name of the tool the limit applies to, for example `search_flights`. Set either `method` or `tool`, not both." - - field: "`limit`" - description: "The number of requests allowed per window. Must be between 1 and 2147483646." - - field: "`window_size`" - description: "The window size in seconds. Must be `1` (second), `60` (minute), `3600` (hour), or `86400` (day)." -{% endtable %} +The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes. +Other [server modes](#server-modes) reject it. -Each limit must set exactly one of `method` or `tool`. -Each combination of target and `window_size` can appear only once. -You can set several windows for the same target, for example a per-hour and a per-day limit on one tool. -If either rule is broken, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. +Each limit targets either an MCP method or a tool name, never both, and sets a request count for a fixed window of one second, one minute, one hour, or one day. +You can define several windows for the same target, but each combination of target and window can appear only once. +If a limit breaks either rule, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently. -In the following example, the AI MCP Server allows 100 `tools/call` requests per minute. +In the following example, the AI MCP Server allows 100 `tools/call` requests per minute across all tools. It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day: ```yaml From 21dc6eac7b00632e8850bfc52fbc92ea06df8493 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Thu, 22 Oct 2026 10:54:14 +0200 Subject: [PATCH 3/7] add warning --- app/_ai_gateway_entities/ai-mcp-server.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 7e22be87179..54565838c31 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -565,8 +565,9 @@ Use `rate_limiting` in the following cases: - **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. - **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool. -The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes. -Other [server modes](#server-modes) reject it. +{:.warning} +> The `rate_limiting` field is supported only in `conversion-listener` and `passthrough-listener` modes. +> Other [server modes](#server-modes) reject it. Each limit targets either an MCP method or a tool name, never both, and sets a request count for a fixed window of one second, one minute, one hour, or one day. You can define several windows for the same target, but each combination of target and window can appear only once. From 2bb9fcc9735e1e2037ce701b769c381a2e0db88a Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Thu, 22 Oct 2026 11:03:38 +0200 Subject: [PATCH 4/7] clarifications --- app/_ai_gateway_entities/ai-mcp-server.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 54565838c31..1595ee6fec0 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -569,11 +569,19 @@ Use `rate_limiting` in the following cases: > The `rate_limiting` field is supported only in `conversion-listener` and `passthrough-listener` modes. > Other [server modes](#server-modes) reject it. -Each limit targets either an MCP method or a tool name, never both, and sets a request count for a fixed window of one second, one minute, one hour, or one day. +Each limit targets either an MCP method or a tool name, never both, and sets a request count for a window of one second, one minute, one hour, or one day. You can define several windows for the same target, but each combination of target and window can appear only once. -If a limit breaks either rule, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. +If a limit breaks either rule, the {{site.konnect_short_name}} API rejects the AI MCP Server configuration with a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. + +A request counts against every limit it matches. +For example, a `tools/call` request for `search_flights` counts against both a `tools/call` limit and a `search_flights` limit. +In a batch request, each message counts separately. Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently. +A client can send up to the configured limit to each node. + +When an MCP client's request exceeds a limit at runtime, the AI MCP Server rejects it with an HTTP `429` status and a JSON-RPC error with code `-32023`. +The error echoes the request ID and includes the `limit` and `window_size` of the limit that was exceeded in its `data` field. In the following example, the AI MCP Server allows 100 `tools/call` requests per minute across all tools. It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day: @@ -593,7 +601,7 @@ rate_limiting: ``` {:.info} -> `rate_limiting` limits requests to the AI MCP Server as a whole. +> `rate_limiting` applies to all requests that reach the AI MCP Server, regardless of the client. > To limit requests per AI Consumer or AI Consumer Group, attach a [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy. ## ACL tool control From a3d24475f951476d914acae332d3e855e5ed85e1 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Thu, 22 Oct 2026 22:11:25 -0700 Subject: [PATCH 5/7] Update app/_ai_gateway_entities/ai-mcp-server.md Co-authored-by: Angel --- app/_ai_gateway_entities/ai-mcp-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 1595ee6fec0..0b5781fcc9a 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -561,7 +561,7 @@ The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) f Use `rate_limiting` in the following cases: -- **Protect an expensive tool**: limit a single tool, such as `search_flights`, to a small number of calls per second, without slowing down the rest of the server. +- **Limit expensive tool usage**: limit the use of a single too to a small number of calls per second, without slowing down the rest of the server. - **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. - **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool. From 49bb740c222ee732dfa0f1808b70fbc82a1c2cd6 Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Thu, 22 Oct 2026 22:11:34 -0700 Subject: [PATCH 6/7] Update app/_ai_gateway_entities/ai-mcp-server.md Co-authored-by: Angel --- app/_ai_gateway_entities/ai-mcp-server.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index 0b5781fcc9a..dc182502a35 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -555,8 +555,6 @@ config: ## Rate limiting -Some MCP operations cost more than others. -A single `tools/call` to a flight search tool can hit a metered upstream API, while `tools/list` is cheap. The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field lets you cap requests for a specific MCP method or [tool](#tools), so an expensive tool or a runaway client can't exhaust your upstream quota. Use `rate_limiting` in the following cases: From a442e359f1cf2ec1932533f24428094e223f579e Mon Sep 17 00:00:00 2001 From: tomek-labuk Date: Fri, 23 Oct 2026 09:01:22 +0200 Subject: [PATCH 7/7] Apply review feedback --- app/_ai_gateway_entities/ai-mcp-server.md | 36 ++++++++++------------- 1 file changed, 16 insertions(+), 20 deletions(-) diff --git a/app/_ai_gateway_entities/ai-mcp-server.md b/app/_ai_gateway_entities/ai-mcp-server.md index dc182502a35..25bde39ca2d 100644 --- a/app/_ai_gateway_entities/ai-mcp-server.md +++ b/app/_ai_gateway_entities/ai-mcp-server.md @@ -555,47 +555,43 @@ config: ## Rate limiting -The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field lets you cap requests for a specific MCP method or [tool](#tools), so an expensive tool or a runaway client can't exhaust your upstream quota. - -Use `rate_limiting` in the following cases: - -- **Limit expensive tool usage**: limit the use of a single too to a small number of calls per second, without slowing down the rest of the server. -- **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls. -- **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool. +The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field caps how many requests the server accepts for a specific MCP method or [tool](#tools). +For example, you can limit a tool to a set number of requests per second and per day, or limit all `tools/call` requests per minute so a retrying agent can't call tools thousands of times. {:.warning} > The `rate_limiting` field is supported only in `conversion-listener` and `passthrough-listener` modes. > Other [server modes](#server-modes) reject it. -Each limit targets either an MCP method or a tool name, never both, and sets a request count for a window of one second, one minute, one hour, or one day. -You can define several windows for the same target, but each combination of target and window can appear only once. -If a limit breaks either rule, the {{site.konnect_short_name}} API rejects the AI MCP Server configuration with a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. +Each limit follows these rules: + +- It targets either an MCP method or a tool name, not both. +- Each combination of target and window can appear only once. +- You can set several windows for the same target, for example per second and per day. + +If a limit breaks any of these rules, the {{site.konnect_short_name}} API rejects the AI MCP Server configuration with a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`. A request counts against every limit it matches. -For example, a `tools/call` request for `search_flights` counts against both a `tools/call` limit and a `search_flights` limit. -In a batch request, each message counts separately. +For example, a `tools/call` request for a tool counts against both a `tools/call` limit and a limit on that tool. +In a JSON-RPC batch request, each message counts separately. Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently. -A client can send up to the configured limit to each node. -When an MCP client's request exceeds a limit at runtime, the AI MCP Server rejects it with an HTTP `429` status and a JSON-RPC error with code `-32023`. -The error echoes the request ID and includes the `limit` and `window_size` of the limit that was exceeded in its `data` field. +When an MCP client's request exceeds a limit, the AI MCP Server rejects it with an HTTP `429` status and a JSON-RPC error with code `-32023`. -In the following example, the AI MCP Server allows 100 `tools/call` requests per minute across all tools. -It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day: +The following example limits a tool per second and per day, and limits all `tools/call` requests per minute: ```yaml rate_limiting: limits: - - method: tools/call - limit: 100 - window_size: 60 - tool: search_flights limit: 10 window_size: 1 - tool: search_flights limit: 1000 window_size: 86400 + - method: tools/call + limit: 100 + window_size: 60 ``` {:.info}