Skip to content

Commit b286c0e

Browse files
committed
bottstrapped notes
1 parent 86de627 commit b286c0e

1 file changed

Lines changed: 58 additions & 1 deletion

File tree

‎app/_ai_gateway_entities/ai-mcp-server.md‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,8 @@ rows:
198198
- use_case: "Secure MCP endpoints with AI Consumer identity or OAuth tokens"
199199
example: "Reference an [AI Auth Strategy](/ai-gateway/entities/ai-auth-strategy/) (`key-auth` or `openid-connect`) in [`access.auth_strategies`](#schema-aigateway-mcpserver-access). Advertise [protected resource metadata](#protected-resource-metadata) through `access.metadata` so MCP clients can discover the authorization server."
200200
- use_case: "Rate limiting"
201-
example: "Use [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy to control MCP request volume per AI Consumer or AI Consumer Group."
201+
example: "Set [`rate_limiting`](#rate-limiting) to cap requests per MCP method or tool.
202+
Use [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy to control MCP request volume per AI Consumer or AI Consumer Group."
202203
- use_case: "Track all MCP traffic and ACL decisions."
203204
example: "Enable request and response logging through [AI logging Policies](/ai-gateway/policies/?category=logging) and audit trails."
204205
- use_case: "Traffic control"
@@ -548,6 +549,62 @@ config:
548549
- kongair-internal-root-ca
549550
```
550551

552+
## Rate limiting {% new_in 2.3 %}
553+
554+
The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field caps how many requests the server accepts for a specific MCP method or [tool](#tools).
555+
For example, you can allow `tools/list` frequently but restrict an expensive `search_flights` tool to a few calls per second.
556+
557+
The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes.
558+
Other [server modes](#server-modes) reject it.
559+
560+
Set a list of `limits`.
561+
Each entry in the list has the following fields:
562+
563+
{% table %}
564+
columns:
565+
- title: Field
566+
key: field
567+
- title: Description
568+
key: description
569+
rows:
570+
- field: "`method`"
571+
description: "The MCP method the limit applies to, for example `tools/call`. Set either `method` or `tool`, not both."
572+
- field: "`tool`"
573+
description: "The name of the tool the limit applies to, for example `search_flights`. Set either `method` or `tool`, not both."
574+
- field: "`limit`"
575+
description: "The number of requests allowed per window. Must be between 1 and 2147483646."
576+
- field: "`window_size`"
577+
description: "The window size in seconds. Must be `1` (second), `60` (minute), `3600` (hour), or `86400` (day)."
578+
{% endtable %}
579+
580+
Each limit must set exactly one of `method` or `tool`.
581+
Each combination of target and `window_size` can appear only once.
582+
You can set several windows for the same target, for example a per-hour and a per-day limit on one tool.
583+
If either rule is broken, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`.
584+
585+
Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently.
586+
587+
In the following example, the AI MCP Server allows 100 `tools/call` requests per minute.
588+
It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day:
589+
590+
```yaml
591+
rate_limiting:
592+
limits:
593+
- method: tools/call
594+
limit: 100
595+
window_size: 60
596+
- tool: search_flights
597+
limit: 10
598+
window_size: 1
599+
- tool: search_flights
600+
limit: 1000
601+
window_size: 86400
602+
```
603+
604+
{:.info}
605+
> `rate_limiting` limits requests to the AI MCP Server as a whole.
606+
> To limit requests per AI Consumer or AI Consumer Group, attach a [Rate Limiting](/ai-gateway/policies/rate-limiting/) or [Rate Limiting Advanced](/ai-gateway/policies/rate-limiting-advanced/) Policy.
607+
551608
## ACL tool control
552609

553610
When exposing MCP servers through {{site.ai_gateway}}, you may need granular control over which authenticated [AI Consumers](/ai-gateway/entities/ai-consumer/) can discover and invoke specific tools. The MCP Server's ACL feature lets you define access rules at both the default level (which applies to all tools) and per-tool level (for fine-grained exceptions).

0 commit comments

Comments
 (0)