Skip to content

Commit 8ba10a9

Browse files
committed
Updates
1 parent b286c0e commit 8ba10a9

1 file changed

Lines changed: 14 additions & 28 deletions

File tree

‎app/_ai_gateway_entities/ai-mcp-server.md‎

Lines changed: 14 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -549,42 +549,28 @@ config:
549549
- kongair-internal-root-ca
550550
```
551551

552-
## Rate limiting {% new_in 2.3 %}
552+
## Rate limiting
553553

554-
The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field caps how many requests the server accepts for a specific MCP method or [tool](#tools).
555-
For example, you can allow `tools/list` frequently but restrict an expensive `search_flights` tool to a few calls per second.
554+
Some MCP operations cost more than others.
555+
A single `tools/call` to a flight search tool can hit a metered upstream API, while `tools/list` is cheap.
556+
The AI MCP Server [`rate_limiting`](#schema-aigateway-mcpserver-rate-limiting) field lets you cap requests for a specific MCP method or [tool](#tools), so an expensive tool or a runaway client can't exhaust your upstream quota.
556557

557-
The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes.
558-
Other [server modes](#server-modes) reject it.
558+
Use `rate_limiting` in the following cases:
559559

560-
Set a list of `limits`.
561-
Each entry in the list has the following fields:
560+
- **Protect an expensive tool**: limit a single tool, such as `search_flights`, to a small number of calls per second, without slowing down the rest of the server.
561+
- **Cap overall tool usage**: limit all `tools/call` requests per minute, regardless of which tool the client calls.
562+
- **Layer short and long windows**: combine a per-second limit to absorb bursts with a per-day limit to control total usage for the same tool.
562563

563-
{% table %}
564-
columns:
565-
- title: Field
566-
key: field
567-
- title: Description
568-
key: description
569-
rows:
570-
- field: "`method`"
571-
description: "The MCP method the limit applies to, for example `tools/call`. Set either `method` or `tool`, not both."
572-
- field: "`tool`"
573-
description: "The name of the tool the limit applies to, for example `search_flights`. Set either `method` or `tool`, not both."
574-
- field: "`limit`"
575-
description: "The number of requests allowed per window. Must be between 1 and 2147483646."
576-
- field: "`window_size`"
577-
description: "The window size in seconds. Must be `1` (second), `60` (minute), `3600` (hour), or `86400` (day)."
578-
{% endtable %}
564+
The `rate_limiting` field is supported in `conversion-listener` and `passthrough-listener` modes.
565+
Other [server modes](#server-modes) reject it.
579566

580-
Each limit must set exactly one of `method` or `tool`.
581-
Each combination of target and `window_size` can appear only once.
582-
You can set several windows for the same target, for example a per-hour and a per-day limit on one tool.
583-
If either rule is broken, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`.
567+
Each limit targets either an MCP method or a tool name, never both, and sets a request count for a fixed window of one second, one minute, one hour, or one day.
568+
You can define several windows for the same target, but each combination of target and window can appear only once.
569+
If a limit breaks either rule, the API returns a `400` error that points to the invalid entry, for example `rate_limiting.limits[1]`.
584570

585571
Each {{site.ai_gateway}} data plane node keeps its own counters and enforces the limits independently.
586572

587-
In the following example, the AI MCP Server allows 100 `tools/call` requests per minute.
573+
In the following example, the AI MCP Server allows 100 `tools/call` requests per minute across all tools.
588574
It also limits the `search_flights` tool to 10 requests per second and 1000 requests per day:
589575

590576
```yaml

0 commit comments

Comments
 (0)