Skip to content

Commit e7367c8

Browse files
authored
feat: add live-model validation for TS MCP tool gating (#43)
1 parent 847da33 commit e7367c8

5 files changed

Lines changed: 744 additions & 0 deletions

File tree

‎typescript/examples/tool_gating/mcp_calendar_admin/README.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,67 @@ prohibit calendar_admin
2323
If a caller still invokes the hidden tool directly, the host blocks execution.
2424

2525
Adversarial request text does not expose the tool or mutate policy state.
26+
27+
The provider-free tests remain the canonical proof for this example.
28+
29+
This example also includes an opt-in live-model comparison that keeps the same
30+
host-owned MCP registry and execution rules while letting a real model choose
31+
from the exposed MCP tool surface.
32+
33+
## Live-model validation
34+
35+
The host builds the exposed MCP tool list from authoritative Context Compiler
36+
state and passes only those tools to the model.
37+
38+
The live-model helper is a plain TypeScript HTTP client. It uses an
39+
OpenAI-compatible request shape for tool calling. It does not use the OpenAI
40+
SDK, LiteLLM, or Ollama's native `/api/chat` route.
41+
42+
Use the same intent across states:
43+
44+
```text
45+
Create an admin calendar event named Quarterly access review on calendar ops-admin.
46+
```
47+
48+
What to observe:
49+
50+
- absent state: the protected admin tool is not exposed and no protected side
51+
effect occurs
52+
- `use calendar_admin`: the protected tool is exposed; the model must select it
53+
for protected execution to occur
54+
- contradiction with `prohibit calendar_admin`: clarification blocks protected
55+
execution before tool use
56+
57+
Run the canonical provider-free tests:
58+
59+
```bash
60+
npm test
61+
```
62+
63+
Run the opt-in live-model validation:
64+
65+
```bash
66+
export RUN_MCP_CALENDAR_ADMIN_LIVE_MODEL=1
67+
export OPENAI_API_KEY=...
68+
export MODEL=gpt-4o-mini
69+
npm test -- --test-name-pattern="live model tool surface changes with authoritative state"
70+
```
71+
72+
HTTP contract:
73+
74+
- `MODEL` is sent as-is to the target OpenAI-compatible endpoint
75+
- `OPENAI_BASE_URL` overrides the default OpenAI-compatible base URL
76+
- if `OPENAI_BASE_URL` is unset, the helper uses
77+
`https://api.openai.com/v1`
78+
- `OPENAI_BASE_URL` overrides the OpenAI-compatible base URL when needed
79+
- `OPENAI_API_KEY` is required for the live-model HTTP call
80+
81+
Run the same opt-in validation with local Ollama:
82+
83+
```bash
84+
export RUN_MCP_CALENDAR_ADMIN_LIVE_MODEL=1
85+
export OPENAI_BASE_URL=http://127.0.0.1:11434/v1
86+
export OPENAI_API_KEY=ollama
87+
export MODEL=qwen2.5:1.5b-instruct
88+
npm test -- --test-name-pattern="live model tool surface changes with authoritative state"
89+
```

0 commit comments

Comments
 (0)