You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1ab5242
Browse filesBrowse the repository at this point in the historyBrowse files
Run every referee scenario in the server conformance leg
## Motivation and Context
The server conformance leg ran the referee's default `active` suite, which leaves out the extension-tagged scenarios,
while the client leg has selected every scenario with `--suite all`. The Skills server implementation
therefore never met the referee's `sep-2640-skills-*` server scenarios, although the conformance server already serves
the `conformance/skills` directory for them, and the `all` suite's other additions (stateless server, SSE polling, caching,
standard header validation, `input-required` results) were not run either.
The server leg now selects `--suite all` the way the client leg does, with a requirement set or a single scenario
still replacing the selection. Of the 34 scenarios this adds, 23 pass. The rest reach three areas the SDK does not implement
and are listed in `expected_failures.yml` the way the TypeScript and Python SDKs list theirs: the Tasks extension (SEP-2663);
the server side of SEP-2243 custom headers, where the server does not bind `Mcp-Param-*` headers to `x-mcp-header` tool parameters
and the conformance server exposes no such tool, so the referee reports the scenario's checks as untestable; and SEP-1699 resumability,
where a POST SSE stream carries no priming event or `retry` field, which the referee reports as warnings that its baseline counts as failures.
`tasks-status-notifications` is left out of the baseline because the referee runs no check in it, and the baseline check would flag
an entry for a passing scenario as stale.
## How Has This Been Tested?
test/conformance/server_runner_test.rb, and `bundle exec rake conformance:test`: the server leg runs 65 scenarios,
the 11 baselined failures reported as expected and the three `sep-2640-skills-*` scenarios passing, where it ran 31 before the change.
`REQUIREMENTS=2026-07-28` still runs the frozen set without a suite.
## Breaking Changes
None.
0 commit comments