Repository navigation
Conversation
Server scenarios gain six checks. enumeration.ts grades nested-skill completeness, a flat listing, -32602 for an unserved skill file, and an informative error message. directory.ts grades ttlMs and cacheScope on the directory result, and resources: [] for an empty directory. A new client file, hosting.ts, adds four scenarios with seven checks: loading a skill by URI after an empty listing, refusing invalid entries, refusing a directory child the entry does not list, and keeping two servers' skills apart. The two-origins scenario starts a second mock server and hands its URL to the client in MCP_CONFORMANCE_CONTEXT, the way the auth scenarios hand over an authorization server. Every new check has a negative control. The sep-2640.yaml header now counts 59 of 89 rows as emitted, up from 46.
commit: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Grades 13 more
sep-2640.yamlrows. The yaml counted 46 of its 89 checks as emitted, and now counts 59.Six server checks join the existing scenarios:
enumeration.ts:sep-2640-resources-nested-completenessandsep-2640-nested-publication-flatfor nested skills, plussep-2640-error-standard-codes(-32602for a skill file the server does not serve) andsep-2640-error-informative-messages(a WARNING when the message only names the code).directory.ts:sep-2640-directory-read-cache-attributes(ttlMsandcacheScopeon the result, from 2026-07-28) andsep-2640-directory-read-empty-dir(an empty directory reads asresources: []).Four client scenarios, in a new
src/scenarios/client/skills/hosting.ts, add seven checks:sep-2640-client-load-by-urihost-load-by-uri,host-no-empty-listing-assumptionsep-2640-client-invalid-entryresources-invalid-entrysep-2640-client-directory-unlisted-childdirectory-read-not-manifest-extensionsep-2640-client-two-originshost-skill-identity-pair,host-cross-origin-no-shadow,host-reads-bound-to-originThe two-origins scenario starts a second mock server and hands its URL to the client in
MCP_CONFORMANCE_CONTEXTassecondServerUrl, the way the auth scenarios hand over an authorization server. The runner needed no change.Prerequisite knowledge
AGENTS.md: fewer scenarios with more checks, one ID for SUCCESS and FAILURE, and a negative control per check.verification.ts's header, on why theunlistedvariant was retired. The new client scenarios are built to avoid the same problem.Reviewer's guide
The client scenarios grade requests that must not arrive, so two rules shape them.
unlistedvariant, would only grade the driver.skills/liston both servers) and reports untestable without it, per Standardize how scenarios report setup/execution failures #248.Read in this order:
src/scenarios/client/skills/mock-host.ts: one configurable SEP-2640 server that records every request.src/scenarios/client/skills/hosting.ts: the four scenarios, each with its contract in the class comment.src/scenarios/client/skills/hosting.test.ts: a conforming scripted client and one rule-breaking client per check. The two-origins tests break it three ways: caching by uri alone, keying skills by name alone, and reading one server's files from the other.enumeration.ts(nestedChecks,errorChecks) anddirectory.ts(cacheCheck,emptyDirCheck), thensrc/scenarios/server/skills/coverage.test.ts.src/seps/sep-2640.yaml: the header's coverage counts and backing-scenario list. No row changes.Decision log
-32002also passeserror-standard-codeson 2025-11-25. That base spec named-32002for an unknown resource. From 2026-07-28 only-32602passes.nested-publication-flatchecks that no entry embeds another. Of "nothing in the listing marks nesting", that is the part a harness can see.resources-dynamic-marker(a harness cannot tell whether content is generated),enumeration-scheme-uniform,skills-get-answers-unlisted(no way to find an unlisted URI on an arbitrary server),read-is-not-a-load, and the MAY rows.Risk / blast radius
examples/skillsat 51 pass / 0 fail / 0 warn / 0 skip. The client scenarios pass against mcpkit's client. One of them found a real mcpkit bug: a single invalid entry failed the wholeskills/listdecode (skills: grade 13 more SEP-2640 rows, and keep a listing with one invalid entry loadable panyam/mcpkit#1728).