Skip to content

Commit 7169291

Browse files
panyampcarleton
andauthored
feat(sep-2640): skills server conformance against the Accepted SEP, enumeration + manifest + directory (#330)
* chore: add SEP-2640 requirement-traceability YAML (Skills Extension) * style: apply yaml formatter pass * chore(sep-2640): record spec source provenance * chore(sep-2640): re-extract against SEP HEAD 556154c (drops mcp-resource-template) Re-extract sep-2640.yaml against current SEP-2640 HEAD. The SEP removed the `mcp-resource-template` index entry type in two commits on 2026-06-04 (fd50cc91 "Remove mcp-resource-template entries from skill index", 556154c0 "Remove remaining resource template mentions from SDK and rationale sections"), after this extraction was first captured on 2026-06-03 at b77fdfe8. Provenance comment bumped accordingly. Resulting changes: - sep-2640-index-entry-type-enum: drop `"mcp-resource-template"` from the enum. - sep-2640-index-name-required: drop the "is Required for ... Omitted for mcp-resource-template" conditional language; the column is now unconditional Required=Yes at SEP HEAD. - sep-2640-index-digest-required: same simplification; also align "hexadecimal" -> "hex" with the HEAD table text. - sep-2640-template-resource-template-registered: removed. The SEP no longer defines a template entry type, so the SHOULD on registering an MCP resource template is gone. - Excluded "Hosts SHOULD surface template entries..." excerpt: removed. The sentence no longer exists in the SEP. Sweep verified: zero "template" mentions in the SEP at HEAD. Security Implications section sentences referenced by the remaining excluded entries are intact. No other check rows touched. * feat(sep-2640): ResourcesDirectoryReadScenario for resources/directory/read Adds the conformance scenario for the SEP-2640 directoryRead surface that landed in spec commit 2e04c48d (2026-06-09). Per AGENTS.md "fewer scenarios, more checks", a single ResourcesDirectoryReadScenario emits 6 ConformanceChecks, one per new sep-2640.yaml requirement row. Class named for the wire method (resources/directory/read), matching the existing ResourcesListScenario / ResourcesReadTextScenario / etc. family in src/scenarios/server/resources.ts. The runner-facing name field stays as 'sep-2640-skills' (umbrella) so mcpkit's conformance/Makefile entry --scenario sep-2640-skills keeps working without a cross-repo race. src/seps/sep-2640.yaml - File-level provenance held at 556154c (the existing PR 330 baseline). - 6 new check rows appended for the directoryRead additions, each carrying a verbatim, grep-F-searchable excerpt from the SEP at 2e04c48da90224000e750ffd54a3611f2824fbc0: - sep-2640-capability-directory-read-flag - sep-2640-directory-read-method-registered - sep-2640-directory-read-subdir-mimetype - sep-2640-directory-read-result-resources-shape - sep-2640-directory-read-invalid-params - sep-2640-directory-read-pagination - forward_reference header comment notes the asymmetry: PR 97 schema rewrite (360123d0, 2026-06-08) made 3 existing rows stale and drifted ~11 others' wording; full re-extraction at SEP HEAD is mcpkit#780's lifecycle. Provenance deliberately holds at 556154c until that lands. src/types.ts - Adds io.modelcontextprotocol/skills to EXTENSION_IDS so the scenario's source: { extensionId: ... } tag type-checks. src/scenarios/server/directory.ts (new) - Capability discovery via wire-observable signal: -32601 method-not-found is the only definitive "server did not declare directoryRead" signal. - Fixture assumption: server exposes skill://acme/billing/refunds/templates with at least one subdirectory child (mcpkit examples/skills layout). No skill:// resources at all -> every check emits SKIPPED so the scenario stays green against upstream's everything-server fixture. - 6 checks (1:1 with the YAML rows above): 1. directoryRead declared (derived from method registration) 2. method registered (happy-path call succeeds) 3. result.resources shape matches resources/list 4. subdir child carries mimeType: "inode/directory" 5. non-directory URI returns -32602 6. nextCursor round-trips (single-page is conformant) src/scenarios/index.ts - Imports + registers ResourcesDirectoryReadScenario in allClientScenariosList, matching the registration name 'sep-2640-skills' that mcpkit's conformance/Makefile already passes to --scenario. Verified end-to-end against mcpkit examples/skills: cd ~/newstack/mcpkit/main && \ MCPCONFORMANCE_SKILLS_PATH=~/newstack/mcpkit/conf-skills \ make -C conformance testconf-skills -> 6/6 SUCCESS, 0 failed, 0 warnings. Out of scope: - PR 97 schema rewrite YAML refresh (separate ticket, mcpkit#780 lifecycle). - Negative-capability fixture (server without directoryRead) - mcpkit examples/skills has no flag for it; ext/skills/client_directory_test.go already covers the SDK pre-call guard. * feat(sep-2640): expand skills conformance — index + manifest scenarios, drift fixes Builds on the directory-read scenario with the server-observable skills surface and brings the whole set up to the current framework conventions. New scenarios (src/scenarios/server/skills/): - SkillsIndexScenario (sep-2640-skills-index): skill://index.json is exposed and readable; every skills[].type is in the SEP enum; every entry has a non-empty name; a present digest matches sha256:{64hex}; entry URLs use skill://. - SkillsManifestScenario (sep-2640-skills-manifest): a skill-md SKILL.md is markdown-typed; frontmatter name/description surface on the resource; the final skill-path segment equals the frontmatter name; skill _meta keys use the reverse-domain prefix. directory.ts (moved into skills/): capability is now read from server/discover instead of inferred from -32601; an undeclared optional capability SKIPs rather than FAILs; the untestable.ts helpers back the not-exercisable paths; all mcpkit-specific fixture URIs are gone — the directory to exercise is discovered dynamically, so the scenario is brand-neutral. Capability gating accepts directoryRead at either extensions[id].directoryRead (SEP-2640's inline example) or extensions[id].config.directoryRead (the SEP-2133 negotiation envelope SEP-2640 defers to); the two SEPs are inconsistent on nesting, so the check accepts either rather than privileging one reading. All three scenarios verified green against a live SEP-2640 server (6/6, 6/6, 7/7); typecheck, eslint, prettier, and the 504-test vitest suite pass. sep-2640.yaml gains a header note mapping each scenario to the rows it now backs. * feat(sep-2640): re-extract against the 2026-08-21 rewrite The prior extraction held at SEP commit 556154c (2026-06-05) by an explicit in-file decision to wait for the index-schema refactor. The 08-21 core-maintainer rewrite made that pin actively wrong rather than merely behind. Removed as no longer normative: the skill://index.json well-known resource (absent from the current text), the entry type enum, the no-nested-skills rule (now reversed - skills MAY nest), and the five archive rows (archives are deferred to an appendix). Reworded the two index-field rows, the empty-listing row and the verify-digest row. Added the new surface: skills/list and skills/get with their pagination, atomicity and caching attributes; the {uri, frontmatter, resources} entry schema; resources completeness, the "dynamic" sentinel and per-file size; the 512-entry and 16 MiB limits; and the host obligations around lazy retrieval, content-bound approval and frontmatter re-verification. index.ts is replaced by enumeration.ts, which drives both new methods. manifest.ts and directory.ts swap their skill://index.json discovery fallback for skills/list. The capability helper no longer accepts a config envelope. SEP-2133 is Final and defines the extensions field as a map of identifiers to settings objects, with no envelope and no slot for id/specVersion/stability; SEP-2640's capability block matches. The earlier leniency was based on a misreading. 77 rows: 65 checks (39 wire-tested by the three server scenarios) and 12 excluded as host-internal. * feat(sep-2640): close the extraction gaps found by a normative-sentence audit The first pass extracted the Specification sections but under-covered Security Implications, which carries 21 of the SEP's 97 RFC-2119 sentences. Sweeping every normative sentence in the source surfaced rows that were missing entirely: - origin tagging at the point content enters model context (the L179 host MUST) - allowed-tools MUST be ignored for MCP-origin skills - reads bound to the originating server, so a skill from A cannot drive a read against B, and any cross-origin read gated behind per-call approval - servers identified by a host-assigned label, never serverInfo.name - the three disk-cache rules: write-isolation or re-hash, exclusion from filesystem-skill discovery paths, and removal when a server is removed - a resources/read of a SKILL.md by another route is not a load - a nested SKILL.md read as supporting content is ordinary markdown - the registry-assembly and virtual-mount no-prefetch restatements - the SDK affordances guidance, which is the row mcpkit itself is measured by Five rows previously filed as excluded are now declared instead. They are real requirements that a host-side harness could check; calling them unobservable was a limitation of this scenario set, not of the requirement. Adds one wire check, names-should-be-unique, since a within-listing collision is observable and tells an operator the listing needs disambiguating. 96 rows: 89 checks (40 wire-tested) and 7 excluded. * style: apply prettier to the SEP-2640 scenarios and yaml Quote-style normalisation in the yaml (escaped single quotes become double-quoted scalars) and one return-type wrap in helpers.ts. The parsed yaml is byte-identical in content; verified by comparing the parse before and after. * docs(sep-2640): note why resultType is not declared in this yaml It appears in all three result examples but is a base-protocol field on the common Result interface, not a SEP-2640 requirement; its caching-hint obligations are already owned by sep-2549.yaml. * fix(sep-2640): follow nextCursor on resources/directory/read The directory scenario issued one resources/directory/read and every check below it inspected that single page: the result-shape check, the subdirectory-mimeType check, and the target selection for the -32602 probe. SEP-2640 says directory-read pagination mirrors resources/list, so a conformant server MAY split a directory across pages, and reading only the first made "no subdirectory here" indistinguishable from "the subdirectory is on page two". Reported by Sam Bloomberg against the go-sdk implementation, which exposes DirectoryOptions.PageSize. Rebuilding that server with PageSize 1 fails sep-2640-directory-read-subdir-mimetype; with this change it passes 7/7, and both servers still pass at their default page size. Only findable against a second implementation. mcpkit hardcoded its page size to zero with no override, so it never emitted a cursor and the defect was invisible to every run against it. listAllResources already did this for resources/list. directoryReadAll applies the same shape to directory reads, with a page bound and a repeated-cursor guard. * feat(sep-2640): client scenario for the no-prefetch MUST NOT The traceability file filed retrieval policy under "host obligations a harness cannot observe", alongside 48 other rows. That was wrong for this one: "hosts MUST NOT retrieve a skill's files ahead of need" reduces to whether a request arrived, which makes it among the most wire-observable rules in the SEP. Adds a Scenario (harness stands up the server, client is the SUT) that serves a listing naming a SKILL.md and a supporting file, then fails if either is read. The client's contract, keyed on MCP_CONFORMANCE_SCENARIO, is to connect, enumerate, and stop. SKIPs rather than passes when the client never calls skills/list, since without a listing there is no window in which prefetching is possible and a pass would prove nothing. Verified both directions against mcpkit's client: SUCCESS as written, and FAILURE with a deliberate ReadFromEntry added to the driver. A check that has never been shown to fail is the same trap as the directory-pagination false negative this suite shipped for months. * feat(sep-2640): client scenarios for the four read-time verification MUSTs Adds digest, size, frontmatter and unlisted-URI variants alongside the no-prefetch scenario. Each serves an internally honest listing, tampers with exactly one thing on the wire, and watches whether the client notices. Rejection is detected by absence, following auth/resource-mismatch: the client's contract is "load the skill, then read its supporting file", and a client that verifies aborts on the load and never reaches the supporting file. Reaching it is the violation. The unlisted variant is proven by the read never appearing on the wire at all, since a conformant client refuses locally. The frontmatter variant recomputes digest and size for the tampered body, so the frontmatter comparison is the only thing left that can fail. Without that the digest check would mask it and the scenario would pass for the wrong reason. allowClientError is set on all four: a verifying client exits non-zero because it correctly aborted, and that is the pass. Verified both directions against mcpkit's client. All four SUCCESS as written; all four FAILURE against a driver rewritten to read through ReadSkillURI, which skips verification entirely. * fix(sep-2640): gate the cache-attributes check on the negotiated version The check graded ttlMs and cacheScope without looking at the protocol version, which made it wrong in both directions. Below 2026-07-28 those fields are not defined by the negotiated schema at all, so a server that omits them is correct and the WARNING was a false positive. From 2026-07-28 they are required, so a missing one is a failure rather than a warning. Now SKIPPED with a version reason below 2026-07-28, and SUCCESS or FAILURE above it. Reported by Sam Bloomberg against the Go SDK, which was correct on both wires while this check mis-graded it on both. Second finding from the same reviewer against this branch, and the second that only a second implementation could surface: mcpkit emits the attributes on every version it supports, so it could never have produced the false positive. Verified three ways: mcpkit at 2026-07-28 SUCCESS, the Go SDK at 2025-11-25 SKIPPED where it previously warned, and a deliberately stripped mcpkit build at 2026-07-28 FAILURE. * docs(sep-2640): how to run the skills scenarios against any implementation Three implementations have now been cross-checked and every one needed a different invocation, so the knowledge was living in chat rather than the repo. Records the two flags and why both are needed, how to read totals that include wire-schema-valid, and the per-SDK setup. The two traps are worth writing down because both have already cost someone time. Extension scenarios do not match --spec-version alone, so --force is always needed. And --spec-version selects the wire lifecycle rather than filtering, so omitting it against a pre-2026-07-28 server produces a -32022 that reads like a missing method. Also records that the go-sdk serves 2026-07-28 only when its streamable transport is Stateless, which is correct per SEP-2575 and which I briefly mistook for a defect. * fix(sep-2640): advertise skills on server/discover, not only initialize Both client scenarios declared the skills extension and `resources` in their `initialize` reply, but `BaseHttpScenario` intercepts `server/discover` before `handlePost` runs. On the 2026-07-28 lifecycle a discover-first client therefore saw the base class default of `{tools: {}}` and had nothing to gate a `skills/list` call on. The scenarios still passed, because `server/discover` is optional for clients and neither client we ran used it. That is the failure mode where a check goes green because its condition never arose, so both lifecycles are now asserted in capabilities.test.ts rather than left to the next implementation to find. Overriding `discoverCapabilities()` covers both paths at once, since `sendInitialize` already defaults to it. * fix(sep-2640): give the capability and naming checks something to fail on `sep-2640-capability-empty-object` reported a hardcoded SUCCESS, so it could not fail and was checking nothing. It now asserts the observable half of "an empty object indicates support for the extension with no optional features": the declared value must be an object at all. That also closes a false SKIP. `skillsCapability` folded a malformed declaration into `undefined`, which downstream read as "extension not declared" and skipped the whole suite, so a server declaring the extension as `true` produced a clean run. `declaredSkillsCapability` separates absent from malformed; the inline/envelope check now SKIPs on a malformed value, since there are no settings to place either way. The base `DiscoverResult` schema rejects a non-object too, so both halves are asserted. `SKILL_NAME_PATTERN` accepted consecutive hyphens. The Agent Skills specification that SEP-2640 defers to for naming lists four rules and we encoded three, so `pdf--processing` passed. Its own valid and invalid examples are now the test cases. Negative controls recorded for all three: with the check back to a hardcoded SUCCESS the four malformed-declaration cases fail, and with the previous regex the consecutive-hyphen case fails. * docs(sep-2640): name skills/list in the discovery prose, not index.json The 2026-08-21 revision replaced the `skill://index.json` well-known resource with `skills/list`, and the code follows it — `discoverDirectory` and the manifest fallback both call `skillsListAll`. Only the prose lagged, including three strings a user actually sees: two untestable-check reasons and the directory scenario's description. The remaining mentions in helpers.ts, enumeration.ts and sep-2640.yaml are deliberate history, describing what was replaced. * fix(sep-2640): grade unlisted reads against the entry, not one fixture URI The `unlisted` client scenario asserted `reads.includes(UNLISTED_URI)`, so it only caught a client that probed the exact path this fixture happens to serve. Any client choosing a different unlisted path passed without being tested, and the check would have reported SUCCESS for a client that reads unlisted files freely. The requirement is "hosts MUST resolve reads of the skill's files only to URIs listed in that entry's resources", so the permitted set now comes from the entry the client was handed and any read outside it is the violation. That is the sentence itself rather than a proxy for it, and it lets a SUT derive its own probe instead of hardcoding ours. Negative control: a client reading a synthesized `mcpkit-unlisted-probe-0.md` is now a FAILURE, where the previous check reported SUCCESS. * fix(sep-2640): require the manifest read, and retire the unlisted scenario Two changes to the client-side verification scenarios, one of which fixes a hole the other one shares. **The manifest read is now a prerequisite.** Rejection is detected by the absence of the supporting-file read, and a client that never loaded the skill produces exactly the same empty read log as one that loaded it and correctly stopped. Both reported SUCCESS, so `listCalled` was the only gate and a client implementing `skills/list` and nothing else passed every variant here. It scored 5/5 on this suite while doing nothing. Absence now means something only after the client has read the tampered `SKILL.md`. A missing prerequisite reports untestable, which per #248 is a FAILURE rather than a SKIP: SKIPPED is excluded from pass/fail counts and exit codes, so it would read as green and hide the gap. Negative control: a driver that lists and exits goes from 1/1 to 0/1 on all three variants. **`-verify-unlisted` is retired**, on Peter Alexander's call. Nothing was tampered in it, so the only thing under test was whether the driver asked for a URI it should not have, which grades the driver rather than the client. It is also the rule an inspection tool answers differently from a host, since refusing a read requires having a load to refuse, which MCP Inspector 2.6.0 reported as "surfaced" rather than passed. The requirement is not deleted. It moves to an `excluded:` row alongside the acting-on window that scopes it, and the reason records that it is excluded by construction rather than in principle: a `SKILL.md` body linking to a file absent from `resources` would make the harness adversarial again, since a host that follows the link fetches an unlisted file on its own initiative. The scenario name and check id are named there so a future reintroduction lines up with historical reports rather than reading as a new requirement. Check rows go from 89 to 88. * feat(sep-2640): check skills/get cache attributes and the resources capability Both requirements landed when the stable spec page merged ext-skills#139 today, and both are checkable on the wire. **`sep-2640-skills-get-cache-attributes`.** `GetSkillResult extends CacheableResult`, so `ttlMs` and `cacheScope` are REQUIRED as they are on `resources/read`. Held out of this PR until the page merged rather than grading against text that was still open: SEP-2640 says of these fields that "whether the result should also carry the base protocol's caching attributes ... is left open", so this is a behaviour change and a server conformant to the SEP text alone omits both. It is a reading not everyone shares. The MCP Inspector's SEP-2640 writeup argues the opposite, that the SEP forecloses it and inventing the requirement would report a reasonable server as broken. Gated the same way as the skills/list check: below 2026-07-28 `CacheableResult` is undefined by the negotiated schema, so omission is correct and the check SKIPs with a version reason rather than failing a conformant legacy server. **`sep-2640-capability-requires-resources`.** A server declaring the extension MUST also declare the base `resources` capability, since skill files are served through `resources/read`. The page states it as a consequence of the base Resources specification rather than a free-standing new rule, and the declaration is on the wire either way. Verified against mcpkit: 32/32 on 2026-07-28 and 30/30 on 2025-11-25, where the get-cache check SKIPs below the floor as designed. Negative control recorded: a build with the attributes stripped from `handleSkillsGet` reports 31/32 with `sep-2640-skills-get-cache-attributes` FAILURE. Check rows go from 88 to 90. * fix(sep-2640): compare frontmatter by content, not key order entry-frontmatter-identical compared each top-level key with JSON.stringify, which is key-order sensitive for nested objects. A server whose encoder sorts map keys (Go's encoding/json does) failed this MUST for any SKILL.md whose metadata keys are not in alphabetical order, although the content is identical. Compare structurally instead. A real difference still fails; both cases are pinned in negative.test.ts. * fix(sep-2640): make the client-scenario mocks answer like a conformant server The skills/list and resources/read replies carried only resultType, so on the 2026-07-28 wire they lacked the ttlMs/cacheScope a ReadResourceResult requires, and skills/list lacked the attributes the enumeration scenario requires of servers. initialize always answered 2026-07-28, while the runner hands these extension scenarios' clients 2025-11-25 by default. A client that checks the negotiated version disconnected before listing, and every verify variant reported SKIPPED whether or not the client verified anything. The mocks now stamp the caching members and answer initialize at the requested version when the suite can speak it. The runner default is left alone because it also applies to the auth extension scenarios. Pinned in draft-result-fields.test.ts next to the other hand-rolled mocks. * docs(sep-2640): run from the published package; --force only with --spec-version The run notes pointed at a fork branch that goes stale on merge, and said --force is always needed. Extension scenarios only SKIP when an explicit --spec-version is passed; without it they run on the draft wire. * docs(sep-2640): drop RUNNING_SEP2640.md Per-run notes and SDK result tables belong in the PR description, not a root-level doc that goes stale on merge. --------- Co-authored-by: Paul Carleton <paulc@anthropic.com>
1 parent a983ba9 commit 7169291

13 files changed

Lines changed: 3767 additions & 3 deletions

‎src/scenarios/client/draft-result-fields.test.ts‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ import type { BaseHttpScenario } from './http-base';
1212
import { RequestMetadataScenario } from './request-metadata';
1313
import { MRTRClientScenario } from './mrtr-client';
1414
import { JsonSchemaRefDerefScenario } from './json-schema-ref-deref';
15+
import { SkillsNoPrefetchScenario } from './skills/no-prefetch';
16+
import { SkillsVerificationScenario } from './skills/verification';
1517

1618
/**
1719
* Pins that the hand-rolled mock servers used by client-direction scenarios
@@ -528,3 +530,96 @@ describe('sep-2322-client-request-state mock results (2026-07-28)', () => {
528530
}
529531
});
530532
});
533+
534+
describe('sep-2640 skills client mock results', () => {
535+
const skillsScenarios: Array<{
536+
name: string;
537+
make: () => BaseHttpScenario;
538+
}> = [
539+
{
540+
name: 'sep-2640-client-no-prefetch',
541+
make: () => new SkillsNoPrefetchScenario()
542+
},
543+
{
544+
name: 'sep-2640-client-verify-digest',
545+
make: () => new SkillsVerificationScenario('digest')
546+
},
547+
{
548+
name: 'sep-2640-client-verify-size',
549+
make: () => new SkillsVerificationScenario('size')
550+
},
551+
{
552+
name: 'sep-2640-client-verify-frontmatter',
553+
make: () => new SkillsVerificationScenario('frontmatter')
554+
}
555+
];
556+
557+
for (const s of skillsScenarios) {
558+
it(`${s.name} carries the draft-required members on skills/list and resources/read`, async () => {
559+
const scenario = s.make();
560+
const { serverUrl } = await scenario.start(
561+
testScenarioContext(DRAFT_PROTOCOL_VERSION)
562+
);
563+
try {
564+
let id = 1;
565+
for (const [method, params] of [
566+
['skills/list', {}],
567+
['resources/read', { uri: 'skill://pdf-processing/SKILL.md' }]
568+
] as const) {
569+
const { status, body } = await post(
570+
serverUrl,
571+
{
572+
jsonrpc: '2.0',
573+
id: id++,
574+
method,
575+
params: { ...params, _meta: meta }
576+
},
577+
{
578+
'mcp-protocol-version': DRAFT_PROTOCOL_VERSION,
579+
'Mcp-Method': method
580+
}
581+
);
582+
expect(status, method).toBe(200);
583+
expect(body.result, method).toMatchObject(CACHEABLE_FIELDS);
584+
if (method === 'resources/read') {
585+
expect(
586+
wireSchemaErrors(DRAFT_PROTOCOL_VERSION, body, method),
587+
method
588+
).toEqual([]);
589+
}
590+
}
591+
} finally {
592+
await scenario.stop();
593+
}
594+
});
595+
596+
// The runner hands these scenarios' clients 2025-11-25 unless
597+
// --spec-version says otherwise, and the extension has no 2026-07-28
598+
// dependency, so the mock answers at the version the client asked for.
599+
it(`${s.name} answers initialize at the requested protocol version`, async () => {
600+
const scenario = s.make();
601+
const { serverUrl } = await scenario.start(testScenarioContext());
602+
try {
603+
let id = 1;
604+
for (const protocolVersion of ['2025-11-25', DRAFT_PROTOCOL_VERSION]) {
605+
const { status, body } = await post(serverUrl, {
606+
jsonrpc: '2.0',
607+
id: id++,
608+
method: 'initialize',
609+
params: {
610+
protocolVersion,
611+
capabilities: {},
612+
clientInfo: { name: 'test', version: '1.0' }
613+
}
614+
});
615+
expect(status, protocolVersion).toBe(200);
616+
expect(body.result.protocolVersion, protocolVersion).toBe(
617+
protocolVersion
618+
);
619+
}
620+
} finally {
621+
await scenario.stop();
622+
}
623+
});
624+
}
625+
});
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
import { describe, test, expect } from 'vitest';
2+
import { testScenarioContext } from '../../../mock-server/testing';
3+
import { sendStatelessRequest } from '../../../connection/stateless';
4+
import { DRAFT_PROTOCOL_VERSION } from '../../../types';
5+
import { SkillsNoPrefetchScenario } from './no-prefetch';
6+
import { SkillsVerificationScenario } from './verification';
7+
8+
/**
9+
* Regression for PR #330 review: both skills client scenarios declared the
10+
* skills extension only in their `initialize` reply, but `BaseHttpScenario`
11+
* intercepts `server/discover` before `handlePost` runs. A 2026-07-28
12+
* discover-first client therefore saw the base class default (`{tools: {}}`)
13+
* and had nothing to gate a `skills/list` call on.
14+
*
15+
* The scenarios passed anyway because `server/discover` is optional for
16+
* clients and the clients we ran did not use it. That is exactly the failure
17+
* mode where a check goes green because its condition never arose, so both
18+
* lifecycles are asserted here rather than left to the next client to notice.
19+
*/
20+
21+
const SKILLS_EXTENSION_ID = 'io.modelcontextprotocol/skills';
22+
23+
type Caps = {
24+
resources?: unknown;
25+
extensions?: Record<string, unknown>;
26+
};
27+
28+
function capsOf(result: unknown): Caps {
29+
return ((result as { capabilities?: Caps })?.capabilities ?? {}) as Caps;
30+
}
31+
32+
function expectSkillsAdvertised(caps: Caps, where: string): void {
33+
expect(caps.extensions, `${where}: no extensions block`).toBeDefined();
34+
expect(
35+
caps.extensions?.[SKILLS_EXTENSION_ID],
36+
`${where}: skills extension not advertised`
37+
).toBeDefined();
38+
// resources/read is how every skill file is fetched, so a client that gates
39+
// on capabilities needs this too.
40+
expect(caps.resources, `${where}: resources not advertised`).toBeDefined();
41+
}
42+
43+
const SCENARIOS: Array<
44+
[string, () => SkillsNoPrefetchScenario | SkillsVerificationScenario]
45+
> = [
46+
['no-prefetch', () => new SkillsNoPrefetchScenario()],
47+
['verify-digest', () => new SkillsVerificationScenario('digest')],
48+
['verify-size', () => new SkillsVerificationScenario('size')],
49+
['verify-frontmatter', () => new SkillsVerificationScenario('frontmatter')]
50+
];
51+
52+
describe('SEP-2640 client scenarios advertise skills on both lifecycles', () => {
53+
test.each(SCENARIOS)('%s: server/discover', async (label, make) => {
54+
const scenario = make();
55+
const { serverUrl } = await scenario.start(testScenarioContext());
56+
try {
57+
const discover = await sendStatelessRequest(serverUrl, 'server/discover');
58+
expect(discover.status).toBe(200);
59+
expectSkillsAdvertised(
60+
capsOf(discover.body?.result),
61+
`${label} discover`
62+
);
63+
} finally {
64+
await scenario.stop();
65+
}
66+
});
67+
68+
test.each(SCENARIOS)('%s: initialize', async (label, make) => {
69+
const scenario = make();
70+
const { serverUrl } = await scenario.start(testScenarioContext());
71+
try {
72+
const init = await sendStatelessRequest(serverUrl, 'initialize', {
73+
protocolVersion: DRAFT_PROTOCOL_VERSION,
74+
capabilities: {},
75+
clientInfo: { name: 'capabilities-test', version: '1.0.0' }
76+
});
77+
expect(init.status).toBe(200);
78+
expectSkillsAdvertised(capsOf(init.body?.result), `${label} initialize`);
79+
} finally {
80+
await scenario.stop();
81+
}
82+
});
83+
});
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
/**
2+
* Results for the hand-rolled SEP-2640 client-scenario mocks, shaped like a
3+
* conformant server's so a strict client is graded on its skills behaviour
4+
* rather than turned away by the harness.
5+
*/
6+
7+
import {
8+
DRAFT_PROTOCOL_VERSION,
9+
NEGOTIABLE_PROTOCOL_VERSIONS
10+
} from '../../../types.js';
11+
import { withRequiredDraftResultFields } from '../../../mock-server/index.js';
12+
13+
/**
14+
* `ListSkillsResult` is cacheable like `resources/list`, but `skills/list` is
15+
* an extension method and not in the shared cacheable set, so the caching
16+
* members are stamped here.
17+
*/
18+
export function skillsListResult(skills: object[]): object {
19+
return { resultType: 'complete', ttlMs: 0, cacheScope: 'private', skills };
20+
}
21+
22+
export function readResult(uri: unknown, text: string): unknown {
23+
return withRequiredDraftResultFields('resources/read', {
24+
contents: [{ uri, mimeType: 'text/markdown', text }]
25+
});
26+
}
27+
28+
/**
29+
* The `initialize` result at the version the client asked for, when the suite
30+
* can speak it. The extension has no 2026-07-28 dependency, and the runner
31+
* hands these scenarios' clients 2025-11-25 unless `--spec-version` says
32+
* otherwise, so a fixed 2026-07-28 reply would turn a correct stateful client
33+
* away before it ever lists.
34+
*/
35+
export function initializeResult(
36+
name: string,
37+
request: { params?: { protocolVersion?: unknown } },
38+
capabilities: object
39+
): object {
40+
const requested = request.params?.protocolVersion;
41+
const protocolVersion =
42+
typeof requested === 'string' &&
43+
NEGOTIABLE_PROTOCOL_VERSIONS.includes(requested)
44+
? requested
45+
: DRAFT_PROTOCOL_VERSION;
46+
return {
47+
resultType: 'complete',
48+
protocolVersion,
49+
serverInfo: { name: `${name}-server`, version: '1.0.0' },
50+
capabilities
51+
};
52+
}

0 commit comments

Comments
 (0)