You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 76b4ae4
Browse filesBrowse the repository at this point in the historyBrowse files
feat(connector-mcp,spec): toolsSnapshot pins each MCP tool; a mismatch degrades through CONNECTOR_UPSTREAM_CHANGED (#28515)
Fixes#27401
Clause-②: yes (widening)
Metadata-protocol: added
Skills-docs: follow-up owed — skills/** is not edited here (Tier H);
whether a published skill should teach `providerConfig.toolsSnapshot` is
for the seat that owns skills/**
## 维护者速读
- 改了什么:MCP 连接器实例可以写 `providerConfig.toolsSnapshot`,按工具名逐个钉住工具定义的
sha256;服务端改写了被钉的工具(描述或参数)或不再提供它,实例进入降级、不暴露任何工具,原因里点名是哪个工具。降级走新错误码
`CONNECTOR_UPSTREAM_CHANGED`,日志写「工具定义与 toolsSnapshot
不符、退避重查」,不再借用「上游不可达」。
-
为什么改:执行你裁的两件事:逐工具钉快照(6506797088,B)和钉子不符要有自己的错误码与措辞(6545635244,B)。`CONNECTOR_UPSTREAM_UNAVAILABLE`
仍只表示上游连不上。
- 风险与回滚:不写 `toolsSnapshot` 的实例行为完全不变;新错误码只有 connector-mcp 抛。回滚就是 revert
本 PR。
- 先看:`packages/services/service-automation/src/plugin.ts`(新的降级分支和 ERROR
措辞)、`packages/spec/src/integration/connector-provider-errors.ts`(新错误类和判定函数)、`packages/connectors/connector-mcp/src/mcp-provider.ts`(`toolsSnapshotDriftError`)。
- 两分钟验证:`pnpm --filter @objectstack/service-automation exec vitest run
src/connector-upstream-changed.test.ts`,预期 5 个测试全过,其中一条断言 ERROR
原文是「serves a tool definition that differs from its toolsSnapshot …
re-checking with backoff, attempt 1.」。
- 需要你做的:无,席位落地(落地前要一次 `CONTRACT_REVIEW_TIER` 契约复核,PM 安排)。
## What changes
Two rulings on this card, both letter B: the per-tool pin (comment
6506797088) and its own degrade marker (comment 6545635244). This PR
picks up the branch's ruled connector-mcp work (head `68d8770f`) and
merges `origin/main` into it.
**`@objectstack/connector-mcp`** (minor): `providerConfig.toolsSnapshot`
on a `provider: 'mcp'` instance is a record of tool name → sha256
(lowercase hex) of the canonical JSON (RFC 8785 form) of the tool's `{
name, description, inputSchema }`. `computeMcpToolsSnapshot` and
`mcpToolDigest` are exported.
- Refused before connecting, as plain configuration faults (fatal at
boot): a malformed record, and a pin on a tool `include` leaves out
(such a pin could never be checked).
- After `tools/list`, each pinned tool is compared on its own. If a
pinned tool changed, or the server no longer serves it, the client is
closed and the factory throws **`ConnectorUpstreamChangedError`**. Its
one-line message names each such tool and the digest the server now
serves.
- When the server stops serving a pinned tool that `include` also lists,
the pin reports it. It is not reported as an `include` typo.
**`@objectstack/spec`** (patch, new code ⇒ `Clause-②: yes (widening)`):
`CONNECTOR_UPSTREAM_CHANGED`, `ConnectorUpstreamChangedError`, and the
structural guard `isConnectorUpstreamChanged`, which reads `code` and
never uses `instanceof`. They are declared beside the outage and
credential-rejected markers in
`integration/connector-provider-errors.ts`. Its row sits in
`ERROR_CODE_LEDGER` under `@objectstack/spec` (`door: 'none'`). The
`ConnectorProviderFactory` TSDoc names it as a degrading marker. The
header block of that file is left alone because open PR #28357 edits it.
**`@objectstack/service-automation`** (patch): this is the one
materializer branch the ruling asks for.
`isConnectorUpstreamChanged(thrown)` calls `degradeConnectorInstance`
with `fault: 'changed'`. The backoff is the same 5 s → 5 min as an
outage, and the husk's reason is the provider's message. Only the
wording differs:
- Materializer ERROR: `… (provider 'P') serves a tool definition that
differs from its toolsSnapshot — instance registered degraded (no
actions); re-checking with backoff, attempt N.`
- Engine degrade WARN (`registerDegradedConnector`, which reads the same
structural guard on the `cause` it already receives, with no signature
change): `… no actions and no handlers until its upstream serves the
definitions its pin names again; … the materializer re-checks with
backoff.`
`CONNECTOR_UPSTREAM_UNAVAILABLE` keeps its declared meaning and its
wording. A control test pins that.
Docs: `content/docs/automation/connectors.mdx` (the key's section and
the fault table row). Generated: `api-surface/integration.json`,
`export-origins/integration.json`, and three
`content/docs/references/api/*` pages, written by `check:generated
--fix`.
## Tests
<details><summary>Package suites at f5e02da (all through the verify
lock, VERDICT command-exit 0)</summary>
- `pnpm --filter @objectstack/service-automation test`: Test Files 248
passed (248) · Tests 3157 passed (3157)
- `pnpm --filter @objectstack/connector-mcp test`: Test Files 7 passed
(7) · Tests 88 passed (88)
- `pnpm --filter @objectstack/spec test`: Test Files 780 passed (780) ·
Tests 23076 passed, 1 todo
- typecheck (`tsc --noEmit` + `check:test-typecheck`) for spec,
service-automation and connector-mcp: exit 0
</details>
<details><summary>New and changed tests</summary>
- `service-automation/src/connector-upstream-changed.test.ts` (new, 5
tests):
- the marker's code, name, and both guards telling it apart from an
outage, including structural matching;
- boot degraded with the provider message as the reason, not held;
- the ERROR message pinned whole, the engine WARN's own clause, and
neither "upstream unavailable" nor "until its upstream is reachable" in
any record;
- re-checked at BASE and then 2·BASE, recovering to `ready` once the
upstream serves the pin again;
- control: an outage keeps its wording.
- `connector-mcp/src/tool-snapshot.test.ts` (24 tests from the branch):
drift cases now assert `isConnectorUpstreamChanged` and
`!isConnectorUpstreamUnavailable`. Configuration faults assert neither
marker.
</details>
<details><summary>Ablation (one, on the behaviour this PR
changes)</summary>
`node scripts/ablation-replace.mjs` on
`service-automation/src/plugin.ts`, anchor `if
(isConnectorUpstreamChanged(thrown)) {` → `if (false && …)`. The anchor
went 1 → 0 and the blob 6698cba → 4dcd50fa.
`connector-upstream-changed.test.ts` then had 3 failed and 2 passed: the
boot-degraded, wording and backoff tests went red, while the marker-unit
and outage-control tests stayed green, as expected. The file was
restored: blob equals HEAD and `git diff HEAD` is empty.
</details>
## Acceptance notes
- **The hasLive case keeps the outage policy.** Suppose an instance is
already serving and its declaration is edited while the server has
drifted from the new pin. The previously materialized connector keeps
serving, and the ERROR says so. This is the same as an unreachable
upstream on an edit, and the same backoff swaps it in on recovery. The
ruling asked for "the same backoff", and nothing here takes a live
connector down.
- **The ERROR wording names `toolsSnapshot`.** That follows the ruling's
quoted text. The marker itself is provider-neutral, and mcp is its only
producer today. A future provider pinning something else would need its
own wording.
- The connector-mcp changeset predates this round and carries the two
declaration lines. The spec and service-automation changesets declare
`Metadata-protocol: none` (an error code and a log wording; no
author-writable key), so they ship `patch`. connector-mcp ships `minor`
for `added`.
## Verification
<details><summary>Gates — `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` at f5e02da</summary>
- 109 commands derived and run, each exit code captured before any pipe.
- 106 exited 0 on the first run. 3 exited 3 (PREREQUISITE NOT MET):
- `check-plugin-teardown-shape.mjs --self-test`: its pinned commit
621a487 was outside the shallow clone. After fetching that one commit,
the rerun exited 0 ("48 cases pass").
- spec `check:skill-examples`: needs built client and client-react.
After building that closure, the rerun exited 0 ("278 prose examples
type-check across 3 surface(s)").
- `check:dual-build-cjs-loads`: needs every workspace package's `dist`.
**NOT MEASURED locally** (a full repo build); CI runs it.
- `--ran` reconciliation: `✓ dispatch-gates --ran: 109 derived
famil(ies) accounted for — 108 run, 1 NOT-MEASURED (1 DERIVED from a
recorded exit 3).`
- `pnpm --filter @objectstack/spec check:generated`: 3 artifacts were
stale (api-surface, export-origins, docs). `--fix` regenerated only
those, and they are committed.
</details>
---
_Generated by [Claude
Code](https://claude.ai/code/session_017xRBcSf5BEfcbq4uCnjEXu)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
fix(service-automation): a connector instance whose upstream no longer matches its pin is degraded in its own words
6
+
7
+
Clause-②: no
8
+
Metadata-protocol: none
9
+
10
+
A provider factory that throws `CONNECTOR_UPSTREAM_CHANGED` (the `mcp` provider does when an instance's `providerConfig.toolsSnapshot` does not match what the server serves) degrades the instance on the same 5 s → 5 min backoff as an unreachable upstream, but its log records now say what happened. The materializer's ERROR reads "… serves a tool definition that differs from its toolsSnapshot — instance registered degraded (no actions); re-checking with backoff, attempt N." instead of "upstream unavailable … retrying with backoff", and the engine's degrade WARN says the instance has no actions "until its upstream serves the definitions its pin names again" instead of "until its upstream is reachable". The instance's reason (`GET /api/v1/automation/connectors`, the `connector_action` refusal) is the factory's message, naming each changed tool, as before.
feat(spec): `ConnectorUpstreamChangedError`, the marker for an upstream that no longer matches the instance's pin
6
+
7
+
Clause-②: yes (widening)
8
+
Metadata-protocol: none
9
+
10
+
`@objectstack/spec/integration` exports `CONNECTOR_UPSTREAM_CHANGED`, `ConnectorUpstreamChangedError` and the structural check `isConnectorUpstreamChanged`, beside the upstream-unavailable and credential-rejected markers. A connector provider factory throws it when the upstream it contacted serves a definition that differs from a pin the instance declares. Today that is the `mcp` provider's `providerConfig.toolsSnapshot`: a pinned tool whose description or `inputSchema` changed, or a pinned tool the server no longer serves. The automation materializer degrades the instance with the factory's message as its reason and re-checks it on the same backoff as an outage, so a server that restores the pinned definition recovers it. `CONNECTOR_UPSTREAM_UNAVAILABLE` keeps its meaning: the upstream could not be reached. The code is registered in the error-code ledger under `@objectstack/spec`; no HTTP door answers with it.
feat(connector-mcp): `providerConfig.toolsSnapshot` pins each tool of an `mcp` instance by name
6
+
7
+
Clause-②: yes (widening)
8
+
Metadata-protocol: added
9
+
10
+
An `mcp` instance's `include` decides which tools it exposes. Until now nothing noticed when the server rewrote one of them, for example a new description that steers an agent or an `inputSchema` that asks for more than it did. `providerConfig.toolsSnapshot` is a record of tool name → the sha256 of that tool's definition. It is keyed by tool name, like `include`, and the two compose.
11
+
12
+
-**The digest.** sha256, lowercase hex, of the UTF-8 canonical JSON of `{ name, description, inputSchema }` from the tool's `tools/list` entry. A member the server does not send is left out. Canonical JSON sorts object keys in UTF-16 code-unit order at every depth, keeps array order, writes primitives as `JSON.stringify` does and has no whitespace: the RFC 8785 (JCS) form for values read from JSON. `computeMcpToolsSnapshot(tools)` and `mcpToolDigest(tool)` are exported and are the functions the check uses.
13
+
- **The check.** On every materialization each pinned tool is hashed and compared on its own. A pinned tool served with a different description or `inputSchema`, or no longer served, degrades the instance: the provider throws `ConnectorUpstreamChangedError` (`CONNECTOR_UPSTREAM_CHANGED`), not the upstream-unavailable marker, so the logs say the pin did not match rather than that the server is down. Its reason names each such tool and the digest the server now serves, and none of the instance's tools are exposed. The instance is re-checked on the retry backoff and recovers once the server serves the pinned definitions again. An edited pin re-materializes it at once. A pinned tool that `include` also lists is reported by the pin when the server stops serving it, not as an `include` typo. When the instance's own credential reached the server (its `auth` over http), a pinned tool the server does not list takes the credential path instead (`ConnectorCredentialRejectedError`, 403, both causes named, a stored credential marked `rejected`), since the server may list only the tools that credential may use; a served pinned tool that hashes differently is still `CONNECTOR_UPSTREAM_CHANGED`.
14
+
-**Refused before connecting.** A `toolsSnapshot` that is not a record of tool name → 64 lowercase hex characters fails the boot, and so does a pin on a tool `include` leaves out, since that pin would never be checked.
15
+
- Without `toolsSnapshot`, an instance behaves exactly as before. A tool the pin does not name is not checked.
16
+
-`McpConnectorOptions.include` now also receives the tool's `tools/list` entry as a second argument. An existing one-argument predicate is unaffected.
A tool's digest is the sha256, in lowercase hex, of the UTF-8 **canonical
307
+
JSON** of `{ name, description, inputSchema }` taken from its `tools/list`
308
+
entry; a member the server does not send is left out (it is not hashed as
309
+
`null`), and no other member (`outputSchema`, annotations) is hashed.
310
+
Canonical JSON is: object members sorted by key in UTF-16 code-unit order
311
+
(JavaScript's default string sort), at every depth; array elements in their
312
+
order; strings, numbers, booleans and `null` written exactly as
313
+
`JSON.stringify` writes them; no whitespace. For values read from JSON that is
314
+
the RFC 8785 (JCS) form, so any JCS implementation reproduces it. In code,
315
+
`computeMcpToolsSnapshot(tools)` from `@objectstack/connector-mcp` builds the
316
+
record from a tool list with the same function the check uses.
317
+
318
+
On every materialization each pinned tool is hashed and compared on its own:
319
+
320
+
- Every pinned tool served with its pinned definition — the instance
321
+
materializes as usual.
322
+
- A pinned tool served with a different description or `inputSchema`, or no
323
+
longer served at all — the instance registers **degraded** and exposes none
324
+
of its tools. Its reason names each such tool, with the digest the server
325
+
now serves, and the logs say the pin did not match (the provider throws
326
+
`CONNECTOR_UPSTREAM_CHANGED`, not the unreachable-server marker). The
327
+
instance is re-checked on the retry backoff, so a server that
328
+
serves the pinned definitions again recovers it on its own; to accept the
329
+
change, review it and set the pin to the served digest (the edit
330
+
re-materializes the instance at once). This holds for a pinned tool
331
+
`include` also lists: once pinned, a tool the server stops serving is
332
+
reported by the pin, not as an `include` typo. When the instance's own
333
+
credential reached the server (its `auth` over http), a pinned tool the
334
+
server does not list may be one hidden from that credential, so it is
335
+
degraded as a credential the upstream refuses instead, naming both causes —
336
+
a served tool that hashes differently is still reported by the pin.
337
+
- A tool the pin does not name is not checked. Pin every tool you depend on,
338
+
and keep tools the server adds out with `include`.
339
+
340
+
A `toolsSnapshot` that is not a record of tool name → 64 lowercase hex
341
+
characters fails the boot before connecting, and so does a pinned tool that
342
+
`include` leaves out — the instance never exposes it, so its pin would never be
343
+
checked.
344
+
277
345
<Callouttype="warn">
278
346
**Declarative stdio transports are denied by default.** Materializing one
279
347
spawns a local process *from metadata* — which a runtime Studio publish can
@@ -794,6 +862,7 @@ retry):
794
862
| Instance `name` collides with a plugin-registered connector |**Hard boot error** — there is no silent precedence between the two worlds |
795
863
| stdio transport without host opt-in |**Hard boot error** with the `declarativeStdio` fix inline — a security rejection is never retried |
796
864
| MCP server unreachable / `tools/list` fails; remote spec URL unreachable or transiently failing (408/429/5xx) |**Not fatal**: the instance registers **degraded** (visible on `GET /api/v1/automation/connectors` with the reason), dispatches fail with that reason, and the platform retries on a 5s→5min backoff — recovery is automatic |
865
+
| An `mcp` instance's `toolsSnapshot` does not match what the server serves (a pinned tool's description or `inputSchema` changed, or a pinned tool is no longer listed) |**Not fatal**: the instance registers **degraded** with no actions, its reason naming each tool and the digest it now serves, and the logs say "serves a tool definition that differs from its toolsSnapshot" (`CONNECTOR_UPSTREAM_CHANGED`); it is re-checked on the same backoff and recovers once the server serves the pinned definitions again — or set the pin to the reviewed digest |
797
866
| A retry of a degraded instance reaches the upstream and meets a configuration fault (an `include` naming a tool or operation the upstream does not offer; on an `mcp` instance that sends a static credential, the credential row above applies instead) |**Not fatal**: the instance stays **degraded**, now with the configuration reason, and is **not retried** — dispatches fail saying to edit its declaration; an edit of the entry (or a restart) re-attempts it |
| An `openapi` operation's parameter declares a `style` OpenAPI 3.x does not define for its location (`deepObject` on a path parameter, `form` on a header) | Configuration fault — **hard boot error** naming the operation and the parameter; correct the document or leave the operation out of `include`|
Copy file name to clipboardExpand all lines: content/docs/references/api/automation-api.mdx
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -356,7 +356,7 @@ const result = AutomationApiErrorCode.parse(data);
356
356
357
357
| Property | Type | Required | Description |
358
358
| :--- | :--- | :--- | :--- |
359
-
|**code**|`Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +355 more>`| ✅ | The inner refusal's code in its WIRE spelling - the ADR-0112 member a record save of the same write answers in `error.code` (e.g. `UNIQUE_VIOLATION`); the closed set `ApiErrorSchema.code` parses against |
359
+
|**code**|`Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +356 more>`| ✅ | The inner refusal's code in its WIRE spelling - the ADR-0112 member a record save of the same write answers in `error.code` (e.g. `UNIQUE_VIOLATION`); the closed set `ApiErrorSchema.code` parses against |
360
360
|**message**|`string`| ✅ | The inner refusal's own diagnostic text, verbatim, without the node and operation prefixes the run's failure text adds |
361
361
362
362
### Allowed Values: `FlowInnerRefusal.code`
@@ -456,6 +456,7 @@ const result = AutomationApiErrorCode.parse(data);
Copy file name to clipboardExpand all lines: content/docs/references/api/contract.mdx
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ const result = ApiErrorSchema.parse(data);
28
28
29
29
| Property | Type | Required | Description |
30
30
| :--- | :--- | :--- | :--- |
31
-
|**code**|`Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +355 more>`| ✅ | Error code (e.g. VALIDATION_ERROR; StandardErrorCode ∪ the ledger the serving side registers — ERROR_CODE_LEDGER for framework packages) |
31
+
|**code**|`Enum<'VALIDATION_ERROR' \| 'INVALID_FIELD' \| 'MISSING_REQUIRED_FIELD' \| 'INVALID_FORMAT' \| 'VALUE_TOO_LONG' \| 'VALUE_TOO_SHORT' \| 'VALUE_OUT_OF_RANGE' \| … +356 more>`| ✅ | Error code (e.g. VALIDATION_ERROR; StandardErrorCode ∪ the ledger the serving side registers — ERROR_CODE_LEDGER for framework packages) |
32
32
|**declaredCode**|`string`| optional | The producer-declared code, verbatim, when it is not a member of the closed `code` vocabulary — the open, author-authored channel (app-specific spellings; ADR-0112) |
|**userMessage**|`string`| optional | Producer-marked user-facing refusal text, verbatim. Present exactly when the producer opted in at throw time; consumers render it to end users and keep their generic substitution for anything unmarked. Status-agnostic; never replaces `message`. |
@@ -135,6 +135,7 @@ const result = ApiErrorSchema.parse(data);
0 commit comments