Skip to content

Commit a81b7cb

Browse files
fix(metadata): a packages[] stack's top-level dataset no body declares registers under the stack's id (#29593)
<!-- claude-projects-attribution: {"github_login":"os-warren"} --> _Requested by **Warren** · [project thread](https://claude.ai/code/project/chan_01U5RdnAAmkSQmZbqh6FtqZ8?thread=cmsg_01U5RdnAAmkSQmZbqh6FtqZ85xEThZCGWjgxwVAkAhF4sq)_ Fixes #29518 Clause-②: no (narrowing) Metadata-protocol: narrowed Skills-docs: none — no file under `skills/**` teaches a `packages[]` stack's top level, its residual, or a `_packageId` stamp on a top-level dataset; every taught dataset sits in a single-package stack, which has no residual. ## 维护者速读 - 改了什么:多包 stack(带 `packages[]`)顶层声明、但没有任何包体声明的 `datasets` 条目,现在按 ADR-0130 D4 的残余规则登记在 stack 自己的 `manifest.id` 下;构建门(spec 镜像)与启动门同步判断它的 `_packageId` 印记。 - 为什么改:之前这种数据集既不登记也不报错,被静默丢弃;引用它的仪表盘或报表要到运行时才失败,离原因很远。 - 风险与回滚:顶层带外包 `_packageId` 印记的残余数据集现在会被拒(之前静默丢弃),故声明 narrowed;另外 artifact 启动时各包体的数据集也会进入元数据服务(封闭运行时 `GET /meta/dataset` 不再为空)。回滚即还原本 PR。 - 先看:`packages/metadata/src/plugin.ts`(`ARTIFACT_FIELD_TO_TYPE` 新增 `datasets`)、`packages/spec/src/stack.zod.ts`(`TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL` 去掉 `datasets`)、新 pin `manifest-service-residual-objects.test.ts` 的 datasets 块。 - 两分钟验证:`pnpm --filter @objectstack/objectql exec vitest run src/manifest-service-residual-objects.test.ts -t datasets`,预期 2 个用例全绿(构建依赖闭包后)。 - 需要你做的:无,席位契约复核后落地。 ## What changes - `packages/metadata/src/plugin.ts`: `ARTIFACT_FIELD_TO_TYPE` gains `datasets: 'dataset'`, with a comment saying why and that the seed `data:` collection stays unmapped. That one map is what the residual rule (`unclaimedTopLevel`) walks, so a `packages[]` stack's top-level dataset that no body declares is now listed as residual. Both boot doors register it under the stack's manifest id and version (the metadata door through `registerUnclaimedTopLevel`, the engine through the `manifest` service's existing residual seam, unchanged), and both apply the owner-stamp refusal they already apply to every other residual item. - The same map is the artifact door's body loop, so every package body's datasets also reach the metadata service from a compiled artifact, with provenance. Before, only the ObjectQL registry took them (its `METADATA_ARRAY_KEYS` loop). - `packages/spec/src/stack.zod.ts`: the build-door mirror `TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL` drops `datasets` and keeps `data` and `jobs`; its docblock says why. Not exported, so no API-surface or generated artifact moves. Triage designated this a cross-domain single PR (6585309156); PR #29514 had merged first. - `packages/spec/src/stack-collection-item-owner-stamp.test.ts`: the "filed by no boot door" case drops `datasets`; a new case pins that a foreign-stamped residual dataset raises the judge's issue at `datasets.0._packageId`. - `packages/metadata/src/owner-stamp-residual-build-boot.test.ts`: the reading becomes `['data', 'jobs']`, and the comment above it no longer says datasets are read off the stack whole (the rider from the contract review of PR #29514, record 6583699079). - `scripts/check-stack-collection-maps.mjs`: the `ARTIFACT_FIELD_TO_TYPE` waiver row loses `datasets` from its keys (the gate reds a stale waiver) and its text no longer says AppPlugin consumes datasets off the bundle; it says why the key left. - New pin block in `packages/objectql/src/manifest-service-residual-objects.test.ts`: a residual dataset registers under the stack's manifest id and version beside a body's own; CONTROL: a top-level dataset a body repeats stays the body's, registered once. - One changeset: `minor` for `@objectstack/spec` and `@objectstack/metadata` (the kind is `narrowed`), carrying the three declaration lines and the fix for an author whose stack now refuses. ## Why ADR-0130 D4's residual rule exists so a top-level item is either registered or refused, never silently lost. `datasets` had no `ARTIFACT_FIELD_TO_TYPE` entry, so the rule never saw it. Measured on a kernel boot (`ObjectKernel` + `ObjectQLPlugin`, the stack handed to the `manifest` service as `AppPlugin` hands it): nothing thrown, `listItems('dataset')` answered the body's dataset only, and no line named the residual one. ## Tests - Red first, at the commit that adds the pin (before the fix): `registers a residual dataset under the stack's manifest id and version, beside a body's own` failed (`expected [ [ 'acme_case_body', …(2) ] ] to deeply equal [ [ 'acme_case_body', …(2) ], …(1) ]`); the CONTROL passed. With the mapping: 62/62 in that file. - Reverse verification of the spec pin, at the final head: putting `datasets` back into `TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL` turns the new spec case red (1 failed, 47 passed); restored with `git checkout HEAD --`, `git diff HEAD` empty. - Before the spec line, the metadata mirror test was red exactly as predicted (`datasets: build verdict == unclaimedTopLevel lists it (true): expected false to be true`); with it, 27/27. - Suites: at head `ae0a25e6` (the last commit adds only the changeset's ADR-0087 marker), each under the verify lock: `@objectstack/spec` 888 files / 25184 tests passed (1 todo); `@objectstack/metadata` 68 / 956 passed; `@objectstack/objectql` `manifest-service-residual-objects.test.ts` 62/62. The full `@objectstack/objectql` suite (535 files / 10398 tests) and `@objectstack/runtime` suite (527 files / 9695 passed, 19 skipped) passed with the mapping before the rebase onto `ff827bcf`; the spec line changes no runtime path they exercise. `typecheck` exit 0 for spec, metadata and objectql. ## Acceptance notes - Artifact boot measured before and after (scratch probe, deleted). `createStandaloneStack` over a multi-package artifact with a residual dataset: `getMetaItems({ type: 'dataset' })` went from the body's dataset only to body + residual (owner `com.probe.release`), and `getMetaItem` for the residual went from a stub with no item to the served item. Single-package artifact: `/meta/dataset` unchanged; the metadata service's own list went from empty to the body dataset. The artifact door alone (as a sealed runtime runs it, no AppPlugin): `manager.list('dataset')` went from empty to every body's dataset plus the residual, each stamped with its package id and version, and the residual `warn` counts the dataset with the other residual items. No boot failed in any reading. - `jobs`, `datasources` and `translations` stay on the waiver row: they are a different fact (AppPlugin does consume them off the bundle) and were not measured here. ## Verification All readings at head `ae0a25e6`, merge base `ff827bcf`. <details><summary>Derived gates: 101 derived, 100 exit 0, 1 NOT MEASURED; plus pnpm lint</summary> - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 101 commands from the 7 changed paths; every one was run and its exit code recorded, and `--ran` reconciled the record: `101 derived famil(ies) accounted for — 100 run, 1 NOT-MEASURED`. - First pass: `check-adr-0087-registration` exited 1 (the narrowed changeset had no ADR-0087 marker); the marker commit makes it exit 0 (`not-required (no-migration-prescription)`), and every changeset-reading gate was rerun on that head (no-major, empty-changeset, changeset-gate-self-tests, doc-authoring, issue-citations, nul-bytes, closing-keyword-parity, stack-collection-maps): all exit 0. `check-plugin-teardown-shape --self-test` exited 3 until its pinned control commit was fetched, then 0. - NOT MEASURED, reason: prerequisite not met (exit 3). `pnpm check:dual-build-cjs-loads` needs every package built; CI's Build Core runs it. - `pnpm lint`: exit 0. `node scripts/check-stack-collection-maps.mjs --self-test`: exit 0 (18 assertions). </details> `test:repo` was not run locally (it belongs to CI's Test Core). Dogfood battery at `dd552104` (owed by the `(narrowing)` arm): closure built, `pnpm --filter @objectstack/dogfood test` under the verify lock, VERDICT command-exit 0; Test Files 397 passed | 3 skipped (400), Tests 3277 passed | 28 skipped (3305), no failures. --- _Generated by [Claude Code](https://claude.ai/code/session_01JQwxbaRjbDqdsjbNUjKQJZ)_ <!-- signed: domain:server#4 · os-warren · session_01JQwxbacSXJQqAztgt4UMrS --> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent dd4a21f commit a81b7cb

7 files changed

Lines changed: 100 additions & 22 deletions
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/metadata': minor
3+
'@objectstack/spec': minor
4+
---
5+
6+
fix(metadata): a `packages[]` stack's top-level dataset that no package body declares is registered under the stack's own manifest id, like every other residual item
7+
8+
Clause-②: no (narrowing)
9+
Metadata-protocol: narrowed
10+
Skills-docs: none — no file under `skills/**` teaches a `packages[]` stack's top level, its residual, or a `_packageId` stamp on a top-level dataset; every taught dataset sits in a single-package stack, which has no residual, so every taught use stays true.
11+
<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes. The narrowed face is the existing owner-stamp judge now reaching one more collection of a `packages[]` stack's top level: a residual dataset wearing another package's `_packageId`. No conversion can rewrite it, because which package owns the dataset is the author's decision (drop the stamp, or move the dataset into the `packages[]` entry of its owner), and the stack that carries it used to lose the dataset at boot anyway, so there is no working stored state for a migration to preserve. -->
12+
13+
A stack that carries `packages[]` registers each package body under its own id, and what its top level carries that no body declares (the residual, ADR-0130 D4) under the stack's `manifest.id`. The residual rule did not walk `datasets` (the ADR-0021 analytics datasets a dashboard or report names), so such a dataset was neither registered nor refused: the boot succeeded, `GET /meta/dataset` listed only the bodies' datasets, and a dashboard or report naming the missing one failed at run time, far from the cause.
14+
15+
- **What changes.** `datasets` now maps to the `dataset` metadata type at the artifact door. A residual dataset is registered under the stack's manifest id and version through both boot doors, in the metadata service and in the ObjectQL registry, and the boot's residual `warn` line counts it with the other residual items. A top-level dataset that a body repeats stays the body's, registered once.
16+
- **What is refused now (narrowed).** A residual dataset wearing a `_packageId` other than the stack's own `manifest.id` is refused, at the build door (`ObjectStackDefinitionSchema`, so `defineStack` / `os build` / `os validate`, with an issue at `datasets.N._packageId`) and at the boot (`VALIDATION_ERROR` 400, the owner-stamp refusal both residual registrars already apply to every other residual item). It used to parse and then be dropped at boot with nothing said. Fix: drop the stamp, or declare the dataset inside the `packages[]` entry of the package that owns it. The build-door mirror `TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL` (`@objectstack/spec`, not exported) loses `datasets` and keeps `data` and `jobs`.
17+
- **What else the artifact door now registers.** Every package body's datasets also reach the metadata service from a compiled artifact, stamped with their package's provenance. A sealed (`bootstrap: 'artifact-only'`) runtime used to answer an empty `GET /meta/dataset` for a package that ships datasets; it now lists them. The seed `data:` collection is unchanged and still never registered as metadata items.

‎packages/metadata/src/owner-stamp-residual-build-boot.test.ts‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -172,9 +172,12 @@ describe('[ADR-0010 · ADR-0130 D4] a residual\'s foreign stamp: refused at the
172172
expect(await bootRefusesStamp(stack), `${key}: the metadata door refuses what it lists`).toBe(listed);
173173
if (!listed) notResidual.push(key);
174174
}
175-
// The reading, so the equality above is not vacuous: the three
176-
// collections the boot reads off the stack whole, never as items.
177-
expect(notResidual.sort()).toEqual(['data', 'datasets', 'jobs']);
175+
// The reading, so the equality above is not vacuous: the two judged
176+
// collections the residual rule does not walk. Seed `data` is applied
177+
// by the seed loader and `jobs` are scheduled, both straight off the
178+
// stack, never registered as items. `datasets` left this list when the
179+
// rule began registering a residual dataset under the stack's id.
180+
expect(notResidual.sort()).toEqual(['data', 'jobs']);
178181
});
179182
});
180183

‎packages/metadata/src/plugin.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,13 @@ const ARTIFACT_FIELD_TO_TYPE: Record<string, string> = {
135135
pages: 'page',
136136
dashboards: 'dashboard',
137137
reports: 'report',
138+
// [ADR-0021, ADR-0130 D4] The analytics datasets a dashboard or report names.
139+
// Mapped so the residual rule walks them too: without this entry a
140+
// `packages[]` stack's top-level dataset that no body declared was neither
141+
// registered nor refused, and the dashboard naming it failed at run time.
142+
// Not the seed `data:` collection, which stays unmapped (see the end of
143+
// this map).
144+
datasets: 'dataset',
138145
actions: 'action',
139146
// `themes: 'theme'` removed by commit 35ad101bc (ADR-0049): the carrier key is
140147
// retired, so a parsed artifact can no longer carry the field — and the

‎packages/objectql/src/manifest-service-residual-objects.test.ts‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -388,6 +388,48 @@ describe('ADR-0130 D4 — the manifest service registers the residual\'s flows',
388388
});
389389
});
390390

391+
/**
392+
* [ADR-0130 D4] A residual DATASET (the ADR-0021 analytics kind, `datasets:`)
393+
* reaches the registry under the stack's manifest id and version, as a body's
394+
* does under its package's. Before, the residual rule did not walk `datasets`
395+
* at all, so a top-level dataset no body declared was neither registered nor
396+
* refused: the boot booted, `listItems('dataset')` answered the bodies' own,
397+
* and a dashboard or report naming it failed at run time, far from the cause.
398+
*/
399+
describe('ADR-0130 D4 — the manifest service registers the residual\'s datasets', () => {
400+
const dataset = (name = 'acme_case_metrics') => ({
401+
name,
402+
label: 'Case metrics',
403+
object: 'acme_case',
404+
dimensions: [{ name: 'subject', field: 'subject', type: 'string' }],
405+
measures: [{ name: 'case_count', aggregate: 'count' }],
406+
});
407+
const datasetOwners = (ql: ObjectQL) =>
408+
(ql.registry.listItems('dataset') as Array<{ name?: string; _packageId?: string; _packageVersion?: string }>)
409+
.map((d) => [d.name, d._packageId, d._packageVersion])
410+
.sort((a, b) => String(a[0]).localeCompare(String(b[0])));
411+
412+
it('registers a residual dataset under the stack\'s manifest id and version, beside a body\'s own', async () => {
413+
const residualEntry = dataset();
414+
const { kernel } = await boot(registering('app', [
415+
residualStack(noteObj(), { datasets: [residualEntry] }, { datasets: [dataset('acme_case_body')] }),
416+
]));
417+
expect(datasetOwners(engineOf(kernel))).toEqual([
418+
['acme_case_body', SERVICE_ID, '2.4.0'],
419+
['acme_case_metrics', RELEASE_ID, '1.0.0'],
420+
]);
421+
// The artifact's own entry is left as it arrived.
422+
expect((residualEntry as { _packageId?: string })._packageId).toBeUndefined();
423+
});
424+
425+
it('CONTROL: a top-level dataset a body repeats is the body\'s, registered once', async () => {
426+
const { kernel } = await boot(registering('app', [
427+
residualStack(noteObj(), { datasets: [dataset()] }, { datasets: [dataset()] }),
428+
]));
429+
expect(datasetOwners(engineOf(kernel))).toEqual([['acme_case_metrics', SERVICE_ID, '2.4.0']]);
430+
});
431+
});
432+
391433
/**
392434
* [ADR-0130 D4] EVERY other residual type a package body's collections take
393435
* into the registry reaches it too, under the stack's manifest id and version:

‎packages/spec/src/stack-collection-item-owner-stamp.test.ts‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@ describe('[ADR-0010 · ADR-0130 D4] a `packages[]` stack\'s residual is judged o
301301
defineStack({ manifest: manifest(ORDERS, { type: 'module' }) } as never),
302302
], { manifest: 'preserve' }) as unknown as Record<string, unknown> & { manifest: { id: string } };
303303

304-
it.each([['data'], ['datasets'], ['jobs']] as const)(
304+
it.each([['data'], ['jobs']] as const)(
305305
'a top-level `%s` item no body declares is filed by no boot door as a residual, so its stamp is not judged at the build door either',
306306
(key) => {
307307
const { item, context } = JUDGED_FIXTURES[key];
@@ -311,6 +311,12 @@ describe('[ADR-0010 · ADR-0130 D4] a `packages[]` stack\'s residual is judged o
311311
},
312312
);
313313

314+
it('a residual dataset IS filed under the artifact\'s id (the residual rule walks `datasets`), so its stamp is judged', () => {
315+
const { item } = JUDGED_FIXTURES.datasets;
316+
expect(stampIssues({ ...artifactOf(), datasets: [{ ...item, _packageId: OTHER }] }).map(([path]) => path))
317+
.toEqual(['datasets.0._packageId']);
318+
});
319+
314320
it('CONTROL: a residual picklist IS filed under the artifact\'s id (the metadata door registers it), so its stamp is still judged', () => {
315321
const { item } = JUDGED_FIXTURES.picklists;
316322
expect(stampIssues({ ...artifactOf(), picklists: [{ ...item, _packageId: OTHER }] }).map(([path]) => path))

‎packages/spec/src/stack.zod.ts‎

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -361,22 +361,20 @@ function applyCollectionItemOwnerStampGate(
361361
* The judged collections a `packages[]` stack's TOP LEVEL never files as
362362
* residual items: the boot's residual rule (`unclaimedTopLevel`,
363363
* `@objectstack/metadata`) walks the collections the metadata door registers
364-
* as items (its `ARTIFACT_FIELD_TO_TYPE`), and these three are not among them,
365-
* so the boot judges no stamp on a top-level item of any of them. Their readers
366-
* differ:
367-
* - seed `data` and `jobs` are read off the stack whole, the top level merged
368-
* with every body (`resolveArtifactCollections`, `@objectstack/runtime`):
369-
* `AppPlugin` hands the seed `data` to the seed loader and schedules the
370-
* `jobs`;
371-
* - `datasets` are read off the package BODIES only, by the engine's
372-
* `registerApp` (its `METADATA_ARRAY_KEYS` loop, `@objectstack/objectql`).
373-
* No boot reader takes a top-level `datasets` item that no body declares.
364+
* as items (its `ARTIFACT_FIELD_TO_TYPE`), and these two are not among them,
365+
* so the boot judges no stamp on a top-level item of either. Both are read off
366+
* the stack whole, the top level merged with every body
367+
* (`resolveArtifactCollections`, `@objectstack/runtime`): `AppPlugin` hands the
368+
* seed `data` to the seed loader and schedules the `jobs`.
369+
* `datasets` is not here: the residual rule walks it, so a top-level dataset
370+
* that no body declares is registered under `manifest.id` and its stamp is
371+
* judged at both doors, like every other residual item.
374372
* A package BODY's items of these collections are still judged, at both doors.
375373
*
376374
* Mirrored, because `@objectstack/spec` cannot import the rule it mirrors; a
377375
* `@objectstack/metadata` test holds the two equal, collection by collection.
378376
*/
379-
const TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL: ReadonlySet<string> = new Set(['data', 'datasets', 'jobs']);
377+
const TOP_LEVEL_COLLECTIONS_NEVER_RESIDUAL: ReadonlySet<string> = new Set(['data', 'jobs']);
380378

381379
/**
382380
* The registry slot one item occupies, the way the boot's residual rule

‎scripts/check-stack-collection-maps.mjs‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -661,17 +661,22 @@ const SITES = [
661661
},
662662
{
663663
direction: 'missing',
664-
keys: ['datasets', 'jobs', 'datasources', 'translations'],
664+
keys: ['jobs', 'datasources', 'translations'],
665665
reason:
666-
'DRIFT with a real, bounded consequence — #6242 row 4(b). All four are consumed '
666+
'DRIFT with a real, bounded consequence — #6242 row 4(b). All three are consumed '
667667
+ 'functionally by AppPlugin straight off the bundle, so boot is not broken; what they never do is '
668668
+ 'register as METADATA ITEMS, so under `bootstrap: \'artifact-only\'` (edge / serverless / '
669-
+ 'immutable image) `GET /meta/job`, `/meta/translation`, `/meta/datasource` and `/meta/dataset` '
670-
+ 'answer empty for a package that ships them. Adding them changes what a sealed runtime serves '
671-
+ 'and must be measured on a real artifact-only boot first — the filing card says so, and this '
672-
+ 'gate does not smuggle it in. `capabilities` was the FIFTH key on this row and left it at '
669+
+ 'immutable image) `GET /meta/job`, `/meta/translation` and `/meta/datasource` answer empty for '
670+
+ 'a package that ships them. Adding them changes what a sealed runtime serves and must be '
671+
+ 'measured on a real artifact-only boot first — the filing card says so, and this gate does not '
672+
+ 'smuggle it in. `datasets` left this row when the residual rule began walking it: it was never '
673+
+ 'the same fact, since AppPlugin reads no dataset off the bundle (the ObjectQL registry takes a '
674+
+ 'package body\'s datasets through its own collection loop), and with no entry here a '
675+
+ '`packages[]` stack\'s top-level dataset that no body declared was neither registered nor '
676+
+ 'refused. Measured on an artifact boot before the entry landed. `capabilities` was the FIFTH key '
677+
+ 'on this row and left it at '
673678
+ '#12892 step 1 (maintainer ruling, option 1: the door owns the registration route for the five '
674-
+ 'security collections). It was never the same fact as the other four: AppPlugin registers it as '
679+
+ 'security collections). It was never the same fact as the other three: AppPlugin registers it as '
675680
+ 'a METADATA ITEM through `registerInMemory`, so `GET /meta/capability` was already non-empty on '
676681
+ 'an artifact-only boot — what was missing from that answer was the strict-parsed shape, the '
677682
+ '`scope` default and the ADR-0010 provenance stamp. Driven on a real artifact-only boot before '

0 commit comments

Comments
 (0)