Repository navigation
Commit 38d416f
fix(metadata-protocol,runtime): a seed load that throws part-way still reports landed ids for the datasets that finished (#30519)
domain:spec#3 · os-support-ai · session_01WGufzfgvGexKn9vLHsy89K
<!-- claude-projects-attribution: {"github_login":"os-support-ai"} -->
_Requested by **Yibo Wang** · [project
thread](https://claude.ai/code/project/chan_01BVUyyPie8gHwoPM18iABwB?thread=cmsg_01BVUyyPie8gHwoPM18iABwBJoqimjkfUcABKduhN9k5hC)_
Fixes #30429
Clause-②: yes (widening: a thrown load's receipt carries `landed` for
the datasets that finished, where it carried none)
Metadata-protocol: semantic
Skills-docs: none owed — no skill, eval or doc teaches `seedApplied`,
`landedIds` or `SeedLoadPartialWrites` (`git grep -n
"seedApplied\|landedIds\|SeedLoadPartialWrites" -- skills content/docs
':!content/docs/references' ':!content/docs/releases'`: 0 hits; control
`git grep -c "publish" -- skills/objectstack-ai/SKILL.md`: 5); the
generated `content/docs/references/api/protocol.mdx` is regenerated
here. The skills lane is paused; nothing is owed to it from this PR.
Prior rulings read: seedapplied,seedloadpartialwrites,landed → 16 hits;
ADR-0038 Decision §4, ADR-0056 D10, ADR-0058 D7, ADR-0060 D5, ADR-0061
D2, ADR-0076 D10, ADR-0090 D10, ADR-0093 D9, ADR-0094 D5; thread: none;
repo: objectstack-ai/objectstack
## 维护者速读
- 改了什么:种子加载中途抛错时,发布回执 `seedApplied` 现在也带 `landed`,列出抛错前已经跑完的数据集落下的行
id(同一个投影、同样分 `inserted`/`matched` 两张表、同样 200
上限);抛错的那个数据集不列出。运行时的兜底种子应用走同一读取与投影,失败回执也补上了 schema 要求的
`inserted`/`updated` 条数,并在 `error` 里点名停在哪个对象。
- 为什么改:#30348 之后,加载一旦中途失败,回执只有条数没有 id,cloud
的「清理示例数据」拿不到前面已落库数据集的行,这些示例行就清不掉。
- 风险与回滚:只在失败回执上多给一个已有的可选字段;成功路径不变。`landed`
缺席的含义收窄为"没有运行加载",抛错前一个数据集都没跑完时给 `[]`。回滚即还原本 PR。
- 先看:`packages/metadata-protocol/src/seed-loader.ts` 的
`SeedLoadPartialWrites.finished` 与
`notePartialWrites`;`packages/metadata-protocol/src/protocol-publish-batch.ts`
的 catch;`packages/runtime/src/domains/packages.ts` 兜底 catch。
- 两分钟验证:`pnpm --filter @objectstack/objectql exec vitest run
src/publish-seed-receipt-landed-ids.test.ts`,预期 10 例全绿,其中"load that
threw"一例的 `landed` 只含先跑完的对象。
- 需要你做的:无,席位落地。
## What changes
- **`@objectstack/metadata-protocol` (`seed-loader.ts`):
`SeedLoadPartialWrites.finished`.** `notePartialWrites` snapshots the
finished datasets' `{ object, landedIds, insertedIds }`, in load order,
off the results it already keeps (`loadProgress.finished`). Nothing new
is tracked; the ids are the ones each finished dataset's returned result
carries. Empty when the throw came before any dataset finished.
- **`@objectstack/metadata-protocol` (`protocol-publish-batch.ts`):
`applySeedBodies`' catch** adds `landed:
seedAppliedLanded(written.finished)` whenever a partial-writes reading
exists. Same projection as the returned path (`seedAppliedLanded`), so
the same two lists, the same per-object fold and the same 200 cap. No
reading (the request parse refused, or the loader module never loaded)
leaves `landed` absent.
- **`@objectstack/metadata-protocol` (`index.ts`)**:
`seedLoadPartialWrites` and `SeedLoadPartialWrites` are exported from
the package root, so the runtime fallback reads the same record.
- **`@objectstack/runtime` (`domains/packages.ts`)**: the route-level
fallback seed apply's catch reads `seedLoadPartialWrites(e)` and reports
the same `landed`. It had the same thrown path (`applyPublishedSeeds` →
`loader.load` throws → catch with `success: false`, `error`). It now
also reports `inserted` / `updated` from the same reading (the response
schema requires both; this catch used to send neither) and names the
object whose seed stopped the load in `error` (`seed apply stopped at
the seed for object 'X': …`), exactly as `applySeedBodies`' catch does;
the caught sentence itself is still withheld unless declared. No
reading: zero counts, the bare sentence, no `landed`.
- **`@objectstack/spec` (`api/protocol.zod.ts`)**: the shared
`seedApplied.landed` description (both publish responses) and the
`PublishMetaItemResponseSchema` docblock now say what `landed` covers on
a thrown load (the datasets that finished, empty when none had; not the
one that threw) and when it is still absent (no load ran, or a producer
that reports no ids). Schema shape unchanged.
`content/docs/references/api/protocol.mdx` regenerated.
- `protocol-publish-batch.ts`: the older comment above the catch's
counters now agrees with the new one beside `landed` (no reading means
no load ran; a reading with zero counts is a load that threw before
writing a row).
- `.changeset/30429-seed-receipt-landed-on-throw.md`: spec minor,
metadata-protocol minor, runtime patch, with a "Supersedes, in the same
release" paragraph for the sentence in the unreleased
`.changeset/30348-seed-receipt-landed-ids.md` that this makes false.
### The dataset that threw: omitted, and why
Measured at `seed-loader.ts`: the dataset that throws can have landed
rows before the throw. Its batched inserts flush every `insertBatchSize`
records (`flushPendingInserts`), so earlier batches are written, and a
throw can come after a row was written (the seed-file settle, pinned in
`seed-loader-seed-files.test.ts`: `pw_doc`'s D1 is in the store when the
storage service throws). Those rows are in `inserted` / `updated` and
the object is named in `error`, but they are **not listed** in `landed`:
- Its ids are only partly known when it throws. A batched insert whose
response was lost may have committed rows whose ids never came back; the
loader re-reads by key on its retry for exactly that reason. A list of
the ids it did see would read as complete while missing rows, which is
the one thing a reader of the ids cannot check.
- A finished dataset's list is complete, so "the datasets that finished"
is a line a reader can rely on. This is the card's default.
A reader that wants to know whether unlisted rows exist compares
`inserted` with the listed `inserted.total`s; `error` names the object.
### What `landed` answers now
| Receipt | `landed` |
| --- | --- |
| load returned | every dataset (unchanged) |
| load threw after some datasets finished | those datasets only |
| load threw in pass 2 (every dataset finished) | every dataset, no
failed object |
| load threw before any dataset finished | `[]` (the loader ran; nothing
finished) |
| no load ran (seed bodies refused at parse, no readable bodies) |
absent |
## Why
#30348 (PR #30427) put the ids on the receipt, but only on the
returned-load path; its contract review named the gap at
`applySeedBodies`' catch. After a partial publish cloud's "clear sample
data" had no ids for the datasets that did land, so those sample rows
could never be cleared by id.
## Verification
Head `4c02bddf8`, under `os-verify-lock`,
`NODE_OPTIONS=--max-old-space-size=4096`:
- `pnpm --filter @objectstack/metadata-protocol test`: Test Files 316
passed, 4 skipped (320), Tests 22425 passed, 26 skipped. `typecheck`
exit 0.
- `pnpm --filter @objectstack/objectql test`: Test Files 551 passed
(551), Tests 10989 passed. `typecheck` exit 0.
- `pnpm --filter @objectstack/runtime test`: Test Files 543 passed
(543), Tests 8647 passed, 19 skipped. `typecheck` exit 0.
- `pnpm --filter @objectstack/spec test`: Test Files 815 passed (815),
Tests 25622 passed, 1 todo. `typecheck` exit 0.
- Pins:
- `packages/objectql/src/publish-seed-receipt-landed-ids.test.ts` (real
protocol, real ObjectQL engine, real loader over an in-memory driver;
ids read back from the engine). The old pin "a load that threw reports
no ids" is replaced by: a good seed (with a customer's pre-existing row
its key matches) lands, a later seed naming a missing object throws; the
receipt lists the good seed's created id under `inserted` and the
customer's row under `matched`, has no entry for the object that threw,
and `PublishPackageDraftsResponseSchema` keeps it. Plus: a load that
throws before any seed finished reports `landed: []`, not absent.
-
`packages/runtime/src/domains/packages-seed-apply-landed-thrown.test.ts`
(new, the fallback door with the real `SeedLoaderService`): positive
control on a returned load; a second dataset's existing-row read throws
after the first dataset wrote, and the receipt lists the first, reports
`inserted: 1`, `updated: 0`, and `error` names the stopped object with
the driver sentence withheld; a failure before any load ran leaves
`landed` absent.
- `packages/metadata-protocol/src/seed-loader-deferred-failure.test.ts`
(+1): a throw in pass 2, after both datasets of a circular pair wrote:
the reading has no `failedObject`, `finished` lists both datasets with
their store ids, and `seedAppliedLanded(finished)` names every row. Pass
2 guards its own driver traffic (a failed back-fill write is counted, a
failed lookup probe is a miss; both measured), so the throw is staged at
its one unguarded call, the `logger.error` its back-fill failure reports
through.
- `packages/metadata-protocol/src/seed-loader-seed-files.test.ts`:
`seedLoadPartialWrites` now carries `finished` with the finished
dataset's store ids, and leaves out the dataset that threw although its
row D1 was written; a throw before any dataset reads `finished: []`.
<details><summary>Reverse verification (two mutations, each restored by
scripts/ablation-replace.mjs to the HEAD blob)</summary>
The objectql and runtime pins resolve `@objectstack/metadata-protocol`
from `dist`, so each mutation was rebuilt and
`ablation-dist-preflight.mjs` confirmed the marker in 2 built files
before reading a result. Directions predicted before running.
1. The loader carries no finished ids (`finished: finished.map(...)` →
`finished: (globalThis as any).ABL30429A ?? []`, blob `a7986c958ece` →
`c3a35b1b3aa0`): objectql landed pins **1 failed | 9 passed** (the
thrown-after-finish case; the before-any-finished `[]` case holds either
way), runtime fallback **1 failed | 2 passed**, `seedLoadPartialWrites`
unit **1 failed | 2 passed**. Restored: blob == HEAD.
2. The catch drops `landed` (`...(written ? { landed: ... } : {})` →
`...((globalThis as any).ABL30429B ?? {})`, blob `eccb3c495621` →
`cb59f9935201`): objectql landed pins **2 failed | 8 passed** (both
thrown cases). Restored: blob == HEAD.
3. Round 1, the runtime catch's counts (`inserted: written?.inserted ??
0,` → `inserted: 0,` in `packages.ts`, blob `39d41e8df5cb` →
`8a37f8d6e007`; runtime resolves its own `src`):
`packages-seed-apply-landed-thrown.test.ts` **1 failed | 2 passed** (the
thrown case). Restored: blob == HEAD, `git diff HEAD` empty.
Restore leg for 1 and 2: rebuilt, `--absent` preflight green for both
markers, `git diff HEAD` empty, the three files back to 10/10, 3/3, 3/3.
A first attempt planted the markers as comments, which the build strips;
the preflight refused it (marker absent from dist) before any test ran,
and it is discarded.
</details>
<details><summary>Gates</summary>
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--ran gates-ran.list`: "105 derived famil(ies) accounted for — 105 run,
0 NOT-MEASURED (a DERIVED zero — all 105 recorded an exit code and none
of them is 3)", exit 0. Every line `exit 0`.
- The serial pass ran at `be4d35b5` (the implementation commit);
`4c02bddf8` adds only the changeset's "Supersedes" paragraph, and the
changeset gates (`check-empty-changeset`, `check-changeset-no-major`)
plus `check:generated` and `check:nul-bytes` were re-run on `4c02bddf8`,
exit 0.
- Three families exited 3 (prerequisite) on the first pass and were
re-run green: `check-plugin-teardown-shape --self-test` (its pinned
fixture commit fetched), `check:skill-examples` and
`check:dual-build-cjs-loads` (after a full `turbo run build`, 119/119).
- `Prior rulings read` is from `node scripts/pm/check-prior-rulings.mjs
--card 30429 --terms seedApplied,SeedLoadPartialWrites,landed`. ADR-0038
Decision §4 (L3 probes attach to the publish response "like
`seedApplied`") does not rule on the receipt's ids; the other hits match
the word "landed" in unrelated decisions. The card's thread holds no
ruling.
- `check-changeset-no-major` with this body as the event: exit 0 on
`Metadata-protocol: semantic` (spec and metadata-protocol `minor`).
Controls: `none` exits 1 (the `minor` entries are unaccounted for);
`added` also exits 0, so the gate checks the level, not the kind. The
kind is `semantic` by reading: no schema key is added or removed (the
schema shape is byte-identical; only the description moves), and an
existing key's absence changes meaning on a thrown load, from "not
reported" to "no load ran".
- No `!` in the title and no `(narrowing)` arm: no full dogfood run. CI
owns the rest of the farm.
</details>
## Acceptance notes
- The seed-ownership claim (`claim-seed-ownership.ts`,
`seed-settlement.ts`) still reads `landedIds`: out of scope per the
card.
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic
(description text only).
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes (CI-owned run).
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: its generated artifacts are
regenerated and committed.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the 维护者速读 block
excepted, per project convention).
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01WGufzht5n1PKh96TqTfrvV
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent e8e924f commit 38d416f
11 files changed
Lines changed: 314 additions & 25 deletions
File tree
- .changeset
- content/docs/references/api
- packages
- metadata-protocol/src
- objectql/src
- runtime/src/domains
- spec/src/api
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2546 | 2546 | | |
2547 | 2547 | | |
2548 | 2548 | | |
2549 | | - | |
| 2549 | + | |
2550 | 2550 | | |
2551 | 2551 | | |
2552 | 2552 | | |
| |||
2647 | 2647 | | |
2648 | 2648 | | |
2649 | 2649 | | |
2650 | | - | |
| 2650 | + | |
2651 | 2651 | | |
2652 | 2652 | | |
2653 | 2653 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
281 | 281 | | |
282 | 282 | | |
283 | 283 | | |
284 | | - | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
285 | 292 | | |
286 | 293 | | |
287 | 294 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
134 | 134 | | |
135 | 135 | | |
136 | 136 | | |
137 | | - | |
138 | | - | |
139 | | - | |
| 137 | + | |
| 138 | + | |
140 | 139 | | |
141 | 140 | | |
142 | 141 | | |
| |||
156 | 155 | | |
157 | 156 | | |
158 | 157 | | |
159 | | - | |
160 | | - | |
161 | | - | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
162 | 163 | | |
163 | 164 | | |
164 | 165 | | |
| |||
168 | 169 | | |
169 | 170 | | |
170 | 171 | | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
171 | 181 | | |
172 | 182 | | |
173 | 183 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
300 | 301 | | |
301 | 302 | | |
302 | 303 | | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
558 | 558 | | |
559 | 559 | | |
560 | 560 | | |
561 | | - | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
562 | 570 | | |
563 | 571 | | |
564 | 572 | | |
| |||
569 | 577 | | |
570 | 578 | | |
571 | 579 | | |
572 | | - | |
| 580 | + | |
573 | 581 | | |
574 | 582 | | |
575 | 583 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
581 | 582 | | |
582 | 583 | | |
583 | 584 | | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
584 | 600 | | |
585 | 601 | | |
586 | 602 | | |
| |||
815 | 831 | | |
816 | 832 | | |
817 | 833 | | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
818 | 837 | | |
819 | 838 | | |
820 | 839 | | |
| |||
0 commit comments