Skip to content

Commit 38d416f

Browse files
fix(metadata-protocol,runtime): a seed load that throws part-way still reports landed ids for the datasets that finished (#30519)
domain:spec#3 · os-support-ai · session_01WGufzfgvGexKn9vLHsy89K <!-- claude-projects-attribution: {"github_login":"os-support-ai"} --> _Requested by **Yibo Wang** · [project thread](https://claude.ai/code/project/chan_01BVUyyPie8gHwoPM18iABwB?thread=cmsg_01BVUyyPie8gHwoPM18iABwBJoqimjkfUcABKduhN9k5hC)_ Fixes #30429 Clause-②: yes (widening: a thrown load's receipt carries `landed` for the datasets that finished, where it carried none) Metadata-protocol: semantic Skills-docs: none owed — no skill, eval or doc teaches `seedApplied`, `landedIds` or `SeedLoadPartialWrites` (`git grep -n "seedApplied\|landedIds\|SeedLoadPartialWrites" -- skills content/docs ':!content/docs/references' ':!content/docs/releases'`: 0 hits; control `git grep -c "publish" -- skills/objectstack-ai/SKILL.md`: 5); the generated `content/docs/references/api/protocol.mdx` is regenerated here. The skills lane is paused; nothing is owed to it from this PR. Prior rulings read: seedapplied,seedloadpartialwrites,landed → 16 hits; ADR-0038 Decision §4, ADR-0056 D10, ADR-0058 D7, ADR-0060 D5, ADR-0061 D2, ADR-0076 D10, ADR-0090 D10, ADR-0093 D9, ADR-0094 D5; thread: none; repo: objectstack-ai/objectstack ## 维护者速读 - 改了什么:种子加载中途抛错时,发布回执 `seedApplied` 现在也带 `landed`,列出抛错前已经跑完的数据集落下的行 id(同一个投影、同样分 `inserted`/`matched` 两张表、同样 200 上限);抛错的那个数据集不列出。运行时的兜底种子应用走同一读取与投影,失败回执也补上了 schema 要求的 `inserted`/`updated` 条数,并在 `error` 里点名停在哪个对象。 - 为什么改:#30348 之后,加载一旦中途失败,回执只有条数没有 id,cloud 的「清理示例数据」拿不到前面已落库数据集的行,这些示例行就清不掉。 - 风险与回滚:只在失败回执上多给一个已有的可选字段;成功路径不变。`landed` 缺席的含义收窄为"没有运行加载",抛错前一个数据集都没跑完时给 `[]`。回滚即还原本 PR。 - 先看:`packages/metadata-protocol/src/seed-loader.ts` 的 `SeedLoadPartialWrites.finished` 与 `notePartialWrites`;`packages/metadata-protocol/src/protocol-publish-batch.ts` 的 catch;`packages/runtime/src/domains/packages.ts` 兜底 catch。 - 两分钟验证:`pnpm --filter @objectstack/objectql exec vitest run src/publish-seed-receipt-landed-ids.test.ts`,预期 10 例全绿,其中"load that threw"一例的 `landed` 只含先跑完的对象。 - 需要你做的:无,席位落地。 ## What changes - **`@objectstack/metadata-protocol` (`seed-loader.ts`): `SeedLoadPartialWrites.finished`.** `notePartialWrites` snapshots the finished datasets' `{ object, landedIds, insertedIds }`, in load order, off the results it already keeps (`loadProgress.finished`). Nothing new is tracked; the ids are the ones each finished dataset's returned result carries. Empty when the throw came before any dataset finished. - **`@objectstack/metadata-protocol` (`protocol-publish-batch.ts`): `applySeedBodies`' catch** adds `landed: seedAppliedLanded(written.finished)` whenever a partial-writes reading exists. Same projection as the returned path (`seedAppliedLanded`), so the same two lists, the same per-object fold and the same 200 cap. No reading (the request parse refused, or the loader module never loaded) leaves `landed` absent. - **`@objectstack/metadata-protocol` (`index.ts`)**: `seedLoadPartialWrites` and `SeedLoadPartialWrites` are exported from the package root, so the runtime fallback reads the same record. - **`@objectstack/runtime` (`domains/packages.ts`)**: the route-level fallback seed apply's catch reads `seedLoadPartialWrites(e)` and reports the same `landed`. It had the same thrown path (`applyPublishedSeeds` → `loader.load` throws → catch with `success: false`, `error`). It now also reports `inserted` / `updated` from the same reading (the response schema requires both; this catch used to send neither) and names the object whose seed stopped the load in `error` (`seed apply stopped at the seed for object 'X': …`), exactly as `applySeedBodies`' catch does; the caught sentence itself is still withheld unless declared. No reading: zero counts, the bare sentence, no `landed`. - **`@objectstack/spec` (`api/protocol.zod.ts`)**: the shared `seedApplied.landed` description (both publish responses) and the `PublishMetaItemResponseSchema` docblock now say what `landed` covers on a thrown load (the datasets that finished, empty when none had; not the one that threw) and when it is still absent (no load ran, or a producer that reports no ids). Schema shape unchanged. `content/docs/references/api/protocol.mdx` regenerated. - `protocol-publish-batch.ts`: the older comment above the catch's counters now agrees with the new one beside `landed` (no reading means no load ran; a reading with zero counts is a load that threw before writing a row). - `.changeset/30429-seed-receipt-landed-on-throw.md`: spec minor, metadata-protocol minor, runtime patch, with a "Supersedes, in the same release" paragraph for the sentence in the unreleased `.changeset/30348-seed-receipt-landed-ids.md` that this makes false. ### The dataset that threw: omitted, and why Measured at `seed-loader.ts`: the dataset that throws can have landed rows before the throw. Its batched inserts flush every `insertBatchSize` records (`flushPendingInserts`), so earlier batches are written, and a throw can come after a row was written (the seed-file settle, pinned in `seed-loader-seed-files.test.ts`: `pw_doc`'s D1 is in the store when the storage service throws). Those rows are in `inserted` / `updated` and the object is named in `error`, but they are **not listed** in `landed`: - Its ids are only partly known when it throws. A batched insert whose response was lost may have committed rows whose ids never came back; the loader re-reads by key on its retry for exactly that reason. A list of the ids it did see would read as complete while missing rows, which is the one thing a reader of the ids cannot check. - A finished dataset's list is complete, so "the datasets that finished" is a line a reader can rely on. This is the card's default. A reader that wants to know whether unlisted rows exist compares `inserted` with the listed `inserted.total`s; `error` names the object. ### What `landed` answers now | Receipt | `landed` | | --- | --- | | load returned | every dataset (unchanged) | | load threw after some datasets finished | those datasets only | | load threw in pass 2 (every dataset finished) | every dataset, no failed object | | load threw before any dataset finished | `[]` (the loader ran; nothing finished) | | no load ran (seed bodies refused at parse, no readable bodies) | absent | ## Why #30348 (PR #30427) put the ids on the receipt, but only on the returned-load path; its contract review named the gap at `applySeedBodies`' catch. After a partial publish cloud's "clear sample data" had no ids for the datasets that did land, so those sample rows could never be cleared by id. ## Verification Head `4c02bddf8`, under `os-verify-lock`, `NODE_OPTIONS=--max-old-space-size=4096`: - `pnpm --filter @objectstack/metadata-protocol test`: Test Files 316 passed, 4 skipped (320), Tests 22425 passed, 26 skipped. `typecheck` exit 0. - `pnpm --filter @objectstack/objectql test`: Test Files 551 passed (551), Tests 10989 passed. `typecheck` exit 0. - `pnpm --filter @objectstack/runtime test`: Test Files 543 passed (543), Tests 8647 passed, 19 skipped. `typecheck` exit 0. - `pnpm --filter @objectstack/spec test`: Test Files 815 passed (815), Tests 25622 passed, 1 todo. `typecheck` exit 0. - Pins: - `packages/objectql/src/publish-seed-receipt-landed-ids.test.ts` (real protocol, real ObjectQL engine, real loader over an in-memory driver; ids read back from the engine). The old pin "a load that threw reports no ids" is replaced by: a good seed (with a customer's pre-existing row its key matches) lands, a later seed naming a missing object throws; the receipt lists the good seed's created id under `inserted` and the customer's row under `matched`, has no entry for the object that threw, and `PublishPackageDraftsResponseSchema` keeps it. Plus: a load that throws before any seed finished reports `landed: []`, not absent. - `packages/runtime/src/domains/packages-seed-apply-landed-thrown.test.ts` (new, the fallback door with the real `SeedLoaderService`): positive control on a returned load; a second dataset's existing-row read throws after the first dataset wrote, and the receipt lists the first, reports `inserted: 1`, `updated: 0`, and `error` names the stopped object with the driver sentence withheld; a failure before any load ran leaves `landed` absent. - `packages/metadata-protocol/src/seed-loader-deferred-failure.test.ts` (+1): a throw in pass 2, after both datasets of a circular pair wrote: the reading has no `failedObject`, `finished` lists both datasets with their store ids, and `seedAppliedLanded(finished)` names every row. Pass 2 guards its own driver traffic (a failed back-fill write is counted, a failed lookup probe is a miss; both measured), so the throw is staged at its one unguarded call, the `logger.error` its back-fill failure reports through. - `packages/metadata-protocol/src/seed-loader-seed-files.test.ts`: `seedLoadPartialWrites` now carries `finished` with the finished dataset's store ids, and leaves out the dataset that threw although its row D1 was written; a throw before any dataset reads `finished: []`. <details><summary>Reverse verification (two mutations, each restored by scripts/ablation-replace.mjs to the HEAD blob)</summary> The objectql and runtime pins resolve `@objectstack/metadata-protocol` from `dist`, so each mutation was rebuilt and `ablation-dist-preflight.mjs` confirmed the marker in 2 built files before reading a result. Directions predicted before running. 1. The loader carries no finished ids (`finished: finished.map(...)` → `finished: (globalThis as any).ABL30429A ?? []`, blob `a7986c958ece` → `c3a35b1b3aa0`): objectql landed pins **1 failed | 9 passed** (the thrown-after-finish case; the before-any-finished `[]` case holds either way), runtime fallback **1 failed | 2 passed**, `seedLoadPartialWrites` unit **1 failed | 2 passed**. Restored: blob == HEAD. 2. The catch drops `landed` (`...(written ? { landed: ... } : {})` → `...((globalThis as any).ABL30429B ?? {})`, blob `eccb3c495621` → `cb59f9935201`): objectql landed pins **2 failed | 8 passed** (both thrown cases). Restored: blob == HEAD. 3. Round 1, the runtime catch's counts (`inserted: written?.inserted ?? 0,` → `inserted: 0,` in `packages.ts`, blob `39d41e8df5cb` → `8a37f8d6e007`; runtime resolves its own `src`): `packages-seed-apply-landed-thrown.test.ts` **1 failed | 2 passed** (the thrown case). Restored: blob == HEAD, `git diff HEAD` empty. Restore leg for 1 and 2: rebuilt, `--absent` preflight green for both markers, `git diff HEAD` empty, the three files back to 10/10, 3/3, 3/3. A first attempt planted the markers as comments, which the build strips; the preflight refused it (marker absent from dist) before any test ran, and it is discarded. </details> <details><summary>Gates</summary> - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran gates-ran.list`: "105 derived famil(ies) accounted for — 105 run, 0 NOT-MEASURED (a DERIVED zero — all 105 recorded an exit code and none of them is 3)", exit 0. Every line `exit 0`. - The serial pass ran at `be4d35b5` (the implementation commit); `4c02bddf8` adds only the changeset's "Supersedes" paragraph, and the changeset gates (`check-empty-changeset`, `check-changeset-no-major`) plus `check:generated` and `check:nul-bytes` were re-run on `4c02bddf8`, exit 0. - Three families exited 3 (prerequisite) on the first pass and were re-run green: `check-plugin-teardown-shape --self-test` (its pinned fixture commit fetched), `check:skill-examples` and `check:dual-build-cjs-loads` (after a full `turbo run build`, 119/119). - `Prior rulings read` is from `node scripts/pm/check-prior-rulings.mjs --card 30429 --terms seedApplied,SeedLoadPartialWrites,landed`. ADR-0038 Decision §4 (L3 probes attach to the publish response "like `seedApplied`") does not rule on the receipt's ids; the other hits match the word "landed" in unrelated decisions. The card's thread holds no ruling. - `check-changeset-no-major` with this body as the event: exit 0 on `Metadata-protocol: semantic` (spec and metadata-protocol `minor`). Controls: `none` exits 1 (the `minor` entries are unaccounted for); `added` also exits 0, so the gate checks the level, not the kind. The kind is `semantic` by reading: no schema key is added or removed (the schema shape is byte-identical; only the description moves), and an existing key's absence changes meaning on a thrown load, from "not reported" to "no load ran". - No `!` in the title and no `(narrowing)` arm: no full dogfood run. CI owns the rest of the farm. </details> ## Acceptance notes - The seed-ownership claim (`claim-seed-ownership.ts`, `seed-settlement.ts`) still reads `landedIds`: out of scope per the card. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic (description text only). - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes (CI-owned run). - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English (the 维护者速读 block excepted, per project convention). - [x] The *Before Submitting* list in CONTRIBUTING.md is done. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01WGufzht5n1PKh96TqTfrvV --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent e8e924f commit 38d416f

11 files changed

Lines changed: 314 additions & 25 deletions

File tree

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/metadata-protocol': minor
4+
'@objectstack/runtime': patch
5+
---
6+
7+
fix(metadata-protocol,runtime): a seed load that throws part-way still reports `seedApplied.landed` for the datasets that finished before the throw
8+
9+
Clause-②: yes (widening: a thrown load's receipt carries `landed` for the datasets that finished, where it carried none)
10+
Metadata-protocol: semantic
11+
12+
- **`@objectstack/spec`: `seedApplied.landed`** (both publish responses) now says what it covers on a load that threw part-way (`success: false` with `error`): only the datasets that FINISHED before the throw. The dataset that threw is not listed (its rows are in `inserted` / `updated`, its object in `error`). Absent `landed` now means the apply failed before any load ran, or the producer reports no ids; a throw before any dataset finished reports `landed: []`.
13+
- **`@objectstack/metadata-protocol`**: `SeedLoadPartialWrites` gains `finished`, the finished datasets' `object` / `landedIds` / `insertedIds` in load order, and `applySeedBodies`' catch folds it through the same `seedAppliedLanded` projection (same two lists, same 200 cap) the returned path uses. `seedLoadPartialWrites` and `SeedLoadPartialWrites` are exported from the package root.
14+
- **`@objectstack/runtime`**: the package-publish door's fallback seed apply reads the same `seedLoadPartialWrites` off a thrown load and reports the same `landed`, plus the `inserted` / `updated` counts the response schema requires (it reported neither before) and the object whose seed stopped the load in `error`, as `metadata-protocol`'s apply does.
15+
- Why the dataset that threw is left out: its ids are only partly known when it throws. A batched insert whose response was lost may have committed rows whose ids never came back, so a list of the ids the loader did see would read as complete while missing rows.
16+
17+
Supersedes, in the same release: `.changeset/30348-seed-receipt-landed-ids.md` says absent `landed` means "not reported (a load that threw, or a producer that reports no ids)". After this change a load that threw part-way reports `landed` for its finished datasets; absent now means no load ran, or a producer that reports no ids.

‎content/docs/references/api/protocol.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2546,7 +2546,7 @@ The authoring surface of a metadata write. Closed: `copilot` is an AI assistant'
25462546
| **updated** | `integer` | ✅ | Rows updated by the externalId-keyed upsert. |
25472547
| **error** | `string` | optional | Single failure message, present when the seed apply threw before the loader ran (including "no readable seed bodies"). |
25482548
| **errors** | `any[]` | optional | Per-record failures reported by the seed loader. Present only when the loader ran and returned a non-empty error list. |
2549-
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. ABSENT means "not reported" — the apply failed before or while loading (see `error`), or the producer reports no ids — never "no rows landed". |
2549+
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. When the load threw part-way (`success: false` with `error`), it lists only the datasets that FINISHED before the throw, and is EMPTY when none had; the dataset that threw is not listed (its rows are in the counts, its object in `error`). ABSENT means "not reported" — the apply failed before any load ran (see `error`), or the producer reports no ids — never "no rows landed". |
25502550

25512551
### Nested Shape: `PublishMetaItemResponse.materializeApplied`
25522552

@@ -2647,7 +2647,7 @@ The authoring surface of a metadata write. Closed: `copilot` is an AI assistant'
26472647
| **error** | `string` | optional | Single failure message, present when the apply failed before the loader ran (including "no readable seed bodies"). When the failure is the seed bodies' own schema refusal, this is a one-sentence headline and the per-path detail rides `issues[]`. |
26482648
| **errors** | `any[]` | optional | Per-record failures reported by the seed loader, plus any seed-body read failures. May be present and empty on a clean load. |
26492649
| **issues** | `{ path: string; message: string; code?: string }[]` | optional | Structured spec-validation findings behind `error`, present when the apply was refused by the seed bodies' own schema — the declared 422 `seedRequestValidationError` mints. The per-path detail lives HERE, once; `error` stays a one-sentence headline. Absent on non-validation failures (driver faults, unreadable bodies), whose whole story is `error` / `errors[]`. |
2650-
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. ABSENT means "not reported" — the apply failed before or while loading (see `error`), or the producer reports no ids — never "no rows landed". |
2650+
| **landed** | `{ object: string; inserted: object; matched: object }[]` | optional | Per object, the stored rows the seed landed on: `inserted` (created by this publish) and `matched` (pre-existing rows a seed key matched), each capped at `SEED_APPLIED_LANDED_IDS_CAP` (200) ids with `total` and `truncated` stating any cut. One entry per object the loader ran, empty lists included. When the load threw part-way (`success: false` with `error`), it lists only the datasets that FINISHED before the throw, and is EMPTY when none had; the dataset that threw is not listed (its rows are in the counts, its object in `error`). ABSENT means "not reported" — the apply failed before any load ran (see `error`), or the producer reports no ids — never "no rows landed". |
26512651

26522652
### Nested Shape: `PublishPackageDraftsResponse.materializeApplied`
26532653

‎packages/metadata-protocol/src/index.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -281,7 +281,14 @@ export {
281281
} from './migrations/recorded-by-sentinel.js';
282282
export type { SentinelHistoryRow } from './migrations/recorded-by-sentinel.js';
283283

284-
export { SeedLoaderService, type SeedLoaderServiceOptions } from './seed-loader.js';
284+
export {
285+
SeedLoaderService,
286+
type SeedLoaderServiceOptions,
287+
// [#30429] The runtime's fallback seed apply reads a thrown load's finished
288+
// datasets through it, so its receipt reports `landed` as this package's does.
289+
seedLoadPartialWrites,
290+
type SeedLoadPartialWrites,
291+
} from './seed-loader.js';
285292
export { runBuildProbes } from './build-probes.js';
286293
export type {
287294
RuntimeBuildIssue,

‎packages/metadata-protocol/src/protocol-publish-batch.ts‎

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -134,9 +134,8 @@ export async function applySeedBodies(
134134
// [#30348] Which rows landed, per object: created by this publish
135135
// vs pre-existing rows a seed key matched — the loader's own ids,
136136
// capped and folded by the one projection both receipt producers
137-
// share. Only on this path: a load that THREW (the catch below)
138-
// reports counts but no ids, so `landed` stays absent there —
139-
// "not reported", never "nothing landed".
137+
// share. [#30429] A load that THREW reports the same key from the
138+
// catch below, for the datasets that finished.
140139
landed: seedAppliedLanded(r.results),
141140
};
142141
} catch (e: any) {
@@ -156,9 +155,11 @@ export async function applySeedBodies(
156155
// The counters describe what was WRITTEN. The loader writes dataset
157156
// by dataset and stops at the one that throws; the rows of the
158157
// datasets ahead of it stay written (nothing rolls them back), so the
159-
// receipt carries the loader's own count of them and names the seed
160-
// that stopped it. No reading means the throw came before the loader
161-
// wrote anything — the request parse, or the loader's import.
158+
// receipt carries the loader's own count of them (plus any rows the
159+
// dataset that threw wrote first) and names the seed that stopped
160+
// it. No reading means no load ran — the request parse refused, or
161+
// the loader's import failed — so nothing was written; a reading
162+
// with zero counts is a load that threw before writing a row.
162163
const written = partialWritesOf?.(e);
163164
const failure = clientFacingFailureText(e, 'seed apply failed');
164165
return {
@@ -168,6 +169,15 @@ export async function applySeedBodies(
168169
error: written?.failedObject
169170
? `seed apply stopped at the seed for object '${written.failedObject}': ${failure}`
170171
: failure,
172+
// [#30429] The rows of the datasets that FINISHED before the
173+
// throw, through the same projection as the returned path — the
174+
// ids a reader clearing a partial publish's sample data needs. The
175+
// dataset that threw is not listed (its ids are only partly known,
176+
// see `SeedLoadPartialWrites.finished`); its rows are in the
177+
// counts above and its object in `error`. Empty when the throw
178+
// came before any dataset finished; absent when there is no
179+
// reading (no load ran, see above): "not reported".
180+
...(written ? { landed: seedAppliedLanded(written.finished) } : {}),
171181
// [#10524] A DECLARED refusal's structured findings ride the
172182
// receipt beside the headline `error` — the message is a
173183
// one-sentence headline now, so this is where the per-key

‎packages/metadata-protocol/src/seed-loader-deferred-failure.test.ts‎

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

33
import { describe, it, expect, vi } from 'vitest';
4-
import { SeedLoaderService } from './seed-loader.js';
4+
import { SeedLoaderService, seedLoadPartialWrites } from './seed-loader.js';
5+
import { seedAppliedLanded } from './seed-applied-landed.js';
56
import type { IDataEngine, IMetadataService } from '@objectstack/spec/contracts';
67
import { assertEngineDeleteDispatch, assertEngineUpdateDispatch, assertEngineFindOnePredicate } from '@objectstack/metadata-core';
78

@@ -300,3 +301,58 @@ describe('seed deferred back-fill failure is reported, not swallowed (framework#
300301
expect(result.summary.totalErrored).toBe(0);
301302
});
302303
});
304+
305+
/**
306+
* [#30429] A load that throws in PASS 2 — after every dataset finished — is
307+
* still a thrown load, and its partial-writes reading lists every dataset as
308+
* finished: no `failedObject`, and the receipt's `landed` (the same
309+
* `seedAppliedLanded` projection the publish catch uses) names every row the
310+
* load wrote.
311+
*
312+
* Pass 2 guards its own driver traffic (a failed back-fill write is counted,
313+
* a failed lookup probe is a miss), so the throw is staged at the one call in
314+
* it that is not guarded: the `logger.error` its back-fill failure reports
315+
* through — a log transport that fails while pass 2 reports. What is pinned is
316+
* the loader's state at that point, not the transport.
317+
*/
318+
describe('a load that throws in pass 2 reports every dataset as finished (#30429)', () => {
319+
it('finished lists both datasets with their store ids; no failed object; landed names every row', async () => {
320+
const { engine, store } = createFaithfulEngine();
321+
const metadata = createMetadata();
322+
const realUpdate = (engine.update as any).getMockImplementation();
323+
(engine.update as any).mockImplementation(async (obj: string, data: any, opts: any) => {
324+
if (obj === 'audit_department') throw new Error('fetch failed');
325+
return realUpdate(obj, data, opts);
326+
});
327+
const logger = createLogger();
328+
logger.error.mockImplementation(() => { throw new Error('the log transport is down'); });
329+
330+
const err = await new SeedLoaderService(engine, metadata, logger)
331+
.load({ seeds: SEEDS, config: CONFIG })
332+
.then(() => undefined, (e: unknown) => e);
333+
334+
expect((err as Error)?.message).toBe('the log transport is down');
335+
// Positive control: the throw came from pass 2's back-fill report, after
336+
// both datasets wrote their row.
337+
expect((engine.update as any).mock.calls.some(([obj]: [string]) => obj === 'audit_department')).toBe(true);
338+
const deptIds = (store.audit_department ?? []).map((r) => String(r.id));
339+
const workerIds = (store.audit_worker ?? []).map((r) => String(r.id));
340+
expect(deptIds).toHaveLength(1);
341+
expect(workerIds).toHaveLength(1);
342+
const partial = seedLoadPartialWrites(err)!;
343+
expect(partial).toEqual({
344+
inserted: 2,
345+
updated: 0,
346+
finished: [
347+
{ object: 'audit_department', landedIds: deptIds, insertedIds: deptIds },
348+
{ object: 'audit_worker', landedIds: workerIds, insertedIds: workerIds },
349+
],
350+
});
351+
expect('failedObject' in partial).toBe(false);
352+
const list = (ids: string[]) => ({ ids, total: ids.length, truncated: false });
353+
expect(seedAppliedLanded(partial.finished)).toEqual([
354+
{ object: 'audit_department', inserted: list(deptIds), matched: list([]) },
355+
{ object: 'audit_worker', inserted: list(workerIds), matched: list([]) },
356+
]);
357+
});
358+
});

‎packages/metadata-protocol/src/seed-loader-seed-files.test.ts‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -558,7 +558,15 @@ describe('seedLoadPartialWrites: the rows a thrown load had written', () => {
558558
expect((err as Error).message).toBe('the storage service is down');
559559
expect((store.pw_tag ?? []).map((r) => r.name).sort()).toEqual(['T1', 'T2']);
560560
expect((store.pw_doc ?? []).map((r) => r.code)).toEqual(['D1']);
561-
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 3, updated: 0, failedObject: 'pw_doc' });
561+
// [#30429] The finished dataset (`pw_tag`) carries its landed ids — the
562+
// store's own; the dataset that threw (`pw_doc`) is in the counts but not
563+
// in `finished`, although its row D1 was written before the throw.
564+
const tagIds = (store.pw_tag ?? []).map((r) => String(r.id));
565+
expect(tagIds).toHaveLength(2);
566+
expect(seedLoadPartialWrites(err)).toEqual({
567+
inserted: 3, updated: 0, failedObject: 'pw_doc',
568+
finished: [{ object: 'pw_tag', landedIds: tagIds, insertedIds: tagIds }],
569+
});
562570
});
563571

564572
it('a load that throws before any dataset wrote: zero, and no object named', async () => {
@@ -569,7 +577,7 @@ describe('seedLoadPartialWrites: the rows a thrown load had written', () => {
569577

570578
expect((err as Error).message).toBe('metadata store unavailable');
571579
expect(store).toEqual({});
572-
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 0, updated: 0 });
580+
expect(seedLoadPartialWrites(err)).toEqual({ inserted: 0, updated: 0, finished: [] });
573581
});
574582

575583
it('an error that did not come out of a load carries no reading', () => {

‎packages/metadata-protocol/src/seed-loader.ts‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ import { isMissingTableError } from '@objectstack/metadata/errors';
4848
// [#25779] `seedFile(path)` values: stored, kept across replays, and owned by
4949
// the row that holds them; an id-shaped file value that names no file refused.
5050
import { SeedDatasetFiles, type SeedFileClaims } from './seed-files.js';
51+
import type { SeedLandingResult } from './seed-applied-landed.js';
5152

5253
interface Logger {
5354
info(message: string, meta?: Record<string, any>): void;
@@ -581,6 +582,21 @@ export interface SeedLoadPartialWrites {
581582
readonly updated: number;
582583
/** The object of the dataset that was loading when the load threw; absent when it threw outside any dataset. */
583584
readonly failedObject?: string;
585+
/**
586+
* [#30429] The datasets that FINISHED before the throw, in load order, each
587+
* with the row ids it landed on (`landedIds` / `insertedIds`, exactly as its
588+
* returned result carries them) — what the publish receipt projects into
589+
* `seedApplied.landed` with the same `seedAppliedLanded` a returned load
590+
* uses. Empty when the throw came before any dataset finished.
591+
*
592+
* The dataset that threw is NOT here, although its rows are in the counts
593+
* above: its ids are only partly known when it throws. A batched insert
594+
* whose response was lost may have committed rows whose ids never came back
595+
* (the loader re-reads by key on its retry for exactly that reason), so a
596+
* list of the ids it did see would read as complete while missing rows. A
597+
* finished dataset's list is complete; that is the line this field draws.
598+
*/
599+
readonly finished: readonly SeedLandingResult[];
584600
}
585601

586602
/** The writes a thrown load had made, keyed by the error it threw. */
@@ -815,6 +831,9 @@ export class SeedLoaderService implements ISeedLoaderService {
815831
inserted: finished.reduce((sum, r) => sum + r.inserted, 0) + (loading?.written.inserted ?? 0),
816832
updated: finished.reduce((sum, r) => sum + r.updated, 0) + (loading?.written.updated ?? 0),
817833
...(loading ? { failedObject: loading.object } : {}),
834+
// A snapshot of the ids alone: the results array is the live one pass 2
835+
// writes its counters into.
836+
finished: finished.map((r) => ({ object: r.object, landedIds: r.landedIds, insertedIds: r.insertedIds })),
818837
};
819838
const carrier = err !== null && typeof err === 'object' ? err : new Error(String(err));
820839
partialWritesByError.set(carrier, partial);

0 commit comments

Comments
 (0)