You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit ce001f9
Browse filesBrowse the repository at this point in the historyBrowse files
test(cli): compile each scaffold sandbox in process, one spawn per distinct lint-eval run, budget the generate/i18n spawn files (#28592)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XSSr8Pe8L9chsNc1qbg5uPE)_
Part of #23181 (slice 27, the generate / scaffold / i18n family in
`packages/cli/test`). The card stays open for later slices.
Clause-②: no
Metadata-protocol: none
## 维护者速读
- **改了什么**:`packages/cli/test` 里 6 个超过 10 s
的测试文件。`lint-eval-generator-refusal-separator` 去掉两次与前面完全相同的 CLI 子进程(6 次变
4 次);`scaffold-emission-typechecks` 不再每个用例起一个 `tsc` 子进程,改为进程内每个脚手架一个独立的
TypeScript Program,共享已解析的 lib 与 spec 声明文件;另外 4 个文件每次子进程都是不同的工作,只加第 2 行
`// test-budget:` 头。
- **为什么改**:卡 #23181 的测试提速职责:超过 10 s 的文件要么提速,要么按裁决 6181973363 B 声明预算。
- **风险与回滚**:只改测试文件,不发布任何东西(cli 的 `files[]` 只有
dist、README、CHANGELOG);没有删改任何断言、用例名、超时或分层;回滚即 revert。
- **先看**:`packages/cli/test/scaffold-emission-typechecks.test.ts` 的
`typecheckProject`(进程内编译与声明文件缓存);`packages/cli/test/lint-eval-generator-refusal-separator.test.ts`
的 `beforeAll`;其余 4 个文件只有第 2 行。
- **两分钟验证**:`pnpm --filter @objectstack/cli exec vitest run
test/scaffold-emission-typechecks.test.ts` → 13/13 通过,canary 仍以 TS2694
变红。
- **需要你做的**:无,席位落地。
## What changes
Slice 23's offender table (the default reporter's per-file line in every
Test Core shard log of the five scheduled `ci.yml` runs `41199450735`,
`41217827347`, `41237094739`, `41255489006`, `41271604267`) names these
six files over 10 s with no header.
**Cause, per file.** I counted child processes against distinct
(command, flags, fixture) triples:
| file | CI max (s) | tests | children | distinct triples | cause | fix
|
|:--|--:|--:|--:|--:|:--|:--|
| `lint-eval-generator-refusal-separator` | 35.7 | 5 | 6 source-CLI
spawns | 4 | the same-door case re-spawned the two `--json` runs the
cases above it already ran | the 4 distinct runs move into one
`beforeAll`; every case reads them. Header 50 |
| `scaffold-emission-typechecks` | 33.0 | 13 | 12 `tsc` children
(canary, 3 `os init` templates, 8 `os g` generators) | 12 sandboxes |
each child loads the compiler and parses and binds the same `lib.*.d.ts`
and `@objectstack/spec` declarations again | in process, one Program per
sandbox from its own `tsconfig.json`, with one shared cache of the
parsed declaration files outside the sandboxes. Header 50 |
| `generate-stack-reach` | 31.0 | 8 | 10 spawns, already in one
`beforeAll` | 10 | distinct real work (five fixtures, each refusal read
against its own byte snapshot) | header 40 |
| `dependency-range-doors` | 27.6 | 5 | 5 spawns | 5 | distinct real
work | header 40 |
| `i18n-extract-check-json` | 22.2 | 5 | 8 spawns (1 setup extract + 7)
| 8 | distinct real work | header 30 |
| `i18n-extract-check-dry-run` | 19.3 | 5 | 7 spawns (1 setup extract +
6) | 7 | distinct real work | header 30 |
**`lint-eval-generator-refusal-separator`.** The same-door case compared
`--json --generator ""` with `--json --generator MISSING_PATH`, two runs
with the same command, flags and fixture as the first and third cases.
It now reads those runs. The pair it compares and every assertion are
unchanged. The `beforeAll` timeout is the old per-case allowance times
four.
**`scaffold-emission-typechecks`.** Each sandbox still compiles as its
own Program, from its own `tsconfig.json` (the one
`renderScaffoldTsconfig` writes), so no scaffold can lean on another's
declarations. The verdict is `tsc --noEmit`'s: `getPreEmitDiagnostics`
(config, options, global, syntactic, semantic, declaration) plus the
emit diagnostics, exit 0 exactly when none is an error, and the output
is those diagnostics in `--pretty false` form. Only files outside the
sandbox are cached, keyed by path and parse options. All sandboxes
compile under the same `SCAFFOLD_TSCONFIG_COMPILER_OPTIONS`; they differ
only in `rootDir` and `include`, which change no parse. This is the
compiler's own reuse model: the language service shares source files
across programs the same way. The compiler is the same `typescript`
package `typescript/bin/tsc` resolved to. The docblock says all this in
a new "One compiler, one Program per sandbox" section.
**Headers** (ruling 6181973363 letter B): N = the slowest of the five CI
readings × 1.25, rounded up to 10 s. Classes: `spawns the real producer
once per single-cause fixture` for the five spawn files, `compiles the
package type graph once` for `scaffold-emission-typechecks`. The two
fixed files keep their pre-fix CI reading. All six parse with
`readTestBudget` (`scripts/measure-test-shard-timings.mjs`).
Not changed: no assertion, case name, per-case timeout, tier, project or
test helper. No source file.
## Tests
- **A/B** (one box, one lock hold, N=3, `--maxWorkers=1`, base and head
interleaved): the base file's bytes were written into the same path for
each base leg and restored from `HEAD` after it, checked by blob hash,
so both legs ran in the same tier and the same
`DECLARATION_READER_FILES` slot (a `zzbase.*` copy beside it would not).
Vitest's `Duration` line, with no per-file line in single-file runs:
| file | base (s) | head (s) | change |
|:--|:--|:--|--:|
| `scaffold-emission-typechecks` | 26.81, 27.16, 25.54 | 15.31, 16.85,
16.52 | −38.8% (mean 26.50 → 16.23); the tests phase 18.2 → 7.3 s |
| `lint-eval-generator-refusal-separator` | 32.29, 32.95, 33.75 | 22.76,
26.32, 25.12 | −25.1% (mean 33.00 → 24.73) |
Every leg exit 0, 13/13 and 5/5 tests on both sides.
- **The in-process verdict matches `tsc` (one-time probe, not
committed):** a copy of the head file ran both the in-process compile
and the old `tsc` child on every sandbox. Same verdict on all 12, the
same error codes (canary `TS2694` on both, none elsewhere). A second
canary run after all the others, with the cache warm, still answers
`TS2694`. `tsc --noEmit` exits 2 on an error under TypeScript 6.0.3, so
the in-process verdict answers 2 too.
- **All six files on head**, in one run: 6/6 files, 41/41 tests.
- `pnpm --filter @objectstack/cli typecheck` (tsc +
`check:test-typecheck`) exit 0.
- No ablation: test-only, no tested behaviour changed.
## Acceptance notes
- **A lever outside this order's fix list.** These five spawn files
start `bin/run-dev.js` through `node_modules/.bin/tsx`. The migrate
files moved to `process.execPath` in slice 12. One `os lint --eval
--json --generator ""` spawn here, without the queue-tier manifest: 9.5
to 11.8 s through tsx, 5.3 to 5.4 s through node, with byte-identical
stdout. Not taken: the order rules the fix set (shared `beforeAll`,
in-process read, header). Carrier: #23181.
- **The two i18n files share work across files.** Both run the same
setup extract and the same `--check` control over an empty `--out`.
Sharing it would need a helper beyond these files or a file merge, so it
was not done.
- **`skip-changeset`.** Test-only. `@objectstack/cli`'s `files[]` is
`dist`, `README.md`, `CHANGELOG.md`, and `test/` is not built into
`dist/`, so no `files[]` content changes.
## Verification
All readings on head `cc5ee375` (`git rev-parse --short HEAD`).
- **Derived gate families:** `node scripts/pm/dispatch-gates.mjs
--commands --repo objectstack-ai/objectstack` gives 48 families. Each
ran with its exit code captured before any pipe. Two first reads were
container prerequisites and were re-read after meeting them:
`check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow
clone, then 0 after `git fetch --depth=1` of its positive-control
commit; `check:dual-build-cjs-loads` exited 3 with no full build, then 0
after `pnpm exec turbo run build --concurrency=2` (118/118 tasks).
`--ran` reconciliation: `✓ dispatch-gates --ran: 48 derived famil(ies)
accounted for — 48 run, 0 NOT-MEASURED`.
- **Hand gates:** `pnpm exec eslint --no-inline-config` over the six
files exit 0 (`@objectstack/cli` has no `lint` script; the repo-wide
`pnpm lint` runs in CI). `node scripts/check-commit-card-trailers.mjs
--range origin/main..HEAD` exit 0.
- **CI** families (Test Core with both cli tiers, the lint farm, the
type-check lanes) are read off this PR's check-runs by the seat.
- Evidence (gate list with exit codes, A/B logs summary, the tsc
equivalence probe output): `pm-orders/server/evidence/23181-s27/` in the
project files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01FEkQ5oeaQtmg5DUi2ojVqs
---------
Co-authored-by: Claude <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: packages/cli/test/dependency-range-doors.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 40 — spawns the real producer once per single-cause fixture: five spawns of the source CLI, each a distinct (command, fixture) pair (`os validate --json` and `os build --json` over the unreadable-range artifact, `os validate --json` and `os lint --json` over the unmet-range artifact, `os validate --json` over the met-range control); slowest CI reading 27.6 s (run 41199450735, over the five scheduled ci.yml runs 41199450735 to 41271604267) x 1.25, rounded up
2
3
3
4
/**
4
5
* ADR-0141 D2, author side — `os validate`, `os build` and `os lint` report
Copy file name to clipboardExpand all lines: packages/cli/test/generate-stack-reach.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 40 — spawns the real producer once per single-cause fixture: ten sequential spawns of the source CLI in one beforeAll, each a distinct (`os g` generator, flags, fixture state) triple over five fixtures (the wired app before and after its object and flow exist, a stack with no `requires`, one requiring `triggers` alone, the pre-fix objects-only config, a bare directory), each refusal read against its own byte snapshot; slowest CI reading 31.0 s (run 41217827347, over the five scheduled ci.yml runs 41199450735 to 41271604267) x 1.25, rounded up
2
3
3
4
/**
4
5
* PIN (#20215) — after writing, `os generate` says whether the scaffold
Copy file name to clipboardExpand all lines: packages/cli/test/i18n-extract-check-dry-run.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 30 — spawns the real producer once per single-cause fixture: seven spawns of the source CLI's `os i18n extract`, the one real extract that writes the in-sync tree each seeded case copies, then six distinct (flags, `--out` state) runs (`--check` and `--check --dry-run` over an empty `--out`, `--check --dry-run` over a stale bundle and over the in-sync tree, `--dry-run` with an empty `--out` and with none); slowest CI reading 19.3 s (run 41217827347, over the five scheduled ci.yml runs 41199450735 to 41271604267) x 1.25, rounded up
2
3
3
4
/**
4
5
* `os i18n extract --check --dry-run` COMPARES, and says what it found (#16480).
Copy file name to clipboardExpand all lines: packages/cli/test/i18n-extract-check-json.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 30 — spawns the real producer once per single-cause fixture: eight spawns of the source CLI's `os i18n extract`, the one real extract that writes the in-sync tree each seeded case copies, then seven distinct (flags, `--out` state) runs (`--check` and `--check --json` over an empty `--out`, `--check --json` over a stale bundle, `--check` and `--check --json` over the in-sync tree, `--check --json` with no `--out`, `--json` without `--check`); slowest CI reading 22.2 s (run 41271604267, over the five scheduled ci.yml runs 41199450735 to 41271604267) x 1.25, rounded up
2
3
3
4
/**
4
5
* `os i18n extract --check --json` COMPARES, and reaches the same verdict as
Copy file name to clipboardExpand all lines: packages/cli/test/lint-eval-generator-refusal-separator.test.ts
+47-22Lines changed: 47 additions & 22 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 50 — spawns the real producer once per single-cause fixture: four spawns of the source CLI in one beforeAll, each a distinct (face, generator) pair (`os lint --eval --json --generator ""`, the same on the human face, and both faces over an unresolvable path); the same-door case reads the two machine-face runs the cases above read instead of spawning them again (six spawns before); slowest CI reading 35.7 s (run 41217827347, over the five scheduled ci.yml runs 41199450735 to 41271604267) x 1.25, rounded up
2
3
3
4
/**
4
5
* [#16359] `os lint --eval --generator ""` printed TWO spaces after our own
@@ -94,6 +95,23 @@ interface Run {
94
95
}
95
96
96
97
letdir: string;
98
+
/** The unresolvable path every negative-control run names; under `dir`, so it never exists. */
99
+
letmissing: string;
100
+
101
+
/**
102
+
* The four distinct runs this file reads, each spawned ONCE in `beforeAll`.
103
+
*
104
+
* The same-door case used to spawn the two machine-face runs a second time,
105
+
* with the same command, flags and fixture as the cases above; it now reads
106
+
* the very runs they read. The pair it compares is unchanged, and so is every
Copy file name to clipboardExpand all lines: packages/cli/test/scaffold-emission-typechecks.test.ts
+88-19Lines changed: 88 additions & 19 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 50 — compiles the package type graph once: the lib and @objectstack/spec declaration files are parsed and bound once in this worker and shared by one TypeScript Program per emitted scaffold (each `os init` template, each `os g` generator, and the canary), each with its own tsconfig and diagnostics, instead of one `tsc` child per case; the file also imports the init and generate command modules to derive both rosters; slowest CI reading 33.0 s (run 41199450735, over the five scheduled ci.yml runs 41199450735 to 41271604267, one tsc child per case) x 1.25, rounded up
2
3
3
4
/**
4
5
* PIN (#15976) — every scaffold this package emits must survive the
@@ -54,9 +55,23 @@
54
55
* scaffolds genuinely did parse, validate and load — they simply did not
55
56
* COMPILE, and nothing here had ever asked a compiler.
56
57
*
57
-
* That is the gap this file fills, and it is why it spawns `tsc` over a
58
-
* materialized project instead of importing the module. The type layer is a
59
-
* separate axis from the schema layer, and it needs its own instrument.
58
+
* That is the gap this file fills, and it is why it runs the TypeScript
59
+
* compiler over a materialized project instead of importing the module. The
60
+
* type layer is a separate axis from the schema layer, and it needs its own
61
+
* instrument.
62
+
*
63
+
* ## One compiler, one Program per sandbox
64
+
*
65
+
* Each sandbox is compiled as its OWN Program, from its own `tsconfig.json`,
66
+
* with the diagnostics `tsc --noEmit -p <root>` reports — so a case still asks
67
+
* exactly "does this emitted project pass its own `typecheck` script", and no
68
+
* scaffold can lean on another's declarations. What the sandboxes share is the
69
+
* read-only part every one of them used to redo in its own `tsc` child: the
70
+
* parsed and bound `lib.*.d.ts` and `@objectstack/spec` declaration files,
71
+
* held in one source-file cache (the compiler's own reuse model: the language
72
+
* service shares source files across programs the same way). The compiler is
73
+
* the `typescript` package `typescript/bin/tsc` resolves to, loaded once in
0 commit comments