Skip to content

Commit 5ea7789

Browse files
ci(test-shards): cut the CLI into 3 Test Core slices, the count the shard wall picks inside pin 3c's band (#31543)
<!-- claude-projects-attribution: {"github_login":"zhuangjianguo"} --> _Requested by **Steedos** · [project thread](https://claude.ai/code/project/chan_018ETKaNwMZCNRYD7D8Q9Ubb?thread=cmsg_018ETKaNwMZCNRYD7D8Q9Ubb7qV7tjQEZS5b86zyDZ9H16)_ Fixes #25870 Clause-②: no Metadata-protocol: none ## 维护者速读 - **改了什么**:Test Core 把 `@objectstack/cli` 的测试从切 2 片改为切 3 片(`scripts/partition-test-shards.mjs` 一行配置 + 上一版映射 + 推导注释),不动 ci.yml、不动任何测试。 - **为什么改**:CLI 的切片在所在分片里排在整包测试之后串行跑,79 次完整运行里 4/8 分片中位 19.4 分钟、1/8 16.8 分钟,其余六片 9.3–14.8 分钟;每个核心 PR 和合并队列条目都在等最慢那片。 - **风险与回滚**:只改分片计划,测试集合不变(三片之并即全部 CLI 文件);多一片多一次 CLI 闭包构建(远程缓存命中,约数秒)。回滚即 revert 本 PR。 - **先看**:`scripts/partition-test-shards.mjs` 的 `FILE_SHARDED_PACKAGES`(cli: 3)与其上方新增的 "RAISED TO 3" 推导段。 - **两分钟验证**:`node scripts/partition-test-shards.mjs --self-test`,预期末行 `self-test OK … file-level slices: @objectstack/cli x3`,whole legs `291/1553/1553/1551/1553/1553/1074/1076s`。 - **需要你做的**:无,席位落地。 ## What changes | File | Change | |---|---| | `scripts/partition-test-shards.mjs` | `FILE_SHARDED_PACKAGES['@objectstack/cli']` 2 → 3. `PREVIOUS_FILE_SHARDED_PACKAGES` 4 → 2 (the outgoing count, so the timings generator still decodes the 2-slice run summaries retained for the next four days). A "RAISED TO 3" paragraph in the derivation docblock with the measurement below. Nothing else: pin 3c, its band, the density cap, pin 3d, `ci.yml` and the CLI's vitest sequencer are untouched. | ## Why A CLI slice runs as its shard's serial leg **after** the whole-package leg (`ci.yml`, "Run this shard's tests"), so it adds to that shard's wall at full weight while whole packages run four wide. With the CLI in 2 slices, the two slice shards are the slowest in the workflow. <details><summary>Measured: per-shard job wall on 79 full-plan runs (origin/main, plan unchanged since 2026-11-04)</summary> Population: every successful `CI` run on `merge_group` (newest 100, 2026-11-08) and `schedule` created after 2026-11-04T16:00Z; kept the 79 where all eight `Test Core (k/8)` jobs ran and passed (57 merge_group, 22 schedule). Wall is the job's `completed_at − started_at`. | shard | median (min) | p90 | closure build median | tests step median | |---|---|---|---|---| | 1/8 | 16.8 | 21.4 | 0.1 | 13.5 | | 2/8 | 12.8 | 16.9 | 0.1 | 10.9 | | 3/8 | 14.8 | 18.2 | 0.1 | 13.0 | | **4/8** | **19.4** | 25.4 | 0.1 | 17.6 | | 5/8 | 9.3 | 13.8 | 0.1 | 7.6 | | 6/8 | 10.0 | 14.7 | 0.1 | 8.4 | | 7/8 | 10.2 | 15.3 | 0.1 | 8.5 | | 8/8 | 9.3 | 12.8 | 0.1 | 7.5 | 4/8 was the slowest in 52 of 79 runs, 1/8 in 18. Leg medians from the full logs of all eight shards of 14 of those runs (112 logs): 4/8 = 313 s whole leg + 873 s `cli 2/2`; 1/8 = 43 s + 766 s `cli 1/2`; 3/8 = 762 s (`runtime:test` alone ~840 s); 2/8 = 698 s (`spec:test:repo` alone ~746 s). The closure build is 0.1–0.4 min median on every shard (remote cache), so it is not what makes these shards long. Control: with the CLI in 4 slices (28 `schedule` runs, 2026-10-28T02:30Z to 11-03T08:13Z) the slowest shard median was 12.1 min and the run-slowest median 16.7 min; with 2 slices (22 `schedule` runs since 11-04) they are 18.1 and 19.2 min. </details> **Why 3, and not more.** Pin 3c, unchanged, judges each count on the committed dataset (CLI 1432.04 s; the heaviest other serial task is `metadata-protocol` at 566.50 s, so the limit is 736.45 s, RAISE past 773.27 s, LOWER when the n − 1 slice is at or under 699.63 s): | n | slice | n − 1 slice | pin 3c | |---|---|---|---| | 2 | 716.02 s | 1432.04 s | passes | | **3** | 477.35 s | 716.02 s | **passes** | | 4 | 358.01 s | 477.35 s | refused, "Lower it to 3" | | 5 | 286.41 s | 358.01 s | refused, "Lower it to 4" | At the CLI's measured ~1640 s (766 + 873, what the next refresh should read), 2 leaves the band ("Raise it to 3") and 3 still passes. So 3 is the one count above 2 that holds on both today's dataset and the next refresh. ## Predicted vs measured shard wall Model: tests step ≈ whole-package sum / `WHOLE_LEG_WALL_DIVISOR` (2.9) + slice. The slices are cut the way the CLI's sequencer cuts them (declaration readers first, then sha1 order, vitest's equal-count ranges) from the per-file times in both slice logs of run 42478508299. That model reproduces the measured `cli 1/2` at 770 s against 776.5 s. Placement comes from the real `planRun()` on the full list, which reproduces CI's current plan byte for byte. | | today, 2 slices | this PR, 3 slices | |---|---|---| | slice shards | 1/8 (64 s whole + 770 s), 4/8 (846 s whole + 874 s) | 1/8 (291 s + 460 s), 7/8 (1074 s + 575 s), 8/8 (1076 s + 609 s) | | slowest CLI shard, modelled tests step | ~1164 s | ~977 s | | slowest shard, job median | **19.4 min measured** (4/8) | **~16.4 min predicted** (the 8/8 CLI shard), ~1.5 min above 3/8's 14.8 | The model reads ~10% high on today's 4/8 (1164 s modelled against 1056 s measured tests step median), and the prediction is scaled by the same factor. **Measured after landing:** the seat reads the card's acceptance, the slowest Test Core shard ≤ ~20 min on 3 consecutive spec-touching `pull_request` runs. ## Tests - `node scripts/partition-test-shards.mjs --self-test`: OK. Bins are 1551.30–1553.28 s, within the 1455.81 s density cap plus its 111.86 s declaration-closure allowance. Whole legs are 291/1553/1553/1551/1553/1553/1074/1076 s, all within the 1868 s cap, and `file-level slices: @objectstack/cli x3`. - `node scripts/measure-test-shard-timings.mjs --self-test`, `node scripts/ci/plan-test-core-shards.mjs --self-test`, `node scripts/ci/select-shard-timings-run.mjs --self-test`, `node scripts/nightly-tiers.mjs --self-test`, `node scripts/check-test-completeness.mjs --self-test`: all exit 0. - No test file changes. The CLI's sequencer takes any count: slice 1/3 holds ~204 files, so the five declaration readers fit. ## Acceptance notes - **The CLI shard stays the slowest, by ~1.5 min.** 4 and 5 slices would bring it to runtime's own floor (~15 min). Modelled the same way, the slowest CLI shard's step is ~933 s at 4 and ~849 s at 5. Pin 3c refuses both because it reads a slice as running beside the other packages' heaviest task rather than after the shard's whole leg. **Possible follow-up (not filed, seat's call):** re-ground pin 3c on the step a slice adds (whole leg / `WHOLE_LEG_WALL_DIVISOR` + slice) and raise the CLI to 5, for a predicted slowest median of ~15 min. 6 and 8 put two slices on one shard, so the planner runs the CLI whole; 7 lands slices on the spec and runtime shards. Changing what pin 3c measures changes a ruled gate (ruling record 6634084947), so it goes to the maintainer. - **Not in this lever:** running the slice concurrently with the whole leg (`ci.yml`, Tier H); weight-aware CLI slicing in `packages/cli/vitest.config.ts` (the sha1 split is 1.11× uneven at 3); splitting `runtime:test` (~840 s) and `spec:test:repo` (~746 s), which set the floor once the CLI is no longer the pole. - **Placement moves** for neighbours: the slice shards change from 1/8 and 4/8 to 1/8, 7/8 and 8/8, and 4/8 becomes a whole-package shard at 1551 s. `objectql` and `plugin-security` still sit apart (pin 3d). - **Run summaries:** for four days the retained 2-slice summaries decode through `PREVIOUS_FILE_SHARDED_PACKAGES` (now 2), as the map's docblock prescribes. ## Verification ```bash node scripts/partition-test-shards.mjs --self-test # self-test OK (... -> 81 shard items, 8 shards, ... whole legs 291/1553/1553/1551/1553/1553/1074/1076s within 1868s, file-level slices: @objectstack/cli x3) ``` ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (not touched) - [x] 2. Every name I added uses the casing for its kind. (no names added) - [x] 3. No formatter ran; `pnpm lint` passes. (CI-owned) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset. (publishes nothing: `skip-changeset`) - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (not touched) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. - [x] The *Before Submitting* list in CONTRIBUTING.md is done. --- _Generated by [Claude Code](https://claude.ai/code/session_01QsR2jUHzPuDW81yN7XfvLG)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent d5fa237 commit 5ea7789

1 file changed

Lines changed: 31 additions & 3 deletions

File tree

‎scripts/partition-test-shards.mjs‎

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -405,6 +405,34 @@ export function timingModeNote(event) {
405405
// density cap equals the 8-shard mean; the slice that reads built declarations
406406
// is admitted by closureAllowance() (#29777 ruling B).
407407
//
408+
// RAISED TO 3 (#25870), on the same dataset, inside that band. Both counts pass
409+
// pin 3c at 1432.04s (2's slice 716.02s is within 773.27s; 3's n - 1 slice,
410+
// 716.02s, is above the 699.63s LOWER edge), so the band admits the choice and
411+
// the shard wall makes it. A slice runs as its shard's serial leg AFTER the
412+
// whole-package leg (ci.yml, "Run this shard's tests"), so it adds to that
413+
// shard's wall at full weight while the whole leg runs four wide. Measured over
414+
// the 79 full-plan runs (57 merge_group, 22 schedule) on 2026-11-04..08 where
415+
// all eight shards ran green, the two slice shards were the slowest:
416+
//
417+
// Test Core (4/8) median job 19.4 min 313s whole leg + 873s cli 2/2
418+
// Test Core (1/8) median job 16.8 min 43s whole leg + 766s cli 1/2
419+
// the other six median job 9.3-14.8 min, no slice
420+
//
421+
// (leg medians from 112 shard logs of 14 of those runs). The CLI measured
422+
// ~1640s alone there, 766 + 873, against its 1432.04s weight -- so at the next
423+
// refresh 2 leaves the band ("Raise it to 3") and 3 stays inside it. Cut 3
424+
// ways the slices land on three shards that hold neither spec's nor runtime's
425+
// suite (shards 1, 7 and 8 on the full list: the reader slice beside 291s of
426+
// whole packages, the other two beside ~1075s), and the slowest CLI shard's
427+
// tests step, modelled as whole sum / WHOLE_LEG_WALL_DIVISOR + slice with the
428+
// slices cut from measured per-file times, falls from ~1164s to ~977s (the
429+
// model reads ~10% high on today's 4/8). Its job median goes ~19.4 -> ~16.4
430+
// min; that shard stays the slowest, ~1.5 min above runtime's 3/8.
431+
// 4 and 5 would close that gap and pin 3c refuses both (4: "Lower it to 3";
432+
// 5: "Lower it to 4"), because it reads a slice beside the other packages' heaviest task rather than
433+
// after the shard's whole leg; re-grounding the pin is a ruled gate's change,
434+
// not this count's.
435+
//
408436
// ⛔ THE COUNT IS CONFIGURED; WHETHER A RUN SLICES IS DECIDED BY THAT RUN. A
409437
// pull_request or merge_group run splits its own affected set, so planShards()
410438
// asks, of the run's own list, whether a configured package is that run's
@@ -425,7 +453,7 @@ export function timingModeNote(event) {
425453
// a change to what that generator can DECODE, which is why the map it replaced
426454
// is kept below.
427455
export const FILE_SHARDED_PACKAGES = Object.freeze({
428-
'@objectstack/cli': 2,
456+
'@objectstack/cli': 3,
429457
});
430458

431459
// THE MAP AS IT STOOD BEFORE ITS LAST CHANGE, read only by the generator's
@@ -454,9 +482,9 @@ export const FILE_SHARDED_PACKAGES = Object.freeze({
454482
// retained summary predates the change and this map decodes nothing.
455483
// The outgoing map of #22075's third round was empty -- the runs before it
456484
// carried no slice. #25870 raised the CLI from 3 slices to 4; #29777 lowered
457-
// it from 4 to 2, so this holds 4.
485+
// it from 4 to 2; #25870 raised it again from 2 to 3, so this holds 2.
458486
export const PREVIOUS_FILE_SHARDED_PACKAGES = Object.freeze({
459-
'@objectstack/cli': 4,
487+
'@objectstack/cli': 2,
460488
});
461489

462490
// ── THE SLICE THAT READS BUILT DECLARATIONS CARRIES THEIR CLOSURE (#25870) ──

0 commit comments

Comments
 (0)