Repository navigation
Commit 5ea7789
ci(test-shards): cut the CLI into 3 Test Core slices, the count the shard wall picks inside pin 3c's band (#31543)
<!-- claude-projects-attribution: {"github_login":"zhuangjianguo"} -->
_Requested by **Steedos** · [project
thread](https://claude.ai/code/project/chan_018ETKaNwMZCNRYD7D8Q9Ubb?thread=cmsg_018ETKaNwMZCNRYD7D8Q9Ubb7qV7tjQEZS5b86zyDZ9H16)_
Fixes #25870
Clause-②: no
Metadata-protocol: none
## 维护者速读
- **改了什么**:Test Core 把 `@objectstack/cli` 的测试从切 2 片改为切 3
片(`scripts/partition-test-shards.mjs` 一行配置 + 上一版映射 + 推导注释),不动
ci.yml、不动任何测试。
- **为什么改**:CLI 的切片在所在分片里排在整包测试之后串行跑,79 次完整运行里 4/8 分片中位 19.4 分钟、1/8 16.8
分钟,其余六片 9.3–14.8 分钟;每个核心 PR 和合并队列条目都在等最慢那片。
- **风险与回滚**:只改分片计划,测试集合不变(三片之并即全部 CLI 文件);多一片多一次 CLI
闭包构建(远程缓存命中,约数秒)。回滚即 revert 本 PR。
- **先看**:`scripts/partition-test-shards.mjs` 的
`FILE_SHARDED_PACKAGES`(cli: 3)与其上方新增的 "RAISED TO 3" 推导段。
- **两分钟验证**:`node scripts/partition-test-shards.mjs --self-test`,预期末行
`self-test OK … file-level slices: @objectstack/cli x3`,whole legs
`291/1553/1553/1551/1553/1553/1074/1076s`。
- **需要你做的**:无,席位落地。
## What changes
| File | Change |
|---|---|
| `scripts/partition-test-shards.mjs` |
`FILE_SHARDED_PACKAGES['@objectstack/cli']` 2 → 3.
`PREVIOUS_FILE_SHARDED_PACKAGES` 4 → 2 (the outgoing count, so the
timings generator still decodes the 2-slice run summaries retained for
the next four days). A "RAISED TO 3" paragraph in the derivation
docblock with the measurement below. Nothing else: pin 3c, its band, the
density cap, pin 3d, `ci.yml` and the CLI's vitest sequencer are
untouched. |
## Why
A CLI slice runs as its shard's serial leg **after** the whole-package
leg (`ci.yml`, "Run this shard's tests"), so it adds to that shard's
wall at full weight while whole packages run four wide. With the CLI in
2 slices, the two slice shards are the slowest in the workflow.
<details><summary>Measured: per-shard job wall on 79 full-plan runs
(origin/main, plan unchanged since 2026-11-04)</summary>
Population: every successful `CI` run on `merge_group` (newest 100,
2026-11-08) and `schedule` created after 2026-11-04T16:00Z; kept the 79
where all eight `Test Core (k/8)` jobs ran and passed (57 merge_group,
22 schedule). Wall is the job's `completed_at − started_at`.
| shard | median (min) | p90 | closure build median | tests step median
|
|---|---|---|---|---|
| 1/8 | 16.8 | 21.4 | 0.1 | 13.5 |
| 2/8 | 12.8 | 16.9 | 0.1 | 10.9 |
| 3/8 | 14.8 | 18.2 | 0.1 | 13.0 |
| **4/8** | **19.4** | 25.4 | 0.1 | 17.6 |
| 5/8 | 9.3 | 13.8 | 0.1 | 7.6 |
| 6/8 | 10.0 | 14.7 | 0.1 | 8.4 |
| 7/8 | 10.2 | 15.3 | 0.1 | 8.5 |
| 8/8 | 9.3 | 12.8 | 0.1 | 7.5 |
4/8 was the slowest in 52 of 79 runs, 1/8 in 18. Leg medians from the
full logs of all eight shards of 14 of those runs (112 logs): 4/8 = 313
s whole leg + 873 s `cli 2/2`; 1/8 = 43 s + 766 s `cli 1/2`; 3/8 = 762 s
(`runtime:test` alone ~840 s); 2/8 = 698 s (`spec:test:repo` alone ~746
s). The closure build is 0.1–0.4 min median on every shard (remote
cache), so it is not what makes these shards long.
Control: with the CLI in 4 slices (28 `schedule` runs, 2026-10-28T02:30Z
to 11-03T08:13Z) the slowest shard median was 12.1 min and the
run-slowest median 16.7 min; with 2 slices (22 `schedule` runs since
11-04) they are 18.1 and 19.2 min.
</details>
**Why 3, and not more.** Pin 3c, unchanged, judges each count on the
committed dataset (CLI 1432.04 s; the heaviest other serial task is
`metadata-protocol` at 566.50 s, so the limit is 736.45 s, RAISE past
773.27 s, LOWER when the n − 1 slice is at or under 699.63 s):
| n | slice | n − 1 slice | pin 3c |
|---|---|---|---|
| 2 | 716.02 s | 1432.04 s | passes |
| **3** | 477.35 s | 716.02 s | **passes** |
| 4 | 358.01 s | 477.35 s | refused, "Lower it to 3" |
| 5 | 286.41 s | 358.01 s | refused, "Lower it to 4" |
At the CLI's measured ~1640 s (766 + 873, what the next refresh should
read), 2 leaves the band ("Raise it to 3") and 3 still passes. So 3 is
the one count above 2 that holds on both today's dataset and the next
refresh.
## Predicted vs measured shard wall
Model: tests step ≈ whole-package sum / `WHOLE_LEG_WALL_DIVISOR` (2.9) +
slice. The slices are cut the way the CLI's sequencer cuts them
(declaration readers first, then sha1 order, vitest's equal-count
ranges) from the per-file times in both slice logs of run 42478508299.
That model reproduces the measured `cli 1/2` at 770 s against 776.5 s.
Placement comes from the real `planRun()` on the full list, which
reproduces CI's current plan byte for byte.
| | today, 2 slices | this PR, 3 slices |
|---|---|---|
| slice shards | 1/8 (64 s whole + 770 s), 4/8 (846 s whole + 874 s) |
1/8 (291 s + 460 s), 7/8 (1074 s + 575 s), 8/8 (1076 s + 609 s) |
| slowest CLI shard, modelled tests step | ~1164 s | ~977 s |
| slowest shard, job median | **19.4 min measured** (4/8) | **~16.4 min
predicted** (the 8/8 CLI shard), ~1.5 min above 3/8's 14.8 |
The model reads ~10% high on today's 4/8 (1164 s modelled against 1056 s
measured tests step median), and the prediction is scaled by the same
factor. **Measured after landing:** the seat reads the card's
acceptance, the slowest Test Core shard ≤ ~20 min on 3 consecutive
spec-touching `pull_request` runs.
## Tests
- `node scripts/partition-test-shards.mjs --self-test`: OK. Bins are
1551.30–1553.28 s, within the 1455.81 s density cap plus its 111.86 s
declaration-closure allowance. Whole legs are
291/1553/1553/1551/1553/1553/1074/1076 s, all within the 1868 s cap, and
`file-level slices: @objectstack/cli x3`.
- `node scripts/measure-test-shard-timings.mjs --self-test`, `node
scripts/ci/plan-test-core-shards.mjs --self-test`, `node
scripts/ci/select-shard-timings-run.mjs --self-test`, `node
scripts/nightly-tiers.mjs --self-test`, `node
scripts/check-test-completeness.mjs --self-test`: all exit 0.
- No test file changes. The CLI's sequencer takes any count: slice 1/3
holds ~204 files, so the five declaration readers fit.
## Acceptance notes
- **The CLI shard stays the slowest, by ~1.5 min.** 4 and 5 slices would
bring it to runtime's own floor (~15 min). Modelled the same way, the
slowest CLI shard's step is ~933 s at 4 and ~849 s at 5. Pin 3c refuses
both because it reads a slice as running beside the other packages'
heaviest task rather than after the shard's whole leg. **Possible
follow-up (not filed, seat's call):** re-ground pin 3c on the step a
slice adds (whole leg / `WHOLE_LEG_WALL_DIVISOR` + slice) and raise the
CLI to 5, for a predicted slowest median of ~15 min. 6 and 8 put two
slices on one shard, so the planner runs the CLI whole; 7 lands slices
on the spec and runtime shards. Changing what pin 3c measures changes a
ruled gate (ruling record 6634084947), so it goes to the maintainer.
- **Not in this lever:** running the slice concurrently with the whole
leg (`ci.yml`, Tier H); weight-aware CLI slicing in
`packages/cli/vitest.config.ts` (the sha1 split is 1.11× uneven at 3);
splitting `runtime:test` (~840 s) and `spec:test:repo` (~746 s), which
set the floor once the CLI is no longer the pole.
- **Placement moves** for neighbours: the slice shards change from 1/8
and 4/8 to 1/8, 7/8 and 8/8, and 4/8 becomes a whole-package shard at
1551 s. `objectql` and `plugin-security` still sit apart (pin 3d).
- **Run summaries:** for four days the retained 2-slice summaries decode
through `PREVIOUS_FILE_SHARDED_PACKAGES` (now 2), as the map's docblock
prescribes.
## Verification
```bash
node scripts/partition-test-shards.mjs --self-test
# self-test OK (... -> 81 shard items, 8 shards, ... whole legs 291/1553/1553/1551/1553/1553/1074/1076s within 1868s, file-level slices: @objectstack/cli x3)
```
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(not touched)
- [x] 2. Every name I added uses the casing for its kind. (no names
added)
- [x] 3. No formatter ran; `pnpm lint` passes. (CI-owned)
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset. (publishes
nothing: `skip-changeset`)
- [x] 6. `packages/spec` touched: its generated artifacts are
regenerated and committed. (not touched)
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English.
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01QsR2jUHzPuDW81yN7XfvLG)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent d5fa237 commit 5ea7789
1 file changed
Lines changed: 31 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
408 | 436 | | |
409 | 437 | | |
410 | 438 | | |
| |||
425 | 453 | | |
426 | 454 | | |
427 | 455 | | |
428 | | - | |
| 456 | + | |
429 | 457 | | |
430 | 458 | | |
431 | 459 | | |
| |||
454 | 482 | | |
455 | 483 | | |
456 | 484 | | |
457 | | - | |
| 485 | + | |
458 | 486 | | |
459 | | - | |
| 487 | + | |
460 | 488 | | |
461 | 489 | | |
462 | 490 | | |
| |||
0 commit comments