Repository navigation
Commit a798ae8
ci(nightly-tiers): four shards — the showcase smoke alone on one, the CLI cut across the rest (#28613)
<!-- claude-projects-attribution: {"github_login":"marchtian"} -->
_Requested by **田騏** · [project
thread](https://claude.ai/code/project/chan_01VmuUhXU9SSUzqJYyXSmYSi?thread=cmsg_01VmuUhXU9SSUzqJYyXSmYSi9paTx1Lu8ENYwwssthfK3h)_
Fixes #28580
Clause-②: no
Metadata-protocol: none
## 维护者速读
- **改了什么**:夜间分层测试从 2 个分片改为 4 个。showcase 浏览器冒烟独占一个分片,CLI 的 e2e 层切成 3
片,放在另外 3 个分片上。分片器新增文档键 `ownShard`,用来表达“这个包独占一个分片”。
- **为什么改**:showcase 的分片要先构建 Console、安装 Chromium,再构建自己的依赖闭包,约 18
分钟,而它在数据集里只按 48 秒计权,所以总被放到最重的 CLI 切片旁边。11-01 这个 1 号分片需要约 52 分钟,在 45
分钟上限处被取消,没有任何测试失败。
- **风险与回滚**:只改夜间 workflow 的分片布局。没有测试被改名、跳过或删除,两个层级都没有收窄。回滚就是 revert 本
PR,回到 2 个分片。
- **先看**:① `.github/workflows/test-nightly-tiers.yml` 的矩阵和头注释里的数字;②
`scripts/partition-test-shards.mjs` 的 `readOwnShard` 和 `planRun`;③
`scripts/nightly-tiers.mjs` 的 `OWN_SHARD_PACKAGES`。
- **两分钟验证**:`node scripts/nightly-tiers.mjs --packages > /tmp/t.json &&
node scripts/partition-test-shards.mjs /tmp/t.json --shard 4/4`,预期
stdout 只打印 `@objectstack/example-showcase`,日志有一行 `own shard: ... alone
on shard 4/4`。
- **需要你做的**:本 PR 改到 `.github/workflows/**`,属于 Tier H
受管面,需要授权审批人批准,之后由席位落地。
## What changes
The nightly tiers job goes from two shards to four. Shards 1–3 each
carry one slice of `@objectstack/cli`'s e2e tier, and shard 4 carries
the showcase browser smoke alone. The timeout stays at 45 minutes.
- `scripts/partition-test-shards.mjs` reads a new document key,
`ownShard`. Each package it names runs whole and alone on one of the
run's last shards. Everything else is planned over the shards left, so a
`sliceAcrossShards` package is cut N − k ways. The key is checked
strictly: it must be an array of distinct listed names, none also in
`sliceAcrossShards`, with fewer reservations than shards. A reserved
package that the run excludes is refused, because its shard would run
nothing. The balance line grades only the shared shards, since a
reserved shard is placed by the document and not by weight.
- `scripts/nightly-tiers.mjs --packages` puts every tier-owning package
in exactly one of the two keys. `OWN_SHARD_PACKAGES` (today only the
showcase) goes to `ownShard`, and the rest go to `sliceAcrossShards`.
- `.github/workflows/test-nightly-tiers.yml`: `matrix.shard` is `[1, 2,
3, 4]`, the job name is `/4`, the partitioner runs with `--shard k/4`,
and the reports artifact is `-of-4`. That is four places. The header
said three, and the artifact name was the one it missed. Two header
statements were false, and the measurements below correct them: "≈ 13
min a shard" and "the closure's build tasks replay".
- New self-test pins (live tree): the workflow's preparation step must
name exactly the `ownShard` packages. The CLI must reach the N − k
shared shards as one distinct slice each, with each reserved package
alone on its own shard. A new battery, `ownShard`, has 12 cases:
identity when the key is absent, 7 refusals, placement at 3 and 4
shards, the control (without the key the showcase rides beside CLI 1/3),
and refusal of an excluded package.
## Why four shards, by measurement
Run 41277375892 (2026-11-01, `890c2a6d`), shard 1 (job 124762188633). It
carried `@objectstack/cli 1/2` and `@objectstack/example-showcase`:
| segment | wall |
|:--|--:|
| setup through "Compute this shard's package set" | 0m52s |
| "Prepare the showcase browser smoke" (Console build + Chromium) |
9m39s |
| CLI 1/2 leg (closure build 6m00s + vitest 1631s) | 33m13s |
| showcase leg | killed 1m43s in, still building its closure |
Run 41009107101 is the reference for a complete showcase leg: it took
8m14s (closure build 5m33s, then 160s of vitest). Shard 1 therefore
needed ≈ 52 min against 45. Shard 2 (CLI 2/2 alone) ran 25m19s. Shard 1
was killed at 45 minutes on 10-28 and 11-01. On 10-29, 10-30 and 10-31
it finished red, in 32–41 min.
Options, each priced from that night's own numbers:
| layout | slowest shard | note |
|:--|--:|:--|
| today: 2 shards, showcase beside CLI 1/2 | ≈ 52 min | over the budget
|
| (a) 2 shards, showcase moved beside CLI 2/2 | ≈ 44 min | about a
minute of headroom; which slice is light depends on file hashes |
| (b) 3 shards, CLI ×3, showcase by weight | ≈ 46 min | LPT breaks the
tie by name and puts the showcase beside CLI 1/3, the heaviest slice |
| 3 shards, CLI ×2, showcase alone | 34.6 min (measured CLI 1/2 shard) |
that shard rose from 31.0 to 34.6 min over 10-28 to 11-01 |
| **4 shards, CLI ×3, showcase alone (this PR)** | **≈ 29 min** | others
≈ 22 / 18 / 19 min; about 16 min of headroom; the whole run is ≈ 30 min,
inside the hour |
| (c) raise the budget, 2 shards | ≈ 52 min plus growth | the whole run
is no longer bounded by the hour |
The CLI rows at three slices are predicted, not measured. I reproduced
vitest's own `--shard` order (sha1 of each file's root-relative path,
cut by file count). It puts exactly the 42 integration files that
11-01's shard 1 ran into slice 1/2. I then summed that night's per-file
durations: slices of 3671 / 2551 / 1759 s of file time, at the 2.86×
parallelism measured that night, plus 6 min of closure build and ≈ 1.5
min of setup.
<details><summary>Per-night shard readings (job logs)</summary>
```
night shard-1 items prepare CLI 1/2 vitest CLI 2/2 vitest shard-1 result
10-26 cli 1/2 - 1141s 788s success 24m42s
10-27 cli 1/2 - 1452s 840s success 30m46s
10-28 cli 1/2 + showcase 8m30s 1476s 1080s cancelled at 45m
10-29 cli 1/2 + showcase 0m31s 1542s 925s failure (a test) 32m35s
10-30 cli 1/2 + showcase 0m21s 1574s 902s failure (showcase) 40m55s
10-31 cli 1/2 + showcase 5m18s 1595s 1127s failure (a test) 38m25s
11-01 cli 1/2 + showcase 9m39s 1631s 1127s cancelled at 45m
```
Same 82 files, per-file time summed: 5532 s on 10-26, 7907 s on 11-01.
</details>
## Tests
Self-tests and gates were run at `a9fdb19a1d`. `origin/main` had not
moved since the branch point (`cd065d3698`).
- `node scripts/partition-test-shards.mjs --self-test`: exit 0. The
roster is 18 batteries; the nightly battery has 17 cases and `ownShard`
has 12.
- `node scripts/nightly-tiers.mjs --self-test`: exit 0, `48 cases pass`
(floor 47 → 48).
- `node scripts/ci/plan-test-core-shards.mjs --self-test`: exit 0. This
is the Test Core caller of `planRun()`, and it is unchanged.
- The preparation-step pin was checked in both directions with a
throwaway probe against the live file: live passes, a renamed package
goes red, a removed step goes red.
- Live partition at 4: shards 1–3 print `@objectstack/cli 1/3`, `2/3`
and `3/3`, and shard 4 prints `@objectstack/example-showcase`.
## Acceptance notes
- **No tier narrowed, no test touched.** The same 86 CLI e2e files and 2
showcase live files run every night, under the same
`OS_TEST_TIERS=nightly`.
- **The real-runner reading is this PR's own Nightly Tiers run.** The
workflow's `pull_request` paths trigger fires on this diff. The ≈ 29 min
figure above is a prediction until that run reports.
- Raise the matrix when the slowest shard's test step nears 35 minutes.
The document re-slices the CLI at the new count with no edit to it.
- Observed, not filed: the CLI e2e tier's per-file time rose 43% over 82
identical files between 10-26 and 11-01 (for example,
`jsx-gate-manifest-notice.e2e.test.ts` went from 208 s to 300 s). This
is the trend that eats any shard's headroom.
- Observed, not filed: the CLI legs replayed 0 of their ~62 build tasks
from the restored Build Core turbo cache on every night measured. The
header now says so; the cause is not diagnosed here.
- Observed, not filed: vitest's `--shard` cuts by file count, so slices
are unequal (4661 vs 3320 s of file time at two). A duration-aware
sequencer in `packages/cli/vitest.config.ts` would flatten that, but it
is outside this card's file surface.
- Pre-existing and left alone: the partitioner's log line `(N of them
file-level slices …)` prints items minus packages, which reads 2 for
three slices.
## Verification
<details><summary>Derived gate list (dispatch-gates --commands) and
--ran reconciliation</summary>
51 families were derived from 3 paths against merge base `cd065d369`.
All 51 were run and every one exited 0, `pnpm check:pm-dispatch-gates`
included (fast tier: 1937 cases, 373 s). The `--ran` reconciliation
reported: `51 derived famil(ies) accounted for — 51 run, 0 NOT-MEASURED
(a DERIVED zero — all 51 recorded an exit code and none of them is 3)`.
</details>
Session: https://claude.ai/code/session_017ihL2gLghEqeDKTbDTC7Zd
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: objectstack-fleet[bot] <332303061+objectstack-fleet[bot]@users.noreply.github.com>1 parent 68a2a21 commit a798ae8
3 files changed
Lines changed: 376 additions & 96 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
21 | 21 | | |
22 | | - | |
23 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
24 | 26 | | |
25 | | - | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
43 | 37 | | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
51 | 67 | | |
52 | 68 | | |
53 | 69 | | |
54 | 70 | | |
55 | | - | |
| 71 | + | |
56 | 72 | | |
57 | 73 | | |
58 | 74 | | |
| |||
148 | 164 | | |
149 | 165 | | |
150 | 166 | | |
151 | | - | |
| 167 | + | |
152 | 168 | | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
157 | 173 | | |
158 | 174 | | |
159 | | - | |
| 175 | + | |
160 | 176 | | |
161 | 177 | | |
162 | 178 | | |
163 | | - | |
| 179 | + | |
164 | 180 | | |
165 | 181 | | |
166 | 182 | | |
| |||
206 | 222 | | |
207 | 223 | | |
208 | 224 | | |
209 | | - | |
| 225 | + | |
210 | 226 | | |
211 | 227 | | |
212 | 228 | | |
| |||
216 | 232 | | |
217 | 233 | | |
218 | 234 | | |
219 | | - | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
220 | 239 | | |
221 | 240 | | |
222 | 241 | | |
223 | | - | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
228 | | - | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
229 | 247 | | |
230 | 248 | | |
231 | 249 | | |
| |||
253 | 271 | | |
254 | 272 | | |
255 | 273 | | |
256 | | - | |
257 | | - | |
258 | | - | |
259 | | - | |
260 | | - | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
261 | 278 | | |
262 | 279 | | |
263 | 280 | | |
| |||
332 | 349 | | |
333 | 350 | | |
334 | 351 | | |
335 | | - | |
| 352 | + | |
336 | 353 | | |
337 | 354 | | |
338 | 355 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
70 | | - | |
| 70 | + | |
71 | 71 | | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
| |||
86 | 87 | | |
87 | 88 | | |
88 | 89 | | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
97 | 103 | | |
98 | 104 | | |
99 | 105 | | |
| |||
217 | 223 | | |
218 | 224 | | |
219 | 225 | | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
220 | 243 | | |
221 | 244 | | |
222 | 245 | | |
223 | 246 | | |
224 | 247 | | |
225 | 248 | | |
226 | 249 | | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
231 | 256 | | |
232 | | - | |
| 257 | + | |
233 | 258 | | |
234 | | - | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
235 | 265 | | |
236 | 266 | | |
237 | 267 | | |
| |||
497 | 527 | | |
498 | 528 | | |
499 | 529 | | |
500 | | - | |
501 | | - | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
502 | 541 | | |
503 | 542 | | |
504 | 543 | | |
| |||
536 | 575 | | |
537 | 576 | | |
538 | 577 | | |
539 | | - | |
| 578 | + | |
540 | 579 | | |
541 | 580 | | |
542 | 581 | | |
| |||
0 commit comments