Repository navigation
Commit 3bd4bf6
ci(ui): port the Test UI per-file timings reporter and its weekly dataset refresh (#29588)
<!-- claude-projects-attribution: {"github_login":"os-warren"} -->
_Requested by **Warren** · [project
thread](https://claude.ai/code/project/chan_01U5RdnAAmkSQmZbqh6FtqZ8?thread=cmsg_01U5RdnAAmkSQmZbqh6FtqZ8AE4cn5ty8rrc9mVmocckfL)_
Fixes #29517
Clause-②: no
Metadata-protocol: none
## 维护者速读
- 改了什么:`ci.yml` 的 Test UI 全量腿加挂逐文件耗时 reporter 并在全量电池事件上传每片报告;新增每周一的
`ui-test-timings-refresh.yml`,把最新合格运行的报告合并成
`ui/scripts/test-shard-timings.json` 并以 fleet App 开 PR。
- 为什么改:ui/ 并入时 reporter 与 refresh 都没搬过来,该数据集自 2026-10-19 冻结,CI 读不到 Test
UI 各文件耗时,#25911 试点与 #29469 的四个折叠无法在 CI 复核,分片也按旧数据切。
- 风险与回滚:只报不判(按裁决无阈值、无新红门);上传步 `continue-on-error`,缺报告只是
warning,不会让分片转红;回滚即 revert 本 PR。
- 先看:`.github/workflows/ci.yml`(test-ui 的 `Run the ui tests`
与新的上传步)、`.github/workflows/ui-test-timings-refresh.yml`、`ui/scripts/test-file-timings.mjs`(`shardJobName`
改为 `Test UI (N/M)`)。
- 两分钟验证:`cd ui && pnpm exec vitest run
scripts/__tests__/test-file-timings.test.ts`,预期 15 passed。
- 需要你做的:受管面(`.github/workflows/**`,Tier H),请你审批并合并;合并后首个全量电池跑完,在 Actions
里手动触发一次 UI Test Timings Refresh 即可拿到首个刷新 PR。
## What changes
- `.github/workflows/ci.yml`, job `test-ui`:
- `Run the ui tests (shard N/16)`: on the whole-suite branch
(`MODE=full`: `push`, `schedule`, `workflow_dispatch`, and a pull
request that failed open) it now also passes
`--reporter="${GITHUB_WORKSPACE}/ui/scripts/test-file-timings.mjs"`
beside `default` and `github-actions`, and the step sets
`OBJECTUI_TEST_TIMINGS_OUT` to
`RUNNER_TEMP/test-file-timings/shard-N.json`. The affected speed passes
no reporter.
- New step `Publish this shard's per-file test timings`:
`actions/upload-artifact@v7`, name `test-file-timings-N`, only when the
event is neither `pull_request` nor `merge_group` (the file's own
deny-list for the full battery), `continue-on-error: true`,
`if-no-files-found: warn`, retention 7 days on `schedule` and 2
otherwise.
- The comment that said the reporter and refresh were not ported is
replaced by one that says what runs and why.
- `.github/workflows/ui-test-timings-refresh.yml` (new), job `Regenerate
the per-file test timings`: Mondays 05:47 UTC, `workflow_dispatch`, and
`pull_request` on its own file and the three scripts it loads. It runs
`test-file-timings.mjs select --shards 16 --limit 6`, downloads each
run's `test-file-timings-*` artifacts with `gh run download`, drops a
run whose download fails or comes back with fewer than 16 reports, runs
`merge` into `ui/scripts/test-shard-timings.json`, and when the bytes
changed pushes `claude/ui-test-timings-refresh-RUN` and opens the PR
with `skip-changeset` as the fleet App. On `pull_request` every outward
act is a dry run. With no eligible run (the state until a full-battery
run uploads reports) it reads NOT MEASURED, warns, and stays green. It
mirrors `shard-timings-refresh.yml` (Test Core, #28978) in shape,
without that lane's ceilings, coverage accumulation and bin pins, which
the ui dataset does not have.
- `ui/scripts/test-file-timings.mjs`: `shardJobName` now spells the root
job, `Test UI (N/M)` (it spelled objectui's `Test (shard N/M)`, so the
selector could never find a shard here). `generated_by` names the
workflow that exists. Header and one comment updated to the root paths.
- `ui/scripts/__tests__/test-file-timings.test.ts`: the pins its
retirement note promised back, now against the root `ci.yml` (parsed)
and the refresh: the shard job name, the reporter on the whole suite and
not on the affected speed, the upload name and path and its report-only
shape, the refresh's `--shards` against the matrix width, its download
pattern, and that the workflow `generated_by` names exists.
- `scripts/pm/dispatch-gates.self-test.mjs`: the live population-marker
census names every declaration by file and line, and its comment asks
the author of a new one to add the row. The refresh's
`no-check-families` declaration is the tenth row (count 9 to 10).
Outside the claim's file surface, and the only file outside it.
## Why
`ui/scripts/test-shard-timings.json` is what the Test UI sequencer
splits on. On `main` `399df5bb` its `measured.at` is `2026-10-19T…`, its
`generated_by` names `.github/workflows/test-shard-timings-refresh.yml`,
which does not exist in this repository, and the `ci.yml` comment above
`Run the ui tests` said the reporter and refresh were not ported. The
writer, selector and generator were all present in
`ui/scripts/test-file-timings.mjs`; only the CI wiring was missing, and
the selector's job name still spelled objectui's.
## Tests
- `cd ui && pnpm exec vitest run
scripts/__tests__/test-file-timings.test.ts
scripts/__tests__/select-affected-tests-wiring.test.ts`: 2 files, 25
tests passed (the wiring test runs the edited step as written).
- The writer under vitest 4.1.12, through the exact CI spelling:
`GITHUB_WORKSPACE=$PWD OBJECTUI_TEST_TIMINGS_OUT=… pnpm --dir ui run
test --shard=1/1 --reporter=default --reporter=github-actions
--reporter="$PWD/ui/scripts/test-file-timings.mjs"
scripts/__tests__/test-file-timings.test.ts` exited 0 and wrote
`{"schema":"objectui.test-file-timings/1","shard":"1/1",…,"files":[{"file":"scripts/__tests__/test-file-timings.test.ts","project":"unit",…}]}`.
- Writer to generator: two files' report fed to `merge --runs-file …
--out COPY --summary …` wrote a valid dataset (the reader's own
`datasetWeights` check runs inside `buildDataset`) carrying the new
`generated_by`.
- No ablation: the diff changes CI wiring and a name, not a tested
behaviour of a package.
## Acceptance notes
- The committed `ui/scripts/test-shard-timings.json` still carries the
old `generated_by` string. It is generator output and is never
hand-edited; the first refresh PR rewrites it from the corrected
generator, together with a measurement date after this card. That
refresh PR is what closes the card's "Done when".
- The reporter is spelled from the workspace root, not as
`./scripts/test-file-timings.mjs` relative to `ui/` (objectui's
spelling): `check-self-test-wired` keys a `./scripts/` path in a
workflow to the root `scripts/`, so the relative spelling minted a key
with no file behind it and redded its `--self-test` (measured on the
first gate pass).
- The refresh carries the `dispatch-gates: no-check-families`
declaration: it is path-filtered and runs a generator, not a named
check; the scripts it loads are pinned by Test UI's
`test-file-timings.test.ts`.
- Push runs feed the dataset as well as scheduled ones: unlike Test
Core, a `push` run that starts Test UI runs the whole ui suite
(`mode=full`), so it is a measurement. The selector already reads both.
- The refresh's `pull_request` leg on this PR will read NOT MEASURED (no
run has uploaded reports yet) and stay green.
- `ui/scripts/dependabot-merge-gate.mjs` lists `Regenerate the per-file
test timings` among its optional contexts; the new job carries that
name. That file is otherwise not touched.
## Verification
All readings at head `82d9d267` (branch off `399df5bb`, then
`origin/main` merged after PR #29556 landed, since that PR also edits
`scripts/pm/dispatch-gates.self-test.mjs`).
<details><summary>dispatch-gates: 69 derived, 69 exit 0</summary>
- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack`: 69 commands, each run with its exit code
captured before any pipe.
- 68 exited 0 on the first pass. `pnpm --dir ui exec vitest run
scripts/` exited 1 on the first pass after the merge (5 cases in
`check-doc-json-fences.test.ts`, `TypeError: fn is not a function`
inside zod, a file this diff does not touch); after re-running the
seven-package `@objectstack` build the ui jobs run (turbo replay, 1 s
under the lock) it exited 0: `Test Files 170 passed | 2 skipped (172)`,
`Tests 4976 passed | 2 skipped (4978)`. Read as stale build state after
the merge.
- `pnpm check:pm-dispatch-gates` exit 0: `dispatch-gates self-test (fast
tier): 1945 cases pass`, including `the live tree carries the ten
declarations measured for this census`.
- `dispatch-gates --ran`: "69 derived famil(ies) accounted for — 69 run,
0 NOT-MEASURED", exit 0.
- Two reds on the first gate pass (head `b117998e`) shaped the final
diff: `check-self-test-wired --self-test` (the `./scripts/` reporter
spelling) and the dispatch-gates coverage guard (the refresh's missing
`no-check-families` declaration). Both are described under Acceptance
notes.
</details>
<details><summary>outside the derived set</summary>
- `pnpm --dir ui run type-check:scripts` (what Type Check UI runs for
`ui/scripts`): exit 0, 0 errors.
- `pnpm --dir ui exec eslint scripts/test-file-timings.mjs
scripts/__tests__/test-file-timings.test.ts`: exit 0.
- Both workflow files parse with the `yaml` package the ui pins use.
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(not touched)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes. (eslint on the two ui
files: exit 0; repo-wide lint is CI's)
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset. (publishes
nothing: CI wiring and a private ui script; `skip-changeset`)
- [x] 6. `packages/spec` touched: n/a.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01JQwxbXf3xRiPQrLhrgsFzw)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent be3cee3 commit 3bd4bf6
5 files changed
Lines changed: 567 additions & 40 deletions
File tree
- .github/workflows
- scripts/pm
- ui/scripts
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3885 | 3885 | | |
3886 | 3886 | | |
3887 | 3887 | | |
3888 | | - | |
3889 | | - | |
3890 | | - | |
3891 | | - | |
3892 | | - | |
| 3888 | + | |
| 3889 | + | |
| 3890 | + | |
| 3891 | + | |
| 3892 | + | |
| 3893 | + | |
| 3894 | + | |
| 3895 | + | |
| 3896 | + | |
| 3897 | + | |
| 3898 | + | |
| 3899 | + | |
| 3900 | + | |
| 3901 | + | |
| 3902 | + | |
| 3903 | + | |
3893 | 3904 | | |
3894 | 3905 | | |
3895 | 3906 | | |
| 3907 | + | |
3896 | 3908 | | |
3897 | 3909 | | |
3898 | 3910 | | |
3899 | 3911 | | |
3900 | 3912 | | |
3901 | 3913 | | |
3902 | | - | |
| 3914 | + | |
3903 | 3915 | | |
3904 | 3916 | | |
| 3917 | + | |
| 3918 | + | |
| 3919 | + | |
| 3920 | + | |
| 3921 | + | |
| 3922 | + | |
| 3923 | + | |
| 3924 | + | |
| 3925 | + | |
| 3926 | + | |
| 3927 | + | |
| 3928 | + | |
| 3929 | + | |
| 3930 | + | |
| 3931 | + | |
| 3932 | + | |
| 3933 | + | |
| 3934 | + | |
| 3935 | + | |
| 3936 | + | |
| 3937 | + | |
3905 | 3938 | | |
3906 | 3939 | | |
3907 | 3940 | | |
| |||
0 commit comments