Skip to content

Commit f923bc1

Browse files
ci(test-ui): select a cross-package test when a package it reads is affected, not on every run (#31540)
<!-- claude-projects-attribution: {"github_login":"zhuangjianguo"} --> _Requested by **Steedos** · [project thread](https://claude.ai/code/project/chan_018ETKaNwMZCNRYD7D8Q9Ubb?thread=cmsg_018ETKaNwMZCNRYD7D8Q9Ubb77KgkSa7rZ7QmbAP6hZdVB)_ Part-of #31520 (slice 1 of 3 planned; the card closes only when the empty-diff count is under 200 and the three-PR shard reading is on it) Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:Test UI 的选测脚本不再把「跨包引用」的测试当作每次必跑;它们只在所引用的包被改动时才跑。空改动下的必跑集从 691 个文件降到 579 个。 - 为什么改:一个只改一行的 UI PR 也要跑完这 691 个必跑文件;其中 112 个只是引用了别的包,包图本可以定位它们。 - 风险与回滚:风险是某个测试在它读的包变了时没被选中;已用 34 个单包改动逐一比对旧新两版,没有一个测试丢掉它该有的选中。回滚即还原这一个脚本。 - 先看:`ui/scripts/select-affected-tests.mjs` 里 `attributed` 那段(依据所触及的包选中),然后是测试文件里新增的 `attributed test files` 一组。 - 两分钟验证:`cd ui && node scripts/select-affected-tests.mjs --base origin/main --explain`,预期 `always-on test files ...: 579`。 - 需要你做的:无,席位落地。 ## What changes `ui/scripts/select-affected-tests.mjs` had four always-on reasons. Two of them, "reaches PKG outside OWNER" and "imports NAME, which OWNER does not depend on", named a workspace package the selector could already place. Those tests now get **input packages**: their own package, every package their relative import closure has a file in, and every workspace package that closure imports by name. A test is selected, as a file filter, whenever one of its input packages is affected. Because the affected set already includes dependents, a change to a package that a reached package depends on selects the test too. What stays always-on is unchanged: tests outside every package, tests whose closure reads the disk or spawns a process, and tests that reach a file outside every package (`scripts/`, the tree root). `--explain` prints the attributed files and the packages that selected each one. The header (new item 2a) says why the edge is attributed rather than declared. Declaring it in `package.json` would make the test's whole package a dependent of the target. On this tree, three of the five undeclared edges would also be cycles: `core`↔`plugin-map`, `core`↔`i18n`, `react`↔`plugin-grid`. Empty-diff count, `cd ui && node scripts/select-affected-tests.mjs --base origin/main --explain`: | head | always-on | selected of tracked | |---|---:|---:| | `f4b1f9ee` (branch base) | 691 | 691 of 6,658 | | `a3ac7fb3` (`origin/main` merged in, main's selector) | 691 | 691 of 6,658 | | `1f0dd719` (this PR, after merging `a3ac7fb3`) | 579 | 579 of 6,658 | ## Why Slice 1 of the plan the `domain:ui#3` seat approved on #31520: class (a) of the card's classification, the 112 always-on tests whose only edges past their package go into other packages. Slices 2 (disk readers whose reads resolve inside named packages) and 3 (`scripts/` and `eslint-rules/` as owner directories) follow as separate PRs. ## Tests In `ui/scripts/__tests__/select-affected-tests.test.ts`, the fixture gains `packages/c/src/reach.test.ts`, which imports a file of `@fx/b`, which in turn depends on `@fx/a`. The existing `undeclared.test.ts` imports `@fx/a` by name without declaring it. - Positive controls: a change in `@fx/a` selects `undeclared.test.ts` (through `@fx/a`). A change in `@fx/b` selects `reach.test.ts` (through `@fx/b`). A change in `@fx/a` also selects `reach.test.ts` (through `@fx/a` and `@fx/b`), which is the case the seat asked for: a package the reached package depends on. - Negative controls: a docs-only change leaves both out of the always-on set and out of the filters. A change in an unrelated package (`@fx/registry`) selects neither. A change in their own package selects them through the package directory, not as attributed files. `escapes.test.ts`, which reaches `scripts/`, stays always-on. - The old pin that `undeclared.test.ts` is always-on is replaced by the attribution pins above. <details><summary>Ablation: the selector at the base, the new controls red</summary> With the fix committed (`80935d1b`), the selector was restored to `f4b1f9ee` on disk (`grep -c attributed` gave 0 on disk against 7 at HEAD). The test file at HEAD was then run: 6 failed, 38 passed. The failures were the six new cases: "does not keep a test whose only edges past its package are into other packages", the three "selects a test when ..." cases, "does not select either ...", and "a change to its own package ...". The selector was then restored from `HEAD`, and `git hash-object` matched the HEAD blob `6b5a98a0`. `git diff HEAD` was empty. </details> <details><summary>Exactness sweep: no test loses a selection it needs</summary> For each workspace package with a tracked `src/` file (38 packages; core, i18n, test-support and types are full-run triggers under both versions), the base selector and this one were each run on a one-file change in that package. Across the 34 affected-mode runs, 1,891 test-file selections were dropped and 0 were added. For every dropped one, an independent check of the test's relative closure confirmed it has no file in an affected package's directory, imports no affected package by name, reads no disk, and reaches no non-package file. </details> ## Acceptance notes - **The card's second acceptance line will read 16 shards on every UI PR, whatever the always-on count.** `node scripts/ci-shard-matrix.mjs` reads `test-ui 16 literal`; Test Core's matrix is `planned`. On PR run 42478811805 (a console-only fix), the 16 Test UI shards took 3,368 runner-seconds, 59% of it in the test step. This PR shrinks only that step. The seat has taken the matrix question to the maintainer. It would be a `ci.yml` change, which is out of scope here. - The step-1 classification of all 691 files is in the card thread's project files (`issue-31520/classification.md`). On that measurement, slices 1 to 3 reach about 393, and under 200 needs a further decision recorded on the card. ## Verification Derived with `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at `80935d1b`: 20 families. All 20 were run locally on that head and reconciled with `--ran`, which reported "20 run, 0 NOT-MEASURED (a DERIVED zero)". `node scripts/check-sdui-manifest.mjs` was also run, as the ui seat requires on every card. <details><summary>Gate exit codes at 80935d1</summary> ```text node scripts/check-closing-keyword-parity.mjs :: exit 0 node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0 node scripts/check-comment-mask-corpus.mjs :: exit 0 pnpm check:cross-package-test-inputs :: exit 0 pnpm check:driver-memory-census :: exit 0 pnpm check:engine-double-contract :: exit 0 pnpm check:gitlink-declared :: exit 0 pnpm check:nul-bytes :: exit 0 pnpm check:published-files :: exit 0 pnpm check:query-options-erasure :: exit 0 pnpm check:refd-timer-probe :: exit 0 pnpm check:registry-log-declared :: exit 0 pnpm check:rest-log-spy-declared :: exit 0 pnpm check:tier-file-adoption :: exit 0 pnpm check:type-check-coverage :: exit 0 pnpm check:type-check-debt :: exit 0 pnpm check:undeclared-dep-imports :: exit 0 pnpm check:watch-hint-literal :: exit 0 pnpm check:where-matcher :: exit 0 node scripts/check-sdui-manifest.mjs :: exit 0 pnpm --dir ui exec vitest run scripts/ :: exit 0 ``` </details> - `pnpm --dir ui exec vitest run scripts/` (under the verify lock): 170 test files passed and 2 were skipped, the skips being existing conditional skips. 5,002 tests passed. - On `1f0dd719`, after merging `a3ac7fb3`: the selector's own two suites passed again (2 files, 54 tests), and `--explain` printed 579. The merge brought in no `ui/scripts/` change. - No changeset: `ui/` is the private `object-ui-monorepo` root, and `ui/scripts/` ships in no package, so this PR carries `skip-changeset`. - CI runs the rest: the PR's required check-runs. --- _Generated by [Claude Code](https://claude.ai/code/session_01QsR2jREy1UXbgGdQRBvZji)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent dabef0f commit f923bc1

2 files changed

Lines changed: 86 additions & 20 deletions

File tree

‎ui/scripts/__tests__/select-affected-tests.test.ts‎

Lines changed: 46 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@ function workspace() {
106106
'packages/c/src/reads.test.ts': "import './helpers/tree';\n",
107107
'packages/c/src/helpers/tree.ts': "import fs from 'node:fs';\nexport const t = fs.existsSync('.');\n",
108108
'packages/c/src/undeclared.test.ts': "import { a } from '@fx/a';\n",
109+
'packages/c/src/reach.test.ts': "import { b } from '../../b/src/index';\n",
109110
'packages/c/src/escapes.test.ts': "import '../../../scripts/helper.mjs';\n",
110111
'packages/heavy-user/package.json': manifest('@fx/heavy-user'),
111112
'packages/heavy-user/src/heavy.test.tsx': 'export {};\n',
@@ -164,12 +165,15 @@ describe('select-affected-tests — the always-on set', () => {
164165
expect(why.get('packages/c/src/reads.test.ts')).toBe('reads the disk or spawns a process');
165166
});
166167

167-
it('keeps a test that imports a workspace package its own package does not depend on', () => {
168-
expect(why.get('packages/c/src/undeclared.test.ts')).toMatch(/imports @fx\/a, which @fx\/c does not depend on/);
168+
it('keeps a test that reaches a file outside every package', () => {
169+
expect(why.get('packages/c/src/escapes.test.ts')).toMatch(/reaches the repository root outside packages\/c/);
169170
});
170171

171-
it('keeps a test that reaches a file outside its own package', () => {
172-
expect(why.get('packages/c/src/escapes.test.ts')).toMatch(/reaches the repository root outside packages\/c/);
172+
it('does not keep a test whose only edges past its package are into other packages (objectstack#31520)', () => {
173+
expect(why.has('packages/c/src/undeclared.test.ts')).toBe(false);
174+
expect(why.has('packages/c/src/reach.test.ts')).toBe(false);
175+
expect(v.attributed).toEqual([]);
176+
expect(v.filters.some((f) => f.includes('undeclared.test') || f.includes('reach.test'))).toBe(false);
173177
});
174178

175179
it('leaves out a test the package graph can place', () => {
@@ -178,6 +182,44 @@ describe('select-affected-tests — the always-on set', () => {
178182
});
179183
});
180184

185+
describe('select-affected-tests — attributed test files (objectstack#31520)', () => {
186+
// `undeclared.test.ts` (in @fx/c) imports @fx/a by name, which @fx/c does
187+
// not declare; `reach.test.ts` (in @fx/c) imports a file of @fx/b, which
188+
// depends on @fx/a. Neither is always-on; each is selected exactly when a
189+
// package it reads is affected.
190+
const { root, tracked } = workspace();
191+
const attributedOf = (changed: string[]) => new Map(affected(root, tracked, changed).attributed.map((a) => [a.file, a.via]));
192+
193+
it('selects a test when the package it imports by name without declaring it changes', () => {
194+
const v = affected(root, tracked, ['packages/a/src/index.ts']);
195+
expect(v.filters).toContain('packages/c/src/undeclared.test.ts');
196+
expect(attributedOf(['packages/a/src/index.ts']).get('packages/c/src/undeclared.test.ts')).toEqual(['@fx/a']);
197+
});
198+
199+
it('selects a test when the package it reaches by a relative import changes', () => {
200+
const v = affected(root, tracked, ['packages/b/src/index.ts']);
201+
expect(v.filters).toContain('packages/c/src/reach.test.ts');
202+
expect(attributedOf(['packages/b/src/index.ts']).get('packages/c/src/reach.test.ts')).toEqual(['@fx/b']);
203+
});
204+
205+
it('selects a test when a dependency of the package it reaches changes, through that package', () => {
206+
expect(affected(root, tracked, ['packages/a/src/index.ts']).filters).toContain('packages/c/src/reach.test.ts');
207+
expect(attributedOf(['packages/a/src/index.ts']).get('packages/c/src/reach.test.ts')).toEqual(['@fx/a', '@fx/b']);
208+
});
209+
210+
it('does not select either when only a package neither reads changes', () => {
211+
const v = affected(root, tracked, ['packages/registry/src/index.ts']);
212+
expect(v.attributed).toEqual([]);
213+
expect(v.filters.some((f) => f.includes('undeclared.test') || f.includes('reach.test'))).toBe(false);
214+
});
215+
216+
it('a change to its own package selects it through the package directory, not as an attributed file', () => {
217+
const v = affected(root, tracked, ['packages/c/src/index.ts']);
218+
expect(v.filters).toContain('packages/c/');
219+
expect(v.attributed).toEqual([]);
220+
});
221+
});
222+
181223
describe('select-affected-tests — what forces a full run', () => {
182224
const { root, tracked } = workspace();
183225

‎ui/scripts/select-affected-tests.mjs‎

Lines changed: 40 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,24 @@
4848
* - it, or a module it reaches through RELATIVE imports, reads the disk
4949
* or spawns a process (`fs`, `child_process`, `import.meta.glob`):
5050
* its input is the tree, and the tree is not a package;
51-
* - it reaches, through relative imports, a file outside its own
52-
* package directory: that edge is in no `package.json`;
53-
* - it imports a workspace package by name that is not its own and not
54-
* in its own package's dependency closure: the root vitest alias map
55-
* resolves every `@object-ui/*` name to source whether or not the
56-
* package declares it, so the edge is real at test time and absent
57-
* from the graph.
51+
* - it reaches, through relative imports, a file outside every
52+
* workspace package (`scripts/`, the tree root): no package change
53+
* reaches that file.
54+
*
55+
* 2a. ATTRIBUTED TEST FILES. A test whose relative closure reads nothing off
56+
* disk and stays inside workspace packages, but reaches past its own
57+
* package -- a relative import into another package's directory, or a
58+
* workspace package imported by name that its own package does not depend
59+
* on (the root vitest alias map resolves every `@object-ui/*` name to
60+
* source whether or not the package declares it) -- has edges no
61+
* `package.json` states. Its INPUT PACKAGES are its own, every package its
62+
* relative closure has a file in, and every workspace package that closure
63+
* imports by name. It is selected, file by file, whenever one of them is
64+
* affected; an affected package already folds in its dependents, so a
65+
* change to a package one of those reads is selected too. Declaring the
66+
* edge instead would make the test's whole package a dependent of the
67+
* target, and three of those edges on this tree would be cycles
68+
* (objectstack#31520).
5869
*
5970
* 3. FULL RUN instead, when the change reaches something every test runs
6071
* through:
@@ -597,7 +608,7 @@ export function readHeavyProject(configText) {
597608
* @param {{ root?: string, changed: string[], outside?: string[], tracked?: string[] }} input
598609
* `changed` relative to `root`; `outside` the changed files outside the tree,
599610
* relative to the workspace root (`changedOutsideTree`)
600-
* @returns {{ mode: 'full', reasons: string[] } | { mode: 'affected', changedPackages: string[], affectedPackages: string[], heavyEdge: string[], alwaysOn: Array<{ file: string, why: string }>, filters: string[], testFileCount: number, selectedTestFileCount: number }}
611+
* @returns {{ mode: 'full', reasons: string[] } | { mode: 'affected', changedPackages: string[], affectedPackages: string[], heavyEdge: string[], alwaysOn: Array<{ file: string, why: string }>, attributed: Array<{ file: string, via: string[] }>, filters: string[], testFileCount: number, selectedTestFileCount: number }}
601612
*/
602613
export function selectAffectedTests({ root = REPO_ROOT, changed, outside = [], tracked = trackedFiles(root) }) {
603614
const packages = listWorkspacePackages(root);
@@ -704,7 +715,10 @@ export function selectAffectedTests({ root = REPO_ROOT, changed, outside = [], t
704715
return owner && !affected.has(owner.name);
705716
});
706717
const summaries = closureSummaries(root, placed, names, dirOf);
718+
const nameOfDir = new Map(packages.map((p) => [p.dir, p.name]));
707719
const alwaysOn = [];
720+
/** @type {Array<{ file: string, via: string[] }>} */
721+
const attributed = [];
708722
for (const file of testFiles) {
709723
const owner = owningPackage(file, packages);
710724
if (!owner) {
@@ -717,23 +731,30 @@ export function selectAffectedTests({ root = REPO_ROOT, changed, outside = [], t
717731
alwaysOn.push({ file, why: 'reads the disk or spawns a process' });
718732
continue;
719733
}
720-
const outside = [...closure.dirs].find((d) => d !== owner.dir);
721-
if (outside !== undefined) {
722-
alwaysOn.push({ file, why: `reaches ${outside === '' ? 'the repository root' : outside} outside ${owner.dir}` });
734+
if (closure.dirs.has('')) {
735+
alwaysOn.push({ file, why: `reaches the repository root outside ${owner.dir}` });
723736
continue;
724737
}
738+
// Item 2a: every edge past the owner's declared closure is an input
739+
// package; one of them affected selects the file.
725740
const allowed = dependencyClosure.get(owner.name) ?? new Set();
726-
const undeclared = [...closure.packageNames].find((n) => n !== owner.name && !allowed.has(n));
727-
if (undeclared) {
728-
alwaysOn.push({ file, why: `imports ${undeclared}, which ${owner.name} does not depend on` });
729-
}
741+
const inputs = new Set([...closure.dirs].map((d) => nameOfDir.get(d)));
742+
for (const n of closure.packageNames) if (!allowed.has(n)) inputs.add(n);
743+
inputs.delete(owner.name);
744+
const via = [...inputs].filter((n) => affected.has(n)).sort();
745+
if (via.length > 0) attributed.push({ file, via });
730746
}
731747

732748
const dirFilters = packages
733749
.filter((p) => affected.has(p.name) && testFiles.some((f) => f.startsWith(`${p.dir}/`)))
734750
.map((p) => `${p.dir}/`);
735751
const heavyFilters = heavyEdge.length > 0 ? [...heavy.tests.filter((t) => tracked.includes(t)), ...heavyConfigDirs] : [];
736-
const filters = dedupeFilters([...dirFilters, ...heavyFilters, ...alwaysOn.map((a) => a.file)]);
752+
const filters = dedupeFilters([
753+
...dirFilters,
754+
...heavyFilters,
755+
...alwaysOn.map((a) => a.file),
756+
...attributed.map((a) => a.file),
757+
]);
737758
const selected = testFiles.filter((f) => filters.some((flt) => f.toLowerCase().includes(flt.toLowerCase())));
738759

739760
return {
@@ -742,6 +763,7 @@ export function selectAffectedTests({ root = REPO_ROOT, changed, outside = [], t
742763
affectedPackages: [...affected].sort(),
743764
heavyEdge,
744765
alwaysOn,
766+
attributed,
745767
filters,
746768
testFileCount: testFiles.length,
747769
selectedTestFileCount: selected.length,
@@ -797,6 +819,8 @@ function main(argv) {
797819
}
798820
console.log(`always-on test files (the package graph cannot place them): ${verdict.alwaysOn.length}`);
799821
if (explain) for (const a of verdict.alwaysOn) console.log(` - ${a.file}: ${a.why}`);
822+
console.log(`attributed test files (an affected package they reach or import past their own): ${verdict.attributed.length}`);
823+
if (explain) for (const a of verdict.attributed) console.log(` - ${a.file}: through ${a.via.join(', ')}`);
800824
console.log(
801825
`selected ${verdict.selectedTestFileCount} of ${verdict.testFileCount} tracked test files through ${verdict.filters.length} filter(s)`,
802826
);

0 commit comments

Comments
 (0)