Skip to content

Commit 2c0412c

Browse files
committed
Reuse verified geometry tests across unchanged deployments
1 parent 4284bf9 commit 2c0412c

5 files changed

Lines changed: 260 additions & 1 deletion

File tree

.github/workflows/pages.yml

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,18 @@ on:
66
pull_request:
77
branches: [main]
88
workflow_dispatch:
9+
inputs:
10+
full_tests:
11+
description: Run all geometry tests even when an exact successful result is cached
12+
type: boolean
13+
default: false
914

1015
permissions:
1116
contents: read
1217

1318
concurrency:
1419
group: pages-${{ github.ref }}
15-
cancel-in-progress: true
20+
cancel-in-progress: false
1621

1722
jobs:
1823
build:
@@ -27,8 +32,46 @@ jobs:
2732
cache: npm
2833
- name: Install dependencies
2934
run: npm ci
35+
- name: Check TypeScript and test contracts
36+
run: npm run typecheck:tests
37+
- name: Check all part packages and presets
38+
run: npm run parts:check
39+
- name: Fingerprint test inputs
40+
id: test-inputs
41+
run: node scripts/ci-test-key.mjs --github-output
42+
- name: Restore exact successful test result
43+
id: test-cache
44+
uses: actions/cache/restore@v4
45+
with:
46+
path: .cache/ci-tests/passed
47+
key: geometry-tests-v1-${{ steps.test-inputs.outputs.key }}
3048
- name: Verify geometry and generators
49+
if: inputs.full_tests || steps.test-cache.outputs.cache-hit != 'true'
3150
run: npm test
51+
- name: Record successful test result
52+
if: success() && (inputs.full_tests || steps.test-cache.outputs.cache-hit != 'true')
53+
env:
54+
TEST_KEY: ${{ steps.test-inputs.outputs.key }}
55+
run: |
56+
mkdir -p .cache/ci-tests
57+
printf '%s\n' "$TEST_KEY" > .cache/ci-tests/passed
58+
- name: Cache successful test result
59+
if: success() && github.event_name != 'pull_request' && steps.test-cache.outputs.cache-hit != 'true'
60+
uses: actions/cache/save@v4
61+
with:
62+
path: .cache/ci-tests/passed
63+
key: geometry-tests-v1-${{ steps.test-inputs.outputs.key }}
64+
- name: Explain test verification
65+
env:
66+
CACHE_HIT: ${{ steps.test-cache.outputs.cache-hit }}
67+
FORCE_TESTS: ${{ inputs.full_tests }}
68+
run: |
69+
if [ "$CACHE_HIT" = "true" ] && [ "$FORCE_TESTS" != "true" ]; then
70+
echo 'Full geometry suite: reused an exact successful result for unchanged test inputs and Node runtime.' >> "$GITHUB_STEP_SUMMARY"
71+
else
72+
echo 'Full geometry suite: executed and passed for these test inputs and Node runtime.' >> "$GITHUB_STEP_SUMMARY"
73+
fi
74+
echo 'TypeScript, package validation and the production build run on every deployment.' >> "$GITHUB_STEP_SUMMARY"
3275
- name: Build static app
3376
run: npm run build
3477
- name: Configure Pages

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ dist/
44
.DS_Store
55
screenshots/
66
test-results/
7+
.cache/ci-tests/
78

89
__pycache__/
910
*.pyc

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,3 +209,13 @@ These components are prototype references. The detailed notes beside each config
209209
The generated recipes are also checked with a local FreeCAD runtime during development. Checks include validity, positive solid volume and agreement with preview dimensions. Gear checks include all defaults/presets; fastener checks exercise head, drive, thread and point variants; spring checks exercise presets, states and end profiles. Runtime checks establish that the recipes execute and produce the intended geometry; they do not establish engineering suitability or exact manufacturer internals. In-memory shape checks do not add objects to user documents. Full macro checks use a temporary document and restore the previously active document.
210210

211211
GitHub Pages deployment follows the [official custom workflow documentation](https://docs.github.com/en/pages/getting-started-with-github-pages/using-custom-workflows-with-github-pages). Push to `main` after enabling the GitHub Actions Pages source to publish. Pull requests run tests and builds without deploying.
212+
213+
### Reusing successful CI verification
214+
215+
Every workflow run installs locked dependencies, checks TypeScript and test contracts, validates all part packages and presets, and builds the production site. The npm download cache speeds up installation; the geometry tests themselves are CPU work.
216+
217+
The full suite is reused only when an exact successful result exists for the same test inputs and Node runtime. `scripts/ci-test-key.mjs` hashes tracked file paths and contents under `src/`, `tests/`, `scripts/`, `data/`, `public/` and `.github/workflows/`, plus the package manifests and root TypeScript configurations. The key also includes the exact Node version, operating system and architecture. Changes, additions, removals and renames invalidate the result. Add any future test inputs outside these paths to the fingerprint contract.
218+
219+
Deployment-only files (`vite.config.ts`, `index.html`, `CNAME`) and documentation outside those directories do not invalidate geometry results; their build and TypeScript checks still run. UI source changes conservatively invalidate the full suite too. A cache miss or eviction runs `npm test` normally. There are no partial-key matches, failed runs never write a success marker, and pull requests can consume the main branch cache but cannot publish one.
220+
221+
The first run after introducing this cache must finish the full suite once. Later deploys with unchanged inputs skip that expensive step. A newer push queues behind a running workflow instead of cancelling its tests and starting from zero; GitHub retains only the newest pending run for that branch. Local `npm test` always runs the complete suite. To force fresh verification in GitHub, choose **Actions → Build and deploy GitHub Pages → Run workflow → full_tests**. The workflow summary says whether tests executed or an exact successful result was reused.

scripts/ci-test-key.mjs

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
import { createHash } from 'node:crypto';
2+
import { execFile } from 'node:child_process';
3+
import { appendFile, lstat, readFile, realpath } from 'node:fs/promises';
4+
import path from 'node:path';
5+
import { fileURLToPath } from 'node:url';
6+
import { promisify } from 'node:util';
7+
8+
const execute = promisify(execFile);
9+
const namespace = 'protolab-full-tests-v1';
10+
11+
// Hash every tracked input used by the full test runner, including source data
12+
// and local reference assets. Deployment-only entry points, Vite's base URL,
13+
// and documentation are excluded; every deployment still builds and typechecks.
14+
const roots = ['src/', 'tests/', 'scripts/', 'data/', 'public/', '.github/workflows/'];
15+
const isTestInput = (file) =>
16+
roots.some((root) => file.startsWith(root)) ||
17+
file === 'package.json' ||
18+
file === 'package-lock.json' ||
19+
/^tsconfig[^/]*\.json$/.test(file);
20+
21+
/** Fingerprint the full suite's tracked files and the exact Node execution platform. */
22+
export async function computeTestKey(
23+
root,
24+
runtime = { version: process.version, platform: process.platform, arch: process.arch },
25+
) {
26+
for (const field of ['version', 'platform', 'arch']) {
27+
if (typeof runtime[field] !== 'string' || !runtime[field])
28+
throw new Error(`A nonempty runtime ${field} is required.`);
29+
}
30+
const repository = await realpath(root);
31+
const { stdout: topLevel } = await execute('git', ['rev-parse', '--show-toplevel'], {
32+
cwd: repository,
33+
});
34+
if ((await realpath(topLevel.trim())) !== repository)
35+
throw new Error('The test fingerprint must run from the Git repository root.');
36+
37+
const { stdout } = await execute('git', ['ls-files', '--cached', '-z'], {
38+
cwd: repository,
39+
maxBuffer: 16 * 1024 * 1024,
40+
});
41+
const files = [...new Set(stdout.split('\0').filter(isTestInput))].sort((a, b) =>
42+
Buffer.compare(Buffer.from(a), Buffer.from(b)),
43+
);
44+
if (!files.length) throw new Error('No tracked full-test inputs were found.');
45+
46+
const hash = createHash('sha256');
47+
// Length-prefix every entry so file boundaries cannot produce ambiguous hashes.
48+
function add(value) {
49+
const bytes = Buffer.isBuffer(value) ? value : Buffer.from(value);
50+
hash.update(`${bytes.length}:`);
51+
hash.update(bytes);
52+
}
53+
add(namespace);
54+
for (const field of ['version', 'platform', 'arch']) add(runtime[field]);
55+
for (const file of files) {
56+
const absolute = path.join(repository, file);
57+
if (!(await lstat(absolute)).isFile())
58+
throw new Error(`Test inputs must be regular files: ${file}`);
59+
add(file);
60+
add(await readFile(absolute));
61+
}
62+
return `${namespace}-${hash.digest('hex')}`;
63+
}
64+
65+
async function main() {
66+
const arguments_ = process.argv.slice(2);
67+
if (arguments_.length > 1 || (arguments_.length && arguments_[0] !== '--github-output'))
68+
throw new Error('Usage: node scripts/ci-test-key.mjs [--github-output]');
69+
if (arguments_.length && !process.env.GITHUB_OUTPUT)
70+
throw new Error('--github-output requires GITHUB_OUTPUT.');
71+
const key = await computeTestKey(process.cwd());
72+
if (arguments_.length) await appendFile(process.env.GITHUB_OUTPUT, `key=${key}\n`);
73+
process.stdout.write(`${key}\n`);
74+
}
75+
76+
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
77+
main().catch((error) => {
78+
process.stderr.write(`${error.message}\n`);
79+
process.exitCode = 1;
80+
});
81+
}

tests/ci-test-key.test.ts

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
import assert from 'node:assert/strict';
2+
import { execFile } from 'node:child_process';
3+
import { mkdtemp, mkdir, readFile, rename, rm, symlink, writeFile } from 'node:fs/promises';
4+
import os from 'node:os';
5+
import path from 'node:path';
6+
import test from 'node:test';
7+
import { fileURLToPath } from 'node:url';
8+
import { promisify } from 'node:util';
9+
10+
// The production helper is also invoked directly by GitHub Actions without tsx.
11+
// @ts-expect-error The dependency-free Node helper has no declaration file.
12+
import { computeTestKey } from '../scripts/ci-test-key.mjs';
13+
14+
const execute = promisify(execFile);
15+
const helper = fileURLToPath(new URL('../scripts/ci-test-key.mjs', import.meta.url));
16+
const runtime = { version: 'v22.14.0', platform: 'linux', arch: 'x64' };
17+
18+
async function fixture(context: { after: (callback: () => Promise<void>) => void }) {
19+
const root = await mkdtemp(path.join(os.tmpdir(), 'protolab-ci-key-'));
20+
context.after(() => rm(root, { recursive: true, force: true }));
21+
await execute('git', ['init', '--quiet'], { cwd: root });
22+
const write = async (name: string, content = name) => {
23+
await mkdir(path.dirname(path.join(root, name)), { recursive: true });
24+
await writeFile(path.join(root, name), content);
25+
};
26+
const stage = () => execute('git', ['add', '--all'], { cwd: root });
27+
await write('src/core/example.ts', 'export const example = 1;');
28+
await stage();
29+
return { root, write, stage, key: () => computeTestKey(root, runtime) as Promise<string> };
30+
}
31+
32+
test('the full-test key tracks contents, additions, removals and path changes', async (context) => {
33+
const { root, write, stage, key } = await fixture(context);
34+
const original = await key();
35+
assert.match(original, /^protolab-full-tests-v1-[a-f0-9]{64}$/);
36+
assert.equal(await key(), original);
37+
await write('src/core/example.ts', 'export const example = 2;');
38+
const changed = await key();
39+
assert.notEqual(changed, original);
40+
await write('src/core/extra.ts', 'export const extra = true;');
41+
assert.equal(await key(), changed, 'Only tracked files are published and fingerprinted.');
42+
await stage();
43+
const added = await key();
44+
assert.notEqual(added, changed);
45+
await rename(path.join(root, 'src/core/extra.ts'), path.join(root, 'src/core/renamed.ts'));
46+
await stage();
47+
assert.notEqual(
48+
await key(),
49+
added,
50+
'A rename changes the input identity, even with identical bytes.',
51+
);
52+
await rm(path.join(root, 'src/core/renamed.ts'));
53+
await stage();
54+
assert.equal(await key(), changed, 'Removing the extra input restores the prior content key.');
55+
});
56+
57+
test('all test input roots and exact runtime versions invalidate the key', async (context) => {
58+
const { root, write, stage, key } = await fixture(context);
59+
for (const name of [
60+
'src/parts/example/presets.json',
61+
'tests/example.test.ts',
62+
'scripts/part-modules/example.ts',
63+
'data/promtehimport-inventory.json',
64+
'public/references/dimensions.png',
65+
'.github/workflows/pages.yml',
66+
'package.json',
67+
'package-lock.json',
68+
'tsconfig.json',
69+
'tsconfig.tests.json',
70+
]) {
71+
const before = await key();
72+
await write(name);
73+
await stage();
74+
assert.notEqual(await key(), before, name);
75+
}
76+
const original = await key();
77+
for (const patch of [{ version: 'v22.14.1' }, { platform: 'darwin' }, { arch: 'arm64' }])
78+
assert.notEqual(await computeTestKey(root, { ...runtime, ...patch }), original);
79+
await assert.rejects(computeTestKey(root, { ...runtime, version: '' }), /runtime version/);
80+
});
81+
82+
test('documentation and deployment-only entry points do not rerun unchanged tests', async (context) => {
83+
const { write, stage, key } = await fixture(context);
84+
const original = await key();
85+
for (const name of ['README.md', 'docs/deployment.md', 'vite.config.ts', 'index.html', 'CNAME'])
86+
await write(name, 'Changed deployment settings or documentation.');
87+
await stage();
88+
assert.equal(await key(), original);
89+
});
90+
91+
test('missing inputs, symlinks, empty repositories and nested working directories fail closed', async (context) => {
92+
const { root, key, stage } = await fixture(context);
93+
const input = path.join(root, 'src/core/example.ts');
94+
await assert.rejects(computeTestKey(path.join(root, 'src'), runtime), /repository root/);
95+
await rm(input);
96+
await assert.rejects(key(), /ENOENT/);
97+
await symlink('missing.ts', input);
98+
await assert.rejects(key(), /regular files/);
99+
await rm(input);
100+
await stage();
101+
await assert.rejects(key(), /No tracked full-test inputs/);
102+
});
103+
104+
test('CLI emits a reusable GitHub output and rejects unsupported arguments', async (context) => {
105+
const { root } = await fixture(context);
106+
const output = path.join(root, 'github-output');
107+
const expected = await computeTestKey(root);
108+
const result = await execute(process.execPath, [helper, '--github-output'], {
109+
cwd: root,
110+
env: { ...process.env, GITHUB_OUTPUT: output },
111+
});
112+
assert.equal(result.stdout, `${expected}\n`);
113+
assert.equal(await readFile(output, 'utf8'), `key=${expected}\n`);
114+
const plain = await execute(process.execPath, [helper], { cwd: root });
115+
assert.equal(plain.stdout, `${expected}\n`);
116+
await assert.rejects(execute(process.execPath, [helper, '--unknown'], { cwd: root }), /Usage:/);
117+
await assert.rejects(
118+
execute(process.execPath, [helper, '--github-output'], {
119+
cwd: root,
120+
env: { ...process.env, GITHUB_OUTPUT: '' },
121+
}),
122+
/requires GITHUB_OUTPUT/,
123+
);
124+
});

0 commit comments

Comments
 (0)