Skip to content

Commit 1c86fea

Browse files
committed
tooling(cutover): scripts/ui-import carries objectui's lockfile resolutions into the merged lock (objectstack#25803)
The splice step of the cutover lock recipe (bill "already on the bill": 5a, objectstack#25726), as the verification branch ran it (cfa1676, used on runs 5-11): splice-lockfile.mjs copies objectui's own resolutions into the root pnpm-lock.yaml instead of letting the root install re-resolve every ui/ range, and README.md is the procedure. The README gains the `pnpm dedupe` step (bill item 16) the verification branch ran after the splice. Measured on objectstack main abebc4e + `git subtree add --prefix=ui` of objectui 9e1fe03 + the root workspace commit + ui patches 51-53, against objectui's own lock: - splice, `pnpm install --lockfile-only` twice, `pnpm dedupe`: 44 objectui rows moved, 2 by the happy-dom floor and 42 by the dedupe; frozen install 0; `pnpm dedupe --check` 0. - `pnpm install --lockfile-only` + `pnpm dedupe` without the splice: 221 rows moved (vite x30, @vitejs/plugin-react x27, vite-plugin-dts x22, lucide-react x22, react-router-dom x8, ...), and copies of @radix-ui/react-slot 1.5.0, @radix-ui/react-use-controllable-state 1.3.0, express 4.22.3 and express-rate-limit 8.7.1 that neither tree had. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CUXv64KLMfYv3ia2sWi6te
1 parent ca283e6 commit 1c86fea

2 files changed

Lines changed: 357 additions & 0 deletions

File tree

‎scripts/ui-import/README.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# ui-import — dependency-neutral lockfile for the `ui/` subtree
2+
3+
Importing objectui under `ui/` must not change which dependency versions objectui's
4+
code runs on. Deleting objectui's lockfile and running `pnpm install` re-resolves every
5+
`ui/*` dependency from its range and moves dozens of them; `splice-lockfile.mjs` instead
6+
carries objectui's own resolutions into the root `pnpm-lock.yaml`. The script header is
7+
the authority on what it writes and refuses.
8+
9+
## Procedure (cutover, from the repository root)
10+
11+
1. `git subtree add --prefix ui <objectui> <sha>`, then the adaptation commits: ui
12+
manifests take `workspace:^` for every `@objectstack/*`, objectui's `pnpm` field
13+
(react pins, security floors, the focus-scope patch) moves to the root
14+
`pnpm-workspace.yaml`.
15+
2. Keep objectui's lockfile for the splice, then delete it from the tree:
16+
`git show <sha>:pnpm-lock.yaml > /tmp/objectui-lock.yaml && git rm ui/pnpm-lock.yaml`.
17+
3. Splice:
18+
`node scripts/ui-import/splice-lockfile.mjs --base pnpm-lock.yaml --import /tmp/objectui-lock.yaml --prefix ui --out pnpm-lock.yaml`
19+
It must end with `moved 0`.
20+
4. Reconcile with pnpm: `pnpm install --lockfile-only`. pnpm applies the root
21+
`overrides` (security floors) to the imported subgraph and re-resolves peer
22+
suffixes; nothing else. Run it a second time: the file must not change. Then
23+
`pnpm dedupe`, so one copy of a package serves both trees wherever both ranges admit it
24+
(one `vite` and one `postcss` for ui/ and packages/, one `ai` 7.0.118 and one
25+
`@ai-sdk/react` 4.0.121; objectstack#25752 item 16). `pnpm dedupe --check` then exits 0.
26+
5. `pnpm install --frozen-lockfile` must pass.
27+
6. Re-measure the reconciled file:
28+
`node scripts/ui-import/splice-lockfile.mjs --measure pnpm-lock.yaml --import /tmp/objectui-lock.yaml --prefix ui`. Every row still moved must be an override the root
29+
`pnpm-workspace.yaml` keeps on purpose, named in its comment block.
30+
7. Regenerate nothing. With the ui importers on objectui's versions no generated file
31+
(for example `ui/packages/components/src/lib/lucide-record-icon-names.ts`) changes;
32+
if a `check:` gate asks for a regeneration, a dependency moved, and step 6 says which.
33+
34+
An override that would move a dependency objectui pins onto another major, or split one
35+
line (react-router under react-router-dom), is scoped in `pnpm-workspace.yaml` with a
36+
comment naming the advisory range it still covers; dependency upgrades for objectui are
37+
their own changes after the cutover.
Lines changed: 320 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,320 @@
1+
#!/usr/bin/env node
2+
// splice-lockfile — carries an imported workspace's own lockfile resolutions into
3+
// this repository's single pnpm-lock.yaml, so that importing a tree (objectui under
4+
// `ui/`) is dependency-neutral: every imported importer keeps the versions its own
5+
// lockfile resolved, instead of whatever a fresh `pnpm install` would pick today.
6+
//
7+
// Usage:
8+
// node scripts/ui-import/splice-lockfile.mjs \
9+
// --base pnpm-lock.yaml --import <objectui pnpm-lock.yaml> --prefix ui --out pnpm-lock.yaml
10+
// [--root <repo root>] where the importers' package.json files live (default: this repo)
11+
// node scripts/ui-import/splice-lockfile.mjs --measure pnpm-lock.yaml --import <objectui lock> --prefix ui
12+
// writes nothing; prints the moved-row count of an existing (e.g. reconciled) lockfile
13+
//
14+
// What the output holds:
15+
// - every importer of --import, re-keyed under --prefix (`.` becomes `ui`,
16+
// `packages/x` becomes `ui/packages/x`), with the specifier and resolved version
17+
// its own lockfile recorded;
18+
// - except a dependency on a package of THIS workspace (any `@objectstack/*` the
19+
// base importers provide): it becomes the workspace link the adapted manifest
20+
// asks for (`link:../../../packages/<name>`, specifier read from the manifest);
21+
// - `snapshots:` and `packages:` entries for everything reachable from the base's
22+
// own importers and from the re-keyed importers. An entry both lockfiles hold
23+
// under the same key is taken from --base, so the base graph stays byte-stable;
24+
// - every importer of --base outside --prefix, unchanged (asserted before writing),
25+
// plus the base's `settings`, `overrides` and `patchedDependencies`.
26+
//
27+
// The output is written in pnpm's own lockfile layout (codepoint-sorted keys, blank
28+
// lines between entries, flow style for resolution/engines/cpu/os/libc); the run
29+
// first proves the writer reproduces --base byte for byte and refuses if it does not.
30+
// The splice is textual: run `pnpm install --lockfile-only` afterwards so pnpm
31+
// reconciles what only it can decide (root overrides and patches that apply to the
32+
// imported subgraph, peer suffixes); the procedure lives in this directory's README.
33+
//
34+
// It ends by printing the moved-row count: importer x dependency rows whose resolved
35+
// version (peer suffix ignored) differs between --import and --out. 0 is the pass;
36+
// `--measure` re-counts a lockfile pnpm has reconciled since.
37+
//
38+
// Exit codes: 0 written, 1 refused (invariant broken), 2 usage error.
39+
40+
import { readFileSync, writeFileSync, existsSync } from 'node:fs';
41+
import { createRequire } from 'node:module';
42+
import path from 'node:path';
43+
import { fileURLToPath } from 'node:url';
44+
import { parseArgs, isDeepStrictEqual } from 'node:util';
45+
46+
const require = createRequire(import.meta.url);
47+
const YAML = require('yaml'); // root devDependency; no new dependency
48+
49+
const SCRIPT_DIR = path.dirname(fileURLToPath(import.meta.url));
50+
const REPO_ROOT = path.resolve(SCRIPT_DIR, '..', '..');
51+
const DEP_SECTIONS = ['dependencies', 'devDependencies', 'optionalDependencies'];
52+
const SPACED_SECTIONS = new Set(['importers', 'packages', 'snapshots']);
53+
const FLOW_PACKAGE_FIELDS = new Set(['resolution', 'engines', 'cpu', 'os', 'libc']);
54+
55+
function fail(code, msg) {
56+
console.error(`splice-lockfile: ${msg}`);
57+
process.exit(code);
58+
}
59+
60+
// ---------- pnpm lockfile layout ----------
61+
62+
const codepoint = (a, b) => (a < b ? -1 : a > b ? 1 : 0);
63+
64+
function sortedObject(obj) {
65+
const out = {};
66+
for (const k of Object.keys(obj).sort(codepoint)) out[k] = obj[k];
67+
return out;
68+
}
69+
70+
function renderEntry(key, value, section) {
71+
const doc = new YAML.Document({ [key]: value });
72+
if (section === 'packages' && value && typeof value === 'object') {
73+
for (const field of FLOW_PACKAGE_FIELDS) {
74+
const node = doc.getIn([key, field], true);
75+
if (node && typeof node === 'object' && 'items' in node) node.flow = true;
76+
}
77+
}
78+
const text = doc.toString({ singleQuote: true, lineWidth: 0, flowCollectionPadding: false });
79+
return text.replace(/^(?=.)/gm, ' ');
80+
}
81+
82+
function renderLockfile(lock) {
83+
const parts = [];
84+
for (const [key, value] of Object.entries(lock)) {
85+
if (value === null || typeof value !== 'object') {
86+
parts.push(YAML.stringify({ [key]: value }, { singleQuote: true, lineWidth: 0 }));
87+
continue;
88+
}
89+
const spaced = SPACED_SECTIONS.has(key);
90+
const entries = Object.entries(value).map(([k, v]) => renderEntry(k, v, key));
91+
parts.push(`${key}:\n${spaced ? '\n' : ''}${entries.join(spaced ? '\n' : '')}`);
92+
}
93+
return parts.join('\n');
94+
}
95+
96+
// ---------- dependency paths (mirrors @pnpm/dependency-path refToRelative) ----------
97+
98+
function refToKey(name, ref) {
99+
if (typeof ref !== 'string' || ref.startsWith('link:')) return null;
100+
if (ref.startsWith('@')) return ref;
101+
const at = ref.indexOf('@');
102+
if (at === -1) return `${name}@${ref}`;
103+
const colon = ref.indexOf(':');
104+
const paren = ref.indexOf('(');
105+
if ((colon === -1 || at < colon) && (paren === -1 || at < paren)) return ref; // npm alias
106+
return `${name}@${ref}`;
107+
}
108+
109+
function packageKey(snapshotKey) {
110+
// `name@version(peer@x)(patch_hash=y)` -> `name@version`; a scoped name starts with '@'.
111+
const paren = snapshotKey.indexOf('(', 1);
112+
return paren === -1 ? snapshotKey : snapshotKey.slice(0, paren);
113+
}
114+
115+
const versionOf = (info) => (info && typeof info === 'object' ? info.version : info);
116+
117+
// ---------- the splice ----------
118+
119+
function importerKeyFor(prefix, imp) {
120+
return imp === '.' ? prefix : `${prefix}/${imp}`;
121+
}
122+
123+
function inPrefix(prefix, key) {
124+
return key === prefix || key.startsWith(`${prefix}/`);
125+
}
126+
127+
function readManifest(root, importerKey) {
128+
const file = path.join(root, importerKey, 'package.json');
129+
return existsSync(file) ? JSON.parse(readFileSync(file, 'utf8')) : null;
130+
}
131+
132+
function splice({ base, imported, prefix, root }) {
133+
const notes = [];
134+
const out = {};
135+
for (const k of Object.keys(base)) out[k] = base[k];
136+
137+
// Workspace packages the base provides, by name -> importer dir.
138+
const workspace = new Map();
139+
for (const key of Object.keys(base.importers)) {
140+
if (inPrefix(prefix, key)) continue;
141+
const manifest = readManifest(root, key);
142+
if (manifest?.name) workspace.set(manifest.name, key);
143+
}
144+
145+
const importers = {};
146+
for (const [key, entry] of Object.entries(base.importers)) {
147+
if (!inPrefix(prefix, key)) importers[key] = entry;
148+
}
149+
for (const [imp, entry] of Object.entries(imported.importers)) {
150+
const key = importerKeyFor(prefix, imp);
151+
const manifest = readManifest(root, key);
152+
if (!manifest) fail(1, `no package.json for imported importer ${key} under ${root}`);
153+
const next = structuredClone(entry);
154+
for (const sect of DEP_SECTIONS) {
155+
for (const [name, info] of Object.entries(next[sect] ?? {})) {
156+
if (workspace.has(name)) {
157+
// Re-placed below: the adapted manifest decides the section and specifier.
158+
delete next[sect][name];
159+
continue;
160+
}
161+
const wanted = manifest[sect]?.[name];
162+
const declared = [...DEP_SECTIONS, 'peerDependencies'].some((s) => manifest[s]?.[name] !== undefined);
163+
if (!declared) {
164+
notes.push(`${key} ${name}: in the imported lock but not in package.json (pnpm reconciles)`);
165+
} else if (wanted !== undefined && wanted !== info.specifier) {
166+
notes.push(`${key} ${name}: package.json asks ${wanted}, the imported lock recorded ${info.specifier} (pnpm reconciles)`);
167+
}
168+
}
169+
}
170+
for (const [name, dir] of workspace) {
171+
// pnpm's precedence: optionalDependencies, then dependencies, then devDependencies;
172+
// a peer-only dependency is auto-installed into dependencies (autoInstallPeers).
173+
const sect = ['optionalDependencies', 'dependencies', 'devDependencies'].find((s) => manifest[s]?.[name] !== undefined)
174+
?? (manifest.peerDependencies?.[name] !== undefined ? 'dependencies' : null);
175+
if (!sect) continue;
176+
const specifier = manifest[sect]?.[name] ?? manifest.peerDependencies[name];
177+
const rel = path.relative(key, dir).split(path.sep).join('/');
178+
next[sect] = sortedObject({ ...(next[sect] ?? {}), [name]: { specifier, version: `link:${rel}` } });
179+
const prior = base.importers[key]?.[sect]?.[name];
180+
if (prior && !isDeepStrictEqual(prior, next[sect][name])) {
181+
notes.push(`${key} ${name}: base lock had ${JSON.stringify(prior)}, writing ${JSON.stringify(next[sect][name])}`);
182+
}
183+
}
184+
for (const sect of DEP_SECTIONS) {
185+
if (next[sect] && Object.keys(next[sect]).length === 0) delete next[sect];
186+
for (const name of Object.keys(manifest[sect] ?? {})) {
187+
if (!workspace.has(name) && !DEP_SECTIONS.some((s) => next[s]?.[name])) {
188+
notes.push(`${key} ${name}: in package.json ${sect} but not in the imported lock (pnpm reconciles)`);
189+
}
190+
}
191+
}
192+
importers[key] = next;
193+
}
194+
out.importers = sortedObject(importers);
195+
196+
// Reachability over the union graph: a key both locks hold resolves to the base entry.
197+
const snapshots = {};
198+
const packages = {};
199+
let sharedDiffering = 0;
200+
const pending = [];
201+
const visit = (name, ref, from) => {
202+
const k = refToKey(name, ref);
203+
if (k === null || k in snapshots) return;
204+
if (k.includes('.objectstack-pin')) fail(1, `${from} still reaches the pin entry ${k}`);
205+
const own = base.snapshots?.[k];
206+
const theirs = imported.snapshots?.[k];
207+
if (own === undefined && theirs === undefined) fail(1, `${from} needs ${k}, which neither lockfile holds`);
208+
if (own !== undefined && theirs !== undefined && !isDeepStrictEqual(own, theirs)) sharedDiffering++;
209+
snapshots[k] = own ?? theirs;
210+
const pk = packageKey(k);
211+
const ownPkg = base.packages?.[pk];
212+
const theirPkg = imported.packages?.[pk];
213+
if (ownPkg && theirPkg && !isDeepStrictEqual(ownPkg.resolution, theirPkg.resolution)) {
214+
fail(1, `${pk} resolves differently in the two lockfiles (${JSON.stringify(ownPkg.resolution)} vs ${JSON.stringify(theirPkg.resolution)})`);
215+
}
216+
if (!ownPkg && !theirPkg) fail(1, `${pk} has a snapshot but no packages entry in either lockfile`);
217+
packages[pk] = ownPkg ?? theirPkg;
218+
pending.push(k);
219+
};
220+
for (const [key, entry] of Object.entries(out.importers)) {
221+
for (const sect of DEP_SECTIONS) {
222+
for (const [name, info] of Object.entries(entry[sect] ?? {})) visit(name, versionOf(info), key);
223+
}
224+
}
225+
while (pending.length) {
226+
const k = pending.pop();
227+
const snap = snapshots[k];
228+
for (const sect of ['dependencies', 'optionalDependencies']) {
229+
for (const [name, ref] of Object.entries(snap[sect] ?? {})) visit(name, ref, k);
230+
}
231+
}
232+
out.packages = sortedObject(packages);
233+
out.snapshots = sortedObject(snapshots);
234+
235+
// Invariant: the base's own importers are untouched.
236+
for (const [key, entry] of Object.entries(base.importers)) {
237+
if (inPrefix(prefix, key)) continue;
238+
if (!isDeepStrictEqual(out.importers[key], entry)) fail(1, `base importer ${key} changed`);
239+
}
240+
const baseOnly = Object.keys(out.importers).filter((k) => !inPrefix(prefix, k)).length;
241+
if (baseOnly !== Object.keys(base.importers).filter((k) => !inPrefix(prefix, k)).length) {
242+
fail(1, 'the set of base importers changed');
243+
}
244+
245+
for (const [name, patch] of Object.entries(imported.patchedDependencies ?? {})) {
246+
const covered = Object.values(base.patchedDependencies ?? {}).some((p) => p.hash === patch.hash);
247+
if (!covered) notes.push(`patch ${name} (${patch.path}) is not in the base patchedDependencies: add it to the root pnpm-workspace.yaml before reconciling`);
248+
}
249+
return { out, notes, sharedDiffering };
250+
}
251+
252+
// ---------- the moved-row measurement ----------
253+
254+
function movedRows(imported, merged, prefix) {
255+
const moved = [];
256+
let same = 0;
257+
for (const [imp, spec] of Object.entries(imported.importers)) {
258+
const target = merged.importers[importerKeyFor(prefix, imp)];
259+
if (!target) continue;
260+
for (const sect of DEP_SECTIONS) {
261+
for (const [name, info] of Object.entries(spec[sect] ?? {})) {
262+
const va = versionOf(info);
263+
const vb = versionOf(target[sect]?.[name]);
264+
if (vb == null || va.startsWith('link:') || va.startsWith('file:')) continue;
265+
const a = packageKey(`x@${va}`).slice(2);
266+
const b = packageKey(`x@${vb}`).slice(2);
267+
if (a !== b) moved.push(`${importerKeyFor(prefix, imp)} ${name} ${a} -> ${b}`);
268+
else same++;
269+
}
270+
}
271+
}
272+
return { moved, same };
273+
}
274+
275+
// ---------- main ----------
276+
277+
const { values } = parseArgs({
278+
options: {
279+
base: { type: 'string' },
280+
import: { type: 'string' },
281+
prefix: { type: 'string', default: 'ui' },
282+
out: { type: 'string' },
283+
root: { type: 'string', default: REPO_ROOT },
284+
measure: { type: 'string' },
285+
},
286+
});
287+
const prefix = values.prefix.replace(/\/+$/, '');
288+
function report(imported, merged) {
289+
const { moved, same } = movedRows(imported, merged, prefix);
290+
for (const m of moved) console.log(`moved: ${m}`);
291+
console.log(`moved ${moved.length} same ${same}`);
292+
}
293+
if (values.measure && values.import) {
294+
report(YAML.parse(readFileSync(values.import, 'utf8')), YAML.parse(readFileSync(values.measure, 'utf8')));
295+
process.exit(0);
296+
}
297+
if (!values.base || !values.import || !values.out) {
298+
fail(2, 'usage: --base <lock> --import <lock> --prefix <dir> --out <path> [--root <repo root>] | --measure <lock> --import <lock> --prefix <dir>');
299+
}
300+
const baseText = readFileSync(values.base, 'utf8');
301+
const base = YAML.parse(baseText);
302+
const imported = YAML.parse(readFileSync(values.import, 'utf8'));
303+
for (const [label, lock] of [['--base', base], ['--import', imported]]) {
304+
if (lock?.lockfileVersion !== '9.0') fail(1, `${label} is lockfileVersion ${lock?.lockfileVersion}, expected 9.0`);
305+
}
306+
if (renderLockfile(base) !== baseText) {
307+
fail(1, '--base does not round-trip through this writer byte for byte; the pnpm layout changed, update renderLockfile');
308+
}
309+
310+
const { out, notes, sharedDiffering } = splice({ base, imported, prefix, root: path.resolve(values.root) });
311+
writeFileSync(values.out, renderLockfile(out));
312+
313+
for (const n of notes) console.log(`note: ${n}`);
314+
const uiImporters = Object.keys(out.importers).filter((k) => inPrefix(prefix, k)).length;
315+
console.log(
316+
`wrote ${values.out}: ${Object.keys(out.importers).length} importers (${uiImporters} under ${prefix}/), ` +
317+
`${Object.keys(out.packages).length} packages, ${Object.keys(out.snapshots).length} snapshots; ` +
318+
`${sharedDiffering} shared snapshot keys taken from --base where the two locks differ`,
319+
);
320+
report(imported, out);

0 commit comments

Comments
 (0)