Skip to content

Commit d91731f

Browse files
cknittclaude
andcommitted
Bump the magic numbers at release time instead of in pull requests
Why: Every pull request that changed a serialized type bumped the .cmi, .cmt or .ast magic number, so parallel pull requests kept conflicting on the same lines of config.ml. `yarn changelog:release` now bumps each counter when compiler/ml, compiler/ext or compiler/flow_parser changed since the previous release tag, and the changelog check rejects pull requests that bump one. The .cmi and .cmt numbers switch to ResCmi/ResCmt prefixes: with OCaml's Caml1999 prefix, they run into the numbers of OCaml releases (OCaml 5.5 writes Caml1999I037), and OCaml tools would take our files for theirs (#8947). Diagnostic modified (.cmi from an earlier compiler): Before (diagnostic): A.cmi is not a compiled interface for this version of OCaml. It seems to be for an older version of OCaml. After (diagnostic): A.cmi is not a compiled interface for this version of ReScript. It seems to be for an older version of ReScript. Review: Why not derive the magic numbers from a digest of the types? Tools that read .cmi files need a stable number per release that only changes when the format may have changed; a digest changes on every release and has no order. Review: What about two dev builds that share a counter? Both build systems clean a package when the bsc hash changes, so only mixing files from different builds by hand reads a mismatched layout. Breaking: Tools that check for Caml1999I/Caml1999T magic numbers have to look for ResCmi/ResCmt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFCgV9acRwi6ohFaj9atR8 Signed-off-by: Christoph Knittel <ck@cca.io>
1 parent c9a267c commit d91731f

13 files changed

Lines changed: 399 additions & 17 deletions

File tree

‎.github/workflows/changelog.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,5 +28,5 @@ jobs:
2828
- uses: actions/setup-node@v7
2929
with:
3030
node-version: 22
31-
- run: node --test scripts/changelog.test.js
31+
- run: node --test scripts/changelog.test.js scripts/magicNumbers.test.js
3232
- run: node scripts/changelog.js check

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -584,7 +584,7 @@ The repository's `npm-release` GitHub environment should have required reviewers
584584
To build a new version and release it on NPM, follow these steps:
585585

586586
1. Verify that the version number is already set correctly for the release. (It should have been incremented after releasing the previous version.)
587-
1. Run `yarn changelog:preview` to review pending notes, then `yarn changelog:release` to collect fragments into `CHANGELOG.md`, remove "(Unreleased)", and delete consumed fragments. Create a PR containing these changes, review the combined release notes, and apply the `changelog:release` label. Existing unreleased entries are preserved. If more fragments are merged before publishing, the `Publish` workflow refuses to run; repeat this step to add them to the same version. Don't run `yarn changelog:release` while a `Publish` run is in progress: it could add notes for changes that the run is not publishing.
587+
1. Run `yarn changelog:preview` to review pending notes, then `yarn changelog:release` to collect fragments into `CHANGELOG.md`, remove "(Unreleased)", and delete consumed fragments. It also bumps the `.cmi`, `.cmt` and `.ast` magic numbers in `compiler/ext/config.ml` when their sources changed since the previous release tag, so fetch the tags first (`git fetch --tags`). Create a PR containing these changes, review the combined release notes, and apply the `changelog:release` label. Existing unreleased entries are preserved. If more fragments are merged before publishing, the `Publish` workflow refuses to run; repeat this step to add them to the same version. Don't run `yarn changelog:release` while a `Publish` run is in progress: it could add notes for changes that the run is not publishing.
588588
1. Once that PR is merged and built successfully, run the `Publish` workflow from the GitHub Actions tab. Use the "Run workflow" branch dropdown to select the release branch (normally `master`, or the relevant maintenance branch) and indicate whether this is the current stable release. The version is read from that branch's `package.json`, and the npm tag is derived automatically:
589589
- A current stable release is published as `latest`.
590590
- A stable maintenance release for an older major is published as `latest-<major>`, for example `latest-12`.

‎changes/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ A bullet needs no PR link: `yarn changelog:release` appends the link of the pull
2323

2424
`yarn changelog:preview` prints pending fragments grouped by category, newest PR first, without changing files. It does not include entries already in `CHANGELOG.md`.
2525

26-
During release preparation, run `yarn changelog:release`. It verifies that the first version heading matches `package.json` and is unreleased, validates every fragment before writing, adds fragments above the existing entries of each section, newest PR first, removes `(Unreleased)`, and deletes consumed files. Review and commit all changes in the release PR. Historical entries and existing unreleased notes are preserved. Releases with no fragments can still finalize existing notes. The publish workflow refuses to release while fragments remain in `changes/`. If fragments are merged after the release PR but before publishing, run `yarn changelog:release` again: as long as Publish has not created the version's `v<version>` tag, it adds them to the finalized version. Once the tag exists, it refuses, and the fragments wait for the next version after `yarn changelog:open`. With no fragments left, running it again fails. Don't run it while a Publish run is in progress, because the tag only appears once that run gets to tagging, and the notes could then describe changes the run is not publishing.
26+
During release preparation, run `yarn changelog:release`. It verifies that the first version heading matches `package.json` and is unreleased, validates every fragment before writing, adds fragments above the existing entries of each section, newest PR first, removes `(Unreleased)`, and deletes consumed files. It also bumps the magic numbers in `compiler/ext/config.ml` whose sources changed since the previous release tag, unless they already differ from that tag, so running it again does not bump them twice. CI rejects pull requests without the `changelog:release` label that bump one. Review and commit all changes in the release PR. Historical entries and existing unreleased notes are preserved. Releases with no fragments can still finalize existing notes. The publish workflow refuses to release while fragments remain in `changes/`. If fragments are merged after the release PR but before publishing, run `yarn changelog:release` again: as long as Publish has not created the version's `v<version>` tag, it adds them to the finalized version. Once the tag exists, it refuses, and the fragments wait for the next version after `yarn changelog:open`. With no fragments left, running it again fails. Don't run it while a Publish run is in progress, because the tag only appears once that run gets to tagging, and the notes could then describe changes the run is not publishing.
2727

2828
After publishing, run `yarn changelog:open` once `package.json` has the next version. It adds an empty unreleased section with every category heading.
2929

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- Give `.cmi` and `.cmt` files ReScript magic numbers (`ResCmi…`, `ResCmt…`) instead of OCaml-style ones, and bump all magic numbers at release time instead of in pull requests.

‎compiler/AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ commands below are relative to the repository root unless stated otherwise.
1111
- Add a source fixture exercising the new syntax to `tests/syntax_tests/data/ast-mapping/`. Every file there is run through `res_parser -test-ast-conversion` (which round-trips the parsetree through the frozen v0 AST before printing) as part of `make test-syntax`; the printed output must match the snapshot in its `expected/` directory.
1212
- For exact-identity invariants (locations, attributes) or the v0 wire shape itself, add cases to `tests/ounit_tests/ounit_ast_mapper0_tests.ml`, which tests `ast_mapper_to0`/`ast_mapper_from0` directly.
1313

14+
- **Don't bump magic numbers** - Changing a serialized type (parsetree, typed tree, `.cmi` signature) needs no edit of the magic numbers in `compiler/ext/config.ml`. `yarn changelog:release` bumps them when `compiler/ml`, `compiler/ext` or `compiler/flow_parser` changed since the previous release, and the changelog check rejects pull requests that bump them.
15+
1416
- **Missing test coverage** - Always add tests for syntax, lambda, and end-to-end behavior
1517

1618
- **Test early and often** - Add tests immediately after modifying each compiler layer to catch problems early, rather than waiting until all changes are complete

‎compiler/ext/config.ml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
1-
let cmi_magic_number = "Caml1999I040"
1+
(* Pull requests never change the magic numbers below: [yarn changelog:release]
2+
bumps their trailing counters when the format sources changed since the
3+
previous release (see scripts/magicNumbers.js). The .cmi and .cmt numbers
4+
must have the same length, since a .cmt reader compares its first bytes with
5+
both. *)
6+
let cmi_magic_number = "ResCmi040"
27

38
(* Magic numbers for marshaled values of the *current* parsetree, whose layout
49
changes across compiler versions. *)
@@ -13,7 +18,7 @@ and ast0_impl_magic_number = "Caml1999M022"
1318

1419
and ast0_intf_magic_number = "Caml1999N022"
1520

16-
and cmt_magic_number = "Caml1999T048"
21+
and cmt_magic_number = "ResCmt048"
1722

1823
let get_load_path () = (Compiler_request_state.current ()).load_path
1924
let set_load_path path = (Compiler_request_state.current ()).load_path <- path

‎compiler/ml/cmi_format.ml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ let input_cmi ic =
3535
let flags = input_value ic in
3636
{cmi_name = name; cmi_sign = sign; cmi_crcs = crcs; cmi_flags = flags}
3737

38+
(* Earlier compilers wrote OCaml-style magic numbers *)
39+
let legacy_magic_prefix = "Caml1999I"
40+
3841
let read_cmi_channel filename ic =
3942
try
4043
let buffer =
@@ -50,6 +53,8 @@ let read_cmi_channel filename ic =
5053
if buffer < Config.cmi_magic_number then "an older" else "a newer"
5154
in
5255
raise (Error (Wrong_version_interface (filename, msg)))
56+
else if String.starts_with ~prefix:legacy_magic_prefix buffer then
57+
raise (Error (Wrong_version_interface (filename, "an older")))
5358
else raise (Error (Not_an_interface filename)));
5459
let cmi = input_cmi ic in
5560
cmi
@@ -111,8 +116,8 @@ let report_error ppf = function
111116
filename
112117
| Wrong_version_interface (filename, older_newer) ->
113118
fprintf ppf
114-
"%a@ is not a compiled interface for this version of OCaml.@.It seems to \
115-
be for %s version of OCaml."
119+
"%a@ is not a compiled interface for this version of ReScript.@.It seems \
120+
to be for %s version of ReScript."
116121
Location.print_filename filename older_newer
117122
| Corrupted_interface filename ->
118123
fprintf ppf "Corrupted compiled interface@ %a" Location.print_filename

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@
6262
"changelog:preview": "node scripts/changelog.js preview",
6363
"changelog:release": "node scripts/changelog.js release",
6464
"changelog:open": "node scripts/changelog.js open",
65-
"test-changelog": "node --test scripts/changelog.test.js",
65+
"test-changelog": "node --test scripts/changelog.test.js scripts/magicNumbers.test.js",
6666
"test-commit-records": "node --test scripts/commitRecords.test.js scripts/recordMarkdown.test.js"
6767
},
6868
"files": [

‎scripts/changelog.js‎

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@ import { execFileSync } from "node:child_process";
22
import fs from "node:fs/promises";
33
import path from "node:path";
44
import { fileURLToPath } from "node:url";
5+
import {
6+
checkMagicNumbers,
7+
configPath,
8+
releaseMagicNumbers,
9+
showAt,
10+
} from "./magicNumbers.js";
511

612
export const categories = {
713
breaking: "#### :boom: Breaking Change",
@@ -280,6 +286,7 @@ export async function run(
280286
command,
281287
root = process.cwd(),
282288
pullRequestOf = name => addedByPullRequest(root, name),
289+
magicNumbersFor = version => releaseMagicNumbers(root, version),
283290
) {
284291
if (!["check", "preview", "release", "open"].includes(command)) {
285292
throw new Error(
@@ -308,6 +315,11 @@ export async function run(
308315
files.push({ status: diff[i], name: diff[i + 1] });
309316
}
310317
checkPullRequest(files, pullRequest);
318+
checkMagicNumbers(
319+
showAt(root, pullRequest.base.sha, configPath),
320+
await fs.readFile(path.join(root, configPath), "utf8"),
321+
pullRequest.labels.map(label => label.name),
322+
);
311323
} else if (process.env.GITHUB_EVENT_NAME === "push") {
312324
// Once merged, every fragment must resolve to its pull request, so a
313325
// merge without "(#N)" in its subject fails here rather than at release.
@@ -339,10 +351,12 @@ export async function run(
339351
linked.length > 0 &&
340352
isFinalized(changelog, version) &&
341353
releaseTagExists(repository.url, version);
342-
await fs.writeFile(
343-
target,
344-
prepareRelease(changelog, version, linked, { tagged }),
345-
);
354+
const released = prepareRelease(changelog, version, linked, { tagged });
355+
// The release bumps the magic numbers of the formats that changed, so that
356+
// pull requests never touch them.
357+
const config = await magicNumbersFor(version);
358+
await fs.writeFile(target, released);
359+
if (config !== null) await fs.writeFile(path.join(root, configPath), config);
346360
for (const fragment of fragments) {
347361
await fs.unlink(path.join(root, "changes", fragment.name));
348362
}

‎scripts/changelog.test.js‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ import {
2222
const note =
2323
"- Fix recovery. https://github.com/rescript-lang/rescript/pull/8667";
2424
const historical = "# 12.0.0\n\nOld notes.\n";
25+
// Bumping the magic numbers needs a checkout of the repository.
26+
const noMagicNumbers = async () => null;
2527
const changelog = `# Changelog\n\n# 13.0.0 (Unreleased)\n\n${Object.values(categories).join("\n\n")}\n\n${historical}`;
2628

2729
test("validates category names and bullet content", () => {
@@ -203,7 +205,7 @@ test("validates before mutation, sorts fragments, consumes only on release", asy
203205
await write("z.fix.md", note);
204206
await write("a.fix.md", note.replace("recovery", "formatting"));
205207
await write("bad.unknown.md", note);
206-
await assert.rejects(run("release", root));
208+
await assert.rejects(run("release", root, undefined, noMagicNumbers));
207209
assert.equal(
208210
await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8"),
209211
changelog,
@@ -216,9 +218,9 @@ test("validates before mutation, sorts fragments, consumes only on release", asy
216218
(await readFragments(root)).map(fragment => fragment.name),
217219
["a.fix.md", "z.fix.md"],
218220
);
219-
await run("release", root);
221+
await run("release", root, undefined, noMagicNumbers);
220222
assert.deepEqual(await fs.readdir(path.join(root, "changes")), ["README.md"]);
221-
await assert.rejects(run("release", root));
223+
await assert.rejects(run("release", root, undefined, noMagicNumbers));
222224
});
223225

224226
test("links bullets without a link to the pull request that added them", () => {
@@ -285,15 +287,28 @@ test("releases fragments without a link with their pull request's link", async t
285287
await fs.writeFile(path.join(root, "changes", "lost.fix.md"), "- Lost.");
286288
const numbers = { "new.fix.md": 9000, "lost.fix.md": null };
287289
await assert.rejects(
288-
run("release", root, name => numbers[name]),
290+
run("release", root, name => numbers[name], noMagicNumbers),
289291
/changes\/lost\.fix\.md/,
290292
);
291293
assert.equal(
292294
await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8"),
293295
changelog,
294296
);
295297
await fs.unlink(path.join(root, "changes", "lost.fix.md"));
296-
await run("release", root, name => numbers[name]);
298+
await fs.mkdir(path.join(root, "compiler", "ext"), { recursive: true });
299+
await run(
300+
"release",
301+
root,
302+
name => numbers[name],
303+
async version => {
304+
assert.equal(version, "13.0.0");
305+
return "bumped";
306+
},
307+
);
308+
assert.equal(
309+
await fs.readFile(path.join(root, "compiler", "ext", "config.ml"), "utf8"),
310+
"bumped",
311+
);
297312
assert.ok(
298313
(await fs.readFile(path.join(root, "CHANGELOG.md"), "utf8")).includes(
299314
`${categories.fix}\n\n- New fix. https://github.com/rescript-lang/rescript/pull/9000\n${note}\n`,

0 commit comments

Comments
 (0)