Repository navigation
Save OPAM environments of dependency changes and bound the CI caches - #9294
Conversation
ChangelogNo changelog entry. Semantic Diff1 commit · Full page ✨ Save OPAM environments of dependency changes and bound the CI caches81a2ded · cknitt + Claude · Nov 13, 2026, 11:47 UTC · new
When an OPAM environment misses, a pull request that changes Review
|
✅ Compiler benchmarksAllocated wordsIn total: 640,783,627 → 640,783,311 (±0.00%). Generated projects: 920,886,686 → 920,886,786 (±0.00%). None of the 265 benchmarks changed by 0.1% or more. InstructionsIn total: 9,209,672,036 → 9,209,672,036 (±0.00%). None of the 33 benchmarks changed by 0.1% or more. Output sizeIn total: 25,838,530 → 25,838,530 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Pull request 81a2ded compared with base 7f416af in this run. |
20e2af7 to
07985f8
Compare
|
@codex review |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9294 |
bcf2ca9 to
1128677
Compare
Why:
A pull request that changed the opam files rebuilt every OPAM
environment on each push (531 s on Linux x64 to 953 s on Windows),
since only pushes to master save caches. The caches exceeded GitHub's
10 GB, so live ones were evicted: the coverage build state, keyed on a
hash of compiler/**, saved a new 266 MB entry on nearly every master
push, earlier generations stayed until GitHub's 7-day eviction, and the
compiler build states grew with dune's shared cache.
When an OPAM environment misses, a pull request that changes `*.opam` saves
the one it built, in the cache scope of its merge ref, so its later pushes
restore it. Each save deletes the earlier environments with the same key
prefix on the same ref, and a new workflow deletes a closed pull request's
caches. The coverage build state is gone, the compiler build states keep
only `_build`, and the Linux system packages are installed with apt instead
of unpacked from a cached tarball that dpkg didn't know about, so opam took
them for missing.
Review: Why not restore the environment of earlier dependencies and update
it? The result would depend on what that environment held: pins,
repository state, installed packages, depext status, global options.
Environments are still only restored on an exact key, so each is the one a
fresh build of its opam files produced.
Review: Why do only pull requests that change the opam files save? Any
other pull request would store a copy of master's environments, 3.3 GB
each.
Review: Was it tested? On this pull request: a temporary opam change built
and saved all six environments, the next push restored them in 9-39 s, a
second change replaced them, and reverting it restored master's again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gGPsF3fHZn9rBxA7Hm
Signed-off-by: Christoph Knittel <ck@cca.io>
ee5fcc8 to
81a2ded
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9294 +/- ##
==========================================
- Coverage 82.07% 82.06% -0.01%
==========================================
Files 455 455
Lines 65361 65361
==========================================
- Hits 53642 53640 -2
- Misses 11719 11721 +2 🚀 New features to boost your workflow:
|
Problems with the caching on master
*.opam, and only push runs save caches, so such a PR misses on every push: 531 s on Linux x64, 643 s on macOS, 953 s on Windows.compiler/**. Nearly every master push saves a new 266 MB entry that no later run matches: 11 entries, 2.9 GB.~/.cache/dune), which each master push restores, extends and saves again.setup-linux-dependenciesunpacks a tarball of the installed files, so dpkg doesn't know those packages and opam takes them for missing. Anopam installin a restored environment then reinstalls them, or rebuilds the whole switch. On a fresh environment the tarball is redundant anyway:ocaml/setup-ocamlinstalls the same packages with apt.Changes
setup-ocamlchecks withgit diffagainst the base whether the PR changes*.opam; if it does, it saves the environment it built. GitHub scopes caches saved by a PR to its merge ref, so the PR's later pushes hit exactly and no other branch sees them. Other PRs still don't save, since each would store a copy of master's environments. The first push per opam change, and the master push after the merge, still build from scratch.setup-ocamldeletes the caches with the same key prefix on the same ref, so each job keeps one environment per ref. A failure only warns.cache_cleanup.ymlruns onpull_request_target: closed, without checking out the PR, and can also be started by hand with a PR number._build, under a newv2key prefix._buildis one tree that dune updates in place; the shared dune cache only accumulates.setup-linux-dependenciesrunsapt-get install bubblewrap musl-tools rsync cmake, so dpkg knows them. The multilib packages are dropped:ocaml/setup-ocamlinstalls them when it builds an environment, and nothing else needs them.Expected steady state: about 5 GB (OPAM environments 3.3 GB, build states about 0.9 GB, Node), plus one set of environments per open PR that changes the opam files.
After the merge, the old
coverage-build-state-*andcompiler-build-state-v1-*caches can be deleted by hand.Validation
Tested on this PR with temporary commits, since squashed away: two edits to
tools.opam(which change the opam hash) and an empty commit.tools.opameditrefs/pull/9294/mergetools.opameditStill to check after the merge: the build time with
_build-only build states, and that storage settles near 5 GB.🤖 Generated with Claude Code
https://claude.ai/code/session_01Kbf3gGPsF3fHZn9rBxA7Hm