Move the pins by cron, and tell autobuilds when they move - #167
Merged
Conversation
Upstream movement used to reach a nightly only when someone edited Components.cmake by hand, so a nightly was as old as the last time anybody remembered. A bot resolves the upstream branches the tracked leaves develop on, writes the exact revisions back into the file, and commits only what actually moved -- its own push can never be a reason for the next run to commit again. Which leaf follows which branch is declared in cmake/pin-tracking.json, and so is every pin the bot must leave alone and why: a released tag, a third-party tag, a revision whose packaging assumptions were reviewed. A pin that appears in neither list stops the bot before it resolves anything, so a source added to the build cannot quietly start or stop being followed. newlib develops on `vita`, not on master; the config says so per leaf rather than assuming.
Nothing connected a commit on this branch to a build of it: the nightly moved when a person went and started one. Every push to the development branch now announces its exact revision to autobuilds, which decides whether it is worth building -- it deduplicates by build_id, so a revision it already built costs one job and no runners. The `run_build` listeners this repository carried were the same idea from the other direction and no sender ever existed. Keeping them while the name means "autobuilds, build this" would leave a dispatch able to start a full build here by accident.
frangarcj
force-pushed
the
next-discovery
branch
2 times, most recently
from
August 25, 2026 21:33
b285c0e to
e358cd3
Compare
Rehearsing the bot against the real upstreams found newlib's `vita` branch 2445 commits behind the pinned revision, missing the 4.3, 4.4, 4.5 and 4.6 merges the SDK is actually built from. Following it would have undone the 4.6 upgrade on the first nightly, at 03:00, and the only sign would have been whatever broke afterwards. That is fixed where it was broken -- `vita` now holds what the SDK builds -- but it was luck that somebody looked. So a pin only moves to a revision that contains it. A force-push upstream, a branch that is not where the work happens, or a pin taken from somewhere else stops the bot and names both revisions. The check asks the remote for commits and nothing else: the whole history of the largest leaf is a few megabytes and about three seconds, and only a pin that is actually moving pays for it.
frangarcj
force-pushed
the
next-discovery
branch
from
August 25, 2026 21:44
e358cd3 to
eef7406
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The nightly moves when somebody moves it. Nothing resolves the upstream branches the leaves develop on, and nothing connects a commit here to a build of it, so "nightly" means "whenever a person remembered". Two pieces here, and a third that came out of rehearsing the first.
The bot.
scripts/bump-pins.pyresolves the upstream branch each tracked leaf develops on, writes the exact revision back intocmake/Components.cmake, and commits only what actually moved — its own push is never a reason for the next run to commit again. Which leaf follows which branch is declared incmake/pin-tracking.json, and so is every pin it must leave alone, with the reason: a released tag (vdpm), a third-party tag (isl), a revision whose packaging assumptions are load-bearing for the core package (vita-makepkg). A pin in neither list stops the bot before it resolves anything, so a source added to the build cannot quietly start or stop being followed.The announcement. Every push to master sends
run_buildtovitasdk/autobuildswith the exact revision. That is discovery; the schedule on the other side is only the backstop for the announcement that never arrives. Therun_buildlisteners this repository carried were the same idea from the other direction and never had a sender — with the name now meaning "autobuilds, build this", a stray dispatch could have started a full build here by accident.What the rehearsal found. Running the bot against the real upstreams, before wiring anything up:
newlib's development branch is
vita, and the pin was not on it:The pin was the head of
newlib-4.6.0, andvitasat 2445 commits behind it with none of the 4.3, 4.4, 4.5 or 4.6 merges — the SDK was being built from a branch nobody was developing on. Followingvitawould have undone the 4.6 upgrade on the first nightly, at 03:00, and the only sign would have been whatever broke afterwards.That is fixed where it was broken rather than worked around here:
vitawas fast-forwarded onto the 4.6 head, so it now holds what the SDK builds and every leaf in the config simply follows its own development branch.It was luck that somebody looked, though, so a pin now only moves to a revision that contains it. A force-push upstream, a branch that is not where the work happens, or a pin taken from somewhere else stops the bot and names both revisions:
The check asks the remote for commits and nothing else (
--filter=tree:0): newlib's whole history arrives in 7.4 MB and about three seconds, and only a pin that is actually moving pays for it.Before this is merged.
vitasdk/buildscriptshas no secrets at all, and both new workflows needREPO_DISPATCH_TOKEN: the bot pushes with it, because a push made withGITHUB_TOKENfires no workflow and the announcement would never happen, and the announcement sends with it. Without the secret the announce job goes red on every push — deliberately, rather than a chain that looks connected and is not. The cron backstop inautobuildskeeps the nightly alive in the meantime.Worth knowing too: this arms a live nightly. Merging is itself a push to master, so it announces its own revision, and the first bot run moves vita-toolchain forward 10 commits.
tests/ci/test-bump-pins.shbuilds real git repositories as upstreams and never reaches the network: convergence, one leaf moving, the commit message naming what moved, a dry run writing nothing, the untracked pins left alone, a branch that does not exist, a rollback refused, a pin declared nowhere, and a tracked name that is not a pin at all.AI tools were used in preparing this PR (Claude Opus 5, Anthropic).