Skip to content

Move the pins by cron, and tell autobuilds when they move - #167

Merged
frangarcj merged 3 commits into
masterfrom
next-discovery
Aug 26, 2026
Merged

Move the pins by cron, and tell autobuilds when they move#167
frangarcj merged 3 commits into
masterfrom
next-discovery

Conversation

@frangarcj

@frangarcj frangarcj commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

The nightly moves when somebody moves it. Nothing resolves the upstream branches the leaves develop on, and nothing connects a commit here to a build of it, so "nightly" means "whenever a person remembered". Two pieces here, and a third that came out of rehearsing the first.

The bot. scripts/bump-pins.py resolves the upstream branch each tracked leaf develops on, writes the exact revision back into cmake/Components.cmake, and commits only what actually moved — its own push is never a reason for the next run to commit again. Which leaf follows which branch is declared in cmake/pin-tracking.json, and so is every pin it must leave alone, with the reason: a released tag (vdpm), a third-party tag (isl), a revision whose packaging assumptions are load-bearing for the core package (vita-makepkg). A pin in neither list stops the bot before it resolves anything, so a source added to the build cannot quietly start or stop being followed.

The announcement. Every push to master sends run_build to vitasdk/autobuilds with the exact revision. That is discovery; the schedule on the other side is only the backstop for the announcement that never arrives. The run_build listeners this repository carried were the same idea from the other direction and never had a sender — with the name now meaning "autobuilds, build this", a stray dispatch could have started a full build here by accident.

What the rehearsal found. Running the bot against the real upstreams, before wiring anything up:

$ python3 scripts/bump-pins.py --dry-run
newlib vita: 892f530fa7d696dbca74abfb8fccfec7d21d269d -> 6cba98129f0a286949391de00519ef39762eff18
toolchain master: c527abce028df33f5281e9ed4994c25fcef53c7d -> eacff34d18e9872a78c0e520e1997ef71900ebb4

newlib's development branch is vita, and the pin was not on it:

$ git merge-base --is-ancestor 892f530fa7d696dbca74abfb8fccfec7d21d269d origin/vita; echo $?
1
$ git rev-list --count origin/vita..892f530fa7d696dbca74abfb8fccfec7d21d269d
2445

The pin was the head of newlib-4.6.0, and vita sat 2445 commits behind it with none of the 4.3, 4.4, 4.5 or 4.6 merges — the SDK was being built from a branch nobody was developing on. Following vita would have undone the 4.6 upgrade on the first nightly, at 03:00, and the only sign would have been whatever broke afterwards.

That is fixed where it was broken rather than worked around here: vita was fast-forwarded onto the 4.6 head, so it now holds what the SDK builds and every leaf in the config simply follows its own development branch.

$ git push origin 892f530fa7d696dbca74abfb8fccfec7d21d269d:refs/heads/vita
   6cba98129..892f530fa  892f530fa7d696dbca74abfb8fccfec7d21d269d -> vita

It was luck that somebody looked, though, so a pin now only moves to a revision that contains it. A force-push upstream, a branch that is not where the work happens, or a pin taken from somewhere else stops the bot and names both revisions:

$ python3 scripts/bump-pins.py --dry-run     # before `vita` was fast-forwarded
bump-pins: newlib: vita is at 6cba98129f0a286949391de00519ef39762eff18, which does not
contain the pinned 892f530fa7d696dbca74abfb8fccfec7d21d269d; moving it would roll the
source back

The check asks the remote for commits and nothing else (--filter=tree:0): newlib's whole history arrives in 7.4 MB and about three seconds, and only a pin that is actually moving pays for it.

Before this is merged. vitasdk/buildscripts has no secrets at all, and both new workflows need REPO_DISPATCH_TOKEN: the bot pushes with it, because a push made with GITHUB_TOKEN fires no workflow and the announcement would never happen, and the announcement sends with it. Without the secret the announce job goes red on every push — deliberately, rather than a chain that looks connected and is not. The cron backstop in autobuilds keeps the nightly alive in the meantime.

Worth knowing too: this arms a live nightly. Merging is itself a push to master, so it announces its own revision, and the first bot run moves vita-toolchain forward 10 commits.

tests/ci/test-bump-pins.sh builds real git repositories as upstreams and never reaches the network: convergence, one leaf moving, the commit message naming what moved, a dry run writing nothing, the untracked pins left alone, a branch that does not exist, a rollback refused, a pin declared nowhere, and a tracked name that is not a pin at all.


AI tools were used in preparing this PR (Claude Opus 5, Anthropic).

Upstream movement used to reach a nightly only when someone edited
Components.cmake by hand, so a nightly was as old as the last time
anybody remembered. A bot resolves the upstream branches the tracked
leaves develop on, writes the exact revisions back into the file, and
commits only what actually moved -- its own push can never be a reason
for the next run to commit again.

Which leaf follows which branch is declared in cmake/pin-tracking.json,
and so is every pin the bot must leave alone and why: a released tag, a
third-party tag, a revision whose packaging assumptions were reviewed.
A pin that appears in neither list stops the bot before it resolves
anything, so a source added to the build cannot quietly start or stop
being followed.

newlib develops on `vita`, not on master; the config says so per leaf
rather than assuming.
Nothing connected a commit on this branch to a build of it: the nightly
moved when a person went and started one. Every push to the development
branch now announces its exact revision to autobuilds, which decides
whether it is worth building -- it deduplicates by build_id, so a
revision it already built costs one job and no runners.

The `run_build` listeners this repository carried were the same idea
from the other direction and no sender ever existed. Keeping them while
the name means "autobuilds, build this" would leave a dispatch able to
start a full build here by accident.
@frangarcj
frangarcj force-pushed the next-discovery branch 2 times, most recently from b285c0e to e358cd3 Compare August 25, 2026 21:33
Rehearsing the bot against the real upstreams found newlib's `vita`
branch 2445 commits behind the pinned revision, missing the 4.3, 4.4,
4.5 and 4.6 merges the SDK is actually built from. Following it would
have undone the 4.6 upgrade on the first nightly, at 03:00, and the only
sign would have been whatever broke afterwards. That is fixed where it
was broken -- `vita` now holds what the SDK builds -- but it was luck
that somebody looked.

So a pin only moves to a revision that contains it. A force-push
upstream, a branch that is not where the work happens, or a pin taken
from somewhere else stops the bot and names both revisions. The check
asks the remote for commits and nothing else: the whole history of the
largest leaf is a few megabytes and about three seconds, and only a pin
that is actually moving pays for it.
@frangarcj
frangarcj merged commit 1a97a87 into master Aug 26, 2026
13 of 19 checks passed
@frangarcj
frangarcj deleted the next-discovery branch August 26, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant