Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 53 additions & 13 deletions .github/workflows/drift.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
name: drift

# Pins the newest server, rebuilds index.db with it from the live wiki and compares its content
# with the committed index. When the content changed, it pushes drift/index with the rebuilt
# index and the next patch version, or the next minor when the schema grew, opens or updates its
# pull request as the tibia-sh App, turns on auto-merge and waits for the merge. The ruleset
# merges it once test and oldest-consumer pass, and that merge publishes through release.yml.
# When only the pin moved, the pull request carries package.json and pnpm-lock.yaml alone, at the
# same version, and its merge publishes nothing. Nothing here publishes by itself. A refresh that
# loses rows, or a table or column, as scripts/drift-guard.ts decides, is held instead: its pull
# request stays open with auto-merge off until a person merges or closes it. A run on main that
# fails or holds a refresh comments on the one alert issue, which alert.yml also writes for
# release.yml, assigned to the maintainer.
# Pins the newest server, rebuilds index.db with it from the live wiki on the tip of main and
# compares its content with the committed index. When the content changed, it pushes drift/index
# with the rebuilt index and the next patch version, or the next minor when the schema grew, opens
# or updates its pull request as the tibia-sh App, turns on auto-merge and waits for the merge.
# The ruleset merges it once test and oldest-consumer pass, and that merge publishes through
# release.yml. When only the pin moved, the pull request carries package.json and pnpm-lock.yaml
# alone, at the same version, and its merge publishes nothing. Nothing here publishes by itself. A
# refresh that loses rows, or a table or column, as scripts/drift-guard.ts decides, is held
# instead: its pull request stays open with auto-merge off until a person merges or closes it. A
# run on main that fails or holds a refresh comments on the one alert issue, which alert.yml also
# writes for release.yml, assigned to the maintainer.

on:
schedule:
Expand All @@ -25,7 +25,9 @@ on:
permissions: {}

# Runs take turns, so two cannot push drift/index at once. One in progress is never
# cancelled. The group is not release.yml's, so a drift run never holds up a publish.
# cancelled. The group is not release.yml's, so a drift run never holds up a publish. A run
# that waited is still given main as it was at its event, and the run before it may have merged
# since, so the build job moves to the tip of main and the pr job to the build job's commit.
concurrency:
group: drift
cancel-in-progress: false
Expand All @@ -40,6 +42,9 @@ jobs:
permissions:
contents: read
outputs:
# The commit the job built on, the tip of main when it started. A dispatch from another branch
# skips the move and hands on nothing, which is safe only while the pr job runs on main alone.
sha: ${{ steps.main.outputs.sha }}
committed: ${{ steps.digests.outputs.committed }}
rebuilt: ${{ steps.digests.outputs.rebuilt }}
sha256: ${{ steps.digests.outputs.sha256 }}
Expand All @@ -59,6 +64,19 @@ jobs:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
# The move to the tip of main, before the install and the scripts read anything, so a run that
# waited behind another one builds on what that run merged. The checkout kept no credentials,
# and the fetch needs none, since the repository is public. Three lines, not one joined with
# &&: under bash -e a failed fetch there would not end the step, which would then hand on the
# event's commit, and the pr job accepts that one. A dispatch from another branch builds that
# branch as it was.
- name: Move to the tip of main
id: main
if: ${{ github.ref == 'refs/heads/main' }}
run: |
git fetch --depth=1 origin main
git switch --detach FETCH_HEAD
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
# No cache, restored or saved, but pnpm/setup's lockfile-verification record. A cache saved
# after the generator ran would carry whatever that run left into later runs. This setup-node
# caches by itself whenever package.json names a packageManager, so it is switched off by name.
Expand Down Expand Up @@ -299,10 +317,32 @@ jobs:
contents: read # the checkout. The App token does every write.
pull-requests: read # the wait for the merge, which reads with github.token
steps:
# No ref: the commit that triggered the run, which is the one the build job rebuilt.
# No ref: the checkout takes the event's commit, which main can have moved past while the run
# waited. The build job built index.db on the tip of main, so the job moves to that commit
# next, and commits on top of the main the refresh was built from.
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
# Before any step reads the checkout. A job output is data, so it is checked whole before git
# reads it. The build job runs the generator, which could hand on any commit a fetch by SHA
# reaches, a pull request's head or an older commit of main among them, and this job commits
# on it with the App token. The build job moved to main's tip after the event, so before the
# switch the commit must be in main's history and descend from GITHUB_SHA, the event's commit,
# which the runner sets. The shallow checkout reaches back to the event's commit only, so the
# history is fetched whole, commits and trees without their files, and the switch fetches the
# files it needs. The trees are small here, since index.db is a file. Not tree:0, which leaves
# the trees out too: git 2.56.0 crashes repacking that fetch once main has moved past the
# checkout's commit, which is the case this move is for.
- name: Move to the build job's commit
id: move
env:
SHA: ${{ needs.build.outputs.sha }}
run: |
[[ $SHA =~ ^[0-9a-f]{40}$ ]] || { echo 'the build job handed on no full commit SHA' >&2; exit 1; }
git fetch --filter=blob:none --unshallow origin +refs/heads/main:refs/remotes/origin/main
git merge-base --is-ancestor "$SHA" refs/remotes/origin/main || { echo 'not a commit of main' >&2; exit 1; }
git merge-base --is-ancestor "$GITHUB_SHA" "$SHA" || { echo "older than the event's commit" >&2; exit 1; }
git fetch origin "$SHA" && git switch --detach "$SHA"
# Switched off by name here too. Left on, it would restore a pnpm store cache.
- uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
Expand Down
34 changes: 22 additions & 12 deletions docs/MAINTAINING.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,11 @@ you run it by hand from the Actions tab, and when tibiawiki-mcp's release workfl
`repository_dispatch` event `server-release`, with `{"version": "<x.y.z>"}` as its payload,
once npm accepted a server publish.

Runs take turns, and a run that waited is given `main` as it was at its event, which the run
before it can have moved with a merge. So the build job's first step after its checkout fetches
the tip of `main` and switches to it, before the install and every script, and hands that commit
to the `pr` job. A run on another branch builds that branch as it is.

Before the generator runs, its `Pin the server` step decides which server builds the index, with
`scripts/server-pin.ts`:

Expand Down Expand Up @@ -193,14 +198,18 @@ index added a table or a column, as [How the index is built](#how-the-index-is-b
and `patch` otherwise. A changed `SCHEMA_VERSION` fails `pnpm test`, since the package major
must equal it, so a new major ends the run red for a person rather than opening a pull request.

The `pr` job first checks each value the build job handed on whole: `changed` and `pin_moved`
are `true` or `false`, the level is `patch` or `minor`, and the server is `x.y.z`, above `main`'s
pin when it moved and `main`'s pin when it did not. The build job ran the generator, so the `pr`
job takes none of its files but the index. When the pin moved, it recomputes the pin itself with
`pnpm add --lockfile-only --ignore-scripts --ignore-pnpmfile`, which installs nothing and runs no
dependency code, requires that the recompute changed `package.json` and `pnpm-lock.yaml` alone,
and `package.json` only in the server's version, and requires both files to hash to what the
build job built with. Its `pnpm/setup` installs pnpm alone and restores no cache.
The `pr` job first moves to the commit the build job built on, before any step reads its
checkout. It switches only once the commit is a full SHA, is in `main`'s history and descends
from the commit of the run's event, so it never commits on a commit outside `main` or on one
older than the event's. It then checks each value the build job handed on whole: `changed` and
`pin_moved` are `true` or `false`, the level is `patch` or `minor`, and the server is `x.y.z`,
above `main`'s pin when it moved and `main`'s pin when it did not. The build job ran the
generator, so the `pr` job takes none of its files but the index. When the pin moved, it
recomputes the pin itself with `pnpm add --lockfile-only --ignore-scripts --ignore-pnpmfile`,
which installs nothing and runs no dependency code, requires that the recompute changed
`package.json` and `pnpm-lock.yaml` alone, and `package.json` only in the server's version, and
requires both files to hash to what the build job built with. Its `pnpm/setup` installs pnpm
alone and restores no cache.

When the content changed, it sets `version` to the next patch or minor npm does not have, as
the level says, pushes the rebuilt `index.db`, with the moved pin if there is one, to the
Expand Down Expand Up @@ -237,10 +246,11 @@ refresh, comments on the issue `Automation needs a look`, or opens it assigned t
[When automation needs a look](RELEASING.md#when-automation-needs-a-look) says what to do.

- A tripped gate, a failing test, a guard that cannot read an index, an unreadable registry, a
dispatch that names no `x.y.z` version, a server npm does not serve within 45 minutes, a
build job output or a recomputed pin the `pr` job does not accept, or a `version` on `main`
that npm does not list yet each end the run red before anything is pushed. A pull request closed without merging, merged at another commit, left with auto-merge
off, or not merged within 60 minutes ends it red after the push.
fetch of `main` that fails, a dispatch that names no `x.y.z` version, a server npm does not
serve within 45 minutes, a build job commit, output or recomputed pin the `pr` job does not
accept, or a `version` on `main` that npm does not list yet each end the run red before
anything is pushed. A pull request closed without merging, merged at another commit, left with
auto-merge off, or not merged within 60 minutes ends it red after the push.
- Each run that finds a change replaces `drift/index`, so an open pull request always carries
the newest rebuild. A held pull request you leave open is not frozen: when a later run's
guard finds no reason to hold, that run turns auto-merge on and it merges by itself.
Expand Down
2 changes: 2 additions & 0 deletions docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,8 +215,10 @@ dropped. Each comment links the run and says what happened:
|---|---|
| `The build job did not succeed.` and its result | `cancelled` means the job hit its 90 minute bound, most often a slow wiki, or someone cancelled the run. Run drift again. For `failure`, open the run and read the red step. A tripped gate in `pnpm build-index`, a failing `pnpm test`, a wiki the build could not read, or `The guard could not compare the indexes.` each end it there, before anything is pushed. Fix the cause, or run drift again once the wiki is back. |
| `The pr job did not succeed.` and its result | `cancelled` means the job hit its 75 minute bound or someone cancelled the run. Auto-merge stays on, so check the drift pull request. For `failure`, open the run and read the red step. `npm does not list X.Y.Z from package.json yet` means the release of the version on `main` is still running or failed, so follow [When a release run fails](#when-a-release-run-fails) and run drift again once npm has it. `was closed without merging` means someone closed the pull request. `merged <commit>, not <commit>` means the pull request merged a commit other than the one this run pushed, so read `git log` on `main` for what landed. `is off. Merge it, or turn auto-merge on again, by hand` means someone or something turned auto-merge off during the wait: merge the pull request with Rebase and merge once its checks pass, or turn auto-merge on again with `gh pr merge <number> --auto --rebase`, and the merge publishes. `failed 3 times in a row` means `gh` could not read the pull request, so check it by hand. `has not merged 3600 seconds after auto-merge was on` means its checks failed or are still running: auto-merge stays on, so it merges by itself once they pass, for example after you re-run a check that failed for a reason outside the repository. A red `Get a token of the tibia-sh App` step, or `Bad credentials` from `gh`, means the App's key or its installation no longer works, as [The tibia-sh App](#the-tibia-sh-app) describes. |
| `The build job did not succeed.`, with a red `Move to the tip of main` | The build job could not fetch the tip of `main`, or switch to it, so it built nothing and handed nothing on. Nothing was pushed. That is most often a passing GitHub outage, so run drift again. |
| `The build job did not succeed.`, with `npm did not serve server X.Y.Z within 2700 seconds.` in `Pin the server` | The run was moving the pin to server X.Y.Z, and for 45 minutes npm either did not list that version's tarball or a `HEAD` on it did not answer 200. Nothing was built or pushed. Check the server's release run in tibiawiki-mcp and `npm view @tibia.sh/tibiawiki-mcp@X.Y.Z dist.tarball`. Once npm serves it, run drift again by hand. A scheduled run also picks it up when it is `latest`. |
| `The build job did not succeed.`, with `The server-release dispatch asks for '…', not an x.y.z version.` in `Pin the server` | A `server-release` dispatch carried a payload version that is not `x.y.z`, and the step used none of it. Nothing was built or pushed. Fix the sender, tibiawiki-mcp's release workflow. Then run drift by hand, which moves the pin to npm's `latest`. |
| `The pr job did not succeed.`, with `the build job handed on no full commit SHA`, `not a commit of main` or `older than the event's commit` in `Move to the build job's commit` | The commit the build job handed on is not a full SHA, is not in `main`'s history, or is older than the commit of the run's event, so the job did not move to it. Nothing was pushed, and no token was minted. The build job moves to the tip of `main` after the event, so the commit it hands on is always a commit of `main` at or after the event's. A value like this means either a mistake in `drift.yml`, such as a misspelt output, or a build job something tampered with. Read the build job's log, and in particular `Move to the tip of main`. Run drift again only once you know which it was. |
| `The pr job did not succeed.`, with a red `Check the build job's outputs` step | The build job handed on a value the `pr` job refuses: `changed`, `pin_moved` or `level` out of range, `Not a server version`, `moved the pin from A to B, which is not above it`, `kept the pin at B, but main pins A`, or `Neither the content nor the pin changed`. Nothing was pushed, and no token was minted. The build job runs the generator, so a value like this means either a mistake in `drift.yml`, such as a misspelt output, or a build job something tampered with. Read the build job's log, and in particular `Pin the server`. Run drift again only once you know which it was. |
| `The pr job did not succeed.`, with `The recompute changed other files than package.json and pnpm-lock.yaml:` or `package.json changed in more than the pin of @tibia.sh/tibiawiki-mcp` in `Keep the recompute to the pin` | The `pr` job's own `pnpm add --lockfile-only` changed more than the server's pin. Nothing was pushed. That usually means a pnpm or `pnpm-workspace.yaml` change rewrote the manifests. Run the same command, `pnpm add -D --save-exact --lockfile-only --ignore-scripts --ignore-pnpmfile @tibia.sh/tibiawiki-mcp@X.Y.Z`, on a clean checkout of `main` and read `git diff`. When the extra change is right, land it in a reviewed pull request of its own, then run drift again. |
| `The pr job did not succeed.`, with `The recomputed package.json hashes to …` or `The recomputed pnpm-lock.yaml hashes to …, but the build job built with …` in `Require the manifests the build job built with` | The manifests the `pr` job recomputed differ from the ones the build job installed and built `index.db` with, so the job refused to commit them. Nothing was pushed. A first-party release published between the two jobs can change the resolution, since `@tibia.sh/*` skips the cooldown, so run drift again. When it repeats, compare a full `pnpm add -D --save-exact` with the `--lockfile-only` one on a clean checkout. When those two agree, suspect the build job. |
Expand Down
Loading
Loading