feat(updates): a build takes the newest bundle it can serve, instead of failing

The build fetched whatever the bundle repos called latest and then refused to
continue if it was too new. That is the right answer for a release whose
version bump was forgotten and the wrong one for everything else: a core built
from an older branch would fail on a console that had simply moved on, when
the console it wants is sitting one release back.

So the build asks the same question the server asks — which release of this
bundle can this core serve — and takes the newest that fits. Against the
current releases, a core at 4.0.0-beta.21 now walks past adminui 6.1.1 and
6.1.0, which both want beta.22, and builds with 6.0.0.

The decision is shared rather than reimplemented. `resolveBundleRelease` moves
out of the update handler into `shared/bundleRelease`, with the network
injected, and both callers use it: the server when somebody presses update,
the build when it assembles an image. An image that bakes in a console the
running server would have refused is the failure this whole mechanism exists
to prevent, so the two must not be able to disagree — and the build imports
the compiled module rather than carrying a copy, which `npm run build` makes
possible by running `tsc` before it fetches. Missing `dist/` degrades to the
static URL, which is what this did before any of it existed.

Moving it out also made it testable. It was private and reached the network,
so nothing covered it; there are now seven cases on it, including the two that
motivated the walk (an older core offered the last bundle built for it, a
release from before manifests ending the walk because nothing below it can be
newer) and the channel rules.

Two things follow from the build no longer being strict:

- The release job checks the tag against package.json. A forgotten bump used
  to fail at the fetch; now it would quietly resolve an older console and ship
  it, so the check moves to where the intended version is actually written
  down twice.
- The build steps are given GITHUB_TOKEN. Release listings come from the API,
  which allows sixty requests an hour to anonymous callers, and a CI runner's
  IP address is not its own.

Author: rudyberends

.github/workflows/release.yml

@@ -40,8 +40,28 @@ jobs:
       - name: Install dependencies
         run: npm ci
 
+      # The build below asks each bundle repo for the newest release this version can serve,
+      # so a forgotten version bump would no longer fail loudly — it would quietly resolve an
+      # older console and ship it. The tag is the one place the intended version is written
+      # down twice, so check them against each other here instead.
+      - name: Check the version matches the tag
+        shell: bash
+        run: |
+          TAG="${{ github.event.release.tag_name }}"
+          PKG=$(node -p "require('./package.json').version")
+          if [ "${TAG#v}" != "$PKG" ]; then
+            echo "Release tag $TAG does not match package.json ($PKG)."
+            echo "Bump the version in the commit being tagged, or tag v$PKG."
+            exit 1
+          fi
+          echo "releasing $PKG as $TAG"
+
       - name: Build
         run: npm run build
+        env:
+          # Release listings come from the API, which allows 60 requests an hour per IP to
+          # anonymous callers — and a CI runner's IP is not its own.
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Run tests
         run: npm test
@@ -142,6 +162,7 @@ jobs:
         run: npm run build
         env:
           BUILD_CHANNEL: dev
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Run tests
         run: npm test
@@ -208,6 +229,8 @@ jobs:
 
       - name: Build
         run: npm run build
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - name: Run tests
         run: npm test

scripts/bundleCompat.mjs

@@ -22,6 +22,7 @@
  */
 import { promises as fs } from 'node:fs';
 import { join } from 'node:path';
+import { pathToFileURL } from 'node:url';
 
 function parseVersion(input) {
   const trimmed = String(input ?? '').trim().replace(/^v/i, '');
@@ -147,3 +148,76 @@ export async function assertBundleFitsCore(dir, coreVersion, label) {
       `       or set SONN_SKIP_BUNDLE_COMPAT=1 if you know this pairing is fine.`,
   );
 }
+
+/**
+ * The compiled resolver, or null when there is nothing compiled yet.
+ *
+ * `npm run build` runs `tsc` before it fetches the bundles, so during a real build `dist/` is
+ * there and the decision about which release fits is the *same code the server runs* — which
+ * is the point: an image must not bake in a console the running server would have refused.
+ *
+ * Running `npm run fetch:admin` on its own in a fresh clone is the case where it is missing.
+ * That degrades to the static `releases/latest` URL, which is exactly what this did before
+ * any of it existed, and `assertBundleFitsCore` still refuses a pairing that cannot work.
+ */
+async function loadResolver(cwd) {
+  try {
+    const url = (rel) => pathToFileURL(join(cwd, 'dist', rel)).href;
+    const [release, upstream] = await Promise.all([
+      import(url('shared/bundleRelease.js')),
+      import(url('adapters/http/adminApi/misc/upstreamJson.js')),
+    ]);
+    if (typeof release.resolveBundleRelease !== 'function') return null;
+    return { release, fetchJson: upstream.fetchUpstreamJson };
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Where to download a bundle from, preferring the newest release this core can actually serve.
+ *
+ * The reason this is not simply `releases/latest`: a bundle repo moves on, and a core being
+ * built from an older branch would otherwise either bake in a console it cannot serve or fail
+ * outright. Reaching one release back is the same answer the server gives an install on an
+ * older core, and it keeps a build of an older core producing a working image.
+ *
+ * Honours the existing pins first — `*_DIST_URL` and `*_RELEASE` name a version, and naming
+ * one is not asking for our opinion.
+ */
+export async function resolveBundleUrl({ cwd, repo, assetName, releaseEnv, distUrlEnv, label }) {
+  const urlOverride = (process.env[distUrlEnv] ?? '').trim();
+  if (urlOverride) return { distUrl: urlOverride, picked: 'pinned' };
+
+  const explicit = (process.env[releaseEnv] ?? '').trim();
+  if (explicit && explicit !== 'latest') {
+    return {
+      distUrl: `https://github.com/${repo}/releases/download/${encodeURIComponent(explicit)}/${assetName}`,
+      picked: 'pinned',
+    };
+  }
+
+  const latest = `https://github.com/${repo}/releases/latest/download/${assetName}`;
+  const loaded = await loadResolver(cwd);
+  if (!loaded) return { distUrl: latest, picked: 'latest' };
+
+  const coreVersion = await readCoreVersion(cwd);
+  try {
+    const resolved = await loaded.release.resolveBundleRelease({
+      repo,
+      assetName,
+      coreVersion,
+      channel: loaded.release.channelFor(coreVersion ?? '', process.env.SERVER_RELEASE_CHANNEL),
+      newestWins: await isDevBuild(cwd),
+      fetchJson: loaded.fetchJson,
+      log: (message, detail) => console.log(`[bundle-compat] ${label}: ${message}`, detail ?? ''),
+    });
+    if (resolved.picked === 'compatible' && resolved.release !== 'latest') {
+      console.log(`[bundle-compat] ${label}: ${resolved.release} is the newest that fits core ${coreVersion}`);
+    }
+    return { distUrl: resolved.distUrl, picked: resolved.picked };
+  } catch (err) {
+    console.warn(`[bundle-compat] ${label}: could not resolve a release (${err?.message ?? err}), using latest`);
+    return { distUrl: latest, picked: 'latest' };
+  }
+}

scripts/fetch-admin-dist.mjs

@@ -5,16 +5,21 @@ import { join } from 'node:path';
 import { pipeline } from 'node:stream/promises';
 import { spawn } from 'node:child_process';
 import https from 'node:https';
-import { assertBundleFitsCore, readCoreVersion } from './bundleCompat.mjs';
+import { assertBundleFitsCore, readCoreVersion, resolveBundleUrl } from './bundleCompat.mjs';
 
 const repo = 'sonn-audio/adminui';
 const assetName = 'admin-dist.tgz';
-const release = process.env.ADMINUI_RELEASE ?? 'latest';
-const distUrl =
-  process.env.ADMINUI_DIST_URL ??
-  (release === 'latest'
-    ? `https://github.com/${repo}/releases/latest/download/${assetName}`
-    : `https://github.com/${repo}/releases/download/${release}/${assetName}`);
+// Not simply `releases/latest`: the bundle repos move on, so a core built from an older
+// branch asks for the newest release it can actually serve. Pins and a full URL still win,
+// and a checkout with no `dist/` yet falls back to exactly what this used to do.
+const { distUrl } = await resolveBundleUrl({
+  cwd: process.cwd(),
+  repo,
+  assetName,
+  releaseEnv: 'ADMINUI_RELEASE',
+  distUrlEnv: 'ADMINUI_DIST_URL',
+  label: 'admin ui',
+});
 
 const targetDir = join(process.cwd(), 'public', 'admin');
 const archivePath = join(tmpdir(), `admin-dist-${Date.now()}.tgz`);

scripts/fetch-player-dist.mjs

@@ -6,18 +6,23 @@ import { fileURLToPath } from 'node:url';
 import { pipeline } from 'node:stream/promises';
 import { spawn } from 'node:child_process';
 import https from 'node:https';
-import { assertBundleFitsCore, readCoreVersion } from './bundleCompat.mjs';
+import { assertBundleFitsCore, readCoreVersion, resolveBundleUrl } from './bundleCompat.mjs';
 
 const comingSoonHtml = join(dirname(fileURLToPath(import.meta.url)), 'player-coming-soon.html');
 
 const repo = 'sonn-audio/player';
 const assetName = 'player-dist.tgz';
-const release = process.env.PLAYER_RELEASE ?? 'latest';
-const distUrl =
-  process.env.PLAYER_DIST_URL ??
-  (release === 'latest'
-    ? `https://github.com/${repo}/releases/latest/download/${assetName}`
-    : `https://github.com/${repo}/releases/download/${release}/${assetName}`);
+// Not simply `releases/latest`: the bundle repos move on, so a core built from an older
+// branch asks for the newest release it can actually serve. Pins and a full URL still win,
+// and a checkout with no `dist/` yet falls back to exactly what this used to do.
+const { distUrl } = await resolveBundleUrl({
+  cwd: process.cwd(),
+  repo,
+  assetName,
+  releaseEnv: 'PLAYER_RELEASE',
+  distUrlEnv: 'PLAYER_DIST_URL',
+  label: 'player',
+});
 
 const targetDir = join(process.cwd(), 'public', 'player');
 const archivePath = join(tmpdir(), `player-dist-${Date.now()}.tgz`);