feat(scenarios): gallery-refresh, the gallery refresh in one deterministic step

.changeset/31121-scenarios-gallery-refresh.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/scenarios': patch
+---
+
+feat(scenarios): `gallery-refresh`, the gallery refresh in one deterministic step
+
+Clause-②: yes — a new `gallery-refresh` entry point on the published `@objectstack/scenarios` widens its surface; the PR body carries the reading
+Metadata-protocol: none
+
+- **`pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR [--cards DIR] [--model M] [--debut] [--work DIR] [--chromium PATH]`** (`src/export/refresh-cli.ts`): reads npm's latest stable `@objectstack/cli` and the `framework.version` of every app the gallery checkout's `index.json` lists. When every listed app states that release it prints one `skip: …` line and exits 0, having run nothing; it also skips while a remote-tracking `export/<version>` branch awaits review. Otherwise it runs every card the gallery may show (origin `generic`, tier `starter` or `delivery`; `--debut`: the first of them, only into an empty gallery) with `run-scenarios --modes scratch --packages published --create-version <version>`, exports with screenshots into the checkout, commits `apps/` and `index.json` on a new local branch `export/<version>` and prints the branch, the commit and a ready PR body. It never pushes and never opens a PR. Every refusal (not a clean checkout, an index that does not validate, a prerelease `latest`, an existing local branch, no app exported, an app on another framework version, a change outside `apps/` and `index.json`) exits 1 with the reason.
+- **`@objectstack/scenarios/export`** also exports the step as a library: `galleryRefresh`, `runnerArgv`, `readGalleryVersions`, `npmLatestVersion`, `refreshPrBody`, `galleryBranch`, `RefreshRefusal`, `FRAMEWORK_PACKAGE`, `STABLE_VERSION` and their types.
+- The live driver's header names ruling D2: a local session or a scheduled cloud Claude Code task on the maintainer's subscription login may run the cards; an API-billed model still never may. The driver's refusal of an API key is unchanged.
+- The README gains "Refreshing the gallery — `gallery-refresh`" and the scheduled task's runbook: what it runs, what it needs, what it never does, and the task's prompt verbatim.

packages/qa/scenarios/README.md

@@ -266,6 +266,78 @@ npx @objectstack/scenarios help                                      # the comma
   dashboards and nav targets. `test/runner/prompt-trace.test.ts` reads the checks off the scorer itself and holds every
   card to it.
 
+## Refreshing the gallery — `gallery-refresh`
+
+```bash
+G="$(realpath ../gallery)"   # pnpm runs the script in the package directory: pass absolute paths
+pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G"                 # every exportable card
+pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G" --debut         # one card: the gallery's first app
+pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G" --model sonnet  # the model every turn asks for
+```
+
+One step from "is there a new stable release?" to a local commit in a checkout of objectstack-ai/gallery
+(`src/export/refresh.ts`, CLI `src/export/refresh-cli.ts`). It reads npm's `latest` `@objectstack/cli` and the
+`framework.version` of every app the gallery's `index.json` lists. When the index lists an app and every app states
+that version, it prints one `skip: …` line and exits 0, having run nothing. It also skips when a remote-tracking
+`export/<version>` branch exists: that refresh was pushed and awaits review. Otherwise it:
+
+1. runs every card the gallery may show (origin `generic`, tier `starter` or `delivery`; `--debut`: the first of them
+   by id) with `run-scenarios --modes scratch --packages published --create-version <version>`, so the apps are built
+   on exactly that release, the turns recorded;
+2. exports with screenshots into the checkout (`export-gallery`'s rules: only a run that passed every half);
+3. creates the branch `export/<version>`, commits `apps/` and `index.json` under the checkout's own git identity, and
+   prints the branch, the commit and a ready PR body.
+
+It refuses, exit 1 with the reason, when: `--gallery` is not the top of a git checkout or has uncommitted changes;
+the index does not validate; npm's `latest` is not a plain `MAJOR.MINOR.PATCH`; a local `export/<version>` branch
+already exists; `--debut` meets an index that lists an app; no card may be exported; no app exported; an exported app
+states another framework version; or anything outside `apps/` and `index.json` changed. The last three leave the
+working tree as the export left it, uncommitted. `--work DIR` keeps the records, projects and recording (default: a
+temp directory, named on a refusal). ⛔ It never pushes, never opens a PR and never talks to GitHub.
+
+### The scheduled task's runbook
+
+Ruling D2 (2026-11-06): a daily scheduled Claude Code task in this project's cloud environment runs the refresh on the
+maintainer's subscription login, and the maintainer merges its PR on the gallery.
+
+- **What it runs**: `gallery-refresh` from a fresh checkout of this repository, into a fresh clone of
+  objectstack-ai/gallery; then `git push` of the one branch it created, and one PR.
+- **What it needs**: both repositories in the environment's resources, with push access to the gallery; npm reachable;
+  the container's `claude` logged in with the subscription (`claude auth status` reports `oauth_token`) and neither
+  `ANTHROPIC_API_KEY` nor `ANTHROPIC_AUTH_TOKEN` set (the live driver refuses both); a permission mode that lets the
+  session launch the headless `claude -p` the runner spawns (Auto mode's classifier refuses it, see
+  [the cloud session settings](#running-the-cards-in-a-claude-code-cloud-session)); Chromium for the screenshots
+  (preinstalled in the cloud container). A full refresh runs one `starter` build after another, longer than the
+  2-hour cap on a background command, so the task starts it detached and waits in the foreground.
+- **What it never does**: push `main` or any branch but `export/<version>`, force-push, merge, edit a gallery file by
+  hand, pass `--packages workspace`, or run on an API key. A refusal is reported, never retried around.
+
+The task's prompt, verbatim:
+
+```text
+Refresh the ObjectStack scenario gallery. Run each step in order and stop at the first that fails.
+
+1. Check the login: `unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN; claude auth status` must report oauth_token.
+   Anything else is a failure (step 6).
+2. In the objectstack checkout: `git fetch origin main && git checkout --detach origin/main`, `pnpm install`,
+   `pnpm --filter '@objectstack/scenarios^...' build`.
+3. Clone the gallery beside it: `git clone https://github.com/objectstack-ai/gallery ../gallery`.
+   If ../gallery/index.json is missing or lists no app, add `--debut` to the command in step 4.
+4. Start the refresh detached and wait for it in the foreground:
+   `nohup sh -c 'pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$(realpath ../gallery)" --work /tmp/gallery-refresh; echo "gallery-refresh exit $?"' > /tmp/gallery-refresh.log 2>&1 & echo $! > /tmp/gallery-refresh.pid`
+   then repeat `timeout 590 tail --pid="$(cat /tmp/gallery-refresh.pid)" -f /dev/null` until the process is gone.
+   The log's last line is `gallery-refresh exit N`; any N but 0 is a failure (step 6).
+5. If a line of the log starts with `skip:`, stop: report that line and do nothing else.
+   Otherwise the log names a `branch: export/VERSION`: push exactly that branch
+   (`git -C ../gallery push origin export/VERSION`) and open one pull request on objectstack-ai/gallery from it into
+   main, titled `Gallery refresh for ObjectStack VERSION`, with the text after "open the PR with this body:" as its
+   body. Never push main, never force-push, never merge, never edit a gallery file by hand.
+6. On any failure (a refusal, a non-zero exit, a failed push or PR): search the open issues of
+   objectstack-ai/objectstack for one titled `Gallery refresh failed`. If one exists, add a comment with today's
+   date, the step that failed and the last 40 lines of /tmp/gallery-refresh.log; otherwise open one issue with that
+   title and the same text. Never open a second one.
+```
+
 ## Other agent tools — Codex and the external driver contract
 
 The same card can measure another agent tool. Every record (`env.driver`) and every gallery app page (`- Driver:`)

packages/qa/scenarios/package.json

@@ -66,7 +66,8 @@
     "test": "vitest run",
     "typecheck": "tsc --noEmit",
     "run-scenarios": "tsx src/runner/cli.ts",
-    "export-gallery": "tsx src/export/cli.ts"
+    "export-gallery": "tsx src/export/cli.ts",
+    "gallery-refresh": "tsx src/export/refresh-cli.ts"
   },
   "dependencies": {
     "@objectstack/core": "workspace:*",

packages/qa/scenarios/src/export/cli.ts

@@ -85,6 +85,12 @@ export function renderSummary(s: ExportSummary): string[] {
   return lines;
 }
 
+/** The real screenshot step: boot the project, sign in as the dev admin, drive Chromium (`--chromium PATH`, or Playwright's own). */
+export function liveScreenshots(chromium?: string): CaptureScreenshots {
+  const browser = playwrightBrowser(chromium ? { executablePath: chromium } : {});
+  return (input) => captureScreenshots(input, { boot: (dir) => bootDev(dir), harness: (url) => new HttpApp({ baseUrl: url, adminEmail: ADMIN.email, adminPassword: ADMIN.password }), browser, admin: ADMIN });
+}
+
 /** Test seam: replace the screenshot step (the CLI's is the real browser). */
 export interface ExportCliDeps {
   screenshots?: CaptureScreenshots;
@@ -102,11 +108,7 @@ export async function main(argv: readonly string[], deps: ExportCliDeps = {}): P
   });
   for (const r of refusals) console.error(formatRefusal(r));
   const recordings = args.recordings.map((f) => JSON.parse(readFileSync(resolve(f), 'utf8')) as Recording);
-  const browser = playwrightBrowser(args.chromium ? { executablePath: args.chromium } : {});
-  const screenshots: CaptureScreenshots | false = !args.screenshots
-    ? false
-    : (deps.screenshots ??
-      ((input) => captureScreenshots(input, { boot: (dir) => bootDev(dir), harness: (url) => new HttpApp({ baseUrl: url, adminEmail: ADMIN.email, adminPassword: ADMIN.password }), browser, admin: ADMIN })));
+  const screenshots: CaptureScreenshots | false = !args.screenshots ? false : (deps.screenshots ?? liveScreenshots(args.chromium));
   const summary = await exportGallery({
     runsDir,
     outDir: resolve(args.out!),

packages/qa/scenarios/src/export/index.ts

@@ -4,14 +4,17 @@
  * `@objectstack/scenarios/export` — the gallery exporter: verified scenario
  * apps written to a local directory in the layout of objectstack-ai/gallery.
  * Private, like the rest of this package. CLI:
- * `pnpm --filter @objectstack/scenarios export-gallery --runs DIR --out DIR`.
+ * `pnpm --filter @objectstack/scenarios export-gallery --runs DIR --out DIR`;
+ * the whole refresh (version check, run, export, commit) is
+ * `pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR`.
  */
 
 export * from './exporter.js';
 export * from './generic.js';
 export * from './manifest.js';
 export * from './pass.js';
 export * from './record.js';
+export * from './refresh.js';
 export * from './screenshots.js';
 export * from './source.js';
 export * from './transcript.js';

packages/qa/scenarios/src/export/refresh-cli.ts

@@ -0,0 +1,131 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * `pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR [flags]`
+ * — the gallery refresh in one step (`refresh.ts`): npm's latest stable
+ * `@objectstack/cli` against the gallery's `index.json`; when they differ, run
+ * the cards on that release, export into the gallery checkout and commit on
+ * `export/<version>`. Ruling D2: a daily scheduled Claude Code task on the
+ * maintainer's subscription login runs it (the package README holds the
+ * task's runbook).
+ *
+ *   --gallery DIR        the gallery checkout, the top of its git working tree (required)
+ *   --cards DIR          the card directory (default: the repo's docs/qa/scenarios);
+ *                        any other directory is loaded as the private overlay,
+ *                        where a `generic` card is refused (`libraryOf`)
+ *   --model M            the model every live turn asks for (`run-scenarios --model`;
+ *                        default: the login's own)
+ *   --debut              run and export one card only, the first exportable card
+ *                        by id: the gallery's first app (refused once the index lists one)
+ *   --work DIR           where the run's records, projects and recording go
+ *                        (default: a new temp directory)
+ *   --chromium PATH      the Chromium executable for the screenshots
+ *
+ * Output: `skip: …` (one line, exit 0) when the gallery is already on the
+ * release; otherwise the branch, the commit and the PR body, exit 0. A refusal
+ * prints why and exits 1. ⛔ It never pushes and never opens the PR.
+ */
+
+import { existsSync, mkdirSync, mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+import { formatRefusal, libraryOf, loadScenarios, readCapabilityAreas } from '../loader.js';
+import { findUp, main as runScenarios } from '../runner/cli.js';
+import { liveScreenshots, renderSummary } from './cli.js';
+import { galleryRefresh, RefreshRefusal, runnerArgv, type RefreshDeps, type RefreshOutcome } from './refresh.js';
+
+export interface RefreshCliArgs {
+  gallery: string;
+  cards?: string;
+  model?: string;
+  debut: boolean;
+  work?: string;
+  chromium?: string;
+}
+
+export function parseRefreshArgs(argv: readonly string[]): RefreshCliArgs {
+  const a: Partial<RefreshCliArgs> & { debut: boolean } = { debut: false };
+  for (let i = 0; i < argv.length; i++) {
+    const flag = argv[i]!;
+    const val = () => {
+      const v = argv[++i];
+      if (v === undefined) throw new Error(`${flag} needs a value`);
+      return v;
+    };
+    if (flag === '--gallery') a.gallery = val();
+    else if (flag === '--cards') a.cards = val();
+    else if (flag === '--model') {
+      const raw = val();
+      if (raw.trim() === '' || raw.startsWith('-')) throw new Error(`--model takes a model alias or name (sonnet, opus, claude-sonnet-4-5), not \`${raw}\``);
+      a.model = raw;
+    } else if (flag === '--debut') a.debut = true;
+    else if (flag === '--work') a.work = val();
+    else if (flag === '--chromium') a.chromium = val();
+    else throw new Error(`unknown flag \`${flag}\``);
+  }
+  if (!a.gallery) throw new Error('--gallery DIR is required: the gallery checkout the refresh exports into and commits in');
+  return a as RefreshCliArgs;
+}
+
+/** What the CLI prints for an outcome, one entry per line. */
+export function renderOutcome(o: RefreshOutcome): string[] {
+  if (o.kind === 'skip') return [o.line];
+  return [
+    ...renderSummary(o.summary),
+    `branch: ${o.branch}`,
+    `commit: ${o.commit}`,
+    'Not pushed. Push the branch and open the PR with this body:',
+    '',
+    o.body,
+  ];
+}
+
+/** Test seam: replace the steps that reach outside (npm, the model, the browser). */
+export type RefreshCliDeps = Partial<RefreshDeps>;
+
+export async function main(argv: readonly string[], deps: RefreshCliDeps = {}, io = { out: (l: string) => console.log(l), err: (l: string) => console.error(l) }): Promise<number> {
+  const args = parseRefreshArgs(argv);
+  const here = dirname(fileURLToPath(import.meta.url));
+  const repo = findUp(here, (d) => existsSync(join(d, 'pnpm-workspace.yaml')));
+  const cardsDir = resolve(args.cards ?? join(repo, 'docs/qa/scenarios'));
+  const { cards, refusals } = loadScenarios(cardsDir, {
+    areas: readCapabilityAreas(join(repo, 'docs/qa/platform-checklist/areas')),
+    library: libraryOf(cardsDir, repo),
+  });
+  for (const r of refusals) io.err(formatRefusal(r));
+  const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'gallery-refresh-')));
+  mkdirSync(work, { recursive: true });
+  const run: RefreshDeps['run'] =
+    deps.run ??
+    (async (plan) => {
+      const code = await runScenarios(runnerArgv(plan));
+      if (code !== 0) throw new RefreshRefusal(`run-scenarios exited ${code}: nothing is exported`);
+    });
+  try {
+    const outcome = await galleryRefresh(
+      { galleryDir: resolve(args.gallery), cards, ...(args.cards ? { cardsDir } : {}), debut: args.debut, ...(args.model ? { model: args.model } : {}), workDir: work },
+      { ...deps, run, screenshots: deps.screenshots ?? liveScreenshots(args.chromium) },
+    );
+    // A skip ran nothing: the temp directory it was given is removed again.
+    if (outcome.kind === 'skip' && args.work === undefined) rmSync(work, { recursive: true, force: true });
+    for (const line of renderOutcome(outcome)) io.out(line);
+    return 0;
+  } catch (e) {
+    if (!(e instanceof RefreshRefusal)) throw e;
+    io.err(`gallery-refresh: refused — ${e.message}`);
+    io.err(`work directory: ${work}`);
+    return 1;
+  }
+}
+
+if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
+  main(process.argv.slice(2)).then(
+    (code) => process.exit(code),
+    (e) => {
+      console.error(e instanceof Error ? e.message : e);
+      process.exit(1);
+    },
+  );
+}