Skip to content

Commit 518d1bd

Browse files
feat(scenarios): gallery-refresh, the gallery refresh in one deterministic step (#31313)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1X7VfYZUwr12nPkXdgocCV7Q)_ domain:devx#3 · os-project-manager · session_01FEkQ5pZeHgu9hYkeNdBNEA Fixes #31121 Clause-②: yes — a new `gallery-refresh` entry point on the published `@objectstack/scenarios` widens its surface; the PR body carries the reading Metadata-protocol: none ## 维护者速读 - 改了什么:`@objectstack/scenarios` 新增 `gallery-refresh --gallery DIR [--cards DIR] [--model M] [--debut]`。它读 npm 上 `@objectstack/cli` 的最新稳定版和画廊 `index.json` 里每个应用的 `framework.version`;一致就只打印一行 `skip: …` 并以 0 退出,什么都不跑。不一致就在这个版本上跑所有可上画廊的卡片(generic + starter/delivery,`--debut` 只跑第一张),导出到画廊检出目录,在本地分支 `export/VERSION` 上提交,打印分支名和现成的 PR 正文。⛔ 永不推送、不开 PR。`driver.ts` 头部写明裁决 D2,README 加了定时任务的运行手册(含任务提示词原文)。 - 为什么改:裁决 D2:每天由云端定时 Claude Code 任务(你的订阅登录)在每次稳定版发布后重跑画廊并在 objectstack-ai/gallery 开 PR,由你一键合并。此前没有任何入口能一步完成「查版本 → 跑 → 导出 → 提交」。 - 风险与回滚:只新增一个入口和若干导出,`run-scenarios`、`export-gallery` 行为不变(`export-gallery` 的截图步骤只是抽成了一个函数)。回滚即还原本 PR。定时任务本身和画廊仓库不在本 PR 内。 - 先看:`packages/qa/scenarios/src/export/refresh.ts`(跳过规则、拒绝条件、提交步骤)、`README.md` 新节「Refreshing the gallery」里的任务提示词、`src/runner/driver.ts` 头部。 - 两分钟验证:`pnpm --filter @objectstack/scenarios exec vitest run --maxWorkers=2 test/export/refresh.test.ts`,预期 17 个用例全过(含「版本一致只打一行 skip」「永不推送:远端引用不变」「版本不符拒绝提交」)。 - 需要你做的:无,席位落地。之后把 objectstack-ai/gallery 加进本项目环境资源,席位才能按 README 原文创建定时任务。 ## What changes - **`gallery-refresh`** (`src/export/refresh.ts`, CLI `src/export/refresh-cli.ts`, package script `gallery-refresh`), in order, each step a refusal (exit 1, the reason on stderr) or a quiet skip (one `skip: …` line, exit 0): 1. `--gallery` must be the top of a git checkout with a clean working tree (untracked files count), and its `index.json` (absent means an empty gallery) must validate as one the exporter wrote. 2. npm's `latest` dist-tag of `@objectstack/cli` (`npm view`, so the caller's npm config applies). A value that is not a plain `MAJOR.MINOR.PATCH` is refused. 3. **Skip** when the index lists at least one app and every app's `framework.version` equals it. 4. **Skip** when a remote-tracking `export/VERSION` branch exists (that refresh was pushed and awaits review). A *local* `export/VERSION` branch is refused: an earlier run's commit is never overwritten. 5. The cards: every loaded card `GALLERY_GUARD` admits (origin `generic`, tier `starter` or `delivery`). With `--debut`, only the first of them by id, and only into a gallery whose index lists no app. 6. The run: `run-scenarios --only CARDS --modes scratch --packages published --create-version VERSION --out WORK/runs --work WORK/work --record WORK/recording.json [--model M] [--cards DIR]` (`runnerArgv`), then `exportGallery` into the checkout with real screenshots. 7. Refused when no app exported, when an exported app's `framework.version` is not the release, or when anything outside `apps/` and `index.json` changed in the checkout. 8. `git switch -c export/VERSION`, `git add --all -- apps index.json`, `git commit` under the checkout's own git identity. It prints the export summary, `branch:`, `commit:` and the PR body (`refreshPrBody`: version, previous version, cards run, model, exported / refused / removed apps). - ⛔ Nothing in the code path pushes, opens a PR or talks to GitHub. - **`--model`** passes through to `run-scenarios --model` (PR #31204's flag), with the same refusal of an empty or flag-shaped value. - **`driver.ts` header** names ruling D2: a local session on the developer's own login, or a scheduled cloud Claude Code task on the maintainer's subscription login. An API-billed model is still never allowed. `ClaudeCodeDriver`'s API-key refusal is unchanged. - **README**: a new section, "Refreshing the gallery — `gallery-refresh`", and "The scheduled task's runbook": what it runs, what it needs, what it never does, and the task's prompt verbatim (clone the gallery, run `gallery-refresh` detached and wait in the foreground, push only `export/VERSION`, open one PR with the printed body, and on any failure comment on or open one `Gallery refresh failed` issue). - `src/export/cli.ts`: the live screenshot wiring moves into `liveScreenshots(chromium?)`, which both CLIs call. No behaviour change. ### Public surface (Clause-②: yes) `@objectstack/scenarios/export` re-exports `refresh.ts`. Read with the TypeScript checker (`getExportsOfModule`) over the built `dist/export.d.ts` on this head: **84 exports, 49 values**. The 15 new names (none of them on `origin/main`, `git grep`): values `galleryRefresh`, `runnerArgv`, `readGalleryVersions`, `npmLatestVersion`, `refreshPrBody`, `galleryBranch`, `RefreshRefusal`, `FRAMEWORK_PACKAGE`, `STABLE_VERSION`; types `RefreshOptions`, `RefreshDeps`, `RefreshOutcome`, `RunPlan`, `GalleryVersions`, `PrBodyInput`. No name is removed or retyped, so the subpath only widens (69 → 84). `refresh-cli.ts` is not an `exports` key and not a `bin` command: the card asks for neither, and the scheduled task runs from a checkout. Nothing in `packages/spec` moves. ## Tests `test/export/refresh.test.ts` (new, 17 cases). The gallery is a real git checkout (`main`, a README, optionally an `index.json` and app directories) with a bare `origin`. The npm reader is a stub. The run step runs the real crm card through `runCard` with the fixture's scripted model inside the runner's own `RecordingDriver` (`test/export/fixture.ts`), and writes the record and the recording where `run-scenarios` would. The crm card is a `control` card, so these cases pass the exporter tests' all-eligible guard double. One case uses the real guard on the repo's cards. - **Skip**: every listed app on npm's latest prints one `skip:` line, calls no run, and leaves HEAD, the working tree and the remote as they were. A mixed index, an empty index and no index do not skip. A remote-tracking `origin/export/18.3.0` skips with its own line. - **Commit**: the commit sits on `export/18.3.0` with `main` as its parent, touches only `index.json` and `apps/crm_account_with_contacts/`, and is authored by the checkout's configured identity. The tree is clean, the index states 18.3.0, and the PR body names the branch, the card and the model. **⛔ Never pushes**: the bare remote's refs are byte-identical before and after. A second refresh then skips. - **Run plan**: `runnerArgv` output parsed by the runner's own `parseArgs` gives exactly that `createVersion`, `packages: published`, `modes: ['scratch']`, the cards, out/work/record and the model, with iterate Run on. No model means no `--model`. With the real guard on the repo's nine cards, the plan is the six `starter` ids, and `--debut` gives the first of them (`asset_lifecycle`). - **Refusals**, each with nothing run or nothing committed: a prerelease, `18.4`, empty or `v`-prefixed `latest`; an npm read that throws; a plain directory; a subdirectory of a checkout; an untracked file; an index that does not validate; a local `export/18.3.0`; `--debut` over a listed app; npm at 18.4.0 while the project pins 18.3.0 (`… states framework 18.3.0, not 18.4.0`); a run with no iterate Run (no app passes); a `.env` written into the checkout. - **CLI**: flag parsing and its refusals (`--gallery` missing, bad `--model`, unknown `--push`). `main` prints exactly one line on a skip. On the repo's cards it commits and prints `branch: export/18.3.0` and the body. A prerelease exits 1 with `gallery-refresh: refused — …`. <details><summary>Ablations (each restored: blob == HEAD, git diff HEAD empty)</summary> All through `scripts/ablation-replace.mjs` on commit `de7c89c5` (anchor x1 → x0, replacement x0 → x1). The suite imports `src/` directly, so there is no dist leg. Each ran `test/export/refresh.test.ts -t NAME`. - **A1**, version check disarmed (`filter((a) => a.framework.version !== latest)` gets `&& String("ABL1").length < 0`): `1 failed | 16 skipped`, the "another framework version" case. - **A2**, the `--debut` refusal disarmed: `1 failed | 16 skipped`, the "--debut into a gallery that already lists an app" case. - **A3**, the skip disarmed (`before.versions[0] === latest`): `1 failed | 16 skipped`, the "every listed app states npm's latest" case. - **A4**, the outside-the-exporter-paths check disarmed: `1 failed | 16 skipped`, the ".env" case. - **A5**, the dirty-checkout refusal disarmed: `1 failed | 16 skipped`, the "top of a clean git checkout" case. </details> ## Acceptance notes - **`index.json` carries `framework.version` per app, not once.** The index schema (`manifest.ts`, strict) is `{ schemaVersion, apps }`, and each app's `framework.version` is the `@objectstack/spec` its project pins (create-objectstack pins its own release exactly, and the fixed group versions `@objectstack/cli` with it). So "the gallery's version" is read as: the index lists an app, and every listed app states that version. No index field was added: that would change the format #31122's check reads. - **After the debut, the full gallery comes with the next release.** A debut leaves one app on version V, so later runs skip until npm's latest moves past V. There is no `--force` flag: the ruling names none. - **The second skip (a pending `export/VERSION` on the remote) is an addition.** Without it, every daily run while the PR awaits review would rerun every card for hours of subscription quota, and then fail to push an existing branch. It reads only the clone's remote-tracking refs, so no network. - **Scratch mode only.** The gallery ships the scratch project, and the exporter's pass rule (`pass.ts`) owes scratch mode's six halves. Template mode would roughly double each card's time and quota without changing what is exported. - **A card that fails is not a refusal of the refresh.** The exporter's own rule applies: it is refused or removed, the commit goes ahead with the rest, and the PR body lists it with its reasons. The refresh is refused only when no app exports. - **`--cards DIR` keeps the meaning it has in `run-scenarios` and `export-gallery`** (the card directory, where any directory but the repo's is the private overlay). A flag of the same name taking card ids in the same package would be a trap. `--debut` therefore picks its card itself: the first exportable id (`asset_lifecycle` today). - **The task's prompt runs the refresh detached** (`nohup` plus repeated `timeout 590 tail --pid`). A full refresh runs one `starter` build after another, longer than the 2-hour cap on a background command in a cloud session. This is written into the runbook; no live run measured it (live runs are paused, cloud#3600). - **Out of scope here:** the scheduled task itself, objectstack-ai/gallery, and #31122's `export-gallery --check` and the gallery workflow. The maintainer still owes adding the gallery to the environment's resources. ## Verification All on head `de7c89c5` (base `7c523aa1`, current `origin/main` when cut), worktree `objectstack-issue-31121`. - **Package suite**, under the verify lock after `pnpm --filter @objectstack/scenarios build`: `pnpm --filter @objectstack/scenarios test` gives `Test Files 31 passed (31)`, `Tests 608 passed (608)`, `VERDICT command-exit 0`. `pnpm --filter @objectstack/scenarios typecheck` (`tsc --noEmit`; `--listFiles` includes `test/export/refresh.test.ts`) exits 0. - **Lint scoped to the package**: `eslint packages/qa/scenarios --no-inline-config` exits 0. - **Derived gates**: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` gives 74 commands, all run on this head, each exit code captured before any pipe. 72 exited 0 on the first pass. Two exited 3, both prerequisites and not reds: - `check-plugin-teardown-shape --self-test`: the clone is shallow. It exits 0 (48 cases) after `git fetch --depth=1` of its pinned sha. - `pnpm check:dual-build-cjs-loads`: PREREQUISITE NOT MET. It exits 0 (122 entry points across 72 packages) after the full `turbo run build` under the lock. - `--ran`: `74 derived famil(ies) accounted for — 74 run, 0 NOT-MEASURED`. - Also run, outside the derivation: `pnpm --filter @objectstack/spec run check:error-code-provenance` exits 0. `check-changeset-no-major --base origin/main` against this body exits 0 (`LEVEL AXIS … clause-② yes … patch`). - **Dogfood battery**: n/a. There is no `!` in the title and no `(narrowing)` arm. - **No live model run** (paused by the maintainer, cloud#3600). No network in the tests: npm is a stub, git is local with a bare remote, and the run step is the fixture's recorded crm run. - CI families are read from this PR's check-runs, not re-run here. <details><summary>The 74 derived gates, as run</summary> ```text node scripts/check-adr-0087-registration.mjs --base origin/main :: exit 0 node scripts/check-adr-0087-registration.mjs --self-test :: exit 0 node scripts/check-changeset-no-major.mjs --base origin/main :: exit 0 node scripts/check-closing-keyword-parity.mjs :: exit 0 node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0 node scripts/check-comment-mask-adoption.mjs :: exit 0 node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0 node scripts/check-comment-mask-corpus.mjs :: exit 0 node scripts/check-console-intercept-disarm.mjs :: exit 0 node scripts/check-console-intercept-disarm.mjs --self-test :: exit 0 node scripts/check-dts-emitted.mjs --self-test :: exit 0 node scripts/check-empty-changeset.mjs --base origin/main :: exit 0 node scripts/check-empty-changeset.mjs --self-test :: exit 0 node scripts/check-issue-citations.mjs :: exit 0 node scripts/check-osv-exemptions.mjs :: exit 0 node scripts/check-osv-exemptions.mjs --self-test :: exit 0 node scripts/check-plugin-teardown-shape.mjs :: exit 0 node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0 node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0 node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0 node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0 node scripts/ci/snapshot-next.mjs --self-test :: exit 0 node scripts/docs-audit/check-affected-docs.mjs :: exit 0 node scripts/docs-audit/check-drift-comment.mjs :: exit 0 node scripts/osv-base-relative.mjs --self-test :: exit 0 node scripts/pm/release-rehearsal-clone.mjs --self-test :: exit 0 node scripts/release-pending-publish.mjs --self-test :: exit 0 pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0 pnpm --filter @objectstack/spec run check:llms-txt :: exit 0 pnpm check:changeset-fixed :: exit 0 pnpm check:changeset-gate-self-tests :: exit 0 pnpm check:ci-filter-parity :: exit 0 pnpm check:closed-ceremony-text :: exit 0 pnpm check:cross-package-test-inputs :: exit 0 pnpm check:doc-authoring :: exit 0 pnpm check:driver-memory-census :: exit 0 pnpm check:dts-closure :: exit 0 pnpm check:dual-build-cjs-loads :: exit 0 pnpm check:engine-double-contract :: exit 0 pnpm check:gitlink-declared :: exit 0 pnpm check:issue-citations :: exit 0 pnpm check:keyed-text-bounds :: exit 0 pnpm check:lean-entry-closure :: exit 0 pnpm check:logger-receiver-detach :: exit 0 pnpm check:manifest-repository-directory :: exit 0 pnpm check:merge-driver :: exit 0 pnpm check:nul-bytes :: exit 0 pnpm check:object-def-param-keys :: exit 0 pnpm check:objectql-double-limit :: exit 0 pnpm check:optional-error-sink :: exit 0 pnpm check:org-identifier :: exit 0 pnpm check:override-consistency :: exit 0 pnpm check:page-declaration-shape :: exit 0 pnpm check:platform-object-tenancy-census :: exit 0 pnpm check:pnpm-flag-spellings :: exit 0 pnpm check:published-files :: exit 0 pnpm check:query-options-erasure :: exit 0 pnpm check:refd-timer-probe :: exit 0 pnpm check:registry-log-declared :: exit 0 pnpm check:rest-log-spy-declared :: exit 0 pnpm check:slot-lookup :: exit 0 pnpm check:sourcemap-no-sources-content :: exit 0 pnpm check:system-context-census :: exit 0 pnpm check:test-source-alias :: exit 0 pnpm check:tier-file-adoption :: exit 0 pnpm check:turbo-task-graph :: exit 0 pnpm check:type-check-coverage :: exit 0 pnpm check:type-check-debt :: exit 0 pnpm check:undeclared-dep-imports :: exit 0 pnpm check:vendor-export-contract-resolve :: exit 0 pnpm check:vouch-for-reference-callers :: exit 0 pnpm check:watch-hint-literal :: exit 0 pnpm check:where-matcher :: exit 0 pnpm check:workspace-manifest-cycles :: exit 0 ``` </details> `patch` changeset `.changeset/31121-scenarios-gallery-refresh.md` (published `@objectstack/scenarios`; `minor` is refused by the launch-window guard), with both declaration lines. --- _Generated by [Claude Code](https://claude.ai/code/session_01FEkQ5pZeHgu9hYkeNdBNEA)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent eddc3ab commit 518d1bd

9 files changed

Lines changed: 911 additions & 10 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
'@objectstack/scenarios': patch
3+
---
4+
5+
feat(scenarios): `gallery-refresh`, the gallery refresh in one deterministic step
6+
7+
Clause-②: yes — a new `gallery-refresh` entry point on the published `@objectstack/scenarios` widens its surface; the PR body carries the reading
8+
Metadata-protocol: none
9+
10+
- **`pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR [--cards DIR] [--model M] [--debut] [--work DIR] [--chromium PATH]`** (`src/export/refresh-cli.ts`): reads npm's latest stable `@objectstack/cli` and the `framework.version` of every app the gallery checkout's `index.json` lists. When every listed app states that release it prints one `skip: …` line and exits 0, having run nothing; it also skips while a remote-tracking `export/<version>` branch awaits review. Otherwise it runs every card the gallery may show (origin `generic`, tier `starter` or `delivery`; `--debut`: the first of them, only into an empty gallery) with `run-scenarios --modes scratch --packages published --create-version <version>`, exports with screenshots into the checkout, commits `apps/` and `index.json` on a new local branch `export/<version>` and prints the branch, the commit and a ready PR body. It never pushes and never opens a PR. Every refusal (not a clean checkout, an index that does not validate, a prerelease `latest`, an existing local branch, no app exported, an app on another framework version, a change outside `apps/` and `index.json`) exits 1 with the reason.
11+
- **`@objectstack/scenarios/export`** also exports the step as a library: `galleryRefresh`, `runnerArgv`, `readGalleryVersions`, `npmLatestVersion`, `refreshPrBody`, `galleryBranch`, `RefreshRefusal`, `FRAMEWORK_PACKAGE`, `STABLE_VERSION` and their types.
12+
- The live driver's header names ruling D2: a local session or a scheduled cloud Claude Code task on the maintainer's subscription login may run the cards; an API-billed model still never may. The driver's refusal of an API key is unchanged.
13+
- The README gains "Refreshing the gallery — `gallery-refresh`" and the scheduled task's runbook: what it runs, what it needs, what it never does, and the task's prompt verbatim.

‎packages/qa/scenarios/README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,78 @@ npx @objectstack/scenarios help # the comma
275275
dashboards and nav targets. `test/runner/prompt-trace.test.ts` reads the checks off the scorer itself and holds every
276276
card to it.
277277

278+
## Refreshing the gallery — `gallery-refresh`
279+
280+
```bash
281+
G="$(realpath ../gallery)" # pnpm runs the script in the package directory: pass absolute paths
282+
pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G" # every exportable card
283+
pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G" --debut # one card: the gallery's first app
284+
pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$G" --model sonnet # the model every turn asks for
285+
```
286+
287+
One step from "is there a new stable release?" to a local commit in a checkout of objectstack-ai/gallery
288+
(`src/export/refresh.ts`, CLI `src/export/refresh-cli.ts`). It reads npm's `latest` `@objectstack/cli` and the
289+
`framework.version` of every app the gallery's `index.json` lists. When the index lists an app and every app states
290+
that version, it prints one `skip: …` line and exits 0, having run nothing. It also skips when a remote-tracking
291+
`export/<version>` branch exists: that refresh was pushed and awaits review. Otherwise it:
292+
293+
1. runs every card the gallery may show (origin `generic`, tier `starter` or `delivery`; `--debut`: the first of them
294+
by id) with `run-scenarios --modes scratch --packages published --create-version <version>`, so the apps are built
295+
on exactly that release, the turns recorded;
296+
2. exports with screenshots into the checkout (`export-gallery`'s rules: only a run that passed every half);
297+
3. creates the branch `export/<version>`, commits `apps/` and `index.json` under the checkout's own git identity, and
298+
prints the branch, the commit and a ready PR body.
299+
300+
It refuses, exit 1 with the reason, when: `--gallery` is not the top of a git checkout or has uncommitted changes;
301+
the index does not validate; npm's `latest` is not a plain `MAJOR.MINOR.PATCH`; a local `export/<version>` branch
302+
already exists; `--debut` meets an index that lists an app; no card may be exported; no app exported; an exported app
303+
states another framework version; or anything outside `apps/` and `index.json` changed. The last three leave the
304+
working tree as the export left it, uncommitted. `--work DIR` keeps the records, projects and recording (default: a
305+
temp directory, named on a refusal). ⛔ It never pushes, never opens a PR and never talks to GitHub.
306+
307+
### The scheduled task's runbook
308+
309+
Ruling D2 (2026-11-06): a daily scheduled Claude Code task in this project's cloud environment runs the refresh on the
310+
maintainer's subscription login, and the maintainer merges its PR on the gallery.
311+
312+
- **What it runs**: `gallery-refresh` from a fresh checkout of this repository, into a fresh clone of
313+
objectstack-ai/gallery; then `git push` of the one branch it created, and one PR.
314+
- **What it needs**: both repositories in the environment's resources, with push access to the gallery; npm reachable;
315+
the container's `claude` logged in with the subscription (`claude auth status` reports `oauth_token`) and neither
316+
`ANTHROPIC_API_KEY` nor `ANTHROPIC_AUTH_TOKEN` set (the live driver refuses both); a permission mode that lets the
317+
session launch the headless `claude -p` the runner spawns (Auto mode's classifier refuses it, see
318+
[the cloud session settings](#running-the-cards-in-a-claude-code-cloud-session)); Chromium for the screenshots
319+
(preinstalled in the cloud container). A full refresh runs one `starter` build after another, longer than the
320+
2-hour cap on a background command, so the task starts it detached and waits in the foreground.
321+
- **What it never does**: push `main` or any branch but `export/<version>`, force-push, merge, edit a gallery file by
322+
hand, pass `--packages workspace`, or run on an API key. A refusal is reported, never retried around.
323+
324+
The task's prompt, verbatim:
325+
326+
```text
327+
Refresh the ObjectStack scenario gallery. Run each step in order and stop at the first that fails.
328+
329+
1. Check the login: `unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN; claude auth status` must report oauth_token.
330+
Anything else is a failure (step 6).
331+
2. In the objectstack checkout: `git fetch origin main && git checkout --detach origin/main`, `pnpm install`,
332+
`pnpm --filter '@objectstack/scenarios^...' build`.
333+
3. Clone the gallery beside it: `git clone https://github.com/objectstack-ai/gallery ../gallery`.
334+
If ../gallery/index.json is missing or lists no app, add `--debut` to the command in step 4.
335+
4. Start the refresh detached and wait for it in the foreground:
336+
`nohup sh -c 'pnpm --filter @objectstack/scenarios gallery-refresh --gallery "$(realpath ../gallery)" --work /tmp/gallery-refresh; echo "gallery-refresh exit $?"' > /tmp/gallery-refresh.log 2>&1 & echo $! > /tmp/gallery-refresh.pid`
337+
then repeat `timeout 590 tail --pid="$(cat /tmp/gallery-refresh.pid)" -f /dev/null` until the process is gone.
338+
The log's last line is `gallery-refresh exit N`; any N but 0 is a failure (step 6).
339+
5. If a line of the log starts with `skip:`, stop: report that line and do nothing else.
340+
Otherwise the log names a `branch: export/VERSION`: push exactly that branch
341+
(`git -C ../gallery push origin export/VERSION`) and open one pull request on objectstack-ai/gallery from it into
342+
main, titled `Gallery refresh for ObjectStack VERSION`, with the text after "open the PR with this body:" as its
343+
body. Never push main, never force-push, never merge, never edit a gallery file by hand.
344+
6. On any failure (a refusal, a non-zero exit, a failed push or PR): search the open issues of
345+
objectstack-ai/objectstack for one titled `Gallery refresh failed`. If one exists, add a comment with today's
346+
date, the step that failed and the last 40 lines of /tmp/gallery-refresh.log; otherwise open one issue with that
347+
title and the same text. Never open a second one.
348+
```
349+
278350
## Other agent tools — Codex and the external driver contract
279351

280352
The same card can measure another agent tool. Every record (`env.driver`) and every gallery app page (`- Driver:`)

‎packages/qa/scenarios/package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,8 @@
6666
"test": "vitest run",
6767
"typecheck": "tsc --noEmit",
6868
"run-scenarios": "tsx src/runner/cli.ts",
69-
"export-gallery": "tsx src/export/cli.ts"
69+
"export-gallery": "tsx src/export/cli.ts",
70+
"gallery-refresh": "tsx src/export/refresh-cli.ts"
7071
},
7172
"dependencies": {
7273
"@objectstack/core": "workspace:*",

‎packages/qa/scenarios/src/export/cli.ts‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,12 @@ export function renderSummary(s: ExportSummary): string[] {
8585
return lines;
8686
}
8787

88+
/** The real screenshot step: boot the project, sign in as the dev admin, drive Chromium (`--chromium PATH`, or Playwright's own). */
89+
export function liveScreenshots(chromium?: string): CaptureScreenshots {
90+
const browser = playwrightBrowser(chromium ? { executablePath: chromium } : {});
91+
return (input) => captureScreenshots(input, { boot: (dir) => bootDev(dir), harness: (url) => new HttpApp({ baseUrl: url, adminEmail: ADMIN.email, adminPassword: ADMIN.password }), browser, admin: ADMIN });
92+
}
93+
8894
/** Test seam: replace the screenshot step (the CLI's is the real browser). */
8995
export interface ExportCliDeps {
9096
screenshots?: CaptureScreenshots;
@@ -102,11 +108,7 @@ export async function main(argv: readonly string[], deps: ExportCliDeps = {}): P
102108
});
103109
for (const r of refusals) console.error(formatRefusal(r));
104110
const recordings = args.recordings.map((f) => JSON.parse(readFileSync(resolve(f), 'utf8')) as Recording);
105-
const browser = playwrightBrowser(args.chromium ? { executablePath: args.chromium } : {});
106-
const screenshots: CaptureScreenshots | false = !args.screenshots
107-
? false
108-
: (deps.screenshots ??
109-
((input) => captureScreenshots(input, { boot: (dir) => bootDev(dir), harness: (url) => new HttpApp({ baseUrl: url, adminEmail: ADMIN.email, adminPassword: ADMIN.password }), browser, admin: ADMIN })));
111+
const screenshots: CaptureScreenshots | false = !args.screenshots ? false : (deps.screenshots ?? liveScreenshots(args.chromium));
110112
const summary = await exportGallery({
111113
runsDir,
112114
outDir: resolve(args.out!),

‎packages/qa/scenarios/src/export/index.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@
44
* `@objectstack/scenarios/export` — the gallery exporter: verified scenario
55
* apps written to a local directory in the layout of objectstack-ai/gallery.
66
* Private, like the rest of this package. CLI:
7-
* `pnpm --filter @objectstack/scenarios export-gallery --runs DIR --out DIR`.
7+
* `pnpm --filter @objectstack/scenarios export-gallery --runs DIR --out DIR`;
8+
* the whole refresh (version check, run, export, commit) is
9+
* `pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR`.
810
*/
911

1012
export * from './exporter.js';
1113
export * from './generic.js';
1214
export * from './manifest.js';
1315
export * from './pass.js';
1416
export * from './record.js';
17+
export * from './refresh.js';
1518
export * from './screenshots.js';
1619
export * from './source.js';
1720
export * from './transcript.js';
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* `pnpm --filter @objectstack/scenarios gallery-refresh --gallery DIR [flags]`
5+
* — the gallery refresh in one step (`refresh.ts`): npm's latest stable
6+
* `@objectstack/cli` against the gallery's `index.json`; when they differ, run
7+
* the cards on that release, export into the gallery checkout and commit on
8+
* `export/<version>`. Ruling D2: a daily scheduled Claude Code task on the
9+
* maintainer's subscription login runs it (the package README holds the
10+
* task's runbook).
11+
*
12+
* --gallery DIR the gallery checkout, the top of its git working tree (required)
13+
* --cards DIR the card directory (default: the repo's docs/qa/scenarios);
14+
* any other directory is loaded as the private overlay,
15+
* where a `generic` card is refused (`libraryOf`)
16+
* --model M the model every live turn asks for (`run-scenarios --model`;
17+
* default: the login's own)
18+
* --debut run and export one card only, the first exportable card
19+
* by id: the gallery's first app (refused once the index lists one)
20+
* --work DIR where the run's records, projects and recording go
21+
* (default: a new temp directory)
22+
* --chromium PATH the Chromium executable for the screenshots
23+
*
24+
* Output: `skip: …` (one line, exit 0) when the gallery is already on the
25+
* release; otherwise the branch, the commit and the PR body, exit 0. A refusal
26+
* prints why and exits 1. ⛔ It never pushes and never opens the PR.
27+
*/
28+
29+
import { existsSync, mkdirSync, mkdtempSync, rmSync } from 'node:fs';
30+
import { tmpdir } from 'node:os';
31+
import { dirname, join, resolve } from 'node:path';
32+
import { fileURLToPath } from 'node:url';
33+
34+
import { formatRefusal, libraryOf, loadScenarios, readCapabilityAreas } from '../loader.js';
35+
import { findUp, main as runScenarios } from '../runner/cli.js';
36+
import { liveScreenshots, renderSummary } from './cli.js';
37+
import { galleryRefresh, RefreshRefusal, runnerArgv, type RefreshDeps, type RefreshOutcome } from './refresh.js';
38+
39+
export interface RefreshCliArgs {
40+
gallery: string;
41+
cards?: string;
42+
model?: string;
43+
debut: boolean;
44+
work?: string;
45+
chromium?: string;
46+
}
47+
48+
export function parseRefreshArgs(argv: readonly string[]): RefreshCliArgs {
49+
const a: Partial<RefreshCliArgs> & { debut: boolean } = { debut: false };
50+
for (let i = 0; i < argv.length; i++) {
51+
const flag = argv[i]!;
52+
const val = () => {
53+
const v = argv[++i];
54+
if (v === undefined) throw new Error(`${flag} needs a value`);
55+
return v;
56+
};
57+
if (flag === '--gallery') a.gallery = val();
58+
else if (flag === '--cards') a.cards = val();
59+
else if (flag === '--model') {
60+
const raw = val();
61+
if (raw.trim() === '' || raw.startsWith('-')) throw new Error(`--model takes a model alias or name (sonnet, opus, claude-sonnet-4-5), not \`${raw}\``);
62+
a.model = raw;
63+
} else if (flag === '--debut') a.debut = true;
64+
else if (flag === '--work') a.work = val();
65+
else if (flag === '--chromium') a.chromium = val();
66+
else throw new Error(`unknown flag \`${flag}\``);
67+
}
68+
if (!a.gallery) throw new Error('--gallery DIR is required: the gallery checkout the refresh exports into and commits in');
69+
return a as RefreshCliArgs;
70+
}
71+
72+
/** What the CLI prints for an outcome, one entry per line. */
73+
export function renderOutcome(o: RefreshOutcome): string[] {
74+
if (o.kind === 'skip') return [o.line];
75+
return [
76+
...renderSummary(o.summary),
77+
`branch: ${o.branch}`,
78+
`commit: ${o.commit}`,
79+
'Not pushed. Push the branch and open the PR with this body:',
80+
'',
81+
o.body,
82+
];
83+
}
84+
85+
/** Test seam: replace the steps that reach outside (npm, the model, the browser). */
86+
export type RefreshCliDeps = Partial<RefreshDeps>;
87+
88+
export async function main(argv: readonly string[], deps: RefreshCliDeps = {}, io = { out: (l: string) => console.log(l), err: (l: string) => console.error(l) }): Promise<number> {
89+
const args = parseRefreshArgs(argv);
90+
const here = dirname(fileURLToPath(import.meta.url));
91+
const repo = findUp(here, (d) => existsSync(join(d, 'pnpm-workspace.yaml')));
92+
const cardsDir = resolve(args.cards ?? join(repo, 'docs/qa/scenarios'));
93+
const { cards, refusals } = loadScenarios(cardsDir, {
94+
areas: readCapabilityAreas(join(repo, 'docs/qa/platform-checklist/areas')),
95+
library: libraryOf(cardsDir, repo),
96+
});
97+
for (const r of refusals) io.err(formatRefusal(r));
98+
const work = resolve(args.work ?? mkdtempSync(join(tmpdir(), 'gallery-refresh-')));
99+
mkdirSync(work, { recursive: true });
100+
const run: RefreshDeps['run'] =
101+
deps.run ??
102+
(async (plan) => {
103+
const code = await runScenarios(runnerArgv(plan));
104+
if (code !== 0) throw new RefreshRefusal(`run-scenarios exited ${code}: nothing is exported`);
105+
});
106+
try {
107+
const outcome = await galleryRefresh(
108+
{ galleryDir: resolve(args.gallery), cards, ...(args.cards ? { cardsDir } : {}), debut: args.debut, ...(args.model ? { model: args.model } : {}), workDir: work },
109+
{ ...deps, run, screenshots: deps.screenshots ?? liveScreenshots(args.chromium) },
110+
);
111+
// A skip ran nothing: the temp directory it was given is removed again.
112+
if (outcome.kind === 'skip' && args.work === undefined) rmSync(work, { recursive: true, force: true });
113+
for (const line of renderOutcome(outcome)) io.out(line);
114+
return 0;
115+
} catch (e) {
116+
if (!(e instanceof RefreshRefusal)) throw e;
117+
io.err(`gallery-refresh: refused — ${e.message}`);
118+
io.err(`work directory: ${work}`);
119+
return 1;
120+
}
121+
}
122+
123+
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
124+
main(process.argv.slice(2)).then(
125+
(code) => process.exit(code),
126+
(e) => {
127+
console.error(e instanceof Error ? e.message : e);
128+
process.exit(1);
129+
},
130+
);
131+
}

0 commit comments

Comments
 (0)