Skip to content

Commit 8ef4d30

Browse files
feat(scenarios): a card names its origin and a zh+en title; the gallery exports generic starter and delivery cards (#31136)
<!-- claude-projects-attribution: {"github_login":"marchtian"} --> _Requested by **田騏** · [project thread](https://claude.ai/code/project/chan_01VmuUhXU9SSUzqJYyXSmYSi?thread=cmsg_01VmuUhXU9SSUzqJYyXSmYSiNjyvCdXYygFqntMNR7S2dW)_ Fixes #30932 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:场景卡新增必填字段 `origin`(generic / customer-derived / held-out),标题改为中英双语 `{ zh, en }`;九张公开卡全部标为 generic 并补上中文标题;画廊导出器只导出 origin 为 generic、tier 为 starter 或 delivery 的卡,index.json 的中英标题都从卡上取。 - 为什么改:画廊是获客主渠道,按 D1 裁决只能展示通用场景;此前卡上没有明确的"通用"标记,导出器一张卡也导不出,index.json 的中文标题一直是 null。 - 风险与回滚:私有包、不发布;私有 overlay 里的卡在补上 origin 和双语标题之前会被拒收,overlay 里标为 generic 的卡会被拒收;反过来,开放库里标为 customer-derived 或 held-out 的卡也会被拒收;私有 overlay 由其维护者迁移到新卡片形态;回滚即 revert 本 PR。 - 先看:`packages/qa/scenarios/src/export/generic.ts`(导出守卫)、`packages/qa/scenarios/src/loader.ts`(双向拒收:overlay 不收 generic,开放库不收 customer-derived/held-out)、`packages/qa/scenarios/src/schema.ts`(origin 与 title 字段)。 - 两分钟验证:`pnpm --filter @objectstack/scenarios exec vitest run test/export/exporter.test.ts`,预期 36 个用例全部通过,其中 generic starter 导出成功,customer-derived、held-out 和 control 都被拒。 - 需要你做的:无,席位落地。 ## What changes - **`origin`, a required card field** (`packages/qa/scenarios/src/schema.ts`): `generic | customer-derived | held-out`, a closed set with a located refusal when missing. It is the explicit marker the exporter reads; `provenance` stays prose for people. - **The loader knows which library it loads, because the caller says so** (`src/loader.ts`). The loader could not tell the overlay from the open library: a card file carries nothing that says where it lives, and the loader reads whatever directory it is handed. So `LoadOptions` gains a required `library: 'open' | 'overlay'`, and the loader refuses a `generic` card loaded as the `overlay`, and a `customer-derived` or `held-out` card loaded as the `open` library (round 2), both at `origin`. Both CLIs (`run-scenarios`, `export-gallery`) answer it with `libraryOf(dir, repo)`: the repository's `docs/qa/scenarios` by real path is `open`, and any other `--cards` directory is `overlay`. - **`title` becomes `{ zh, en }`**, both required and non-blank, strict (a third language is an unknown key; the retired one-string shape is refused with the new shape named). Every reader picks a language explicitly: - the gallery `index.json`: `title.en` and `title.zh` from the card (`manifest.ts` no longer allows `zh: null`); - the app README the exporter writes: `title.en` as the heading, `title.zh` under it; - `transcript.md`: `title.en` (its body is English); - the runner's `CardRun.title` (the run record): carries the `{ zh, en }` object whole, as data; nothing renders it. - The prompts, the scorecard and the runner's console output read no title (checked: no reader there). - **The exporter guard** (`src/export/generic.ts`): `NO_GENERIC_MARKER` is replaced by `GALLERY_GUARD`, which passes a card only when `origin === 'generic'` AND `tier` is `starter` or `delivery` (the tier rule from the epic ruling comment 6687463653 on #30349), and names every rule a card fails. The option is renamed `isGeneric` to `guard`; the CLI still offers no flag to pass another guard. - **The nine open-library cards**: each gets `"origin": "generic"` and a Chinese title. Every card's `provenance` says it was written from the domain alone or derived mechanically from a golden corpus case; none says it came from a customer, so none is `customer-derived`. - **Docs**: `docs/qa/scenarios/README.md` (the format's reference) gains an Origin section and the title shape, and its stale "the exporter reads no tier" paragraph is replaced; `packages/qa/scenarios/README.md`'s loader example passes `library`. ## Tests - `test/refusals.test.ts`: no origin; an origin outside the set; the old one-string title; no title; no `zh`; no `en`; an empty `en`; a blank `zh`; a third language. A `generic` card loaded as the overlay is refused at `origin` (directory and single file), with `customer-derived` and `held-out` in the overlay as the green controls. Round 2: a `customer-derived` and a `held-out` card loaded as the open library are each refused at `origin`. `libraryOf` answers `open` for the repository's directory, through a non-canonical spelling and through a symlink, and `overlay` elsewhere. - `test/library.test.ts`: every open card is `generic` and carries a `zh` title with CJK text that is not the English one. - `test/export/exporter.test.ts`, on the recorded crm run (`RecordingDriver` over a scripted double, no model): with the default guard a generic `starter` and a generic `delivery` card export, and the index carries both titles; `customer-derived` and `held-out` (starter and delivery) and `control` are each refused with an empty index, and each refusal's control leg exports the same run once the guard lets it through. Over the real library the guard passes the six starter cards and refuses the three control cards for their tier. The CLI end to end refuses the crm card for its tier, and a `--cards` directory outside the repository is loaded as the overlay, where the loader refuses the generic card before the export. - Ablations (each mutation landed and was restored by `scripts/ablation-replace.mjs`, blob equal to HEAD after restore): the loader's overlay rule disabled turns 2 refusal tests red; the open-library rule (round 2) disabled turns its test red (1 failed, 87 passed); the guard's origin rule disabled turns 4 exporter tests red; the guard's tier rule disabled turns 4 exporter tests red. ## Acceptance notes - **Consequence for the private overlay.** `origin` and the `{ zh, en }` title are required, so every overlay card is refused until it gains both, and an overlay card marked `generic` is refused. The overlay lives outside this repository; nothing here could migrate it. - **Both directions (round 2, seat ruling Q1 A):** a `customer-derived` or `held-out` card found in the OPEN library is refused too, so a non-generic card cannot sit in the public tree. The overlay owner migrates its cards to the new shape (Q2 A, one line in `packages/qa/scenarios/README.md`), and the loader keeps no lenient conversion. `libraryOf` is unchanged (Q3 A). - `skip-changeset`: `packages/qa/scenarios` is `private: true` and `docs/qa/scenarios` ships in no package, so nothing published moves. - Out of scope, untouched: the runner's `owner_id` and seed logic, and `personas[].record`. ## Verification Head: see the report comment on #30932 for the head sha the gates were run on. <details><summary>Package suite and typecheck</summary> - `pnpm --filter @objectstack/scenarios build && pnpm --filter @objectstack/scenarios test && pnpm --filter @objectstack/scenarios typecheck` (after the merge of `origin/main`): Test Files 23 passed (23), Tests 454 passed (454); `tsc --noEmit` exit 0, its program covering the 24 files under `test/`. </details> <details><summary>Derived gates</summary> - `node scripts/pm/dispatch-gates.mjs --commands` derives 54 commands for this diff; every one was run on the final head, each recorded as `CMD :: exit N`, and reconciled with `--ran`. The list and the exit codes are in the report comment on #30932. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- signed: domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8 --> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 449c858 commit 8ef4d30

35 files changed

Lines changed: 446 additions & 124 deletions

‎docs/qa/scenarios/README.md‎

Lines changed: 35 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,10 @@ is seeded with, the data questions an agent must answer about that data and the
66
the seed determines, who may see what, the writes an agent performs and the guardrails
77
those writes must respect, and one follow-up change request.
88

9-
The cards here are the **open** library. Every card is generic: ⛔ no card in this
10-
directory is derived from a customer's app, data or wording. Held-out and customer-derived
11-
cards live in a private overlay outside this repository.
9+
The cards here are the **open** library. Every card is generic, and says so in its
10+
`origin` ([below](#origin)): ⛔ no card in this directory is derived from a customer's app,
11+
data or wording. Held-out and customer-derived cards live in a private overlay outside this
12+
repository.
1213

1314
The schema and loader are `@objectstack/scenarios` (`packages/qa/scenarios`, private, not
1415
published). This format is a devx instrument, not a protocol: `packages/spec` carries no
@@ -37,9 +38,11 @@ question's computed truth; a card that does not load is red in CI.
3738
| 6 | `iterate` | A one-sentence change request plus the `expectDelta` it should add to `expect`. |
3839

3940
Besides the layers a card carries `id` (its basename), `code`, `tier` (what the card is for,
40-
[below](#tiers)), `title`, `provenance` (where the
41-
scenario comes from) and, when a question uses a time window, `asOf`: the fixed "today"
42-
(`YYYY-MM-DD`) windows are computed against. `code` is the app code: 2–8 lowercase letters,
41+
[below](#tiers)), `origin` (where it comes from, [below](#origin)), `title`, `provenance` (where
42+
the scenario comes from, in prose for people) and, when a question uses a time window, `asOf`:
43+
the fixed "today" (`YYYY-MM-DD`) windows are computed against. `title` is
44+
`{ "zh": "…", "en": "…" }`, both required and non-blank: the gallery lists every app with a
45+
Chinese and an English title, and every reader picks one of the two explicitly. `code` is the app code: 2–8 lowercase letters,
4346
unique across the library. The app lives in the namespace `demo` + code (`democrm`) wherever
4447
it is built, scored or exported, and every item it declares carries that prefix.
4548

@@ -69,8 +72,27 @@ refuses a `starter` or `delivery` card under any line of it, at the field that f
6972
`countsReason` does not lower on a gallery card;
7073
- at least 3 `run.agentActions`.
7174

72-
The gallery exporter's filter on `tier` is a change of its own: until it lands the exporter
73-
reads no tier, and it exports no card at all (`packages/qa/scenarios/src/export/generic.ts`).
75+
The gallery exporter exports a `starter` or `delivery` card only, and only a `generic` one
76+
([Origin](#origin)); a `control` card is never exported, whatever it scores
77+
(`packages/qa/scenarios/src/export/generic.ts`).
78+
79+
## Origin
80+
81+
Every card names its `origin`, a closed set; a card without one is refused. It is the marker
82+
the gallery exporter reads: neither `provenance` (prose) nor the directory a card sits in is
83+
one, because the loader reads whatever directory it is handed.
84+
85+
| `origin` | Where the card comes from | Lives in | Exported |
86+
|:---|:---|:---|:---|
87+
| `generic` | Written from the domain alone; anyone could write it. | this open library | when its `tier` is `starter` or `delivery` |
88+
| `customer-derived` | Derived from a customer's app, data or wording. | the private overlay | ⛔ never |
89+
| `held-out` | Withheld from the open set, so a score on it is not one anything was tuned against. | the private overlay | ⛔ never |
90+
91+
The caller tells the loader which library it is loading (`LoadOptions.library`: `open` or
92+
`overlay`), and the loader refuses, at `origin`, a `generic` card in the overlay and a
93+
`customer-derived` or `held-out` card in the open library. The runner and
94+
the exporter CLIs answer it from the directory: the repository's `docs/qa/scenarios` (by real
95+
path) is the open library, and any other `--cards` directory is the overlay.
7496

7597
## Seed rows
7698

@@ -192,8 +214,11 @@ the loader enforces; each card's own test pins the rest:
192214

193215
1. Write `<id>.json` here; `id` equals the basename, `code` is one no other card uses, and
194216
`tier` says what the card is for. A new gallery card is `starter` and meets the gallery
195-
floor; `control` is kept for the golden cases, which `test/library.test.ts` pins.
196-
2. Keep it generic — a scenario anyone could write from the domain alone.
217+
floor; `control` is kept for the golden cases, which `test/library.test.ts` pins. Give it
218+
a `title` in both languages.
219+
2. Keep it generic — a scenario anyone could write from the domain alone — and say so:
220+
`"origin": "generic"`. A card derived from a customer, or held out, goes to the private
221+
overlay instead, with that `origin`.
197222
3. Run `pnpm --filter @objectstack/scenarios test`: a refusal names the file and the field
198223
path. Then pin the card's hand-worked truths, per persona too, in its own
199224
`packages/qa/scenarios/test/<id>.test.ts` (model: `travel-expense.test.ts`). The three control

‎docs/qa/scenarios/asset_lifecycle.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "asset_lifecycle",
44
"code": "asset",
55
"tier": "starter",
6-
"title": "Asset lifecycle: register, checkout and return, repairs, monthly depreciation and a scrap approval",
6+
"origin": "generic",
7+
"title": { "zh": "固定资产全生命周期:资产登记、领用与归还、维修、按月折旧和报废审批", "en": "Asset lifecycle: register, checkout and return, repairs, monthly depreciation and a scrap approval" },
78
"provenance": "Mid-tier scenario 资产全生命周期 (asset lifecycle) of the open library, written for this card from the domain alone: generic names, equipment and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

‎docs/qa/scenarios/crm_account_with_contacts.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
"id": "crm_account_with_contacts",
33
"code": "crm",
44
"tier": "control",
5-
"title": "CRM: accounts and independently-living contacts",
5+
"origin": "generic",
6+
"title": { "zh": "客户管理:客户及可独立维护的联系人", "en": "CRM: accounts and independently-living contacts" },
67
"provenance": "Golden case `crm_account_with_contacts` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
78
"prompt": {
89
"short": {"en": "Build a simple CRM with accounts and contacts."},

‎docs/qa/scenarios/invoice_with_line_items.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
"id": "invoice_with_line_items",
33
"code": "invoice",
44
"tier": "control",
5-
"title": "Invoicing: invoices with inline line items and a summed total",
5+
"origin": "generic",
6+
"title": { "zh": "发票管理:发票随单录入明细行,自动汇总金额", "en": "Invoicing: invoices with inline line items and a summed total" },
67
"provenance": "Golden case `invoice_with_line_items` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
78
"prompt": {
89
"short": {"en": "Build an invoicing app with invoices and their line items."},

‎docs/qa/scenarios/leave_balances.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "leave_balances",
44
"code": "leave",
55
"tier": "starter",
6-
"title": "Leave and balances: leave types, yearly balances, requests with day lines and a route set by the days",
6+
"origin": "generic",
7+
"title": { "zh": "请假与假期余额:假期类型、年度余额、按天明细的请假单,按天数分级审批", "en": "Leave and balances: leave types, yearly balances, requests with day lines and a route set by the days" },
78
"provenance": "Mid-tier scenario 请假与假期余额 (leave and balances) of the open library, written for this card from the domain alone: generic names, cities and dates, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

‎docs/qa/scenarios/order_to_cash.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "order_to_cash",
44
"code": "order",
55
"tier": "starter",
6-
"title": "Order to cash: accounts, an activity stream, sales orders with lines, a discount approval, invoices, payments and receivables aging",
6+
"origin": "generic",
7+
"title": { "zh": "订单到收款:客户、跟进活动、销售订单及明细、折扣审批、发票、收款和应收账龄", "en": "Order to cash: accounts, an activity stream, sales orders with lines, a discount approval, invoices, payments and receivables aging" },
78
"provenance": "Mid-tier scenario 订单到收款 (order to cash) of the open library, written for this card from general business knowledge alone: generic account names, cities, products and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

‎docs/qa/scenarios/project_with_tasks.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
"id": "project_with_tasks",
33
"code": "project",
44
"tier": "control",
5-
"title": "Project management: projects owning inline-edited tasks with roll-ups",
5+
"origin": "generic",
6+
"title": { "zh": "项目管理:项目下行内编辑任务,汇总任务数与工时", "en": "Project management: projects owning inline-edited tasks with roll-ups" },
67
"provenance": "Golden case `project_with_tasks` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
78
"prompt": {
89
"short": {"en": "Build a project management app with projects and tasks."},

‎docs/qa/scenarios/requisition_to_receipt.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "requisition_to_receipt",
44
"code": "purchase",
55
"tier": "starter",
6-
"title": "Requisition to receipt: purchase requisitions, an amount-threshold approval, purchase orders with lines and goods receipts",
6+
"origin": "generic",
7+
"title": { "zh": "采购申请到收货:采购申请、按金额分级审批、采购单及明细、收货登记", "en": "Requisition to receipt: purchase requisitions, an amount-threshold approval, purchase orders with lines and goods receipts" },
78
"provenance": "Mid-tier scenario 采购申请到收货 (requisition to receipt) of the open library, written for this card from the procurement domain alone: generic names, suppliers, items and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

‎docs/qa/scenarios/service_desk.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "service_desk",
44
"code": "helpdesk",
55
"tier": "starter",
6-
"title": "Service desk: tickets with comment threads, SLA policies, time-based escalation and satisfaction surveys",
6+
"origin": "generic",
7+
"title": { "zh": "客服工单:工单及沟通记录、SLA 策略、超时升级和满意度调查", "en": "Service desk: tickets with comment threads, SLA policies, time-based escalation and satisfaction surveys" },
78
"provenance": "Mid-tier scenario 客服工单 (service desk) of the open library, written for this card from the domain alone: invented company and agent names, example.com addresses and invented figures, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

‎docs/qa/scenarios/travel_expense.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"id": "travel_expense",
44
"code": "travel",
55
"tier": "starter",
6-
"title": "Travel expense: trip requests, claims with lines, grade-based standards and a two-step approval",
6+
"origin": "generic",
7+
"title": { "zh": "差旅报销:出差申请、报销单及明细、按职级的差旅标准,两级审批", "en": "Travel expense: trip requests, claims with lines, grade-based standards and a two-step approval" },
78
"provenance": "Mid-tier scenario 差旅报销 (travel expense) of the open library, written for this card from the domain alone: generic names, cities and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
89
"prompt": {
910
"short": {

0 commit comments

Comments
 (0)