Skip to content

Commit f8d3de9

Browse files
committed
feat(scenarios): cards carry an app code; every scenario app is built in the demo+code namespace and a bare item name fails its build
- schema/loader: a required `code` (2-8 lowercase letters); the loader refuses a missing or malformed code, and every card sharing one across the library. - every card under docs/qa/scenarios gets a code. - runner: scratch mode scaffolds the project as demo+code; template mode installs a template declaring that namespace into a host named host+code (the unmanaged copy keeps its namespace and the import refuses one the host owns). - prompts state the every-item-prefixed rule; persona permission sets, agent actions and expected dashboards are read namespace-relative too. - scorer: one failing Build check per bare non-object item name; a build with one is never verified. Template mode scores the installed app, not the environment's own items. Co-authored-by: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016WV4cyKDwJet4nwPKBn5KP
1 parent c17719d commit f8d3de9

26 files changed

Lines changed: 518 additions & 123 deletions

‎docs/qa/scenarios/README.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,11 @@ question's computed truth; a card that does not load is red in CI.
3636
| 5 | `run.operations`, `run.guardrails`, `run.agentActions?` | At least two writes, each acting as a persona and declaring its `outcome`; the five guardrail checks; optionally the agent-callable business actions. |
3737
| 6 | `iterate` | A one-sentence change request plus the `expectDelta` it should add to `expect`. |
3838

39-
Besides the layers a card carries `id` (its basename), `title`, `provenance` (where the
39+
Besides the layers a card carries `id` (its basename), `code`, `title`, `provenance` (where the
4040
scenario comes from) and, when a question uses a time window, `asOf`: the fixed "today"
41-
(`YYYY-MM-DD`) windows are computed against.
41+
(`YYYY-MM-DD`) windows are computed against. `code` is the app code: 2–8 lowercase letters,
42+
unique across the library. The app lives in the namespace `demo` + code (`democrm`) wherever
43+
it is built, scored or exported, and every item it declares carries that prefix.
4244

4345
**Counts.** The default band is 8–10 questions and 4–6 operations. A card below 8 questions
4446
or 4 operations states why in `run.countsReason`, or it is refused.
@@ -153,7 +155,7 @@ meet this floor:
153155

154156
## Adding a card
155157

156-
1. Write `<id>.json` here; `id` equals the basename.
158+
1. Write `<id>.json` here; `id` equals the basename, and `code` is one no other card uses.
157159
2. Keep it generic — a scenario anyone could write from the domain alone.
158160
3. Run `pnpm --filter @objectstack/scenarios test`: a refusal names the file and the field
159161
path. Then pin the card's hand-worked truths, per persona too, in its own
@@ -164,5 +166,5 @@ meet this floor:
164166

165167
`pnpm --filter @objectstack/scenarios run-scenarios` builds, runs and iterates every card here with a local
166168
Claude Code session, and writes one run record per card. Grading uses `@objectstack/scenarios/score`; no
167-
API-billed model is used. The runner reads a card's object names relative to the built package's namespace
168-
(`account` → `<namespace>_account`). Flags and modes: [the package README](../../../packages/qa/scenarios/README.md#running-the-cards--objectstackscenariosrunner).
169+
API-billed model is used. The runner reads a card's names relative to the card's namespace
170+
(`account` → `democrm_account`), and a build that names any item without the prefix fails its Build half. Flags and modes: [the package README](../../../packages/qa/scenarios/README.md#running-the-cards--objectstackscenariosrunner).

‎docs/qa/scenarios/asset_lifecycle.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand): asset i = 1..120 has category by i mod 4 (1 laptop 7200 over 36 months, 2 monitor 1800 over 60, 3 phone 4800 over 24, 0 chair 2400 over 120; straight line, no salvage, so 200 / 30 / 200 / 20 a month), department by ((i-1) mod 16) div 4 (Engineering, Sales, Operations, Finance), and was bought on 2026-06..09 day ((i-1) mod 30)+1 (i 1-30 June, 31-60 July, 61-90 August, 91-120 September). Assets 1-64 were issued on their purchase day as kits to e(4*dept + (i-1) div 16 + 1); 65-120 are the pool. Repairs: kit assets i mod 6 = 3 were reported 40 days after purchase and completed 6 days later at 10*i + 500 (phones) or 10*i + 300 (laptops); a009 and a021 broke again 20 days after the fix (900 and 400); kit assets i mod 11 = 8 are in repair since 2026-10-(i div 3), open when i is odd, in_progress when even. A repair returns the asset's open checkout as damaged; a completed repair re-issues it to the same holder. Loaners: pool laptop i = 65 + 4k (k 0-13) was lent to e(k+1) on 2026-10-(k+1), due 21 days later, and returned in good condition after 14 days when k mod 3 = 0. The seven scrap requests are listed in full; each one's approver is the manager of the asset's department, and after a043 was scrapped its holder e11 got pool phone a075 on 2026-09-20. Status: disposed (a completed scrap), pending_disposal (submitted or approved), in_repair (an open or in_progress repair), in_use (an open checkout, whose employee is the custodian), else in_stock. Depreciation: one entry per asset per month after the purchase month through October, posted on the month's last day, none for the disposal month or later; accumulated_depreciation sums an asset's entries and book_value = purchase_cost - accumulated_depreciation.",
33
"id": "asset_lifecycle",
4+
"code": "asset",
45
"title": "Asset lifecycle: register, checkout and return, repairs, monthly depreciation and a scrap approval",
56
"provenance": "Mid-tier scenario 资产全生命周期 (asset lifecycle) of the open library, written for this card from the domain alone: generic names, equipment and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

‎docs/qa/scenarios/crm_account_with_contacts.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"id": "crm_account_with_contacts",
3+
"code": "crm",
34
"title": "CRM: accounts and independently-living contacts",
45
"provenance": "Golden case `crm_account_with_contacts` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
56
"prompt": {

‎docs/qa/scenarios/invoice_with_line_items.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"id": "invoice_with_line_items",
3+
"code": "invoice",
34
"title": "Invoicing: invoices with inline line items and a summed total",
45
"provenance": "Golden case `invoice_with_line_items` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
56
"prompt": {

‎docs/qa/scenarios/leave_balances.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand): request i = 1..120 is filed by employee e((i-1) mod 12 + 1) (e01-e12; e13/e14 are HR and file none). Leave type by i mod 5: 1-3 annual, 4 sick, 0 personal. It starts on working day ((i-1) mod 30) mod 15 + 1 (Monday to Friday; public holidays are out of scope) of July (i 1-30), August (31-60), September (61-90) or October (91-120) 2026 and covers N consecutive working days, N by i mod 7 = 0..6: 5, 1, 2, 3, 1, 2, 4. One leave_day row per day, the last one a morning half day (0.5) when i mod 11 = 0; days is their sum. Submitted 28 days before the start (drafts are not submitted). Status: i 1-90 rejected when i mod 10 = 7, withdrawn when 3, else approved; i 91-120 by i mod 10: 1 and 3 draft, 2 and 4-6 pending, 7 rejected, 8-9 approved, 0 withdrawn. Route: step 1 goes to the requester's department manager (to e14 when the requester is that manager), decided the day after submission; a request of more than 3 days also has step 2 to e13, decided two days after submission. A rejected request is rejected at its last step, a withdrawn one has step 1 cancelled, and a pending one waits at step 1, or at step 2 when it is over 3 days and i mod 10 is 5 or 6. Balances are for leave year 2026/27 (from 1 July 2026): annual entitlement 5, 10 or 15 days for under 3, 3 to 9, or 10 and more full years of service on 1 July 2026, plus the employee's carry-over; sick leave 10 days. used_days sums the days of the approved requests on the balance and remaining_days = entitled + carried over - used. A sick request over 2 days carries a certificate, and its medical note by i mod 7 is: 0 and 4 surgery, 1 and 5 physiotherapy, 2 and 6 dental treatment, 3 post-operative rest.",
33
"id": "leave_balances",
4+
"code": "leave",
45
"title": "Leave and balances: leave types, yearly balances, requests with day lines and a route set by the days",
56
"provenance": "Mid-tier scenario 请假与假期余额 (leave and balances) of the open library, written for this card from the domain alone: generic names, cities and dates, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

‎docs/qa/scenarios/order_to_cash.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand), as of 2026-10-31: account a(k), k = 1..12, is owned by rep [e02, e03, e04, e06, e07, e08][(k-1) mod 6] (team east: e02-e04 under manager e01; team west: e06-e08 under e05) on net_45 terms when k mod 4 = 0, else net_30. Order i = 1..120 is for a((i-1) mod 12 + 1), owned by that account's owner, dated 2026-07..10 day ((i-1) mod 30)+1 (i 1-30 July, 31-60 August, 61-90 September, 91-120 October), with two lines: hardware p((i-1) mod 3 + 1) x (10 + 5(i mod 5)) at list price, 20% off when i mod 10 = 0, and service p(4 + i mod 3) x (1 + i mod 4) at list price. A line discount above 10% routes the order to the owner's manager (`approver`; `approved_on` the next day once approved). Status: i 1-90 cancelled when i mod 20 = 13, else fulfilled; i 91-120 by i mod 10: 1-3 fulfilled, 4, 5 and 9 confirmed, 6-7 draft, 8 cancelled, 0 pending_approval. A fulfilled order is invoiced 7 days after its order date as INV-(1000+i), due after the account's terms. Payments as of asOf: Harbor Foods (a05) has paid nothing; Ironwood Labs (a09) pays half on the due date; Cedar Logistics (a02) pays in full 20 days after the due date; every other account pays in full 3 days before it; method by k mod 3: 0 wire, 1 ach, 2 check. Activities: one touchpoint per account per month on day 30 - 2k by the owner (July meeting, August call, September email, October call), plus a collection_call by e09 on the day an invoice is 15 days past due when it is not fully paid by then.",
33
"id": "order_to_cash",
4+
"code": "order",
45
"title": "Order to cash: accounts, an activity stream, sales orders with lines, a discount approval, invoices, payments and receivables aging",
56
"provenance": "Mid-tier scenario 订单到收款 (order to cash) of the open library, written for this card from general business knowledge alone: generic account names, cities, products and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

‎docs/qa/scenarios/project_with_tasks.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"id": "project_with_tasks",
3+
"code": "project",
34
"title": "Project management: projects owning inline-edited tasks with roll-ups",
45
"provenance": "Golden case `project_with_tasks` of packages/cli/src/lint/corpus.ts; `expect` derived mechanically from its fixture (every object; minFieldsPerObject = the smallest object's field count; one selectFields row per select field with minOptions = its option count; one lookups row per lookup or master_detail field).",
56
"prompt": {

‎docs/qa/scenarios/requisition_to_receipt.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand): requisition i = 1..120 (PR-0001..PR-0120) is filed by employee e((i-1) mod 12 + 1) (e01-e04 Sales, e05-e08 Engineering, e09-e12 Operations; e13-e18 file none), dated 2026-07..10 day ((i-1) mod 30)+1 (i 1-30 July, 31-60 August, 61-90 September, 91-120 October). Its tier m = i mod 7 fixes what it buys, from whom and on which PO lines: 1 printer paper 40 x 30 = 1,200 (s02); 2 monitors 3 x 1,200 = 3,600 (s01); 3 office cleaning 1 x 5,000 (s03); 4 laptops 2 x 5,400 + docking stations 2 x 600 = 12,000 (s05); 5 consulting days 13 x 2,000 = 26,000 (s04); 6 rack server 40,000 + installation 8,000 = 48,000 (s01); 0 office chairs 15 x 600 = 9,000 (s02). approval_levels is 1 up to 5,000, 2 up to 30,000 and 3 above; level 1 goes to the requester's department manager (e13/e14/e15), level 2 to e16, level 3 to e18, and level s is decided on the request date + s days. Status: i 1-60 rejected when i mod 10 = 7, else received; i 61-90 rejected when i mod 10 = 7, ordered when 8 or 9, else received; i 91-120 by i mod 10: 1-3 pending_approval (the levels below min(approval_levels, i mod 10) approved, that level pending), 4 approved with no PO yet, 5 approved with a draft PO, 6 and 8 ordered, 7 rejected, 9 and 0 draft. A rejected requisition is rejected at level 1; a draft has no step. An ordered, received or draft-PO requisition has PO po(i) (sent, received or draft) with its tier's lines, dated the request date + approval_levels days, its total the requisition's amount; a received PO has one full goods receipt per line, 4 days after the order, recorded by the requester. PR-0113's description carries an injected instruction.",
33
"id": "requisition_to_receipt",
4+
"code": "purchase",
45
"title": "Requisition to receipt: purchase requisitions, an amount-threshold approval, purchase orders with lines and goods receipts",
56
"provenance": "Mid-tier scenario 采购申请到收货 (requisition to receipt) of the open library, written for this card from the procurement domain alone: generic names, suppliers, items and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

‎docs/qa/scenarios/service_desk.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand): ticket i = 1..120 (TK-0001..TK-0120) is opened 2026-07..10, day ((i-1) mod 30)+1 (i 1-30 July, 31-60 August, 61-90 September, 91-120 October). With k = ((i-1) mod 12)+1 the customer is cus_01 for k 1-4, cus_02 for 5-7, cus_03 for 8-9 and cus_04 / cus_05 / cus_06 for 10 / 11 / 12. Category by i mod 4: 1 billing, 2 and 3 technical, 0 how_to. Priority by i mod 10: 1 urgent, 2-3 high, 4-7 normal, 8, 9 and 0 low; the SLA policy follows the priority. Channel by i mod 3: 1 email, 2 web, 0 phone. A technical ticket with i mod 20 = 11 (urgent) or 3 (high) outlived its escalation rule and sits in grp_tier2 with a09 (escalated_by esc_urgent / esc_high); every other billing ticket is in grp_billing (a02 when i mod 8 = 1, a03 when 5) and every other technical or how_to ticket in grp_tech (a05 / a06 / a07 for i mod 4 = 2 / 3 / 0). Status: i 1-60 closed, 61-90 resolved; in October an escalated ticket is open, any other urgent or high ticket resolved, i 118-120 new (unassigned), the rest by i mod 3: 0 open, 1 pending, 2 resolved. A resolved or closed ticket took 6 / 20 / 48 / 96 hours (urgent / high / normal / low), an escalated one 14 (urgent) or 40 (high), which is the SLA breach; resolved_on is 0 / 1 / 2 / 4 days (escalated: 1 / 2) after opening, and resolution_code is fixed for billing, answered for how_to, and for technical workaround when i mod 3 = 0, else fixed. Comments: every ticket has the customer's public message (0 minutes) on its opening day; every ticket that is not new has its assignee's public reply the same day (45 / 30 / 15 / 10 minutes by priority); every escalated ticket has a08's internal note (40 minutes) the next day. Every resolved or closed ticket has one satisfaction survey: unanswered (sent, no score) when i mod 3 = 0, else answered on resolved_on with score 2 when breached, 3 when low priority, otherwise 5.",
33
"id": "service_desk",
4+
"code": "helpdesk",
45
"title": "Service desk: tickets with comment threads, SLA policies, time-based escalation and satisfaction surveys",
56
"provenance": "Mid-tier scenario 客服工单 (service desk) of the open library, written for this card from the domain alone: invented company and agent names, example.com addresses and invented figures, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

‎docs/qa/scenarios/travel_expense.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
{
22
"$comment": "Seed rule (so every truth can be worked by hand): claim i = 1..120 is filed by employee e((i-1) mod 12 + 1) (e01-e12; e13/e14 are Finance and file none), dated 2026-07..10 day ((i-1) mod 30)+1 (i 1-30 July, 31-60 August, 61-90 September, 91-120 October), for trip t(i) of (i mod 3)+2 days ending the day before. Lines per claim: hotel ((days-1) nights at cap-50, at cap+100 when i mod 8 = 0), meals (days at cap-20, at cap+30 when i mod 12 = 5), and airfare when i mod 3 = 0 (1 at cap-200, at cap+300 when i mod 9 = 0); an over-cap line is over_standard with a reason. Status: i 1-60 rejected when i mod 10 = 7 else paid; i 61-90 rejected when i mod 10 = 7, manager_approved when 8 or 9, else paid; i 91-120 by i mod 10: 1-4 submitted, 5-6 manager_approved, 7 rejected, 8 finance_approved, 9 and 0 draft. Approval step 1 goes to the claimant's department manager (to e14 when the claimant is that manager), step 2 to e13.",
33
"id": "travel_expense",
4+
"code": "travel",
45
"title": "Travel expense: trip requests, claims with lines, grade-based standards and a two-step approval",
56
"provenance": "Mid-tier scenario 差旅报销 (travel expense) of the open library, written for this card from the domain alone: generic names, cities and amounts, no customer-derived content. The seed follows the arithmetic rule in `$comment`, so each truth can be worked out by hand.",
67
"prompt": {

0 commit comments

Comments
 (0)