Repository navigation
Commit c0f88b9
feat(scenarios)!: cards declare user-handle fields and persona records; seed dependence is decided per turn (#31275)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1X4Jt9kqe7MNvcC7UcRXr6hv)_
Fixes #30914
Clause-②: yes (narrowing) — the published card schema
(`ScenarioCardSchema`, `.` entry) gains `run.userHandleFields` and
`personas[].record`; the loader refuses cards it loaded before (a scope
rule ending on an undeclared field, a scoped persona naming no person);
`./runner`'s `seedDependence` takes the loaded scenario and asks per
turn.
Metadata-protocol: none
Skills-docs: none — no skill or docs page teaches the scenario card
format; its own documentation, `docs/qa/scenarios/README.md`, is updated
here.
## 维护者速读
- 改了什么:场景卡片格式新增两项声明——`run.userHandleFields`(哪些种子字段的值是"用户句柄",如
`account.owner = u_sam`)与 `personas[].record`(有行范围的角色对应哪一条种子行,如员工
`e06`);加载器拒收"范围规则落在既非句柄、也非查找/选项/`id` 的字段上"以及"有范围却没指明是谁"的卡;跑测记录新增
`seed.rewritten`;种子行被应用拒收时,只把"预期答案因此改变"的那几轮问答记为未测,其余照常评分。
- 为什么改:句柄字段原来靠推断,新卡可能被悄悄写错种子;一行被拒就整对象作废,遮掉了本可评分的结果(#30882 契约复审的
S1、S2);`record` 是维护者 2026-11-07 13:34 对 #30913 裁决的卡片格式那一半(S3)。
- 风险与回滚:只动 `@objectstack/scenarios`(QA
工具包)与九张卡;九张卡全部照常加载,种子改写的字段集合与改前完全相同(`account.owner`、`invoice.owner`、`task.assignee`);私有覆盖目录里的卡若没有这两项声明会被拒收,补声明即可。回滚即
revert 本 PR。
- 先看:`packages/qa/scenarios/src/loader.ts`(新拒收与
`truthsOver`)、`packages/qa/scenarios/src/runner/orchestrate.ts`(`seedDependence`
按轮判断)、`docs/qa/scenarios/service_desk.json`(三个角色的 `record`)。
- 两分钟验证:`pnpm --filter @objectstack/scenarios exec vitest run
test/refusals.test.ts test/runner/units.test.ts`,预期全绿;其中 ct4 用例显示
`contacts_per_account` 以 `sdr` 身份仍被评分。
- 需要你做的:无,席位落地。
## What changes
**S2 — user-handle fields are declared.**
- `run.userHandleFields: [{ object, field }]` (optional, at least one
entry) lists the seed fields whose text stands for a user.
`userHandleFields` in `src/runner/prompts.ts` now reads this list and
nothing else; it no longer infers a handle from "a direct scope field
that is no lookup or select".
- The loader refuses a persona `scope` rule whose last field (the last
hop of a relation path) is not `id`, a declared lookup, a declared
select or a declared handle field. It also refuses a handle declaration
on the row id, a declared lookup, a declared select, an object the card
does not seed, a field no seed row carries, and a duplicate.
- The run record's `seed.rewritten` lists each `object.field` the seed
loaded with a user id in place of the handle text (`RunHalf.seed`,
`src/runner/orchestrate.ts`).
- The three control cards declare their handle field. Before and after,
measured over the nine cards: before (`origin/main`, inferred)
`crm_account_with_contacts:account.owner`,
`invoice_with_line_items:invoice.owner`,
`project_with_tasks:task.assignee`; after (declared) the same three.
**S1 — seed dependence per turn.**
- `seedDependence(scenario, refused)` keeps its row walk (the refused
rows plus their children through the declared lookups), then computes
every truth again over the seed less those rows with `truthsOver` (new
in `src/loader.ts`, the loader's own resolver, row-scope test and
withheld rule, factored out as `seedReader` and `withholds`).
`question(q, persona)` is not measured only when that turn's expected
answer changes, or is no longer defined (`seed: its expected answer is
not defined without …`).
- `runHalf` asks it per turn (inside the persona loop). Operations are
unchanged.
- The ct4 case: refusing contact `ct4` on `crm_account_with_contacts`
used to leave 5 turns not measured (`contacts_per_account/sdr`,
`unattached_contacts`, `best_covered_account`, `tech_contacts`,
`contacts_by_industry`); it now leaves 2 (`unattached_contacts`,
`contacts_by_industry`, the two whose answer ct4 changes).
`contacts_per_account` as `sdr` stays measured.
**S3 — `personas[].record = { object, id }`.**
- Required on a persona with a `scope`, unless that persona is named by
a direct scope rule on a declared handle field; optional on a global
persona. The loader refuses a record naming no seeded row (including a
row of an object the card does not seed). `PERSON_USER_FIELD = 'user'`
fixes the person object's user field by name.
- Filled on the six cards the ruling names: `asset_lifecycle` (employee
e06, dept_manager e05), `leave_balances` (e07, e05), `order_to_cash`
(e03, e01), `requisition_to_receipt` (e02, e13), `travel_expense` (e02,
e01), `service_desk` (agent a02, agent a04, customer cus_03). Each id is
the one the persona's `role` already names.
- Format and loader only: the runner does not read `record` (#30913's
half).
**Namespacing and docs.** `namespacedScenario`
(`src/runner/namespace.ts`) prefixes the object of each handle
declaration and each `record`. `docs/qa/scenarios/README.md` documents
both declarations and the new refusals; the package README documents
per-turn dependence and `seed.rewritten`. One `patch` changeset.
## Tests
- `test/refusals.test.ts`: 12 new refusal pins (one refusal each, exact
path) and two controls: a scoped persona naming its record (and a global
persona naming one) loads clean; deleting `run.userHandleFields` from
the crm card refuses both of `sdr`'s scope rules and its missing person.
The gallery-floor fixture now scopes its second persona on a declared
select and names its record.
- `test/runner/units.test.ts`: `userHandleFields` reads the declaration
(a card declaring none has none); across the nine cards the handle set
is the three fields and every record is pinned; `seedDependence` per
turn on the ac1 case, the ct4 case and a truth the rest of the seed no
longer defines.
- `test/runner/orchestrate.test.ts`: `seed.rewritten` in the record; the
ct4 run (7/7 measured, 2 not measured; the driver is asked
`contacts_per_account` as `sdr`); the all-accounts-refused run keeps the
three turns no account feeds, and a half with every turn not measured is
still `not-run`.
Ablations, each through `scripts/ablation-replace.mjs` on committed HEAD
`33a20989`: mutation landed (anchor 1 → 0, replacement 0 → 1, blob
changed), suite red, restored to the HEAD blob; `git diff HEAD` empty
after all legs. The tests import `src/` directly (no `dist/` in the
path), so no rebuild leg.
| Leg | Mutation | Red |
|:--|:--|:--|
| A1 | scope rule ending on an undeclared field not refused | 2 failed
(the refusal pin; the no-declaration test) |
| A2 | handle declared on a lookup, select or id not refused | 3 failed
(the three pins) |
| A3 | handle on an unseeded object not refused | 1 failed |
| A4 | handle field no seed row carries not refused (`checkField` call
deleted) | 1 failed |
| A5 | duplicate handle declaration not refused | 1 failed |
| A6 | record naming no seeded row not refused | 2 failed (row missing;
object unseeded) |
| A7 | scoped persona with no person not refused | 2 failed (the pin;
the no-declaration test) |
| A8 | per-turn comparison removed (every turn on an out row not
measured) | 4 failed (both runCard dependence tests, both units
dependence tests) |
| A9 | `seed.rewritten` not recorded | 2 failed (both seed-record pins)
|
## Acceptance notes
- **Reading of "required for a persona with a scope".** The S3 ruling
says required for a scoped persona; the #30913 ruling (Q1 = A′, item 5)
keeps the user-handle route for crm, invoice and project, whose scoped
personas are users with no person row. Both hold here: `record` is
required unless the persona is named by a direct scope rule on a
declared handle field. On the nine cards that is exactly the three
controls.
- **Global personas carry no record.** It is optional for them and the
ruling names only the scoped ones; whether #30913's owner rule wants one
on `asset_admin`, `hr_officer` and the like is that card's call.
- **The terminal-field rule reads the last hop of a path.**
`contact.account.owner` ends on `account.owner`, a declared handle;
`order_line.order.owner.team` ends on `employee.team`, a declared
select. The card's wording is "a scope on a field"; reading only direct
rules would leave a path ending on an undeclared field to inference,
which is what S2 removes.
- **File surface.** `src/runner/namespace.ts` is outside the claim's
list: without it the namespaced card the runner reads keeps bare object
names in the two new keys and `userHandleFields` answers `account.owner`
where the seed loop looks for `democrm_account.owner`.
`src/runner/seed.ts` needed no change. `orchestrate.ts` changes
`seedDependence`, its caller in `runHalf`, the `RunHalf.seed` type and
one line of the seed loop (`rewritten`).
- **Serial with #31159.** No change to `runner/driver.ts` or
`runner/cli.ts`. `origin/main` moved by two commits since the branch
point, neither in `packages/qa/scenarios`.
## Verification
- Heads: code and tests at `33a20989`; `4b4df50a` adds one sentence to
the changeset only. The derived loop started on `33a20989`; the
changeset-reading gates were run again on `4b4df50a` (below). Base
`bdefd49f`; `origin/main` has moved two commits since, neither in
`packages/qa/scenarios`.
- `pnpm --filter @objectstack/scenarios exec vitest run`: Test Files
28/28, Tests 571/571 (after `pnpm --filter '@objectstack/scenarios^...'
build`, VERDICT command-exit 0, and the package's own build, which
`test/build/*` read).
- `pnpm --filter @objectstack/scenarios typecheck` (`tsc --noEmit`, its
program includes `test/**`): exit 0.
- `node --stack-size=4000 node_modules/eslint/bin/eslint.js
packages/qa/scenarios --no-inline-config` (the root `pnpm lint`, scoped;
the package has no lint script): exit 0.
- On `4b4df50a`: `check-changeset-no-major.mjs --base origin/main` with
this body as the event: exit 0 ("`yes (narrowing)` and metadata-protocol
`none` … a non-protocol change at `patch`");
`check-adr-0087-registration.mjs --base origin/main`: exit 0 (the
`no-migration-prescription` disposition read);
`check-empty-changeset.mjs --base origin/main`: exit 0;
`check-partof-closing-keyword.mjs` with `PR_BODY` set to this body: exit
0.
- **Not measured:** a live model run (paused by the maintainer) and a
recorded-driver replay (no recording in this container); the per-turn
rule is pinned on the real crm card through the fake-app `runCard` tests
instead.
<details><summary>Derived gates (dispatch-gates --commands, 62
commands)</summary>
```text
node scripts/check-adr-0087-registration.mjs --base origin/main :: exit 0
node scripts/check-adr-0087-registration.mjs --self-test :: exit 0
node scripts/check-adr-links.mjs :: exit 0
node scripts/check-adr-links.mjs --self-test :: exit 0
node scripts/check-changeset-no-major.mjs --base origin/main :: exit 0
node scripts/check-closing-keyword-parity.mjs :: exit 0
node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0
node scripts/check-comment-mask-adoption.mjs :: exit 0
node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0
node scripts/check-comment-mask-corpus.mjs :: exit 0
node scripts/check-dts-emitted.mjs --self-test :: exit 0
node scripts/check-empty-changeset.mjs --base origin/main :: exit 0
node scripts/check-empty-changeset.mjs --self-test :: exit 0
node scripts/check-issue-citations.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0
node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0
node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0
node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0
node scripts/ci/snapshot-next.mjs --self-test :: exit 0
node scripts/docs-audit/check-affected-docs.mjs :: exit 0
node scripts/docs-audit/check-drift-comment.mjs :: exit 0
node scripts/pm/release-rehearsal-clone.mjs --self-test :: exit 0
node scripts/release-pending-publish.mjs --self-test :: exit 0
pnpm --filter @objectstack/lint run check:doc-formula-expressions :: exit 0
pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0
pnpm check:changeset-gate-self-tests :: exit 0
pnpm check:ci-filter-parity :: exit 0
pnpm check:closed-ceremony-text :: exit 0
pnpm check:cross-package-test-inputs :: exit 0
pnpm check:dispatcher-error-vocabulary :: exit 0
pnpm check:doc-authoring :: exit 0
pnpm check:driver-memory-census :: exit 0
pnpm check:dts-closure :: exit 0
pnpm check:dual-build-cjs-loads :: exit 3
pnpm check:engine-double-contract :: exit 0
pnpm check:gitlink-declared :: exit 0
pnpm check:issue-citations :: exit 0
pnpm check:keyed-text-bounds :: exit 0
pnpm check:lean-entry-closure :: exit 0
pnpm check:logger-receiver-detach :: exit 0
pnpm check:nul-bytes :: exit 0
pnpm check:objectql-double-limit :: exit 0
pnpm check:org-identifier :: exit 0
pnpm check:page-declaration-shape :: exit 0
pnpm check:platform-object-tenancy-census :: exit 0
pnpm check:published-files :: exit 0
pnpm check:query-options-erasure :: exit 0
pnpm check:refd-timer-probe :: exit 0
pnpm check:registry-log-declared :: exit 0
pnpm check:rest-log-spy-declared :: exit 0
pnpm check:slot-lookup :: exit 0
pnpm check:sourcemap-no-sources-content :: exit 0
pnpm check:system-context-census :: exit 0
pnpm check:test-source-alias :: exit 0
pnpm check:tier-file-adoption :: exit 0
pnpm check:type-check-coverage :: exit 0
pnpm check:type-check-debt :: exit 0
pnpm check:undeclared-dep-imports :: exit 0
pnpm check:vouch-for-reference-callers :: exit 0
pnpm check:watch-hint-literal :: exit 0
pnpm check:where-matcher :: exit 0
```
- `dispatch-gates --ran`: "62 derived famil(ies) accounted for — 61 run,
1 NOT-MEASURED".
- NOT MEASURED: `pnpm check:dual-build-cjs-loads`, exit 3 PREREQUISITE
NOT MET (it loads every package's `dist/`; only the scenarios closure
was built). CI's lint job runs it on a full build.
- `check-plugin-teardown-shape.mjs --self-test` answered exit 3 on the
shallow clone until `git fetch --depth=1 origin 621a487…`; rerun exit 0
(recorded above as the rerun).
- `check:type-check-debt` re-measured under the lock (414 s): "0 raw tsc
error(s) total, none above its recorded number".
</details>
<details><summary>Gates outside the derived set: the artifact-roster and
declared-wide families (47 commands)</summary>
```text
node scripts/check-closing-target-claim.mjs :: exit 2
node scripts/check-partof-closing-keyword.mjs :: exit 2
node scripts/check-platform-checklist-watchdog.mjs :: exit 0
node scripts/check-published-list-mirrors.mjs :: exit 0
node scripts/check-single-claim-paths.mjs :: exit 2
node scripts/check-skills-token-ratchet.mjs :: exit 0
node scripts/pm/check-governed-queue-guard.mjs --toolchain-needed :: exit 0
node scripts/pm/check-governed-queue-guard.mjs :: exit 1
pnpm --filter @objectstack/spec run check:error-code-provenance :: exit 0
pnpm --filter @objectstack/spec run check:meta-url-spelling :: exit 0
pnpm --filter @objectstack/spec run check:react-blocks :: exit 0
pnpm --filter @objectstack/spec run check:strictness-ledger :: exit 0
pnpm check:auth-mount-ledger :: exit 0
pnpm check:authz-resolver :: exit 0
pnpm check:changeset-fixed :: exit 0
pnpm check:cli-examples-parity :: exit 0
pnpm check:engine-double-contract :: exit 0
pnpm check:error-code-casing :: exit 0
pnpm check:error-status-conformance :: exit 0
pnpm check:facade-delegation-only :: exit 0
pnpm check:filter-alias-parity :: exit 0
pnpm check:future-spec-major :: exit 0
pnpm check:i18n-stale-fill :: exit 0
pnpm check:init-service-contract :: exit 0
pnpm check:live-db-isolation :: exit 0
pnpm check:meta-type-normalized :: exit 0
pnpm check:object-def-param-keys :: exit 0
pnpm check:optional-error-sink :: exit 0
pnpm check:overlay-whitelist-table :: exit 0
pnpm check:pm-governed-prose :: exit 0
pnpm check:pm-label-desc-cap :: exit 0
pnpm check:pm-settings-deny-roster :: exit 0
pnpm check:published-readme-exports :: exit 0
pnpm check:resume-authority-declared :: exit 0
pnpm check:route-envelope :: exit 0
pnpm check:route-ledger-census :: exit 0
pnpm check:runner-env-posture :: exit 0
pnpm check:scaffold-emission-policy :: exit 0
pnpm check:select-gate-families :: exit 0
pnpm check:select-shard-packages :: exit 0
pnpm check:settings-bind-window :: exit 0
pnpm check:skill-top-level-keys :: exit 0
pnpm check:stack-collection-maps :: exit 0
pnpm check:startup-registry-verdict :: exit 0
pnpm check:tenant-chokepoint :: exit 0
pnpm check:verify-stand-in :: exit 0
pnpm check:wildcard-fallthrough :: exit 0
```
- The four non-zero rows judge a pull request, not the tree, and refuse
without its context (exit 2 "handed no pull request", exit 1 for the
event file): NOT MEASURED locally, read from this PR's check-runs.
`check-partof-closing-keyword.mjs` with `PR_BODY` answered exit 0
(above).
- `pnpm --filter @objectstack/spec run check:error-code-provenance` is
among them, exit 0: the diff adds no error-code constant.
</details>
Every CI family is read from this PR's check-runs, not re-run here.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01FEkQ5nVm4yfBmbggVVSu4R)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent b3bd75d commit c0f88b9
20 files changed
Lines changed: 425 additions & 89 deletions
File tree
- .changeset
- docs/qa/scenarios
- packages/qa/scenarios
- src
- runner
- test
- runner
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
| 36 | + | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| |||
159 | 159 | | |
160 | 160 | | |
161 | 161 | | |
162 | | - | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
163 | 179 | | |
164 | 180 | | |
165 | 181 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
767 | 767 | | |
768 | 768 | | |
769 | 769 | | |
| 770 | + | |
770 | 771 | | |
771 | 772 | | |
772 | 773 | | |
| |||
787 | 788 | | |
788 | 789 | | |
789 | 790 | | |
| 791 | + | |
790 | 792 | | |
791 | 793 | | |
792 | 794 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
49 | 50 | | |
50 | 51 | | |
51 | 52 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
817 | 817 | | |
818 | 818 | | |
819 | 819 | | |
| 820 | + | |
820 | 821 | | |
821 | 822 | | |
822 | 823 | | |
| |||
828 | 829 | | |
829 | 830 | | |
830 | 831 | | |
| 832 | + | |
831 | 833 | | |
832 | 834 | | |
833 | 835 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
838 | 838 | | |
839 | 839 | | |
840 | 840 | | |
| 841 | + | |
841 | 842 | | |
842 | 843 | | |
843 | 844 | | |
| |||
850 | 851 | | |
851 | 852 | | |
852 | 853 | | |
| 854 | + | |
853 | 855 | | |
854 | 856 | | |
855 | 857 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
848 | 848 | | |
849 | 849 | | |
850 | 850 | | |
| 851 | + | |
851 | 852 | | |
852 | 853 | | |
853 | 854 | | |
| |||
862 | 863 | | |
863 | 864 | | |
864 | 865 | | |
| 866 | + | |
865 | 867 | | |
866 | 868 | | |
867 | 869 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
691 | 691 | | |
692 | 692 | | |
693 | 693 | | |
| 694 | + | |
694 | 695 | | |
695 | 696 | | |
696 | 697 | | |
| |||
703 | 704 | | |
704 | 705 | | |
705 | 706 | | |
| 707 | + | |
706 | 708 | | |
707 | 709 | | |
708 | 710 | | |
| |||
714 | 716 | | |
715 | 717 | | |
716 | 718 | | |
| 719 | + | |
717 | 720 | | |
718 | 721 | | |
719 | 722 | | |
| |||
0 commit comments