Repository navigation
Commit de3996b
feat(scenarios): every Build check traces to the build prompt (#31185)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1X93oALNL86RLkdGBScHovs1)_
Fixes #31144
Clause-②: no
Metadata-protocol: none
## 维护者速读
- 改了什么:场景跑测的建造提示词现在按名写明 Build
半场逐条检查的结构(查找字段指向、单选字段最少选项数、仪表盘名、应用导航目标),迭代提示词同样写明新增对象与字段;五张把审批历史当普通数据种入的卡片改为“审批即数据”,去掉
`approvals.*` 标签并在描述里说清楚;带行内编辑标签的卡片在描述里写明明细在主表单上内联编辑。
- 为什么改:首次完整中档跑测里,忠实按描述建造的应用因为猜不中仪表盘名、选项个数、审批引擎而丢分;原则是每条 Build
检查都必须能在提示词里找到出处。
- 风险与回滚:只动私有包 `@objectstack/scenarios` 与 `docs/qa/scenarios` 卡片,不发布;回滚即
revert 本 PR。代价是开放卡库暂时没有一张卡考察平台审批引擎(三条审批标签行保留在表里)。
- 先看:`packages/qa/scenarios/src/runner/prompts.ts`(`structureLines` /
`changeContract`)、`packages/qa/scenarios/test/runner/prompt-trace.test.ts`、`docs/qa/scenarios/order_to_cash.json`
的描述与标签。
- 两分钟验证:`pnpm --filter @objectstack/scenarios exec vitest run
test/runner/prompt-trace.test.ts`,预期 40 个用例全绿。
- 需要你做的:无,席位落地。
## What changes
**The rule.** Every Build check traces to text the build prompt carries;
otherwise a faithful build fails for not guessing a name.
1. **Item 1, dashboard / report names: the route chosen is "state each
name in the data contract".** `dataContract`
(`packages/qa/scenarios/src/runner/prompts.ts`) gains a `Structure:`
block, rendered by the new `structureLines`, that names every `expect`
row the exact-name scorer (`scoreGraph`) checks beyond the seed: each
lookup's target, each single-choice field with its minimum option count,
each dashboard, and each app navigation target. Why this route and not
"check by function":
- The counted Build score is the exact-name scorer; the name-agnostic
one (`scoreGraphShape`) is reported, not counted. Every other `expect`
row already traces by name (objects and fields through the seed, persona
sets as `NAMESPACE_PERSONA`); dashboards and nav targets were the only
rows that did not.
- "By function" already exists where it belongs: the `dashboards.report`
and `dashboards.widget-measures` tag rows judge a report or widget over
an object the card names. A functional `expect.dashboards` would
duplicate them and change the card schema, the loader's gallery floor,
`appNavTargets` resolution, the iterate delta and the screenshot
exporter (`export/screenshots.ts` picks the card's dashboard by name).
- The cost is that a card prescribes the name. The naming rule already
prescribes the prefix of every item, and the run showed builds follow
contract names verbatim (every persona set was built by its contract
name).
- Measured on main `8ef4d309`: six cards pin a dashboard and its nav
entry, not three. `requisition_to_receipt` pins
`purchasing_spend_by_department` and `service_desk` pins
`support_performance`, besides the three the card names. The rule
applies to all six.
2. **Item 2, option minimums: the route chosen is "state the minimum in
the contract".** Each `expect.selectFields` row is stated as a
single-choice field with its count, for example ``
`demopurchase_supplier.category` (4) ``. On a card carrying
`records-forms.select-options`, one more line states that tag's check:
every other single-choice field the build declares offers two options or
more. That is the check behind the `asset_category.depreciation_method`
failure: it is the tag row
`select:asset_category.depreciation_method->options>=2`, not an `expect`
row. Every row is kept: in five rows the minimum is above what the seed
shows (`leave_day.portion` 3/2, `invoice.status` 4/3,
`invoice.aging_bucket` 5/4, `purchase_order.status` 4/3,
`escalation_rule.priority` 4/3), and the prose names each extra option
(afternoon, written off, over 90, cancelled, low).
3. **Item 3, approval tags: all five cards model approvals as data.** I
decided each card separately; the table is in Acceptance notes. The
card's stop condition does not apply, because modelling approvals as
data keeps every approval-history question answerable. For
`asset_lifecycle`, `leave_balances`, `order_to_cash`,
`requisition_to_receipt` and `travel_expense`:
- **Tags:** `approvals.approval-flow`, `approvals.multi-step-approval`
and `approvals.dynamic-approver-routing` come off the cards.
- **Prose (zh and en):** says where each decision is recorded (an
approval step record, or the approver and decision date on the request)
and that deciding updates that record. It no longer lists "an approval
process" among the preferred declarative tools.
- **Seed:** unchanged. It already modelled approvals this way.
4. **Item 4, `master-detail-inline-edit`: it is stated in the prompt.**
The prose of every card carrying the tag now says the lines are details
of their master, edited inline on its form: `order_to_cash`,
`leave_balances`, `requisition_to_receipt` and `travel_expense`, plus
the control card `invoice_with_line_items` (one clause).
`project_with_tasks` already said it.
5. **Bounded in-place fix in the same class: the iterate prompts.**
`iteratePrompt` and `templateIteratePrompt` now append `changeContract`.
The iterate re-score checked `iterate.expectDelta` names that the
iterate prompt never stated (`demoorder_credit_note`,
`demoasset_asset_transfer.to_department`, …). The prompts now state the
new objects (with the card's field minimum) and the delta's lookups and
option fields. The fix meets all four in-place conditions: the same
defect class, the renderer from item 1, `prompts.ts` held by this claim,
and the same gate family.
6. **Docs:**
- `docs/qa/scenarios/README.md` gains "Build checks trace to the prompt"
(names, tags, approvals).
- The package README gains a "Structure contract" bullet.
## Why
The first complete mid-tier run (cloud#3600, main `5f003744`) lost Build
points on checks the build was never told about: dashboard names, option
counts, an approval engine the cards' own data never runs through, and
inline edit.
## Tests
- New `packages/qa/scenarios/test/runner/prompt-trace.test.ts` (40
tests). It reads the exact-name checks off the scorer itself, as
`scoreGraph` over an empty graph, and requires each to name text in the
build prompt (or in both iterate prompts, for the delta's checks) for
all 9 cards. A check kind the contract does not know is reported as
untraced. It also pins the two-option floor line where the tag is
carried, the inline-edit clause in each language of a tagged card, and
that a card seeding a decision date (`decided_on` / `approved_on`)
carries no `approvals.*` tag and asks for no approval process.
- `test/score/scenario-tags.test.ts`: the library pin "each row is
declared by a card" now exempts the two approval-engine rows, which no
open card declares, and says why. The rows stay in the table.
- Ablations, run through `scripts/ablation-replace.mjs`. Each mutation
landed on disk, was restored, and the blob equals HEAD (`4dee8b93ba84`,
`git diff HEAD` empty):
- With the `Structure:` block emptied, the suite read 18 failed of 40
(the build trace for 9 cards and the select floor for 9).
- With `changeContract`'s structure rows emptied, it read 18 failed of
40 (the iterate trace and the floor).
## Acceptance notes
<details><summary>Item 3: the per-card approval reading</summary>
| Card | Approval history in the seed | 问数 reading it | Operations
grading approval | Under the engine route | Decision |
|:--|:--|:--|:--|:--|:--|
| asset_lifecycle | `disposal_request.approver`, `decided_on`, `status`
| `dr_0005_waiting_on` | `manager_approves_scrap`,
`employee_approves_own_scrap` update the request | the waiting-on
question loses its rows; the approver is an `employee` row, not a user |
data |
| leave_balances | `approval_step` (141 rows) | `lr_0105_waiting_on`,
`pending_by_approver` | `manager_approves_step_one` updates an approval
step | the approval step object leaves the seed, with two questions and
two persona scopes | data |
| order_to_cash | `sales_order.approver`, `approved_on`, `status` |
`so_0120_waiting_on` | `approve_discount`, `rep_approves_own_discount`
update the order | the approver is an `employee` lookup, which an engine
approver cannot route to | data |
| requisition_to_receipt | `approval_step` (184 rows) |
`pr_0111_waiting_on` | `manager_approves_level_one`,
`manager_approves_finance_level` update an approval step | the same as
leave_balances | data |
| travel_expense | `approval_step` (204 rows) | `ec_0093_waiting_on` |
`manager_approves_step_one`, `employee_approves_own_claim` update an
approval step | the same as leave_balances | data |
Each card's `agentActions` also decide approvals on the data object:
`decide_approval_step`, `decide_scrap_request`,
`decide_discount_approval`.
</details>
- **No open card exercises the platform approval engine now.** The three
tag rows stay in `CAPABILITY_TAG_CHECKS`, judged and tested, for a card
written for the engine: approvers as users, no seeded engine history,
and approving through the engine's decide door. Writing such a card is
outside this card.
- **The control card `invoice_with_line_items` now differs from its
corpus case.** Its prose gains ", inline on its form". The golden case
in `packages/cli/src/lint/corpus.ts` keeps its wording: the card's
`provenance` derives only `expect` from it, and `scenario-cards.test.ts`
pins only that.
- **Card premise readings:**
- The `select:…>=Nopts` spelling is real for `expect` rows. The fourth
example the card gives is the tag row's spelling
`select:OBJECT.FIELD->options>=2`.
- The PM hypothesis "requisition_to_receipt may pin none" is disproved:
it pins one dashboard.
- **Out of scope:** the runner's `owner_id` and seed logic (#30914 /
#30913), and #31145's run-grading items.
## Verification
- Head `115474c9`, base `8ef4d309`.
- `pnpm --filter @objectstack/scenarios test`: Test Files 24/24, Tests
495/495 on head `115474c9` (after `pnpm --filter
'@objectstack/scenarios^...' build` and the package build).
- `pnpm --filter @objectstack/scenarios typecheck` (`tsc --noEmit`, its
`include` covers `test/**`): exit 0.
- `node --stack-size=4000 node_modules/eslint/bin/eslint.js
packages/qa/scenarios --no-inline-config` (the root `pnpm lint` command,
scoped; the package has no lint script, and the cards are JSON, which
the config does not lint): exit 0.
- **Not measured:**
- A recorded-driver replay. No recording is in this container, and the
recorded driver replays turns by card, mode and phase, not by prompt.
- A live model run. Live runs are paused by the maintainer.
<details><summary>Derived gates (dispatch-gates --commands on head, 52
commands)</summary>
```text
node scripts/check-adr-links.mjs :: exit 0
node scripts/check-adr-links.mjs --self-test :: exit 0
node scripts/check-closing-keyword-parity.mjs :: exit 0
node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0
node scripts/check-comment-mask-adoption.mjs :: exit 0
node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0
node scripts/check-comment-mask-corpus.mjs :: exit 0
node scripts/check-dts-emitted.mjs --self-test :: exit 0
node scripts/check-issue-citations.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs :: exit 0
node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0
node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0
node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0
node scripts/docs-audit/check-affected-docs.mjs :: exit 0
node scripts/docs-audit/check-drift-comment.mjs :: exit 0
pnpm --filter @objectstack/lint run check:doc-formula-expressions :: exit 0
pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0
pnpm check:ci-filter-parity :: exit 0
pnpm check:closed-ceremony-text :: exit 0
pnpm check:cross-package-test-inputs :: exit 0
pnpm check:doc-authoring :: exit 0
pnpm check:driver-memory-census :: exit 0
pnpm check:dts-closure :: exit 0
pnpm check:dual-build-cjs-loads :: exit 3
pnpm check:engine-double-contract :: exit 0
pnpm check:gitlink-declared :: exit 0
pnpm check:issue-citations :: exit 0
pnpm check:keyed-text-bounds :: exit 0
pnpm check:lean-entry-closure :: exit 0
pnpm check:logger-receiver-detach :: exit 0
pnpm check:nul-bytes :: exit 0
pnpm check:objectql-double-limit :: exit 0
pnpm check:org-identifier :: exit 0
pnpm check:page-declaration-shape :: exit 0
pnpm check:platform-object-tenancy-census :: exit 0
pnpm check:published-files :: exit 0
pnpm check:query-options-erasure :: exit 0
pnpm check:refd-timer-probe :: exit 0
pnpm check:registry-log-declared :: exit 0
pnpm check:rest-log-spy-declared :: exit 0
pnpm check:slot-lookup :: exit 0
pnpm check:sourcemap-no-sources-content :: exit 0
pnpm check:system-context-census :: exit 0
pnpm check:test-source-alias :: exit 0
pnpm check:tier-file-adoption :: exit 0
pnpm check:type-check-coverage :: exit 0
pnpm check:type-check-debt :: exit 0
pnpm check:undeclared-dep-imports :: exit 0
pnpm check:vouch-for-reference-callers :: exit 0
pnpm check:watch-hint-literal :: exit 0
pnpm check:where-matcher :: exit 0
node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0
```
- `dispatch-gates --ran`: 52 derived, 51 run, 1 NOT MEASURED
(`check:dual-build-cjs-loads` exit 3, PREREQUISITE NOT MET: it needs
every package's `dist/`), 0 unrun.
- `check-plugin-teardown-shape.mjs --self-test` first answered exit 3 on
the shallow clone; after `git fetch --depth=1 origin 621a487…` it
answered exit 0 (48 cases).
- Every CI-run family is read from this PR's check-runs.
</details>
<details><summary>The rendered structure block, order_to_cash</summary>
```text
Structure: besides the objects and fields above, the build is checked for these, by the exact names shown:
- Lookups, each referencing the object shown: `demoorder_employee.manager` → `demoorder_employee`, `demoorder_account.owner` → `demoorder_employee`, `demoorder_sales_order.account` → `demoorder_account`, `demoorder_sales_order.owner` → `demoorder_employee`, `demoorder_sales_order.approver` → `demoorder_employee`, `demoorder_order_line.order` → `demoorder_sales_order`, `demoorder_order_line.product` → `demoorder_product`, `demoorder_invoice.order` → `demoorder_sales_order`, `demoorder_invoice.account` → `demoorder_account`, `demoorder_payment.invoice` → `demoorder_invoice`, `demoorder_activity.account` → `demoorder_account`, `demoorder_activity.invoice` → `demoorder_invoice`, `demoorder_activity.owner` → `demoorder_employee`.
- Single-choice (select) fields, each offering at least the number of options shown: `demoorder_employee.role` (4), `demoorder_employee.team` (3), `demoorder_account.payment_terms` (2), `demoorder_product.category` (3), `demoorder_sales_order.status` (5), `demoorder_invoice.status` (4), `demoorder_invoice.aging_bucket` (5), `demoorder_payment.method` (3), `demoorder_activity.type` (4).
- Every other single-choice field you declare offers at least two options: a field with one option is no choice.
- Dashboards, each a dashboard named exactly: `demoorder_receivables_aging_report`. A report the description asks for may be shown on one; it does not stand in for it.
- The app's navigation has an entry for each of: `demoorder_account`, `demoorder_sales_order`, `demoorder_invoice`, `demoorder_payment`, `demoorder_activity`, `demoorder_receivables_aging_report`.
---ITERATE---
Add credit notes for returns and allowances: a credit note is issued against an invoice and reduces its balance.
The change is checked for these, by the exact names shown:
- New objects, named exactly: `demoorder_credit_note`, each with at least 4 fields.
- Lookups, each referencing the object shown: `demoorder_credit_note.invoice` → `demoorder_invoice`, `demoorder_credit_note.account` → `demoorder_account`.
- Every single-choice field you declare offers at least two options: a field with one option is no choice.
```
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(untouched)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes (scoped to the touched
paths; the repo-wide run is CI's).
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset: this one does not
publish (`@objectstack/scenarios` is `private`), `skip-changeset`.
- [x] 6. `packages/spec` touched: no.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the 维护者速读 block is the
lane's Chinese summary).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01FEkQ5n81cbmj4ZjYYxQQAs)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent bd81a7a commit de3996b
11 files changed
Lines changed: 255 additions & 25 deletions
File tree
- docs/qa/scenarios
- packages/qa/scenarios
- src/runner
- test
- runner
- score
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
210 | 210 | | |
211 | 211 | | |
212 | 212 | | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
213 | 234 | | |
214 | 235 | | |
215 | 236 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
16 | | - | |
| 15 | + | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
63 | 62 | | |
64 | 63 | | |
65 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
0 commit comments