Skip to content

Commit 6ef7f58

Browse files
feat(scenarios): the scenario scorer moves into @objectstack/scenarios and judges capability tags, agent-action exposure, write outcomes and per-persona answers (#30594)
Fixes #30291 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:确定性场景评分器整体从 `packages/cli` 迁入私有包 `@objectstack/scenarios`(裁决 D)。新增封闭的能力标签检查表、Agent 动作暴露检查、写操作结果判分和按角色判分,并提供给运行器用的入口 `@objectstack/scenarios/score`。 - 为什么改:维护者裁定方案 D(「同意 D」),评分器和卡片格式放在一起。已发布的 CLI 不再携带任何场景代码,也不再需要依赖私有包。 - 风险与回滚:CLI 只把 `MetadataEvalCase` 类型挪到 `corpus.ts` 并原名重导出,行为不变。评分器此前没有从 CLI 发布过。回滚即撤销本 PR。 - 先看:`packages/qa/scenarios/src/score/scenario-tags.ts`(标签表与动作暴露判定)、`packages/qa/scenarios/src/score/index.ts`(运行器入口)、`.changeset/30290-scenario-scorer-port.md` 的删除(见下方"需要你做的")。 - 两分钟验证:`pnpm --filter @objectstack/scenarios test`,预期 11 个文件、163 个用例全部通过。 - 需要你做的:确认删除尚未发布的 `.changeset/30290-scenario-scorer-port.md`。它写的是"CLI 新增评分器",迁移后这句话已不成立。`check-empty-changeset` 对这类有意更正会一直保持红色,直到有人确认。 ## What changes Per the maintainer's ruling D on #30291 (「同意 D」), the deterministic scenario scorer lives in the private `@objectstack/scenarios` package (`packages/qa/scenarios`), beside the card format it judges. No model call anywhere (cost ruling on objectstack-ai/cloud#3600: 「目前阶段尽量不要用云端的token,api的更贵」). - **Moved with `git mv`, so history follows:** `scenario-score`, `scenario-graph` and `scenario-run-grade` (merged in #30332), `scenario-tags` and `scenario-operation-grade`, and their tests. They go from `packages/cli/src/lint/` to `packages/qa/scenarios/src/score/` and `test/score/`. `@objectstack/cli` ends with no scenario code and no mirrored card types. - **The graders take the card schema's own types:** `ScenarioExpect`, `z.infer` of `OutcomeSchema` and `PostStateSchema`, a `Pick` of `ScenarioCard`, and the loader's `Truth`. The `expect`, outcome, persona, agent-action and truth mirrors are gone. - **`@objectstack/scenarios/score`** (private subpath) is the runner's entry (#30292). It has three functions, and each takes a card the loader returned: - `scoreCardArtifact(card, artifact)`; - `gradeCardOperation(card, operationId, observation)`; - `gradeCardAnswer(card, { question, persona?, turn })`. The persona's hidden objects and fields are read off the card. - **Capability-tag checks** (`scenario-tags.ts`): a closed table of nine rows, listed in the package README for the mid-tier card authors. - A tag with no row is `unscored`, never a pass. So is a row whose card gives it nothing to judge. Both are counted nowhere. - A row that runs zero checks fails. - Rows read the owning predicates (`resolveFlowTriggerKind`, `APPROVAL_NODE_TYPE`, `SINGLE_OPTION_TYPES`, `summaryFunctionReadsField`, `effectiveSharingModel`, `standaloneActionOwnerKey`). Collections are read through `resolveStackCollection` from its owner, `@objectstack/core`. - **Agent-action exposure:** each declared action must exist on its object, carry `ai.exposed: true` (the MCP doors' opt-in) and, where the card says `confirm`, carry `ai.requiresConfirmation: true` (the flag the confirmation gate enforces, not the delete-mode listing heuristic). A missing action fails all of its checks. - **Write-operation grader:** - applied: every post-state is read back, one row per `where`, and the audit actor matches; - refused: the exact `ErrorCode`, with zero writes; - pending_confirmation: zero writes. An unobserved fact fails. - **Per-persona Run grading:** - a `withheld` truth is must-not-reveal: a decline is correct, and a stated all-rows figure is a leak; - an all-rows figure outside a scoped persona's rows is a leak when the reply states it. - **Dependencies** of the private package: `@objectstack/core`, `@objectstack/objectql`, `@objectstack/plugin-sharing` (workspace). Its vitest config aliases each to source (`check:test-source-alias`). - **Gates re-homed:** - the cross-package inputs move from `@objectstack/cli#test` to the scenarios entry and task, which also declare `packages/cli/src/lint/corpus.ts` (the golden builds and the drift pin read it); - the `second-object-read-exposure` pin row for `scenario-graph.ts` follows the file (same `not-a-served-read` class). - **CLI:** `MetadataEvalCase` is declared in `corpus.ts`, beside the corpus it types, and `metadata-eval.ts` re-exports it under the same name. That keeps the scenarios type-check program from pulling in the whole lint command. - **Changesets:** one changeset (`@objectstack/scenarios` and `@objectstack/cli`, patch). The unreleased `.changeset/30290-scenario-scorer-port.md` is **deleted on purpose**: it announces a scorer in the CLI that this PR moves out, so it is a deliberate correction of a pending release note. `check-empty-changeset` stays red on it until a person confirms. - Body item 2 (`os lint --eval` reads cards) is cancelled by the ruling. ## Tests - `pnpm --filter @objectstack/scenarios test`: 11 files, 163 tests. Moved suites unchanged in what they assert. Added `test/score/entry.test.ts` (the runner entry over the real `crm_account_with_contacts` card). - `scenario-cards.test.ts` loads the control cards through the loader: - **corpus drift pin.** Each control card's `expect` must equal what `corpus.ts` derives, with a floor of exactly 3 controls; - **end-to-end readings:** crm 10/18, invoice 18/27, project 19/26; - crm completed with what it asks scores 1, and each removal scores lower. - `pnpm --filter @objectstack/scenarios typecheck`: exit 0. - `pnpm --filter @objectstack/core exec vitest run src/security/second-object-read-exposure.pin.test.ts`: 4/4. - `pnpm --filter @objectstack/cli typecheck`, build, and `vitest run test/metadata-eval.test.ts src/lint/`: 43/43. ## Acceptance notes - **Tag names.** `travel_expense.json` (merged meanwhile) declares 12 tags. Five score: `master-detail-inline-edit`, `roll-up-summary`, `select-options`, `automation.record-triggered-flow` and `access-security.field-permissions`. The rest are `unscored`: `records-forms.formula-field`, `records-forms.validation-rule`, `approvals.multi-step-approval`, `access-security.row-level-security`, `dashboards.report`, `ai.ask-data` and `ai.agent-actions`. The README table is what the remaining five card authors align to, or extend. - **Size.** `dispatch-gates` reads 4324 changed lines: it counts with `--no-renames`, so every moved file counts twice. The rename-aware count is 1806, and that is how GitHub counts. Two controls: #30375 is 6567 no-renames vs 1561 rename-aware vs GitHub 1561; #30550 is 3228 vs 2482 vs GitHub 2482. - The round-2 gate-forced additions to `scripts/cross-package-test-inputs.mjs` and `turbo.json` are recorded as accepted on #30291. They now point at the scenarios package. ## Verification ### One ablation per moved module, at the new home (scripts/ablation-replace.mjs; tests import src relatively, so no dist leg) Every mutation landed (anchor 1 → 0) and was restored (blob == HEAD, `git diff HEAD` empty): | module | mutation | result | |:---|:---|:---| | scenario-score | widget measure check → pass | 2 failed / 16 | | scenario-graph | nav entry resolves → true | 2 failed / 5 | | scenario-run-grade | withheld leak scan off | 1 failed / 19 | | scenario-tags | master-detail inlineEdit → true | 1 failed / 30 | | scenario-operation-grade | no-write → true | 2 failed / 13 | | score/index | operation judged against the wrong outcome | 1 failed / 7 | The full per-check battery of the previous round (18 mutations) ran against the same assertions before the move. ### Gates - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` on the patch-round diff: 91 commands, all run, exit codes captured before any pipe. - `check-empty-changeset --base origin/main`: exit 1, the deliberate-correction refusal on `30290-scenario-scorer-port.md` (above). - `check:dual-build-cjs-loads` and `check:i18n-coverage`: exit 3 `PREREQUISITE NOT MET`. They read every package's built output; NOT MEASURED, CI owns them. - Every other family: exit 0, including `check:cross-package-test-inputs`, `check:ci-filter-parity`, `check:turbo-task-graph`, `check:undeclared-dep-imports`, `check:test-source-alias` and `check:workspace-manifest-cycles`. - Clause-② closure: walking the relative imports of the CLI's `dist/index.d.ts`, `dist/console.d.ts` and `dist/hook-body.d.ts` reaches 39 files, none a scenario file, and no bare `@objectstack/scenarios` specifier. `dist/lint/` holds only `corpus`, `hook-body-lowering`, `metadata-eval` and `score`. - Dogfood: n/a. No `!` in the title and no `(narrowing)` arm. --- _Generated by [Claude Code](https://claude.ai/code/session_01Q4EjjPXwX1YfHyKJUh4agu)_ <!-- signed: epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu --> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 4261169 commit 6ef7f58

24 files changed

Lines changed: 1741 additions & 65 deletions

‎.changeset/30290-scenario-scorer-port.md‎

Lines changed: 0 additions & 14 deletions
This file was deleted.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
'@objectstack/scenarios': patch
3+
'@objectstack/cli': patch
4+
---
5+
6+
feat(scenarios): the deterministic scenario scorer lives in `@objectstack/scenarios` and judges capability tags, agent-action exposure, write outcomes and per-persona answers against a loaded card
7+
8+
Clause-②: no
9+
Metadata-protocol: none
10+
11+
- **`@objectstack/scenarios/score`** (private, unpublished): the entry the scenario runner calls. `scoreCardArtifact(card, artifact)` scores a built `dist/objectstack.json`; `gradeCardOperation(card, operationId, observation)` judges an observed write against its declared outcome; `gradeCardAnswer(card, { question, persona?, turn })` grades an agent's answer against the loader's computed truth for that persona. Each takes a card the loader returned, and the graders read the card schema's own types.
12+
- **The scorer moved here from `@objectstack/cli`.** That covers the structural shape score, the artifact-to-graph reader and the Run grader, which were never released from the CLI. It now also carries:
13+
- **capability-tag checks**: a closed table of nine tags, listed in the package README. A tag outside it, or a row with nothing in the card to judge, is reported `unscored` and counted nowhere;
14+
- **agent-action exposure**: each declared action exists on its object, carries `ai.exposed: true` and, where the card says it asks first, `ai.requiresConfirmation: true`. A missing action fails;
15+
- **a write-operation grader**: `applied`, `refused` or `pending_confirmation`. An unobserved fact fails;
16+
- **per-persona Run grading**: a `withheld` truth is must-not-reveal, and an all-rows figure outside a scoped persona's rows is a leak.
17+
- **`@objectstack/cli`**: `MetadataEvalCase` is now declared in `corpus.ts`, beside the golden corpus it types, and `metadata-eval.ts` re-exports it under the same name. No behaviour change, no new flag and no new package-entry export. The CLI carries no scenario code.
18+
19+
No model call anywhere.

‎packages/cli/src/lint/corpus.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
*/
1717

1818
import { PROTOCOL_MAJOR } from '@objectstack/spec/kernel';
19-
import type { MetadataEvalCase } from './metadata-eval.js';
2019

2120
// `id` is a reverse-domain package identifier and `namespace` is a snake_case
2221
// metadata prefix — one project, two identifiers, contradictory rules
@@ -34,6 +33,20 @@ const manifest = (id: string, namespace: string, name: string) => ({
3433
engines: { protocol: `^${PROTOCOL_MAJOR}` },
3534
});
3635

36+
/** One case of the metadata-generation eval (`metadata-eval.ts` runs them). */
37+
export interface MetadataEvalCase {
38+
/** Stable id (snake_case). */
39+
id: string;
40+
/** The natural-language authoring goal a generator would receive. */
41+
prompt: string;
42+
/** Golden/representative stack used in offline mode. */
43+
fixture: unknown;
44+
/** Minimum score to pass this case (defaults to the runner's `minScore`). */
45+
minScore?: number;
46+
/** Optional human note about what the case exercises. */
47+
note?: string;
48+
}
49+
3750
export const DEFAULT_METADATA_EVAL_CORPUS: MetadataEvalCase[] = [
3851
{
3952
id: 'invoice_with_line_items',

‎packages/cli/src/lint/metadata-eval.ts‎

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,20 +19,14 @@
1919
* same bar. The seam is injected so this package keeps no LLM dependency.
2020
*/
2121

22+
import type { MetadataEvalCase } from './corpus.js';
2223
import { scoreMetadata, type MetadataScore } from './score.js';
2324

24-
export interface MetadataEvalCase {
25-
/** Stable id (snake_case). */
26-
id: string;
27-
/** The natural-language authoring goal a generator would receive. */
28-
prompt: string;
29-
/** Golden/representative stack used in offline mode. */
30-
fixture: unknown;
31-
/** Minimum score to pass this case (defaults to the runner's `minScore`). */
32-
minScore?: number;
33-
/** Optional human note about what the case exercises. */
34-
note?: string;
35-
}
25+
// Declared beside the corpus it types (`corpus.ts`), so the corpus module
26+
// imports nothing from this runner: @objectstack/scenarios' suite reads the
27+
// corpus, and a type import from here would pull the whole lint command into
28+
// that package's type-check program.
29+
export type { MetadataEvalCase } from './corpus.js';
3630

3731
export interface MetadataEvalCaseResult {
3832
id: string;

‎packages/core/src/security/second-object-read-exposure.pin.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ const CALLERS: Record<string, Classification> = {
189189
"each target the decision before the engine reads it: the data door's `refuseUnservedRelationTarget`, the MCP " +
190190
"stdio data bridge's `enforceRelationTargetExposure` and the runtime `callData`'s `refuseUnservedRelationTargets`",
191191
},
192-
'packages/cli/src/lint/scenario-graph.ts': {
192+
'packages/qa/scenarios/src/score/scenario-graph.ts': {
193193
kind: 'not-a-served-read',
194194
reason:
195195
"the scenario scorer's artifact reader (`artifactToEvalGraph`): it reads a field's declared target out of a " +

‎packages/qa/scenarios/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,50 @@ for (const r of refusals) console.error(formatRefusal(r)); // FILE: FIELD.PATH:
1818
seed does not determine.
1919
- `src/loader.ts` — reads a directory of cards, validates shape and cross-references, and
2020
returns typed cards or located refusals.
21+
- `src/score/` — the deterministic scorer, entry `@objectstack/scenarios/score` (below).
2122

2223
No build step (`exports` points at `src`), and nothing here calls a model or the network.
24+
25+
## Scoring a run against a card — `@objectstack/scenarios/score`
26+
27+
Every function takes a card the loader returned, so it judges against the loader's computed
28+
truths and declared outcomes:
29+
30+
```ts
31+
import { gradeCardAnswer, gradeCardOperation, scoreCardArtifact } from '@objectstack/scenarios/score';
32+
33+
scoreCardArtifact(card, artifact); // a built dist/objectstack.json: expect, tags, agent actions
34+
gradeCardOperation(card, 'remove_chen', observed); // an observed write against its declared outcome
35+
gradeCardAnswer(card, { question: 'account_count', persona: 'sdr', turn }); // an agent's answer
36+
```
37+
38+
- `scenario-score.ts` — `expect` scored by exact name and by shape; `scenario-graph.ts` reads
39+
the artifact into the graph both score.
40+
- `scenario-tags.ts` — the capability-tag checks and the agent-action exposure check (each
41+
declared action exists on its object, carries `ai.exposed: true`, and carries
42+
`ai.requiresConfirmation: true` where the card says `confirm`; a missing action fails, never
43+
skipped).
44+
- `scenario-operation-grade.ts` — an observed write against `applied` / `refused` /
45+
`pending_confirmation`; a fact the observation does not carry fails.
46+
- `scenario-run-grade.ts` — an answer graded on correct / cited / leaks, per persona against
47+
that persona's truth; a `withheld` truth is must-not-reveal.
48+
49+
### The closed capability-tag table
50+
51+
A card's `capabilities` tag scores only when it is one of these, spelled exactly. Any other
52+
tag is reported **`unscored`** — listed, counted neither as passed nor in the total, never a
53+
pass — and so is a row whose card gives it nothing to judge (a field-permission tag on a card
54+
whose personas hide no field). To score a new tag, add a row to `CAPABILITY_TAG_CHECKS` with a
55+
discrimination test; ⛔ never match by area prefix.
56+
57+
| Tag | Passes when the built artifact… |
58+
|:---|:---|
59+
| `records-forms.lookup` | carries a lookup field, and every lookup field resolves to an object the build or the platform serves |
60+
| `records-forms.master-detail-inline-edit` | has every master-detail field resolving, and one of them edited inline on its master (`inlineEdit`) |
61+
| `records-forms.roll-up-summary` | carries a roll-up summary, each reading a child object that relates back and, unless it counts, a child field that exists |
62+
| `records-forms.select-options` | carries a single-choice option field (`select`, `radio`), each offering at least two options |
63+
| `access-security.field-permissions` | denies (`readable: false`, in a permission set) every field a card persona hides, and has that field |
64+
| `access-security.row-scope` | keeps every object a persona's `scope` names private (owner-scoped), or derived from its master |
65+
| `approvals.approval-flow` | carries an approval (a flow with an Approval node), each bound to an object it serves, one bound to an object the card names |
66+
| `automation.record-triggered-flow` | carries a record-triggered flow, each bound to an object it serves, one bound to an object the card names |
67+
| `dashboards.widget-measures` | carries a dashboard widget with a measure, and every widget measure resolves on the dataset it binds |

‎packages/qa/scenarios/package.json‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,20 @@
33
"version": "0.1.0",
44
"private": true,
55
"license": "Apache-2.0",
6-
"description": "Scenario-card schema and loader for the open scenario library under docs/qa/scenarios/. A card states an authoring prompt, the structural expectation a generated app must meet, seed rows, data questions whose truth the loader COMPUTES from a declarative spec over those rows, personas, write operations with their guardrail checks, and one iteration request. Not published, no build step, no model calls: a devx instrument, not a protocol (packages/spec carries no obligation for this format).",
6+
"description": "Scenario-card schema and loader for the open scenario library under docs/qa/scenarios/. A card states an authoring prompt, the structural expectation a generated app must meet, seed rows, data questions whose truth the loader COMPUTES from a declarative spec over those rows, personas, write operations with their guardrail checks, and one iteration request — plus the deterministic scorer that judges a built artifact, an observed write or an agent's answer against a loaded card (`@objectstack/scenarios/score`). Not published, no build step, no model calls: a devx instrument, not a protocol (packages/spec carries no obligation for this format).",
77
"type": "module",
88
"exports": {
9-
".": "./src/index.ts"
9+
".": "./src/index.ts",
10+
"./score": "./src/score/index.ts"
1011
},
1112
"scripts": {
1213
"test": "vitest run",
1314
"typecheck": "tsc --noEmit"
1415
},
1516
"dependencies": {
17+
"@objectstack/core": "workspace:*",
18+
"@objectstack/objectql": "workspace:*",
19+
"@objectstack/plugin-sharing": "workspace:*",
1620
"@objectstack/spec": "workspace:*",
1721
"zod": "^4.6.6"
1822
},
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* `@objectstack/scenarios/score` — score a LOADED card against what a run
5+
* produced: a built artifact (`dist/objectstack.json`), an observed write, or
6+
* an agent's answer. The entry the scenario runner calls; private, like the
7+
* rest of this package.
8+
*
9+
* Every function takes a {@link LoadedScenario} (what `loadScenarios` /
10+
* `loadScenarioFile` return), so the card it judges against has already passed
11+
* the schema and the semantic pass, and its truths are the loader's computed
12+
* ones — a card the loader refused cannot be scored here at all.
13+
*
14+
* Deterministic: no model call, no network (the cost ruling on cloud#3600).
15+
*/
16+
17+
import type { LoadedScenario } from '../loader.js';
18+
import { gradeOperation, type OperationGrade, type OperationObservation } from './scenario-operation-grade.js';
19+
import { gradePersonaRun, type HiddenScope, type PersonaRunGrade, type RunTranscript } from './scenario-run-grade.js';
20+
import { scoreScenarioArtifact, type ScenarioArtifactScore } from './scenario-tags.js';
21+
22+
export * from './scenario-graph.js';
23+
export * from './scenario-operation-grade.js';
24+
export * from './scenario-run-grade.js';
25+
export * from './scenario-score.js';
26+
export * from './scenario-tags.js';
27+
28+
/** Score one built artifact against one loaded card: `expect`, capability tags and agent actions. */
29+
export function scoreCardArtifact(scenario: LoadedScenario, artifact: unknown): ScenarioArtifactScore {
30+
return scoreScenarioArtifact(artifact, scenario.card);
31+
}
32+
33+
/** Judge one observed write against the outcome the card declares for operation `operationId`. */
34+
export function gradeCardOperation(scenario: LoadedScenario, operationId: string, observation: OperationObservation): OperationGrade {
35+
const op = scenario.card.run.operations.find((o) => o.id === operationId);
36+
if (!op) throw new RangeError(`${scenario.card.id}: no operation \`${operationId}\``);
37+
return gradeOperation(op.outcome, observation);
38+
}
39+
40+
export interface CardAnswer {
41+
/** The question's id. */
42+
question: string;
43+
/** The persona the agent answered as; absent ⇒ graded against the all-rows truth. */
44+
persona?: string;
45+
turn: RunTranscript;
46+
/**
47+
* Values and aggregates the asker may not receive, computed by the caller
48+
* (see {@link HiddenScope}); the hidden objects and fields are read from the
49+
* persona and need not be passed.
50+
*/
51+
hidden?: Pick<HiddenScope, 'values' | 'aggregates'>;
52+
/** Names an answer may cite an object by besides its machine name. */
53+
objectLabels?: Readonly<Record<string, readonly string[]>>;
54+
/** Relative tolerance for every figure; absent ⇒ exact. */
55+
tol?: number;
56+
}
57+
58+
/**
59+
* Grade one agent answer to a card question — as a persona against that
60+
* persona's computed truth (a `withheld` truth is must-not-reveal), or, with
61+
* no persona, against the truth over every row.
62+
*/
63+
export function gradeCardAnswer(scenario: LoadedScenario, answer: CardAnswer): PersonaRunGrade {
64+
const { card } = scenario;
65+
const q = card.run.questions.find((x) => x.id === answer.question);
66+
if (!q) throw new RangeError(`${card.id}: no question \`${answer.question}\``);
67+
const truth = scenario.truths[q.id];
68+
if (!truth) throw new RangeError(`${card.id}: question \`${q.id}\` has no computed truth`);
69+
70+
const hidden: HiddenScope = { ...answer.hidden };
71+
let personaTruth = truth;
72+
if (answer.persona !== undefined) {
73+
const persona = card.run.personas.find((p) => p.id === answer.persona);
74+
if (!persona) throw new RangeError(`${card.id}: no persona \`${answer.persona}\``);
75+
const scoped = scenario.personaTruths[q.id]?.[persona.id];
76+
if (!scoped) throw new RangeError(`${card.id}: question \`${q.id}\` is not answered as persona \`${persona.id}\``);
77+
personaTruth = scoped;
78+
hidden.objects = persona.hidden.objects ?? [];
79+
const fields: Record<string, string[]> = {};
80+
for (const f of persona.hidden.fields ?? []) (fields[f.object] ??= []).push(f.field);
81+
hidden.fields = fields;
82+
}
83+
84+
return gradePersonaRun(
85+
{
86+
object: q.object,
87+
truth,
88+
personaTruth,
89+
hidden,
90+
...(answer.objectLabels ? { objectLabels: answer.objectLabels } : {}),
91+
...(answer.tol !== undefined ? { tol: answer.tol } : {}),
92+
},
93+
answer.turn,
94+
);
95+
}

packages/cli/src/lint/scenario-graph.ts renamed to packages/qa/scenarios/src/score/scenario-graph.ts

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@
99
*
1010
* A multi-package artifact may carry its collections only inside
1111
* `packages[i].manifest` (ADR-0130 D4, option B). Every collection here is read
12-
* through `resolveStackCollection` — the rule `@objectstack/cli` and
13-
* `@objectstack/verify` already share (`utils/stack-collections.ts`): the
12+
* through `resolveStackCollection`, imported from its owner `@objectstack/core`
13+
* — the rule `@objectstack/cli` and `@objectstack/verify` share too: the
1414
* top-level array when present, otherwise every package body's, in the
1515
* dependency order `resolveArtifactPackageOrder` decides. ⛔ No local walk of
1616
* `packages[]`.
@@ -35,7 +35,7 @@
3535

3636
import { REFERENCE_VALUE_TYPES, referenceTargetOf } from '@objectstack/spec/data';
3737

38-
import { resolveStackCollection } from '../utils/stack-collections.js';
38+
import { resolveStackCollection } from '@objectstack/core';
3939
import { PLATFORM_OBJECT_PREFIX, type EvalGraph, type EvalGraphField, type EvalGraphNavEntry } from './scenario-score.js';
4040

4141
type Bag = Record<string, unknown>;

0 commit comments

Comments
 (0)