Skip to content

Commit 465c2be

Browse files
feat(scenarios): score the seven capability tags the mid-tier cards declare and the closed tag table left unscored (#31098)
Fixes #30984 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:场景评分器的封闭能力标签表新增 7 行(公式字段、校验规则、多级审批、动态审批人、行级安全、报表、AI 可调用动作),六张中阶卡声明的这 7 个标签不再显示 `unscored`。 - 为什么改:中阶卡正是为考这些能力而写,原先 Build 半场完全不检查它们,得分说明不了 app 是否真有卡片要考的能力。 - 风险与回滚:只影响私有 devx 包 `@objectstack/scenarios` 的评分,不发布;新增声明依赖 `@objectstack/formula`(本就在 full 评分入口的依赖闭包内);回滚即 revert 本 PR。 - 先看:`packages/qa/scenarios/src/score/scenario-tags.ts`(7 行新规则,风险最高的是 `rowSecurityGap` 与 `approvalSequence`)、`src/score/scenario-expressions.ts`(「能编译」一律问平台自己的校验器)、`test/score/scenario-tags.test.ts`。 - 两分钟验证:`pnpm --filter @objectstack/scenarios test`,预期 23 个文件全绿;其中 `the library` 一组证明六张中阶卡上这 7 个标签都被评分(空 build 上为 `fail`,不是 `unscored`)。 - 需要你做的:无,席位落地。 ## What changes Seven rows join `CAPABILITY_TAG_CHECKS`, one per tag the mid-tier cards declare and the closed table left unscored. Each row reads the built artifact through the platform's own schemas and helpers, and each carries the table's discrimination test: the rich build passes it, and one defect per stated reason fails exactly that tag. No area prefix is matched anywhere. | Tag | Checks | Read through | |:--|:--|:--| | `records-forms.formula-field` | `computed-field-exists`; per computed field `formula:OBJECT.FIELD->compiles` | `validateExpression('value', …)` at `record` scope over the object's declared and injected columns (`resolveInjectedSystemColumns`) and declared field types, `traversalHydration` on: the validator `os build` runs on a formula | | `records-forms.validation-rule` | per rule with a predicate `validation:OBJECT.RULE->compiles` (`condition`, a conditional rule's `when`, nested `then` / `otherwise` rules); `validation-rule-binds-expected-object` (an active, `severity: 'error'` rule on an object the card names) | `validateExpression('predicate', …)`, same hint | | `approvals.multi-step-approval` | `multi-step-approval-binds-expected-object`: a flow bound to a card-named object the build carries, where a second Approval node follows the first on its `approve` branch | `APPROVAL_NODE_TYPE`, `APPROVAL_BRANCH_LABELS`; the walk mirrors the engine's `traverseNext` (approve-labelled out-edges, or every out-edge when none is labelled; never `fault`) | | `approvals.dynamic-approver-routing` | per record-routed approver `approver:FLOW.NODE[i]->routes`; `dynamic-approver-binds-expected-object` | `APPROVER_VALUE_BINDINGS` sources `trigger-field` / `auto` / `expression` are record-routed; a `field` (or a `manager`'s subject: its `value` field, else `owner_id`, as plugin-approvals resolves it) must hold a user (`referenceTargetOf`, injected columns via `injectedSystemColumnDefs`); an `expression` must parse over `APPROVER_EXPRESSION_ROOTS` (`collectCelRootIdentifiers`) | | `access-security.row-level-security` | per scoped object `row-level-security:OBJECT->enforced`, for every persona that scopes it | row-scope's three routes, each held to enforce: private (`effectiveSharingModel`); `controlled_by_parent` only when its master (`controlledByParentMasterOf`) is itself kept so for that persona; or a read policy on the persona's set. Every applicable read policy must lower to a filter (`isSupportedRlsExpression`) | | `dashboards.report` | per report `report:NAME->resolves` (its dataset exists and declares every selected measure and dimension; a joined report per block); `report-binds-expected-object` (a report's dataset `object` is card-named) | `ReportSchema` `dataset` / `values` / `rows` / `columns` / `blocks`, `DatasetSchema` `object` / `measures` / `dimensions` (ADR-0021) | | `ai.agent-actions` | the checks `gradeAgentActions` runs for `run.agentActions` (exists, `ai.exposed`, `ai.requiresConfirmation` where `confirm`) | unchanged reader, so the tag and the agent-action half judge an action by one rule | Where measurement sharpened the card's suggested checks (the suggestions were not rulings): - **Formula field accepts a stored computed field.** The spec's `expression` on a non-`formula` field is computed and stored "so a `summary` can roll the column up"; a `formula` field is virtual. The mid-tier cards sum computed line amounts in roll-ups, so a faithful build writes the stored kind, and a `type: 'formula'`-only check would fail it. - **"Compiles" is the shared validator's verdict:** CEL syntax, `record.FIELD` column existence and the one-hop traversal rule. The build's further site rules over the same source (null guards, type soundness under the site's fault outcome, a hop into a field the target lacks, `label()` over an optionless field) are not re-run; `scenario-expressions.ts` names that gap in its header. - **Validation rule:** an advisory severity lets the write through, so the binding check counts an active `severity: 'error'` rule; a rule with no predicate (state machine, format, JSON schema) counts toward the binding and has nothing to compile. - **Row-level security keeps row-scope's routes, as the card asks (#30883 item 4 as corrected).** The platform's own checklist item titled "Row-level security" (`access-security.rls-both-sides`) is an OWD-private object, so a private route is RLS in this platform's vocabulary. What the row adds over row-scope is enforcement: a read policy whose `using` does not lower matches no row (the spec's own `using` description), and a scoped detail whose master the persona reads in full keeps nothing to it. Both are pinned as cases where row-scope passes and this row fails. - **Dynamic routing counts `manager`.** plugin-approvals resolves it from the record (`record[value] ?? record.owner_id`), so it is read off the request; every directory or catalog binding (user, position, team, department, queue, membership tier) is the same slate for every request and does not count. - **Report means a `report` item.** A report binds a dataset, never an object, so the binding is judged through the dataset's `object`. A dashboard widget is the sibling tag `dashboards.widget-measures`. Outside the claim's named surface, each the minimum the change needs: - `packages/qa/scenarios/package.json` and `pnpm-lock.yaml` (+3 lines): `@objectstack/formula` declared, because the expression verdicts are its functions. It was already in the `./score/full` runtime graph (objectql and plugin-sharing depend on it); the narrow `./score` entry does not reach it (`test/build/narrow-score-graph.test.ts` stays green). - `packages/qa/scenarios/vitest.config.ts`: one anchored alias so the tests judge the formula source in this checkout (`pnpm check:test-source-alias` went red without it). - `packages/qa/scenarios/README.md`: the closed-table section lists the seven rows (a different hunk from the tier lines #31091 landed). No changeset: `@objectstack/scenarios` is `private: true` and nothing in this diff ships from a released package, so the PR carries `skip-changeset`. ## Tests `test/score/scenario-tags.test.ts`: - The rich build gains a `formula` field, a script validation rule, a two-step approval (a `field` approver over a `user` field, then a position), a dataset dimension and a report. `passes every row of the table` now covers all 16 rows at score 1. - `TAG_DEGRADATIONS` adds 19 one-defect cases for the seven rows; `ALSO_FAILS` names the five defects that break a second row by construction (the approval flow's three binding defects take the steps and the routed approver along; a public scoped object fails both row tags). - Per-row suites: a stored computed field counts; two hops refused, one accepted; a `formula` field with no expression fails; a conditional rule's nested predicate is judged; a state machine binds with nothing to compile; a decision between steps is followed, an unlabelled approve edge is walked as the engine walks it, alternative branches are not steps; `manager` with and without a `value`, `expression` over `current` passes and over `record` or unparseable fails, seven fixed approver kinds count nothing; RLS fails where row-scope passes (a non-lowering policy, on a public and a private object; a scoped detail whose master is read in full); a joined report draws per block; a report with no dataset fails; the agent-actions tag carries exactly the agent-action checks. - `the library`: loads `docs/qa/scenarios/` through the loader; each of the seven has a row and is declared by a card; on every card that declares one, an empty build grades it `fail`, never `unscored`. <details><summary>Package suite and typecheck at 37d8ec1</summary> ``` pnpm --filter @objectstack/scenarios build check-dts-emitted: 10/10 declared declaration file(s) present pnpm --filter @objectstack/scenarios test Test Files 23 passed (23) Tests 402 passed (402) pnpm --filter @objectstack/scenarios typecheck tsc --noEmit exit 0 (--listFiles includes test/score/scenario-tags.test.ts) eslint --no-inline-config on the four changed .ts files exit 0 ``` </details> <details><summary>Ablations (one per tested behaviour, via scripts/ablation-replace.mjs; tests import src, so no build leg)</summary> | Mutation | Red | Restore | |:--|:--|:--| | `rowSecurityGap`: the non-lowering policy refusal disabled | 1: RLS fails where row-scope passes | blob == HEAD, `git diff HEAD` empty | | `approvalSequence`: the approve-label filter dropped (every out-edge walked) | 1: second approval on the reject branch | blob == HEAD | | `recordExpressionRefusal`: the column list removed from the hint | 3: formula / rule over a missing field, nested conditional predicate | blob == HEAD | | `approverRouteRefusal`: the holds-a-user test forced true | 3: non-user field, missing field, `manager` over a non-user field | blob == HEAD | The first attempt at the second mutation was refused by the tool (the replacement text already occurred inside the anchor, so the count could not prove the write); it ran nothing and was redone with a distinct replacement. </details> ## Acceptance notes - `ai.agent-actions` counts the agent-action checks under the tag as well, as `dashboards.widget-measures` already counts the measure checks `expect` runs; a card carrying the tag weighs a missing action twice in the Build total. - Card-side observation, not filed (read only): the five cards tagged `dashboards.report` ask for "a report of …" while `expect.dashboards` names that report as a dashboard the build prompt never names, and `leave_balances` tags the same wording `dashboards.widget-measures`. - The row-scope row does not ask whether a read policy lowers; that is now this PR's RLS row's question, and an artifact `os build` wrote cannot carry one that does not (`rls-predicate-unenforceable` refuses it). ## Verification All at head `37d8ec1c` (origin/main `b6315733` merged, #31091 included; whole tree rebuilt first: turbo 119/119, 73 cached). - `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`: 62 commands derived; all 62 run, each exit code captured before any pipe: **62 exit 0**. `--ran` with the recorded codes: "62 derived, 62 run, 0 NOT-MEASURED, 0 UNRUN" (a derived zero). - `pnpm check:test-source-alias` was red on the first run (`@objectstack/formula` resolved to `dist/` in the tests); fixed by the alias in `vitest.config.ts`, green at this head. - `pnpm check:dual-build-cjs-loads` answered PREREQUISITE NOT MET before the whole-tree build; at this head: 116 require entry points across 71 packages load. - Not run locally, CI's: the repo-wide lint scan, the Test Core / Build Core / type-check lanes, the dogfood batteries (no `!` in the title, Clause-② no). <details><summary>The 62 derived commands, each with its exit code</summary> ``` node scripts/check-closing-keyword-parity.mjs :: exit 0 node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0 node scripts/check-comment-mask-adoption.mjs :: exit 0 node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0 node scripts/check-comment-mask-corpus.mjs :: exit 0 node scripts/check-console-intercept-disarm.mjs :: exit 0 node scripts/check-console-intercept-disarm.mjs --self-test :: exit 0 node scripts/check-dts-emitted.mjs --self-test :: exit 0 node scripts/check-issue-citations.mjs :: exit 0 node scripts/check-osv-exemptions.mjs :: exit 0 node scripts/check-osv-exemptions.mjs --self-test :: exit 0 node scripts/check-plugin-teardown-shape.mjs :: exit 0 node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0 node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0 node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0 node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0 node scripts/docs-audit/check-affected-docs.mjs :: exit 0 node scripts/docs-audit/check-drift-comment.mjs :: exit 0 node scripts/osv-base-relative.mjs --self-test :: exit 0 pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0 pnpm --filter @objectstack/spec run check:llms-txt :: exit 0 pnpm check:changeset-fixed :: exit 0 pnpm check:ci-filter-parity :: exit 0 pnpm check:closed-ceremony-text :: exit 0 pnpm check:cross-package-test-inputs :: exit 0 pnpm check:doc-authoring :: exit 0 pnpm check:driver-memory-census :: exit 0 pnpm check:dts-closure :: exit 0 pnpm check:dual-build-cjs-loads :: exit 0 pnpm check:engine-double-contract :: exit 0 pnpm check:gitlink-declared :: exit 0 pnpm check:issue-citations :: exit 0 pnpm check:keyed-text-bounds :: exit 0 pnpm check:lean-entry-closure :: exit 0 pnpm check:logger-receiver-detach :: exit 0 pnpm check:manifest-repository-directory :: exit 0 pnpm check:merge-driver :: exit 0 pnpm check:nul-bytes :: exit 0 pnpm check:objectql-double-limit :: exit 0 pnpm check:org-identifier :: exit 0 pnpm check:override-consistency :: exit 0 pnpm check:page-declaration-shape :: exit 0 pnpm check:platform-object-tenancy-census :: exit 0 pnpm check:pnpm-flag-spellings :: exit 0 pnpm check:published-files :: exit 0 pnpm check:query-options-erasure :: exit 0 pnpm check:refd-timer-probe :: exit 0 pnpm check:registry-log-declared :: exit 0 pnpm check:rest-log-spy-declared :: exit 0 pnpm check:slot-lookup :: exit 0 pnpm check:sourcemap-no-sources-content :: exit 0 pnpm check:system-context-census :: exit 0 pnpm check:test-source-alias :: exit 0 pnpm check:tier-file-adoption :: exit 0 pnpm check:turbo-task-graph :: exit 0 pnpm check:type-check-coverage :: exit 0 pnpm check:type-check-debt :: exit 0 pnpm check:undeclared-dep-imports :: exit 0 pnpm check:vendor-export-contract-resolve :: exit 0 pnpm check:watch-hint-literal :: exit 0 pnpm check:where-matcher :: exit 0 pnpm check:workspace-manifest-cycles :: exit 0 ``` </details> --- _Generated by [Claude Code](https://claude.ai/code/session_012MmDsRzowAnXEmogPGSEKA)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2178d80 commit 465c2be

8 files changed

Lines changed: 840 additions & 21 deletions

File tree

‎packages/core/src/security/second-object-read-exposure.pin.test.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,13 @@ const CALLERS: Record<string, Classification> = {
196196
'built metadata artifact offline, only to score whether the reference resolves to an object the build carries; ' +
197197
'it reads no record and serves nothing of the target to any caller',
198198
},
199+
'packages/qa/scenarios/src/score/scenario-tags.ts': {
200+
kind: 'not-a-served-read',
201+
reason:
202+
"the scenario scorer's capability-tag checks: they read a field's declared target out of a built metadata " +
203+
"artifact offline (whether an approver's field holds a user, a controlled-by-parent detail's master), only to " +
204+
'judge a capability check; they read no record and serve nothing of the target to any caller',
205+
},
199206
'packages/lint/src/object-graph.ts': {
200207
kind: 'not-a-served-read',
201208
reason: 'authoring-time static analysis of metadata; reads no record',

‎packages/qa/scenarios/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,11 +76,18 @@ discrimination test; ⛔ never match by area prefix.
7676
| `records-forms.master-detail-inline-edit` | has every master-detail field resolving, and one of them edited inline on its master (`inlineEdit`) |
7777
| `records-forms.roll-up-summary` | carries a roll-up summary, each reading a child object that relates back and, unless it counts, a child field that exists |
7878
| `records-forms.select-options` | carries a single-choice option field (`select`, `radio`), each offering at least two options |
79+
| `records-forms.formula-field` | carries a computed field (a `formula` field, or a stored field with an `expression`), each expression compiling at `record` scope against its object's columns |
80+
| `records-forms.validation-rule` | carries an active, refusing (`severity: 'error'`) validation rule on an object the card names, every rule's predicates (nested ones included) compiling against its object |
7981
| `access-security.field-permissions` | denies (`readable: false`, in a permission set) every field a card persona hides, and has that field |
8082
| `access-security.row-scope` | keeps every object a persona's `scope` names private (owner-scoped) or derived from its master, or narrows it for each such persona by a read policy (`rowLevelSecurity`, `select` or `all`, no `positions`, a `using` that is not `true`) on that persona's permission set |
83+
| `access-security.row-level-security` | keeps every object a persona's `scope` names to that persona by a route that enforces: private, a detail whose master is itself kept so, or a narrowing read policy on its permission set; and every read policy that applies there lowers to a query filter |
8184
| `approvals.approval-flow` | carries an approval (a flow with an Approval node), each bound to an object it serves, one bound to an object the card names |
85+
| `approvals.multi-step-approval` | carries an approval flow, bound to an object the card names, in which a second Approval node follows the first on its `approve` branch |
86+
| `approvals.dynamic-approver-routing` | routes an Approval node of a flow bound to an object the card names to an approver read off the record (`field`, `manager`, `expression`), each one able to name someone: a field holding a user, an expression over `current` / `trigger` / `vars` |
8287
| `automation.record-triggered-flow` | carries a record-triggered flow, each bound to an object it serves, one bound to an object the card names |
8388
| `dashboards.widget-measures` | carries a dashboard widget with a measure, and every widget measure resolves on the dataset it binds |
89+
| `dashboards.report` | carries a report whose dataset binds an object the card names, every report selecting only measures and dimensions its datasets declare |
90+
| `ai.agent-actions` | carries every action in `run.agentActions` on its object, AI-exposed and confirm-gated where the card says `confirm` (the agent-action checks) |
8491

8592
## Running the cards — `@objectstack/scenarios/runner`
8693

‎packages/qa/scenarios/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@
6868
},
6969
"dependencies": {
7070
"@objectstack/core": "workspace:*",
71+
"@objectstack/formula": "workspace:*",
7172
"@objectstack/objectql": "workspace:*",
7273
"@objectstack/plugin-sharing": "workspace:*",
7374
"@objectstack/spec": "workspace:*",
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* What the platform's own expression front end (`@objectstack/formula`) says of
5+
* the expressions a built artifact carries: the reads behind the capability-tag
6+
* checks that ask whether an expression COMPILES or ENFORCES
7+
* (`scenario-tags.ts`). Each answer is the verdict of the function the platform
8+
* runs at that site, never a parser of this package's:
9+
*
10+
* - a computed field's `expression` and a validation rule's predicates:
11+
* `validateExpression`, the shared validator `os validate` / `os build` run
12+
* over every formula and predicate (`@objectstack/lint`'s
13+
* `validate-expressions.ts`), at `record` scope against the object's
14+
* columns (its declared fields and the system columns the registry injects,
15+
* `resolveInjectedSystemColumns`) and its declared field types, with a
16+
* relationship traversal judged the way the engine serves one at both sites
17+
* (`traversalHydration`: one hop through a reference, never two);
18+
* - an `expression` approver: `collectCelRootIdentifiers` over the closed root
19+
* set `APPROVER_EXPRESSION_ROOTS`, the pre-check plugin-approvals runs at
20+
* node entry and the `approval-expression-invalid` lint mirrors;
21+
* - an RLS `using`: `isSupportedRlsExpression`, the one shape gate
22+
* plugin-security compiles with. `false` there is a predicate that never
23+
* lowers to a filter, so its policy matches no row.
24+
*
25+
* What it does NOT re-run is the build's further site rules over the same
26+
* sources: the null guard, type soundness under the site's fault outcome, a
27+
* hop into a field the target object lacks, `label()` over an optionless
28+
* field. An artifact `os build` wrote has passed those; a source that parses
29+
* and reads only real columns passes here even where the build would also
30+
* have flagged one of them.
31+
*
32+
* Deterministic: no model call, no network.
33+
*/
34+
35+
import { collectCelRootIdentifiers, isSupportedRlsExpression, validateExpression } from '@objectstack/formula';
36+
import { APPROVER_EXPRESSION_ROOTS } from '@objectstack/spec/automation';
37+
import { resolveInjectedSystemColumns } from '@objectstack/spec/data';
38+
39+
type Bag = Record<string, unknown>;
40+
41+
const isBag = (value: unknown): value is Bag => !!value && typeof value === 'object' && !Array.isArray(value);
42+
43+
/** The source text of an expression slot: a bare string or the `{ dialect, source }` envelope the parse writes. */
44+
export function expressionSource(raw: unknown): string | undefined {
45+
const source = isBag(raw) ? raw.source : raw;
46+
return typeof source === 'string' && source.trim() !== '' ? source : undefined;
47+
}
48+
49+
/** The columns a `record`-scoped expression on this object can read: its declared fields and the injected ones. */
50+
function objectColumns(def: Bag): string[] {
51+
const declared = isBag(def.fields) ? Object.keys(def.fields) : [];
52+
return [...new Set([...declared, ...resolveInjectedSystemColumns(def).names])];
53+
}
54+
55+
/** Declared field name → its type, the index the build's pass hands the validator (a reference type is a hop). */
56+
function declaredFieldTypes(def: Bag): Record<string, string> {
57+
const types: Record<string, string> = {};
58+
for (const [name, field] of isBag(def.fields) ? Object.entries(def.fields) : []) {
59+
if (isBag(field) && typeof field.type === 'string') types[name] = field.type;
60+
}
61+
return types;
62+
}
63+
64+
/**
65+
* Why a `record`-scoped expression of `object` does not compile, or
66+
* `undefined` when it does: `value` for a computed field, `predicate` for a
67+
* validation rule. An absent or blank source is a refusal here, never the
68+
* validator's vacuous pass.
69+
*/
70+
export function recordExpressionRefusal(role: 'value' | 'predicate', raw: unknown, object: string, def: Bag): string | undefined {
71+
const source = expressionSource(raw);
72+
if (source === undefined) return 'declares no expression';
73+
const verdict = validateExpression(role, isBag(raw) ? (raw as { dialect?: string; source?: string }) : source, {
74+
objectName: object,
75+
fields: objectColumns(def),
76+
fieldTypes: declaredFieldTypes(def),
77+
scope: 'record',
78+
traversalHydration: true,
79+
});
80+
return verdict.ok ? undefined : (verdict.errors[0]?.message ?? 'does not compile');
81+
}
82+
83+
/** Why an `expression` approver cannot route, or `undefined`: it parses and reads only `current` / `trigger` / `vars`. */
84+
export function approverExpressionRefusal(raw: unknown): string | undefined {
85+
const source = expressionSource(raw);
86+
if (source === undefined) return 'declares no expression';
87+
const parsed = collectCelRootIdentifiers(source);
88+
if (!parsed.ok) return `does not parse as CEL: ${parsed.error}`;
89+
const roots = new Set<string>(APPROVER_EXPRESSION_ROOTS);
90+
const foreign = parsed.roots.filter((r) => !roots.has(r));
91+
return foreign.length === 0 ? undefined : `reads ${foreign.join(', ')}, and an approver expression reads only ${[...roots].join(', ')}`;
92+
}
93+
94+
/** Whether an RLS `using` lowers to a query filter, so its policy enforces (`isSupportedRlsExpression`). */
95+
export const rlsPredicateEnforces = (using: string): boolean => isSupportedRlsExpression(using);

0 commit comments

Comments
 (0)