Skip to content

Commit 802c98c

Browse files
feat(scenarios): a local scenario runner builds, runs and iterates every card with a local Claude Code session (#30746)
Fixes #30292 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:在私有包 `@objectstack/scenarios` 里新增本地场景运行器(`run-scenarios`)。每张卡跑完 Build → Run(每个角色经应用自带的 MCP 问数和执行写操作)→ Iterate,另有"模板定制"模式(已验证的构建以 unmanaged 方式安装后再改),每张卡写一份运行记录。 - 为什么改:卡片要变成可比的运行记录,而且不花云端或 API 计费的 token(「目前阶段尽量不要用云端的token,api的更贵」)。评分全部走 `@objectstack/scenarios/score`(裁决 D)。 - 风险与回滚:只动私有、不发布的包和两份 README,不碰运行时代码;回滚就是 revert 本 PR。 - 先看:`packages/qa/scenarios/src/runner/observe.ts`(写操作结果的判定顺序)、`src/runner/namespace.ts`(卡片对象名按命名空间前缀解读,这是一个需要你拍板的分叉)、`src/runner/driver.ts`(拒绝 API 计费的守卫)。 - 两分钟验证:`pnpm --filter @objectstack/scenarios exec vitest run test/runner`,应看到 29 个测试全绿,其中两个分别证明"降级的构建"和"泄露的回答"会出现在记分卡上。 - 需要你做的:看一眼报告里的命名空间分叉,选 A 或 B。默认按 A 实现(运行器把卡片对象名读成带前缀的形式)。 ## What changes A local scenario runner, `pnpm --filter @objectstack/scenarios run-scenarios`, at the private entry `@objectstack/scenarios/runner`. It runs every card under `docs/qa/scenarios/` end to end and writes one run record per card in the `docs/qa/platform-checklist/runs/` shape. Records stay out of git, per that README. - **scratch mode:** `create-objectstack` scaffolds a project. The agent builds from the card prompt plus the runner's data contract (`prompts.ts`: the seed rows, one permission set per persona named by its id, the agent actions). Then `objectstack validate`, `build`, and `verify` with proofs chosen from the tags, then the score. `objectstack dev --fresh` boots the app. The runner seeds the card's rows with their own ids and creates one user, permission-set grant and API key per persona. Every question runs as every persona it names, and every operation as its persona, over the app's `/api/v1/mcp`. Then the iterate change, a rebuild, a re-score and a re-run. - **template mode** (the ruling on the card, comment 6659969691): a blank project boots, the card's verified build is installed **unmanaged** (`POST /api/v1/marketplace/install-local`, `installMode: 'unmanaged'`), the agent applies the iterate change to the installed copy, and then come the score, the seed and the Run half. A refused install is a finding in the record. The scorecard puts the two modes side by side and adds the AI change size in metadata items. - **Driver** (`driver.ts`): pluggable. `ClaudeCodeDriver` spawns the local `claude -p --output-format stream-json`. It refuses to start when `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` is set, or the CLI reports an API-key login, and it strips both from every child it spawns. `RecordedDriver` replays a recording, and `RecordingDriver` saves one (`--record`). The record states the driver and the login. - **Observation** (`observe.ts`): the write grader gets only what the runner reads itself. That is the agent's write calls and the door's coded answer, the object's rows before and after the turn (rows added, removed or changed), and the audit trail's actor mapped back to a persona. The answer is decided in a documented order. An unattempted write gets a prose label outside the error-code catalogue, so it can never grade as the refusal a card declared. - **Namespace-relative cards** (`namespace.ts`): against a package with namespace NS, a card's `account` is `NS_account`. See the fork under Acceptance notes. - **Scorecard** (`scorecard.ts`): Build, Run-问数, Run-operations and Iterate per mode, a per-tag table, and the actual question, persona-turn and operation counts. A degraded build and a leaking answer are flagged. ## Why The card needs a runner that turns cards into run records with zero cloud-billed or API-billed tokens. Ruling D on #30291 puts the scorer and loader in this package, and the runner calls them. Nothing here re-implements scoring or card parsing. ## Tests `packages/qa/scenarios/test/runner/` holds 29 tests and needs no model and no network. - `units.test.ts`: the stream-json reader, using the shape measured from Claude Code 2.1 (pairing, the MCP prefix, error flags, a torn line); the observation rules one by one; observation feeding `gradeCardOperation` on the real crm card; the cost guard (three refusals plus env stripping); replay, including a recorded file that escapes the project being refused; the prompts (no question text and no outcome code in a build prompt); seed order; template manifest; change size; the namespace transform. - `orchestrate.test.ts`: the runner end to end on the real crm card, with a fake scaffold, an in-memory app and a scripted replay driver. Both modes appear side by side with counts 8 questions, 9 persona turns, 5 operations and 2 personas. A degraded build shows `⚠ degraded (failed: validate)` and template mode is skipped for want of a verified build. A leaking answer shows `8/9 ⚠ 1 leaking` while its figures are correct. An `UNCOPYABLE` install refusal is a finding. A namespaced project seeds `crm_account` and bare-name citations still count. <details><summary>Package suite on the final head 15a58cf</summary> `bash scripts/pm/os-verify-lock.sh -c "pnpm --filter @objectstack/scenarios test && pnpm --filter @objectstack/scenarios typecheck"`: Test Files 15 passed (15), Tests 203 passed (203), `VERDICT command-exit 0`. `eslint --no-inline-config` on the new files: exit 0. `node scripts/check-nul-bytes.mjs`: exit 0. </details> ## Acceptance notes - **Live end-to-end run of one control card: NOT MEASURED.** `claude` 2.1.318 is installed in the container and logged in with `oauth_token` (subscription, firstParty). One small live `claude -p` turn ran against a booted app's MCP with a persona key; that is where the stream-json shape was measured. The full run (`run-scenarios --only crm_account_with_contacts --record …`) was **refused by this session's auto-mode permission classifier** ("Create Unsafe Agents"): it would spawn a headless Claude Code agent with Bash and edit permissions. I did not route around the refusal. The command is ready for a maintainer's own machine. - **What WAS measured live, without a model**, against a `create-objectstack@18.3.0` project with a hand-authored crm app on `objectstack dev --fresh`, through the runner's own `HttpApp`, `runHalf`, observation and graders. A deterministic MCP stand-in replaced the model; it is not committed. - Persona provisioning is `admin/create-user`, then a forced password change, then `POST /keys`, then the `sys_user_permission_set` grant. Both personas were created with their sets assigned. Self sign-up is closed on the default `invite_only` audience. A user without an app permission set reads `PERMISSION_DENIED` on everything. - Seed: 8/8 rows inserted with the card's own ids, and ownership was mapped from the persona handles. - **`VALIDATION_FAILED` is real**: `create_record` with `email: 'gil at acme'` answers `{"error":{"code":"VALIDATION_FAILED","message":"Email must be a valid email address"}}`, and `contact_bad_email` grades PASS. **There is a coupling, though**: when the persona's hidden field is also non-editable, the security gate answers `PERMISSION_DENIED` ("Field write denied") before validation runs. A card that hides a field from a persona and also expects that persona to write it bad data depends on the build granting the write. - The five crm operations graded 4/5. `add_contact` applied, `remove_chen` was pending confirmation, `contact_bad_email` was refused with `VALIDATION_FAILED`, and `partner_edits_account` was refused with `PERMISSION_DENIED`. `attach_dee` failed: see the card finding below. - Template install: the namespaced crm build installed unmanaged with `200`, `copiedCount: 5` and **zero `UNCOPYABLE`**. - **Fork, a card-format contract, recorded and not silently decided: card object names vs the platform's namespace prefix.** A scaffolded project's `defineStack` refuses an object not named `NS_short`. Dropping `namespace` validates with a warning, but the unmanaged install still refuses bare names with `422 NAMESPACE_PREFIX`, deriving the namespace from the package id. So a bare-name build can never be a template. (A) Implemented here: the runner reads cards as namespace-relative, and answers may cite either spelling. (B) The cards themselves adopt a namespace convention. Recommendation: A, because it keeps the cards portable and the cost is one transform. - **Card finding** for the crm card's author lane, noted and not filed: `attach_dee` expects the `sdr` to edit `ct4`, a contact with no account. The card's own `sdr` scope (`contact.account.owner = u_sam`) puts that row outside what the sdr sees, so a faithful build refuses it. Carrier: the scenario-card lane. - The build prompt's data contract tells the agent the seed rows, the persona sets named by persona id, and each persona's hidden objects and fields and row scope. It never contains a question, a truth or an outcome. The question turn asks for a `Source:` line; that is the answer format the citation axis grades. - Template mode supplies `installModes: ['unmanaged']` when the build did not declare it, and records `installModesSupplied`. - `run-scenarios` runs through `tsx` from the workspace root, the same way other private packages run their scripts. No new dependency was added and the lockfile is unchanged. ## Verification `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands`, run on head d6b1607 after merging `origin/main`, derived 74 families. All 74 ran (`--ran` reconciliation: 74 derived, 74 run, 0 UNRUN). <details><summary>Exit codes</summary> - 70 exited 0 on the first run. - `pnpm check:doc-authoring` exited 1: a tracker id inside a runtime string in `driver.ts`. Fixed by moving the id into a comment, then re-run: exit 0. - `pnpm check:dispatcher-error-vocabulary` exited 1: the observation labels were SCREAMING constants at a `code:` position. Fixed by making them prose labels in camelCase constants, then re-run: exit 0. - `node scripts/check-plugin-teardown-shape.mjs --self-test` exited 3: NOT MEASURED, because its fixture is pinned to a commit outside this shallow clone. It is unrelated to this diff, and CI runs it. - `pnpm check:dual-build-cjs-loads` exited 3: NOT MEASURED, because the gate reads built output and this container has not run a full `pnpm build`. CI runs it. </details> Size: 17 files, +2757 / −3 against `origin/main`, under the 3,000-line threshold. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (`packages/spec` is untouched.) - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran. eslint is clean on the new files; repo-wide `pnpm lint` is left to CI. - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A changeset (`.changeset/30292-scenario-runner.md`) carries both declaration lines. - [x] 6. `packages/spec` is not touched. - [x] 7. No hand edits to generated or release-owned files. - [x] 8. Title, body and comments are in English. The two verbatim rulings are quoted in their original language. - [x] The *Before Submitting* list in CONTRIBUTING.md is done. --- _Generated by [Claude Code](https://claude.ai/code/session_01Q4EjjR1A9qzqfcVHPQrYmN)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Q4EjjR1A9qzqfcVHPQrYmN --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 561d532 commit 802c98c

17 files changed

Lines changed: 2766 additions & 3 deletions

File tree

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
'@objectstack/scenarios': patch
3+
---
4+
5+
feat(scenarios): a local scenario runner builds, runs and iterates every card with a local Claude Code session and writes one run record per card
6+
7+
Clause-②: no
8+
Metadata-protocol: none
9+
10+
- **`pnpm --filter @objectstack/scenarios run-scenarios`** (private, unpublished; entry `@objectstack/scenarios/runner`): for each card under `docs/qa/scenarios/`, scaffold a project with `create-objectstack`, let the agent build it from the card prompt, run `objectstack validate`, `build` and `verify` (opt-in proofs chosen from the card's tags), boot it with `objectstack dev --fresh`, seed the card's rows, create one user, permission-set grant and API key per persona, and run every question and operation over the app's own MCP endpoint as that persona. Then apply the card's iterate change and score again.
11+
- **Template mode**: the card's verified build is installed unmanaged (`POST /api/v1/marketplace/install-local` with `installMode: 'unmanaged'`), the agent applies the iterate change to the installed copy, and the scorecard reports both modes side by side with the size of the change. A refused install is recorded as a finding.
12+
- **Scorecard and run record**: the scorecard reports Build, Run-问数, Run-operations and Iterate per mode, plus a per-tag table, with the real question and operation counts. A degraded build and a leaking answer are flagged. Records use the `docs/qa/platform-checklist/runs/` shape and stay out of git.
13+
- **Card object names are read namespace-relative.** Against a package with namespace `ns`, the card's `account` means `ns_account`. A scaffolded project refuses bare object names, and so does the unmanaged install (`NAMESPACE_PREFIX`).
14+
- **No API-billed model.** The live driver spawns the local `claude` CLI and refuses to start when `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` is set or the CLI is logged in with an API key. A recorded driver replays turns, so the runner's own tests need no model and no network. All grading is `@objectstack/scenarios/score`.

‎docs/qa/scenarios/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,3 +159,10 @@ meet this floor:
159159
path. Then pin the card's hand-worked truths, per persona too, in its own
160160
`packages/qa/scenarios/test/<id>.test.ts` (model: `travel-expense.test.ts`). The three control
161161
cards keep theirs in `test/library.test.ts`.
162+
163+
## Running the cards
164+
165+
`pnpm --filter @objectstack/scenarios run-scenarios` builds, runs and iterates every card here with a local
166+
Claude Code session, and writes one run record per card. Grading uses `@objectstack/scenarios/score`; no
167+
API-billed model is used. The runner reads a card's object names relative to the built package's namespace
168+
(`account` → `<namespace>_account`). Flags and modes: [the package README](../../../packages/qa/scenarios/README.md#running-the-cards--objectstackscenariosrunner).

‎packages/qa/scenarios/README.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,3 +65,31 @@ discrimination test; ⛔ never match by area prefix.
6565
| `approvals.approval-flow` | carries an approval (a flow with an Approval node), each bound to an object it serves, one bound to an object the card names |
6666
| `automation.record-triggered-flow` | carries a record-triggered flow, each bound to an object it serves, one bound to an object the card names |
6767
| `dashboards.widget-measures` | carries a dashboard widget with a measure, and every widget measure resolves on the dataset it binds |
68+
69+
## Running the cards — `@objectstack/scenarios/runner`
70+
71+
```bash
72+
pnpm --filter @objectstack/scenarios run-scenarios # every card, both modes
73+
pnpm --filter @objectstack/scenarios run-scenarios --only service_desk --modes scratch
74+
pnpm --filter @objectstack/scenarios run-scenarios --record /tmp/rec.json # also save every turn
75+
pnpm --filter @objectstack/scenarios run-scenarios --driver recorded:/tmp/rec.json # replay, no model
76+
```
77+
78+
Per card and mode the runner writes `docs/qa/platform-checklist/runs/<date>-scenario-<card>.json`
79+
(git-ignored) and prints the scorecard.
80+
81+
| Mode | Steps |
82+
|:---|:---|
83+
| `scratch` | `create-objectstack` → the agent builds from the card prompt and the runner's data contract (`prompts.ts`) → `objectstack validate` / `build` / `verify` (proofs chosen from the tags) → score → `objectstack dev --fresh` → seed, one user + permission-set grant + API key per persona → every question as every persona it names and every operation as its persona, over the app's MCP → the iterate change → rebuild, re-score, re-run |
84+
| `template` | a blank project → boot → install the card's verified build **unmanaged** (`/api/v1/marketplace/install-local`, `installMode: 'unmanaged'`) → the agent applies the iterate change to the installed copy → score → seed → Run. A refused install is a finding in the record |
85+
86+
- **Driver** (`driver.ts`): the local `claude` CLI on your own login (`claude -p --output-format stream-json`). It
87+
refuses to start when `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN` is set or the CLI is logged in with an API key
88+
(the cost ruling on cloud#3600), and the record states the driver and login. `RecordedDriver` replays a recording.
89+
- **Observation** (`observe.ts`): the write grader is fed what the runner reads itself: the agent's write calls and
90+
the door's answer (a coded refusal), the object's rows before and after the turn, and the audit trail's actor.
91+
- **Namespaces** (`namespace.ts`): a card's object names are read relative to the package namespace (`account` →
92+
`ns_account`), because a scaffolded project and the unmanaged install both refuse bare names.
93+
- **Persona contract**: the build is asked for one permission set per persona, named by the persona's id. The runner
94+
grants it to that persona's user, and makes a seed row whose scope field names a persona's handle (`owner: u_sam`)
95+
owned by that persona's user.

‎packages/qa/scenarios/package.json‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,17 @@
33
"version": "0.1.0",
44
"private": true,
55
"license": "Apache-2.0",
6-
"description": "Scenario-card schema and loader for the open scenario library under docs/qa/scenarios/. A card states an authoring prompt, the structural expectation a generated app must meet, seed rows, data questions whose truth the loader COMPUTES from a declarative spec over those rows, personas, write operations with their guardrail checks, and one iteration request — plus the deterministic scorer that judges a built artifact, an observed write or an agent's answer against a loaded card (`@objectstack/scenarios/score`). Not published, no build step, no model calls: a devx instrument, not a protocol (packages/spec carries no obligation for this format).",
6+
"description": "Scenario-card schema and loader for the open scenario library under docs/qa/scenarios/. A card states an authoring prompt, the structural expectation a generated app must meet, seed rows, data questions whose truth the loader COMPUTES from a declarative spec over those rows, personas, write operations with their guardrail checks, and one iteration request \u2014 plus the deterministic scorer that judges a built artifact, an observed write or an agent's answer against a loaded card (`@objectstack/scenarios/score`). Not published, no build step, no model calls: a devx instrument, not a protocol (packages/spec carries no obligation for this format).",
77
"type": "module",
88
"exports": {
99
".": "./src/index.ts",
10-
"./score": "./src/score/index.ts"
10+
"./score": "./src/score/index.ts",
11+
"./runner": "./src/runner/index.ts"
1112
},
1213
"scripts": {
1314
"test": "vitest run",
14-
"typecheck": "tsc --noEmit"
15+
"typecheck": "tsc --noEmit",
16+
"run-scenarios": "tsx src/runner/cli.ts"
1517
},
1618
"dependencies": {
1719
"@objectstack/core": "workspace:*",
Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* The booted app as the runner reaches it: plain HTTP against `os dev`'s
5+
* server, through the same public doors a person uses. Every path here was
6+
* measured against a `create-objectstack` blank project on `os dev --fresh`
7+
* (the runner's PR records the reading):
8+
*
9+
* sign in POST /api/v1/auth/sign-in/email → session cookie
10+
* a persona's user POST /api/v1/auth/admin/create-user (self sign-up is closed
11+
* on the default `invite_only` audience) → the new user must change
12+
* the password before anything else, so: sign in, POST
13+
* /api/v1/auth/change-password, sign in again
14+
* its API key POST /api/v1/keys (a SESSION door: a key cannot mint a key) →
15+
* the raw key, returned once; it acts as its owner
16+
* its access POST /api/v1/data/sys_user_permission_set { user_id,
17+
* permission_set, organization_id } — a user with no app permission
18+
* set reads `PERMISSION_DENIED` on every app object
19+
* a seed row POST /api/v1/data/:object with the card's own `id` (accepted
20+
* verbatim, so a lookup value in the seed names the real row)
21+
* rows GET /api/v1/data/:object?limit=…
22+
* the audit actor GET /api/v1/data/sys_audit_log?record_id=… (`user_id`)
23+
* unmanaged install POST /api/v1/marketplace/install-local { packageId, manifest,
24+
* installMode: 'unmanaged' } (ADR-0131 D6)
25+
* metadata GET /api/v1/meta/:type
26+
*
27+
* The data API's refusal is the flat `{ error, code }` envelope; the auth and
28+
* install doors answer `{ success: false, error: { code, message } }`. Both
29+
* are read by {@link codeOf}.
30+
*/
31+
32+
import { PLURAL_TO_SINGULAR } from '@objectstack/spec/meta-spelling';
33+
34+
export interface HttpAnswer {
35+
ok: boolean;
36+
status: number;
37+
body: unknown;
38+
code?: string;
39+
message?: string;
40+
}
41+
42+
/** The refusal code of either envelope this app answers with. */
43+
export function codeOf(body: unknown): { code?: string; message?: string } {
44+
if (typeof body !== 'object' || body === null) return {};
45+
const b = body as Record<string, unknown>;
46+
if (typeof b.code === 'string') return { code: b.code, ...(typeof b.error === 'string' ? { message: b.error } : {}) };
47+
const e = b.error;
48+
if (typeof e === 'object' && e !== null) {
49+
const ee = e as Record<string, unknown>;
50+
return {
51+
...(typeof ee.code === 'string' ? { code: ee.code } : {}),
52+
...(typeof ee.message === 'string' ? { message: ee.message } : {}),
53+
};
54+
}
55+
return {};
56+
}
57+
58+
export interface PersonaAccount {
59+
persona: string;
60+
userId: string;
61+
email: string;
62+
apiKey: string;
63+
/** The permission set the runner assigned, or why it could not. */
64+
permissionSet: { name: string; assigned: boolean; code?: string; message?: string };
65+
}
66+
67+
/** What the orchestrator needs from a booted app — the seam the tests replace. */
68+
export interface AppHarness {
69+
readonly baseUrl: string;
70+
readonly mcpUrl: string;
71+
signInAdmin(): Promise<void>;
72+
adminUserId(): string;
73+
organizationId(): string;
74+
/** One user, its own permission set (named after the persona) and one API key. */
75+
createPersona(persona: string): Promise<PersonaAccount>;
76+
/** An API key for the admin, for the questions no persona is named on. */
77+
adminApiKey(): Promise<string>;
78+
insertRow(object: string, row: Record<string, unknown>): Promise<HttpAnswer>;
79+
listRows(object: string): Promise<HttpAnswer & { rows: Array<Record<string, unknown>> }>;
80+
/** The `user_id` the newest audit row for this record names; `null` when none names one. */
81+
auditActorUserId(object: string, recordId: string): Promise<string | null>;
82+
installUnmanaged(manifest: Record<string, unknown>): Promise<HttpAnswer>;
83+
/** The environment's metadata of every collection the template carries, as an artifact. */
84+
metadataAsArtifact(collections: readonly string[]): Promise<Record<string, unknown[]>>;
85+
}
86+
87+
export interface HttpAppOptions {
88+
baseUrl: string;
89+
adminEmail: string;
90+
adminPassword: string;
91+
/** Password given to every persona user (dev only; the server is the run's own throwaway). */
92+
personaPassword?: string;
93+
}
94+
95+
class Session {
96+
private cookies = new Map<string, string>();
97+
98+
absorb(res: Response): void {
99+
for (const c of res.headers.getSetCookie()) {
100+
const [pair] = c.split(';');
101+
const i = pair!.indexOf('=');
102+
if (i > 0) this.cookies.set(pair!.slice(0, i).trim(), pair!.slice(i + 1).trim());
103+
}
104+
}
105+
106+
header(): string {
107+
return [...this.cookies].map(([k, v]) => `${k}=${v}`).join('; ');
108+
}
109+
}
110+
111+
export class HttpApp implements AppHarness {
112+
readonly baseUrl: string;
113+
readonly mcpUrl: string;
114+
private readonly admin = new Session();
115+
private adminId = '';
116+
private orgId = '';
117+
private readonly opts: HttpAppOptions;
118+
119+
constructor(opts: HttpAppOptions) {
120+
this.opts = opts;
121+
this.baseUrl = opts.baseUrl.replace(/\/$/, '');
122+
this.mcpUrl = `${this.baseUrl}/api/v1/mcp`;
123+
}
124+
125+
private async call(
126+
method: string,
127+
path: string,
128+
body?: unknown,
129+
auth?: { session?: Session; apiKey?: string },
130+
): Promise<HttpAnswer> {
131+
const headers: Record<string, string> = { accept: 'application/json' };
132+
if (body !== undefined) headers['content-type'] = 'application/json';
133+
if (auth?.session) headers.cookie = auth.session.header();
134+
if (auth?.apiKey) headers['x-api-key'] = auth.apiKey;
135+
const res = await fetch(`${this.baseUrl}${path}`, { method, headers, ...(body !== undefined ? { body: JSON.stringify(body) } : {}) });
136+
auth?.session?.absorb(res);
137+
const text = await res.text();
138+
let parsed: unknown = text;
139+
try {
140+
parsed = text ? JSON.parse(text) : null;
141+
} catch {
142+
/* keep the text */
143+
}
144+
const ok = res.ok && !(typeof parsed === 'object' && parsed !== null && (parsed as { success?: unknown }).success === false);
145+
return { ok, status: res.status, body: parsed, ...(ok ? {} : codeOf(parsed)) };
146+
}
147+
148+
private async signIn(session: Session, email: string, password: string): Promise<HttpAnswer> {
149+
return this.call('POST', '/api/v1/auth/sign-in/email', { email, password }, { session });
150+
}
151+
152+
async signInAdmin(): Promise<void> {
153+
const r = await this.signIn(this.admin, this.opts.adminEmail, this.opts.adminPassword);
154+
if (!r.ok) throw new Error(`admin sign-in refused: ${r.status} ${r.code ?? ''} ${r.message ?? ''}`.trim());
155+
this.adminId = String((r.body as { user?: { id?: unknown } }).user?.id ?? '');
156+
const s = await this.call('GET', '/api/v1/auth/get-session', undefined, { session: this.admin });
157+
const session = (s.body as { session?: { activeOrganizationId?: unknown } } | null)?.session;
158+
this.orgId = typeof session?.activeOrganizationId === 'string' ? session.activeOrganizationId : '';
159+
}
160+
161+
adminUserId(): string {
162+
return this.adminId;
163+
}
164+
165+
organizationId(): string {
166+
return this.orgId;
167+
}
168+
169+
async createPersona(persona: string): Promise<PersonaAccount> {
170+
const email = `${persona}@scenario.test`;
171+
const first = `${this.opts.personaPassword ?? 'Scenario-Run-1'}a`;
172+
const password = this.opts.personaPassword ?? 'Scenario-Run-1';
173+
const created = await this.call('POST', '/api/v1/auth/admin/create-user', { email, password: first, name: persona, role: 'user' }, { session: this.admin });
174+
if (!created.ok) throw new Error(`create-user ${persona}: ${created.status} ${created.code ?? ''} ${created.message ?? ''}`.trim());
175+
const data = (created.body as { data?: { user?: { id?: string }; organizationId?: string } }).data;
176+
const userId = String(data?.user?.id ?? '');
177+
if (!this.orgId && data?.organizationId) this.orgId = data.organizationId;
178+
179+
const s = new Session();
180+
await this.signIn(s, email, first);
181+
const changed = await this.call('POST', '/api/v1/auth/change-password', { currentPassword: first, newPassword: password, revokeOtherSessions: false }, { session: s });
182+
if (!changed.ok) throw new Error(`change-password ${persona}: ${changed.status} ${changed.code ?? ''}`.trim());
183+
const again = await this.signIn(s, email, password);
184+
if (!again.ok) throw new Error(`sign-in ${persona}: ${again.status} ${again.code ?? ''}`.trim());
185+
186+
const grant = await this.call(
187+
'POST',
188+
'/api/v1/data/sys_user_permission_set',
189+
{ user_id: userId, permission_set: persona, organization_id: this.orgId },
190+
{ session: this.admin },
191+
);
192+
const key = await this.call('POST', '/api/v1/keys', { name: `scenario-${persona}` }, { session: s });
193+
const raw = (key.body as { data?: { key?: unknown } } | null)?.data?.key;
194+
if (!key.ok || typeof raw !== 'string') throw new Error(`mint key ${persona}: ${key.status} ${key.code ?? ''}`.trim());
195+
return {
196+
persona,
197+
userId,
198+
email,
199+
apiKey: raw,
200+
permissionSet: {
201+
name: persona,
202+
assigned: grant.ok,
203+
...(grant.code ? { code: grant.code } : {}),
204+
...(grant.message ? { message: grant.message.slice(0, 300) } : {}),
205+
},
206+
};
207+
}
208+
209+
async adminApiKey(): Promise<string> {
210+
const key = await this.call('POST', '/api/v1/keys', { name: 'scenario-admin' }, { session: this.admin });
211+
const raw = (key.body as { data?: { key?: unknown } } | null)?.data?.key;
212+
if (!key.ok || typeof raw !== 'string') throw new Error(`mint admin key: ${key.status} ${key.code ?? ''}`.trim());
213+
return raw;
214+
}
215+
216+
insertRow(object: string, row: Record<string, unknown>): Promise<HttpAnswer> {
217+
return this.call('POST', `/api/v1/data/${encodeURIComponent(object)}`, row, { session: this.admin });
218+
}
219+
220+
async listRows(object: string): Promise<HttpAnswer & { rows: Array<Record<string, unknown>> }> {
221+
const r = await this.call('GET', `/api/v1/data/${encodeURIComponent(object)}?limit=1000`, undefined, { session: this.admin });
222+
const records = (r.body as { records?: unknown } | null)?.records;
223+
return { ...r, rows: Array.isArray(records) ? (records as Array<Record<string, unknown>>) : [] };
224+
}
225+
226+
async auditActorUserId(object: string, recordId: string): Promise<string | null> {
227+
const q = `object_name=${encodeURIComponent(object)}&record_id=${encodeURIComponent(recordId)}&limit=200`;
228+
const r = await this.call('GET', `/api/v1/data/sys_audit_log?${q}`, undefined, { session: this.admin });
229+
const rows = ((r.body as { records?: unknown } | null)?.records ?? []) as Array<Record<string, unknown>>;
230+
const mine = rows.filter((x) => x.record_id === recordId && x.object_name === object);
231+
mine.sort((a, b) => String(b.created_at).localeCompare(String(a.created_at)));
232+
const actor = mine[0]?.user_id;
233+
return typeof actor === 'string' ? actor : null;
234+
}
235+
236+
installUnmanaged(manifest: Record<string, unknown>): Promise<HttpAnswer> {
237+
const packageId = typeof manifest.id === 'string' ? manifest.id : 'scenario_template';
238+
return this.call('POST', '/api/v1/marketplace/install-local', { packageId, manifest, installMode: 'unmanaged' }, { session: this.admin });
239+
}
240+
241+
async metadataAsArtifact(collections: readonly string[]): Promise<Record<string, unknown[]>> {
242+
const out: Record<string, unknown[]> = {};
243+
for (const plural of collections) {
244+
const type = PLURAL_TO_SINGULAR[plural];
245+
if (!type) continue;
246+
const r = await this.call('GET', `/api/v1/meta/${type}`, undefined, { session: this.admin });
247+
const items = (r.body as { items?: unknown } | null)?.items;
248+
out[plural] = Array.isArray(items) ? items.filter((i) => !String((i as { name?: unknown }).name ?? '').match(/^(sys_|ai_)/)) : [];
249+
}
250+
return out;
251+
}
252+
}

0 commit comments

Comments
 (0)