Skip to content

Commit b1b153a

Browse files
docs(skills): objectstack-ui gains an eval for a report's dateRange and exportOptions (#26774)
<!-- claude-projects-attribution: {"github_login":"os-steve"} --> _Requested by **Steve** · [project thread](https://claude.ai/code/project/chan_01FdL1XeUyvVNxLreh3F4Qi9?thread=cmsg_01FdL1XeUyvVNxLreh3F4Qi9T13SmMGaz8cQkchZBGh7J2)_ Fixes #26633 Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:`objectstack-ui` 技能新增第三个 eval 文件 `evals/report-period-export.json`,只含一条报表编写题:要求报表在 `type: 'date'` 的数据集维度上声明 `dateRange`(沿用仪表盘的预设名),并用 `exportOptions` 声明 csv / xlsx 下载;禁止 `joined` 容器、手写 `runtimeFilter` 日期窗、第二张预设表和自造的导出权限键。`evals/README.md` 列出该文件;token 棘轮两行按实测钉住;`@objectstack/skills` 补丁 changeset。 - 为什么改:#26628 给报表加了这两个键,但两个既有 eval 文件都在 token 上限(余量 0),没有任何 eval 检查 agent 会用这两个键;本卡是它欠下的 eval。 - 风险与回滚:只加一个 JSON 夹具、三行 README 和两行棘轮;不改任何 schema、键或运行时行为。回滚即 revert 本 PR。 - 先看:`skills/objectstack-ui/evals/report-period-export.json`(题面与断言)、`scripts/check-skills-token-ratchet.mjs`(README 行 289 → 335、新文件行 457,算式在注释里)、`skills/objectstack-ui/evals/README.md`。 - 两分钟验证:`node scripts/check-skills-token-ratchet.mjs`,预期四个 `objectstack-ui/evals` 文件全部 ✓ 且余量 0;`node -e "JSON.parse(require('fs').readFileSync('skills/objectstack-ui/evals/report-period-export.json','utf8'))"` 无输出即解析通过。 - 需要你做的:批准本 PR(`skills/**` 受管面,Tier H)。 ## What changes - **New eval file** `skills/objectstack-ui/evals/report-period-export.json` (same fixture shape as its two siblings, one eval). The prompt names a dataset with a `type: 'date'` `close_date` dimension and asks for a summary report that opens on the current quarter, lets the viewer pick another period, and lets users who may already export the base object download the table as CSV or Excel. The assertions require `defineReport`, `dateRange`, `dimension: 'close_date'`, `defaultRange: 'this_quarter'`, `exportOptions`, `'csv'`, `'xlsx'` and `allowExport` (the existing permission-set axis, named in words), and refuse `type: 'joined'` / `blocks:` (either key on a joined container), `close_date: {` (a hand-rolled `runtimeFilter` window on the date column), `presets:` (a second preset table), `permissions.export` / `canExport` / `allowExport: true` (an invented report permission key) and `'pdf'`. Every key in the prompt, the expected output and the assertions is read off `packages/spec/src/ui/report.zod.ts` and `list-view-export-options.ts` on `origin/main` dc9d1ff (`dateRange: { dimension?, defaultRange, allowCustomRange }`, `exportOptions: { formats?, includeHeaders?, fileNamePrefix? }`, formats `csv` / `xlsx`, preset `this_quarter` from `DATE_RANGE_PRESETS`); none is invented. - **Why a third file.** Both sibling eval files sit at headroom 0, and the open draft PR #26739 (another lane) edits `analytics-inline-vs-dataset.json` and its ratchet row, so the prompt gets a new file and a new row instead of a serial conflict. - **`skills/objectstack-ui/evals/README.md`** lists the new file in one three-line entry, between its two siblings. - **`scripts/check-skills-token-ratchet.mjs`**: a new row for the eval file pinned AT the measurement (1828 bytes → 457 tokens, headroom 0) and the README row moved 289 → 335 (1143 → 1339 bytes), each with its arithmetic comment and the standing maintainer ruling quoted verbatim beside the sibling row: 「A:先削重复,余额按实测提高」. Paid down first as the ruling asks: the new file is fixtures and restates nothing; the README's four sections share no sentence; so 0 bytes were available to cut and the whole addition is raised to the measurement, not past it. No other row moves. - **Changeset** `.changeset/26633-report-period-export-eval.md`: `@objectstack/skills` patch, both declaration lines at line start (the published skills bundle is built from `skills/**`). **Skills readings** (`skills/**` touched; before = `origin/main` dc9d1ff, after = head): | Reading | Before | After | | --- | --- | --- | | `skills/objectstack-ui/evals/report-period-export.json` lines / tokens | — (new) | 16 / 457 (ceiling 457, headroom 0) | | `skills/objectstack-ui/evals/README.md` lines / tokens | 31 / 286 (ceiling 289, headroom 3) | 34 / 335 (ceiling 335, headroom 0) | | `skills/objectstack-ui/**` lines | 2259 | 2278 | | all `SKILL.md` lines | 4600 | 4600 | | whole `skills/**` tree lines | 19191 | 19210 | | bundle total tokens (ratchet report) | 213404 | 213910 | ## Why A report can now declare its period and its export (`dateRange` / `exportOptions`, PR #26628), and the skill's rule example shows `dateRange`, but no eval scored an agent that hand-rolls a `runtimeFilter` date window, invents a `permissions.export` bit, redeclares the preset table, or puts either key on a `joined` container. This file is the instrument that makes those wrong answers fail at scoring. ## Tests - `node scripts/check-skills-token-ratchet.mjs` at head: `82 authored bundle file(s) within their ceilings; 10 generator-owned file(s) measured, not ratcheted`, exit 0; the four `objectstack-ui/evals` rows each ✓ at headroom 0. - `node scripts/check-skills-token-ratchet.mjs --self-test`: `79 cases pass`, exit 0. - The eval file parses (`JSON.parse`, no output). - No test file edited; no forced test edit. ## Acceptance notes - **The skill text teaches `dateRange` but not `exportOptions`.** `skills/objectstack-ui/rules/dashboards.md` § Report Configuration carries the `dateRange` example from PR #26628; `exportOptions` is taught only on the docs page `content/docs/ui/reports.mdx` § Period and export. This eval therefore asserts a key the published skill does not yet teach in its rules. Noted, not filed — the card's file surface is the eval, its README line, the ratchet rows and the changeset; the teaching line is the skills lane's call (that file sits at headroom 0 too). - The `allowExport` must-contain is a review cue as much as a scorer: an answer that explains the export gate as "the base object's export permission" without the key name scores as a miss and deserves a read, the way eval 2 of `analytics-inline-vs-dataset.json` documents its strict `dateGranularity`. - The eval is not run by any gate: the repo has no eval runner; `check:skill-docs` / `check:skill-refs` / `check:skill-examples` read `SKILL.md`, `references/` and the rules' code fences, none the eval JSON. The gates this diff does reach are listed under Verification. ## Verification - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` at head `78362c8b` (the final commit) derived 61 commands (51 matched by path, 2 by change kind, 8 whole-tree). All 61 ran in the worktree after that commit, each exit 0 captured before any pipe — among them `node scripts/check-skills-token-ratchet.mjs` and its `--self-test` (79 cases), `pnpm check:pm-governed-merges`, `pnpm check:pm-dispatch-gates`, `pnpm check:published-files`, `node scripts/check-empty-changeset.mjs --base origin/main`, `node scripts/check-adr-0087-registration.mjs --base origin/main`, `node scripts/check-changeset-no-major.mjs --base origin/main`, `pnpm check:skill-identifier-liveness`, `pnpm check:skill-frame-sync`, `pnpm check:skill-compatibility`, `pnpm check:doc-authoring`, `pnpm check:ratchet-remedy-authority`, `pnpm check:nul-bytes`. - `dispatch-gates --ran`: `61 derived famil(ies) accounted for — 61 run, 0 NOT-MEASURED (a DERIVED zero — all 61 recorded an exit code and none of them is 3)`. - Three families take a value from the workflow and are NOT MEASURED locally (the two shard attestations and `check-test-completeness`); the 11 wide-population families and the repo-wide `pnpm lint` are CI's runs. - Build closure before the gates: `turbo run build --concurrency=2 --filter='./packages/**'` under `scripts/pm/os-verify-lock.sh`, `VERDICT command-exit 0` (held 8m42s). - The edited gate script's own test suite is its `--self-test` (no other test file names `check-skills-token-ratchet`); no package source is touched, so no package `test` / `typecheck` is owed. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (No schema touched.) - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes (the repo-wide run is CI's; no source file edited). - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. - [x] 6. `packages/spec` touched: n/a, not touched. - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. - [x] The *Before Submitting* list in CONTRIBUTING.md is done. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01ADHkmSMxfTnb6u1AsKaTcU --- _Generated by [Claude Code](https://claude.ai/code/session_01ADHkmSMxfTnb6u1AsKaTcU)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 4ae66a2 commit b1b153a

4 files changed

Lines changed: 52 additions & 2 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/skills': patch
3+
---
4+
5+
docs(skills): objectstack-ui gains an eval for a report's `dateRange` and `exportOptions`
6+
7+
Clause-②: no
8+
Metadata-protocol: none
9+
10+
- **Before.** The skill taught a report's `dateRange` in `rules/dashboards.md`, but no eval checked that an agent authoring a report reaches for `dateRange` on a `type: 'date'` dataset dimension and for `exportOptions` with `csv` / `xlsx`. An agent could hand-roll a `runtimeFilter` window on the date column, invent a report permission key for export, declare a second preset table, or place either key on a `joined` container, and nothing scored that wrong.
11+
- **Now.** A third eval file, `evals/report-period-export.json`, carries one report-authoring prompt whose assertions require `dateRange: { dimension, defaultRange }` and `exportOptions` with `'csv'` / `'xlsx'`, name the existing permission-set `allowExport` as the only export gate, and refuse a `joined` container, a `close_date` filter window, a `presets:` table and an invented permission key. `evals/README.md` lists the file.
12+
- No schema, key or behaviour changes.

‎scripts/check-skills-token-ratchet.mjs‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -679,10 +679,18 @@ export const CEILINGS = new Map([
679679
// objectstack-automation
680680
['skills/objectstack-automation/evals/approvals/test-revise-loop.md', 1329],
681681

682-
// objectstack-ui — the two authored eval files; its `references/react-blocks.md`
682+
// objectstack-ui — the three authored eval files; its `references/react-blocks.md`
683683
// is generator-owned and carries no row here on purpose (see the boundary
684684
// section). Its JSON twin was retired by #14296 item 3 and is gone entirely.
685-
['skills/objectstack-ui/evals/README.md', 289],
685+
// 289 -> 335 (#26633): the README lists the third eval file,
686+
// `report-period-export.json`, in one three-line entry (+196 bytes). The
687+
// README's four sections share no sentence, so there was no duplicate to cut
688+
// (0 bytes). On `origin/main` the file was 1143 bytes (286 tokens, headroom
689+
// 3); the LANDED file is 1339 bytes -> 335 tokens, ceiling +46, pinned AT the
690+
// measurement. Same ruling as the `analytics-inline-vs-dataset.json` row
691+
// below (2026-10-27, tap A: 「A:先削重复,余额按实测提高」).
692+
// ceil($(wc -c < skills/objectstack-ui/evals/README.md) / 4) == ceil(1339 / 4)
693+
['skills/objectstack-ui/evals/README.md', 335],
686694
// 1102 -> 1229: the aggregate × field-type rule and the multi-value-dimension
687695
// rule (card #24425, the skills half of cloud#2861 ruling B) each gain one
688696
// regression case — a `formula` under `sum` and a `multiselect` as the
@@ -710,6 +718,17 @@ export const CEILINGS = new Map([
710718
// Maintainer ruling 2026-10-27 (Warren, decision card 「skills 修复卡的教学和 eval 都撞 token 天花板,怎么付?」,
711719
// option A tapped at 01:21Z), verbatim and untranslated: 「A:先削重复,余额按实测提高」.
712720
['skills/objectstack-ui/evals/analytics-inline-vs-dataset.json', 1348],
721+
// INITIAL measurement (#26633): the third authored eval file — ONE
722+
// report-authoring prompt for `dateRange` on a `type: 'date'` dataset
723+
// dimension and `exportOptions` csv / xlsx under the base object's
724+
// `allowExport`, the eval owed by the spec PR that added both keys. A new
725+
// file because both sibling eval files sat at headroom 0 and the analytics
726+
// file was held by an open PR. Nothing in it restates anything (0 bytes
727+
// cut); 1828 bytes -> 457 tokens, pinned AT the measurement — zero headroom,
728+
// so the next token added here is paid for by deleting one from this file.
729+
// Same ruling as the row above (2026-10-27, tap A: 「A:先削重复,余额按实测提高」).
730+
// ceil($(wc -c < skills/objectstack-ui/evals/report-period-export.json) / 4) == ceil(1828 / 4)
731+
['skills/objectstack-ui/evals/report-period-export.json', 457],
713732

714733
// objectstack-ui `rules/` — the five files the #14296 item 1 split created.
715734
// Each row is an INITIAL measurement AT its landed count, so every one starts

‎skills/objectstack-ui/evals/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ view / page / dashboard / report metadata.
99
dashboard/report widgets: when a data need fits a `defineDataset`, when it
1010
must escalate to a stored field or app code, and when an ad-hoc
1111
in-page `<ObjectChart>` needs no dataset at all.
12+
- `report-period-export.json` — a non-`joined` report's `dateRange` (one
13+
`type: 'date'` dataset dimension) and `exportOptions` (csv / xlsx under the
14+
base object's `allowExport` permission).
1215
- `views-apps-actions-pages.json` — `defineView` containers, `App.create`
1316
navigation, `defineAction` surfaces, record / interface pages, package
1417
docs, wizard forms.
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"skill_name": "objectstack-ui",
3+
"description": "Report period and export (ADR-0021 dataset reports): a non-joined report windows ONE `type: 'date'` dataset dimension through `dateRange` and declares its download in `exportOptions`; who may export stays the base object's `allowExport` permission.",
4+
"evals": [
5+
{
6+
"id": 1,
7+
"prompt": "The opportunity_metrics dataset (base object opportunity) declares the amount_sum measure and the stage and close_date dimensions (close_date is type: 'date', dateGranularity: 'quarter'). Add a pipeline-by-stage summary report that opens on the current quarter, lets the viewer pick another period, and lets users who may already export opportunities download the table as CSV or Excel. Emit the metadata.",
8+
"expected_output": "One non-joined defineReport bound to opportunity_metrics (rows: ['stage'], values: ['amount_sum']) with `dateRange: { dimension: 'close_date', defaultRange: 'this_quarter', allowCustomRange: true }` — the dataset's `type: 'date'` dimension and the dashboard's own preset names, never a second preset table or a hand-rolled `runtimeFilter` window on close_date — and `exportOptions: { formats: ['csv', 'xlsx'] }` (optionally `includeHeaders` / `fileNamePrefix`). Who may download is the base object's existing permission-set `allowExport`, named in words: no report permission key is invented, and neither key is placed on a `joined` container (refused).",
9+
"files": [],
10+
"assertions": {
11+
"must_contain": ["defineReport", "dateRange", "dimension: 'close_date'", "defaultRange: 'this_quarter'", "exportOptions", "'csv'", "'xlsx'", "allowExport"],
12+
"must_not_contain": ["type: 'joined'", "blocks:", "close_date: {", "presets:", "permissions.export", "canExport", "allowExport: true", "'pdf'"]
13+
}
14+
}
15+
]
16+
}

0 commit comments

Comments
 (0)