Repository navigation
Commit b618224
fix(service-analytics, lint)!: a dataset or cube dimension over a formula field is refused where it is authored, instead of on the first query that groups by it (#30815)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} -->
_Requested by **Elon** · [project
thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9D1W98tqSQjC4jC6VTdbuasE)_
domain:server#3 · os-elon-musk · session_016WV4cytQjG9X1CrCgytUM5
Fixes #30773
Clause-②: no (narrowing)
Metadata-protocol: narrowed
Skills-docs: updated here —
`content/docs/deployment/validating-metadata.mdx` names
`dimension-virtual-field-refused` beside
`dimension-json-stored-field-refused` and in its cube paragraph
(seat-approved surface addendum); follow-up #30824 —
`skills/objectstack-data/rules/field-types.md`'s formula row (Tier H,
the skills lane).
## 维护者速读
- 改了什么:数据集(或
cube)的维度如果指向公式字段(不落库的虚拟字段),现在在"写的时候"就被拒:数据集编译(`registerDataset` /
`queryDataset`)给 `DATASET_INVALID` / 400,点名数据集、维度、位置和修法;`os validate` /
`os build` / `os lint` 与运行时保存数据集(Studio、REST `/meta`、MCP 共用的那道门)报新规则
`dimension-virtual-field-refused`。
- 为什么改:#29304 之后,按公式字段分组的查询在所有分析入口都会
400,但数据集本身照样能保存、发布,错误要等看板读者打开图表才出现。度量侧(对公式字段求和)早已在编译和 lint 两处拒绝,维度侧缺这一半。
-
风险与回滚:只收窄"维度指向公式字段"这一种写法;含这种维度的整个数据集将无法编译,连它的其它存储字段维度也不再出数(这是收窄的代价,changeset
的 BREAKING 段写明)。存储字段维度、日期维度和所有度量判定不变。回滚即 revert。
- 先看:`packages/services/service-analytics/src/dataset-compiler.ts` 的
`assertDimensionFieldStored`;`packages/lint/src/validate-dataset-measure-aggregates.ts`
的 `virtualDimensionFinding` 及数据集、cube
两腿的调用点;`preview-date-bucket-type.test.ts` 里公式维度从共享数据集拆出(原 #29167 /
#29304 的数据集面钉现在钉编译期拒绝)。
- 两分钟验证:`pnpm --filter @objectstack/service-analytics exec vitest run
src/__tests__/aggregate-nontemporal-measure-refusal.test.ts`,预期全部通过,含 6
条 `[#30773]` 用例。
- 需要你做的:无;契约复核后席位落地。
## What changes
**`@objectstack/service-analytics`: the compile refuses the dimension.**
- `compileDataset`'s dimension loop now calls
`assertDimensionFieldStored` after `assertDeclared`, the way the measure
loop calls its type gate after the join check. A dimension whose `field`
the base object declares VIRTUAL, judged by the spec's
`isVirtualSearchField` (`SEARCH_VIRTUAL_TYPES`, today `formula`),
whatever its `returnType` and with or without `dateGranularity`, throws
`datasetInvalidError` (`DATASET_INVALID` / 400). The message names the
dataset, the dimension, its position (`dimensions.N`), the field and its
declared type, and the remedy the query-time doors name: denormalise the
value onto the object (a stored field, written when the source changes)
and point the dimension at that field.
- It reaches every door the measure refusal (#24426) reaches, because
both run inside `compileDataset` through the service's one `compile()`:
`registerDataset` (the configuration door), `queryDataset` on every face
(native, ObjectQL, the draft preview), and the build probes that query
through it.
- Tiering is the measure gate's ("cannot answer, do not block"): no
`declaredValueShape` hook, a field the hook does not resolve, or a
dotted relationship path (the hook answers for the base object only) is
not judged here. A dotted dimension over a joined formula stays the
query-time doors' (#29304) and the lint leg's.
**`@objectstack/lint`: `dimension-virtual-field-refused`.**
- A new id beside `dimension-json-stored-field-refused` in
`validate-dataset-measure-aggregates.ts`, so it rides the one registry
entry: gating, on `os validate` / `os build` / `os lint`, and at the
runtime `dataset` write door (`422`). One verdict for a dataset
dimension (`datasets[N].dimensions[M].field`) and an authored cube
dimension (`analyticsCubes[N].dimensions.KEY.sql`), resolved on the
object graph exactly as the JSON-stored leg resolves it, dotted paths
included (judged on the leaf's object), with the same skips. The
predicate is the spec's `isVirtualSearchField`, never a local list. A
formula is never JSON-stored, so the two dimension ids never fire on one
member.
- One-line verdict and fix, as the #22161 siblings. Verdict: groups by
field "FIELD" (formula on object "OBJECT"), a value computed on read and
never stored, so no query can group by it. Fix: Store the value in a
stored field of "OBJECT" (written when its source changes) and point
this dimension at that field, or remove the dimension. The reasoning
(which doors refuse it, why a computed value is no group key) is `os
explain dimension-virtual-field-refused` (`rule-explanations.ts`). The
id's constant is exported from the package barrel beside its two
siblings.
**Measured before the change** (on `origin/main` `2d04cd2b`, probes
since deleted):
- (a) A dataset dimension over a `formula`: `registerDataset` ACCEPTED;
`queryDataset` selecting a stored dimension of it SERVED (1 statement);
selecting the formula dimension refused only at query time.
- (b) A cube dimension whose `sql` names a formula column: no authoring
door judged it (cubes are registered without a compile; the query-time
door refuses it since #29304).
- (c) `runAuthoringRules` on validate / build / lint: zero findings for
either (the only finding on the probe stack was an unrelated
`expression-invalid` on the probe's own formula, fixed in the probe).
## Tests
- `service-analytics` `aggregate-nontemporal-measure-refusal.test.ts`
(the #24426 harness), new top-level describe `[#30773]`: the request
door refuses `DATASET_INVALID` / 400 with the exact message and no SQL
emitted, for a selection naming the formula dimension and for one naming
only the stored dimension; the configuration door (`registerDataset`)
refuses in the same words; a date dimension bucketed by month over a
formula is refused at its position; controls (a stored, a date-bucketed
and a numeric dimension compile and query); the three cannot-answer
tiers do not block; the measure gate over the same formula is unchanged
(a `count` compiles, a `sum` keeps the stored-number remedy).
- `service-analytics` `preview-date-bucket-type.test.ts`: the shared
`DATASET` declared three formula dimensions beside its stored ones, so
every case in the file went red under the compile gate (19 red,
measured). The formula dimensions moved to `FORMULA_DATASET` /
`FORMULA_TEXT_DATASET`; the #28852 / #28977 cases are byte-unchanged and
green, and the five #29167 / #29304 dataset-face cases now pin the
compile refusal alike on every face (native, ObjectQL, preview), plus a
case for a selection naming only the stored dimension. The query-time
refusal itself stays pinned where no dataset compile stands in front of
it: the engine door, `objectql-echo-non-date-bucket.test.ts` (the dry
run) and `fallback-delegate-date-bucket-type.test.ts` (both untouched,
green).
- `lint` `validate-dataset-measure-aggregates.test.ts`, new top-level
describe for the id: the dataset pin with exact message and hint plus a
text control; a date-typed dimension bucketed over a formula of
`returnType: 'date'`; a sweep over every `FieldType` agreeing with
`SEARCH_VIRTUAL_TYPES` (floors on both sides); the later position beside
a stored dimension, never under the JSON-stored id; a dotted dimension
judged on the joined object; a cube dimension over a formula column plus
a stored control; the skips; the measure leg over a formula unchanged;
through `runAuthoringRules` on all three commands. The #22161 one-line
verdict / fix blocks now cover three ids.
- `lint` `runtime-gate.dataset-writes.test.ts`: a LIT case through the
real dataset write door (`runRuntimeAuthoringRules`), refusal at
`datasets.acme_invoice_metrics.dimensions[1].field`; `OBJECTS` gained
one formula field, every other case unchanged and green.
- `dogfood` `analytics-formula-dimension-authoring.dogfood.test.ts` on
the stock showcase (`showcase_project.budget_remaining`, a formula):
`PUT /meta/dataset` answers 422 naming the id at `dimensions[1].field`,
and `POST /analytics/dataset/query` with the inline dataset answers `400
DATASET_INVALID` naming the dimension and the remedy, for a selection
with and without the formula dimension; controls save and serve the
dataset without it. 5/5, plus the #29304 sibling 5/5.
<details><summary>Package runs</summary>
- At `b100c406` (the implementation commit, before the `origin/main`
merge; the commits after it touch only the dogfood pin and the
changeset, and the merge brought in no file of either package):
`@objectstack/service-analytics` full suite 211 files / 5015 passed, 300
skipped (lock VERDICT command-exit 0); `@objectstack/lint` full suite
195 files / 8236 passed (VERDICT 0); `typecheck` for both packages exit
0.
- Before the fix, the compile gate alone on the same suite: 1 file red,
`preview-date-bucket-type.test.ts`, 19 cases (the shared fixture
declared formula dimensions), which is what the fixture split answers.
- The dogfood pin with its #29304 sibling, 10/10 at the pre-tightening
head `6c77bf99` (the merge commit); the pin alone 5/5 at `0dd6d595`, and
both inside the full battery below.
</details>
<details><summary>Ablations (fix committed first;
`scripts/ablation-replace.mjs`, which restores to HEAD and checks `git
diff HEAD` is empty)</summary>
- Compile gate: deleted the `assertDimensionFieldStored(...)` call in
`compileDataset` (anchor 1 → 0, blob `9bf5497fcd8b` → `6e7d8fe55a4a`).
10 red: the three new request / configuration / bucket door cases in
`aggregate-nontemporal-measure-refusal.test.ts` and the seven `[#30773]`
cases in `preview-date-bucket-type.test.ts`; 66 green, including the
controls, the tiers and every measure pin. Restored: blob == HEAD.
- Lint dataset leg: replaced `if (isVirtualSearchField(shape)) {` with a
constant-false guard (blob `5f981455daa6` → `5e2fc683fa4f`). 7 red: six
cases of the new describe and the runtime write-door LIT case; the cube
case stayed green (its own leg). Restored: blob == HEAD.
- Lint cube leg: the same on `if (isVirtualSearchField(column.shape)) {`
(blob → `0e485f1cf802`). 1 red, the cube case. Restored: blob == HEAD.
- Both packages' tests import the subjects from `src/` by relative path,
so no `dist/` rebuild was needed.
</details>
## Acceptance notes
- **The preview's own virtual arm is now pre-empted for a bare dataset
dimension.** `queryDataset` compiles before the draft preview evaluates,
and both read the same `sourceFieldMeta` hook, so the preview's call to
`assertDatasetGroupTargetsStored` no longer sees a direct formula
dimension (and its `declaredType` reads the base object only, so it
never judged a dotted one). It is left in place as the date-bucket
door's partner; carrier: none (noted, not filed).
- **Docs, skills and the upgrade guide** (the seat's rulings on contract
review 6677341880): the `validating-metadata.mdx` paragraph is folded in
here at `a97ed9f0` as a surface addendum; the skills formula row is
follow-up #30824 (Tier H, the skills lane); an ADR-0087 semantic
upgrade-guide entry mirroring
`18.dataset-measure-aggregate-field-type-refused` is follow-up #30822
(the spec lane), to land before the release that ships this change.
- **The WIP commit `098b2b92` carries a `Claude-Session:` trailer naming
the wrong session id** (the coordinator's); later commits carry this
run's. The branch is a `claude/project-thread-*` branch, so it is not
rewritten; the seat ruled it has no landing effect (squash merge).
## Verification
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` at `0dd6d595` (after merging `origin/main`
at `172eebb2`, `pnpm install --frozen-lockfile` and a whole-tree `turbo
run build`, 119/119 tasks) derived 64 commands. All 64 ran and exited 0;
`--ran` reports 64 run, 0 NOT-MEASURED, 0 UNRUN.
`check-adr-0087-registration` reads the changeset as
`[BREAKING+bang+clause-②-narrowing+metadata-protocol-narrowed]
not-required (no-migration-prescription)`.
After the docs commit `a97ed9f0` (`content/docs` only), the derivation
grew to 89 families; the 25 new ones plus seven docs-reading families
from the first pass were run at `a97ed9f0`, all exit 0, and `--ran` over
the combined record reports 89 run, 0 NOT-MEASURED, 0 UNRUN. The other
families read no file the docs commit changed and keep their `0dd6d595`
readings.
Dogfood battery (title carries `!`): `0dd6d595` · `pnpm --filter
@objectstack/dogfood test` under the verify lock, VERDICT command-exit 0
· Test Files 424 passed, 3 skipped (427) · Tests 3537 passed, 41 skipped
(3578) · no failure.
---
_Generated by [Claude
Code](https://claude.ai/code/session_016WV4d1SZmabyrNd3YLCq3J)_
<!-- signed: domain:server#3 · os-elon-musk ·
session_016WV4d1SZmabyrNd3YLCq3J -->
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 1120389 commit b618224
11 files changed
Lines changed: 666 additions & 84 deletions
File tree
- .changeset
- content/docs/deployment
- packages
- lint/src
- qa/dogfood/test
- services/service-analytics/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
247 | 247 | | |
248 | 248 | | |
249 | 249 | | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
250 | 261 | | |
251 | 262 | | |
252 | 263 | | |
253 | 264 | | |
254 | | - | |
| 265 | + | |
| 266 | + | |
255 | 267 | | |
256 | 268 | | |
257 | 269 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| 67 | + | |
67 | 68 | | |
68 | 69 | | |
69 | 70 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2360 | 2360 | | |
2361 | 2361 | | |
2362 | 2362 | | |
| 2363 | + | |
| 2364 | + | |
| 2365 | + | |
| 2366 | + | |
| 2367 | + | |
| 2368 | + | |
| 2369 | + | |
| 2370 | + | |
| 2371 | + | |
| 2372 | + | |
| 2373 | + | |
| 2374 | + | |
| 2375 | + | |
| 2376 | + | |
| 2377 | + | |
| 2378 | + | |
| 2379 | + | |
| 2380 | + | |
| 2381 | + | |
2363 | 2382 | | |
2364 | 2383 | | |
2365 | 2384 | | |
| |||
5737 | 5756 | | |
5738 | 5757 | | |
5739 | 5758 | | |
| 5759 | + | |
5740 | 5760 | | |
5741 | 5761 | | |
5742 | 5762 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| 64 | + | |
| 65 | + | |
64 | 66 | | |
65 | 67 | | |
66 | 68 | | |
| |||
229 | 231 | | |
230 | 232 | | |
231 | 233 | | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
232 | 249 | | |
233 | 250 | | |
234 | 251 | | |
| |||
0 commit comments