Skip to content

Commit f64724a

Browse files
fix(service-analytics)!: the dataset draft preview refuses a runtimeFilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do (#28264)
<!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9DLMAJ9kZMBicdMLygftCdLd)_ domain:server#3 · os-elon-musk · session_016WV4cyQNVxpxREGfKkTsqs Fixes #28257 Clause-②: no (narrowing) — a draft-preview dataset query (`queryDataset` with `previewDrafts: true` over a pending seed draft) now refuses a `runtimeFilter` member whose field the object does not have, and a grouped dimension over a `json` column (#20807), both of which the native and ObjectQL strategies refuse and the preview served before. No export, answer member, status or error code is added: both refusals are the live faces' existing `INVALID_FIELD` / 400. Metadata-protocol: none ## 维护者速读 - 改了什么:数据集草稿预览遇到过滤对象上不存在的字段(如 `runtimeFilter: { bogus: 'x' }`),过去返回 `[]`,放在 `$or` 里还会返回另一分支的行;现在与实时两种策略一样按 400 `INVALID_FIELD`(`param: where`)拒,消息逐字相同。triage 并入的 #20807:按 `json` 字段分组,预览过去按对象值分组,实时拒;实测确有差异,同一 PR 一并对齐。 - 为什么改:草稿里过滤字段拼错只显示空图,发布后才被拒;这是同一接缝上 #28210(度量)、#28259(维度)之后的第三处缺口。 - 风险与回滚:只影响草稿预览;对真实字段的过滤(已声明维度、未声明的真实字段、`$in`)三面照旧返回同样的行,实时策略不变。回滚即 revert。 - 先看:`packages/services/service-analytics/src/analytics-service.ts` 里 `answerDataset` 的 `previewService.query`(新增两道闸门及其位置)。 - 两分钟验证:`pnpm --filter @objectstack/service-analytics exec vitest run src/__tests__/preview-filter-field-json-dimension-parity.test.ts`,预期 11 passed。 - 需要你做的:无,席位落地。 ## What changes **The draft preview's gate list (`analytics-service.ts`, `answerDataset` → `previewService.query`).** Two gates `ensureCube` asks on the live path are now asked on the preview, on the cube with the minted measures, in the order #28259 set up: 1. measure fields (#4437) 2. dimension fields (#5520) 3. **structured-JSON dimension door `assertDimensionsGroupScalarColumns` (#20807), new here** 4. aggregate × field-type (#21044) 5. record currency (ADR-0144 D6) 6. **where fields `assertWhereFields` (#5669), new here, last of `ensureCube`'s** 7. then `query()`'s: caller-measure column reference (#21156), stored body (#21120), internal fields, field-level read (#20917). The where gate takes the compiled cube's declared dimensions as its vocabulary, as live does, so a filter on a declared dimension or on any of the object's own fields stays legal. It reads every leaf of the normalized filter tree, so the object spelling, an `$or` arm and the array triple are refused alike. The JSON door refuses a `json` column whether the dataset declares the dimension over it or the caller names the field. `preview-evaluator.ts` is unchanged: the gates run before any seed row is read. ## Why Measured on `main` `4159c701`, real SQLite engine, object fields `region` (text), `amount` (number), `payload` (json); dataset dimensions `region`, `payload_dim` (over `payload`), measures `total` (sum of `amount`), `deals` (count): | selection | native | ObjectQL | draft preview (before) | draft preview (after) | |:---|:---|:---|:---|:---| | `runtimeFilter: { bogus: 'x' }` | `INVALID_FIELD` / 400, `param: where` | same | `[]` | same refusal, same message | | `runtimeFilter: { $or: [{ bogus: 1 }, { region: 'amer' }] }` | `INVALID_FIELD` / 400 | same | amer, 150 | same refusal, same message | | `runtimeFilter: [['bogus', '=', 'x']]` | `INVALID_FIELD` / 400 | same | `[]` | same refusal, same message | | filter `region`, `amount: { $gt: 40 }`, `region: { $in }` | rows | same rows | same rows | same rows | | dims `['payload']` (json, undeclared) | `INVALID_FIELD` / 400, `param: dimensions` | same | one group per object value | same refusal, same message | | dims `['payload_dim']` (json, declared) | `INVALID_FIELD` / 400, `param: dimensions` | same | one group per object value | same refusal, same message | | dims `['payload']` + filter `{ ghost: 1 }` | dimension 400 | dimension 400 | `[]` | dimension 400 | | `amount + 1` + filter `{ ghost: 1 }` | where 400 | where 400 | column-reference 403 | where 400 | ## Verification - `packages/services/service-analytics/src/__tests__/preview-filter-field-json-dimension-parity.test.ts` (new, 11 cases, each over native / ObjectQL / preview through `queryDataset` on real SQLite): the unknown filter field refused `INVALID_FIELD` / 400 with `param: 'where'`, `field` and a message identical on all three faces, in three spellings; CONTROLS: three real-field filters serve the same rows on all three; the json dimension refused with `param: 'dimensions'` and one message, declared and undeclared; three two-mistake cases answer the same refusal first on all three (json dimension + unknown filter → dimension; json dimension + `region_sum` → the JSON door ahead of the field-type door; `amount + 1` + unknown filter → where 400 ahead of the 403). - Ablations (source changed, the new file rerun): `analytics-service.ts` reset to `main` ⇒ 8 red (all but the three controls); `assertWhereFields` removed ⇒ 4 red (three spellings, `amount + 1` + filter); `assertDimensionsGroupScalarColumns` removed ⇒ 4 red (both json dimensions, both json two-mistake cases); the where gate moved after the caller-measure gate ⇒ 1 red (`amount + 1` + filter); the JSON door moved after the currency door ⇒ 1 red (json dimension + `region_sum`). - `pnpm --filter @objectstack/service-analytics exec vitest run --maxWorkers=2` at head: 200 files, 4806 passed, 282 skipped, 0 failed (the #28210 and #28259 parity files included). `pnpm --filter @objectstack/service-analytics typecheck` and eslint on the touched files clean. - Every `dispatch-gates --commands` family, `pnpm lint` and the dogfood battery run at this head: the readings are posted on the card with the os-dev report. Skills-docs: none. `git grep -n -i -E "previewDrafts|draft (data )?preview" -- skills content/docs` (references and releases excluded) finds only `content/docs/data-modeling/currency.mdx:146`, still true; `git grep -n runtimeFilter` finds `content/docs/data-modeling/analytics.mdx` and `content/docs/ui/{dashboards,reports}.mdx`, which describe the filter vocabulary and say the Studio preview shares the governed path ("so the numbers match everywhere", `analytics.mdx:263`), which this PR makes true for filters; nothing describes the preview serving an unknown filter field. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes. (No formatter ran; the lint reading is posted on the card with the gate battery.) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. - [x] 6. `packages/spec` touched: not touched. - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English (the 维护者速读 block is the lane's maintainer summary). --- _Generated by [Claude Code](https://claude.ai/code/session_016WV4cyQNVxpxREGfKkTsqs)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016WV4cyQNVxpxREGfKkTsqs Co-authored-by: Claude <noreply@anthropic.com>
1 parent ba5c754 commit f64724a

3 files changed

Lines changed: 212 additions & 0 deletions

File tree

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
'@objectstack/service-analytics': patch
3+
---
4+
5+
fix(service-analytics)!: the dataset draft preview refuses a runtimeFilter on a field the object does not have, and a dimension over a JSON column, as the live strategies do, instead of serving an empty or object-keyed answer
6+
7+
Clause-②: no (narrowing) — a draft-preview dataset query (`queryDataset` with `previewDrafts: true` over a pending seed draft) now refuses a `runtimeFilter` member whose field the object does not have, and a grouped dimension over a `json` column, both of which the native and ObjectQL strategies refuse and the preview served before. Nothing is widened: no export, answer member, error code or status is added.
8+
Metadata-protocol: none
9+
10+
<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes: the draft preview now answers an unknown filter field and a JSON dimension the way the published dataset already answers them. Both refusals are the live faces' existing ones, so there is nothing for a migration to rewrite and no tombstone. -->
11+
12+
**BREAKING** accept-set narrowing on the draft preview of `queryDataset` (`@objectstack/service-analytics`).
13+
14+
- **Before.** The preview evaluated a `runtimeFilter` over the seed rows without asking whether the object has each filtered field. A misspelled `{ bogus: 'x' }` matched no row and served `[]`; under an `$or`, `{ $or: [{ bogus: 1 }, { region: 'amer' }] }` served the other arm's rows. Both live strategies refuse it `INVALID_FIELD` / 400 with `param: 'where'` ("Filter member 'bogus' in 'where' on cube … constrains field 'bogus', which object … does not have …"). The preview also grouped by a dimension over a `json` column, one group per object value, where live refuses it `INVALID_FIELD` / 400 with `param: 'dimensions'` (#20807).
15+
- **Now.** The preview asks both live gates before it reads a seed row, so each mistake is refused with the same envelope and message on all three faces: the filter in every spelling (object, `$or` arm, array triple), the JSON dimension whether the dataset declares it or not.
16+
- **Gate order.** Both gates sit where `ensureCube` asks them: the structured-JSON door after the dimension-field gate and before the aggregate × field-type door, the `where` gate last among `ensureCube`'s gates (after the record-currency door) and before `query()`'s caller-measure column-reference gate. So a query with two mistakes is answered about the same one first on every face: a JSON dimension beside an unknown filter field answers the dimension; `amount + 1` beside an unknown filter field answers the filter (400), where the preview answered the column-reference gate (403).
17+
- **Unchanged.** A filter on a declared dimension, on an undeclared field the object has (`amount: { $gt: 40 }`) and an `$in` filter serve the same rows as before on every face. The live strategies are unchanged.
18+
- **Who sees it.** A Studio author whose draft widget filters on a field name the object does not have, or groups by a JSON field: the mistake now shows the refusal it meets once published, instead of an empty chart or one bar per object value.
19+
20+
**The remedy.** Filter on one of the object's own fields, spelled as the object declares it; group by a scalar field, not a `json` one.
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#28257] A `runtimeFilter` on a field the dataset object does not have, and
5+
* a dimension over a structured-JSON column (#20807), through `queryDataset` on
6+
* the native strategy, the ObjectQL strategy and the draft preview.
7+
*
8+
* ## What these pins hold
9+
*
10+
* - A filter member whose field the object does not have is refused
11+
* `INVALID_FIELD` / 400 with `param: 'where'` on every face, with the same
12+
* message, whether it is the whole filter, one arm of an `$or` or the array
13+
* spelling. Before, the preview matched no row and served `[]`, or under the
14+
* `$or` served the other arm's rows.
15+
* - A dimension over a `json` column, declared or not, is refused
16+
* `INVALID_FIELD` / 400 with `param: 'dimensions'` on every face, with the
17+
* same message. Before, the preview grouped by each row's object value.
18+
* - CONTROL: a filter on a declared dimension, on an undeclared real field and
19+
* an `$in` filter serve the same rows on every face.
20+
* - The preview asks both gates in the live order: the JSON door after the
21+
* dimension-field gate and before the field-type door, the `where` gate last
22+
* among `ensureCube`'s and before the column-reference gate.
23+
*/
24+
25+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
26+
import { ObjectQL } from '@objectstack/objectql';
27+
import { SqlDriver } from '@objectstack/driver-sql';
28+
import { DatasetSchema } from '@objectstack/spec/ui';
29+
import type { AnalyticsService } from '../analytics-service.js';
30+
import { AnalyticsServicePlugin } from '../plugin.js';
31+
import { installDefaultOrganization } from './default-organization.testkit.js';
32+
33+
const OBJECT = 'os28257_deal';
34+
35+
const ROWS = [
36+
{ id: 'd1', region: 'amer', amount: 100, payload: { k: 1 } },
37+
{ id: 'd2', region: 'amer', amount: 50, payload: { k: 2 } },
38+
{ id: 'd3', region: 'emea', amount: 30, payload: { k: 1 } },
39+
] as const;
40+
41+
const DATASET = DatasetSchema.parse({
42+
name: 'os28257_deals',
43+
label: 'Deals',
44+
object: OBJECT,
45+
dimensions: [
46+
{ name: 'region', field: 'region', type: 'string' },
47+
{ name: 'payload_dim', field: 'payload', type: 'string' },
48+
],
49+
measures: [
50+
{ name: 'total', aggregate: 'sum', field: 'amount' },
51+
{ name: 'deals', aggregate: 'count' },
52+
],
53+
});
54+
55+
type Face = 'native' | 'objectql' | 'preview';
56+
const FACES: readonly Face[] = ['native', 'objectql', 'preview'];
57+
58+
const quiet = { debug() {}, info() {}, warn() {}, error() {}, trace() {}, fatal() {}, child() { return quiet; } };
59+
60+
type Refusal = Error & { code?: string; status?: number; param?: string; field?: string };
61+
62+
const refusalOf = (p: Promise<unknown>): Promise<Refusal> =>
63+
p.then<never, Refusal>(
64+
() => {
65+
throw new Error('expected a refusal, but the query resolved');
66+
},
67+
(e) => e as Refusal,
68+
);
69+
70+
/** A `'protocol'` service answering one pending seed draft, which routes `previewDrafts` onto the preview path. */
71+
const pendingSeedProtocol = {
72+
getMetaItems: async () => [{ name: 'os28257_seed', object: OBJECT }],
73+
getMetaItem: async () => ({ item: { records: ROWS } }),
74+
};
75+
76+
describe('[#28257] an unknown filter field and a json dimension on the native, ObjectQL and draft-preview faces', () => {
77+
let engine: ObjectQL;
78+
const services: Partial<Record<'native' | 'objectql', AnalyticsService>> = {};
79+
80+
const ask = (face: Face, selection: Record<string, unknown>) =>
81+
services[face === 'preview' ? 'objectql' : face]!.queryDataset(
82+
DATASET,
83+
selection as never,
84+
undefined,
85+
face === 'preview' ? { previewDrafts: true } : undefined,
86+
);
87+
88+
beforeAll(async () => {
89+
const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true } as never);
90+
engine = new ObjectQL({ logger: quiet } as never);
91+
engine.registerDriver(driver as never, true);
92+
await engine.init();
93+
engine.registry.registerObject({
94+
name: OBJECT,
95+
label: 'Deal',
96+
fields: {
97+
region: { name: 'region', type: 'text' },
98+
amount: { name: 'amount', type: 'number' },
99+
payload: { name: 'payload', type: 'json' },
100+
},
101+
} as never);
102+
await engine.syncSchemas();
103+
await installDefaultOrganization(engine as never);
104+
for (const row of ROWS) await engine.insert(OBJECT, { ...row } as never);
105+
106+
for (const [face, caps] of [
107+
['native', undefined],
108+
['objectql', () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false })],
109+
] as const) {
110+
const registered: Record<string, unknown> = {};
111+
const provided: Record<string, unknown> = { data: engine, protocol: pendingSeedProtocol };
112+
await new AnalyticsServicePlugin({ ...(caps ? { queryCapabilities: caps } : {}) } as never).init({
113+
getService: (name: string) => provided[name] ?? registered[name],
114+
registerService: (name: string, svc: unknown) => { registered[name] = svc; },
115+
replaceService: (name: string, svc: unknown) => { registered[name] = svc; },
116+
hook: () => {},
117+
logger: quiet,
118+
} as never);
119+
services[face] = registered.analytics as AnalyticsService;
120+
}
121+
});
122+
123+
afterAll(async () => {
124+
try { await engine?.destroy(); } catch { /* noop */ }
125+
});
126+
127+
const refusedAlike = async (selection: Record<string, unknown>, param: string, field: string) => {
128+
const refusals = await Promise.all(FACES.map((face) => refusalOf(ask(face, selection))));
129+
for (const err of refusals) {
130+
expect(err.code, err.message).toBe('INVALID_FIELD');
131+
expect(err.status).toBe(400);
132+
expect(err.param).toBe(param);
133+
expect(err.field).toBe(field);
134+
}
135+
const [native, objectql, preview] = refusals;
136+
expect(preview.message).toBe(native.message);
137+
expect(objectql.message).toBe(native.message);
138+
return native;
139+
};
140+
141+
for (const [label, runtimeFilter] of [
142+
['the whole filter', { bogus: 'x' }],
143+
['one arm of an $or', { $or: [{ bogus: 1 }, { region: 'amer' }] }],
144+
['the array spelling', [['bogus', '=', 'x']]],
145+
] as const) {
146+
it(`refuses a filter on 'bogus' as ${label}, INVALID_FIELD / 400 param where, with one message on every face`, async () => {
147+
const err = await refusedAlike({ dimensions: ['region'], measures: ['total'], runtimeFilter }, 'where', 'bogus');
148+
expect(err.message).toContain("Filter member 'bogus' in 'where' on cube 'os28257_deals' constrains field 'bogus'");
149+
});
150+
}
151+
152+
for (const [label, runtimeFilter, expected] of [
153+
['a declared dimension', { region: 'amer' }, [{ key: 'amer', total: 150 }]],
154+
['an undeclared real field', { amount: { $gt: 40 } }, [{ key: 'amer', total: 150 }]],
155+
['an $in on a declared dimension', { region: { $in: ['emea'] } }, [{ key: 'emea', total: 30 }]],
156+
] as const) {
157+
it(`CONTROL: a filter on ${label} serves the same rows on every face`, async () => {
158+
for (const face of FACES) {
159+
const result = await ask(face, { dimensions: ['region'], measures: ['total'], runtimeFilter });
160+
const rows = (result.rows as Array<Record<string, unknown>>).map((r) => ({ key: r.region, total: Number(r.total) }));
161+
expect(rows, face).toEqual(expected);
162+
}
163+
});
164+
}
165+
166+
for (const dimension of ['payload', 'payload_dim'] as const) {
167+
it(`refuses the json dimension '${dimension}' INVALID_FIELD / 400 param dimensions, with one message on every face`, async () => {
168+
const err = await refusedAlike({ dimensions: [dimension], measures: ['total'] }, 'dimensions', 'payload');
169+
expect(err.message).toContain('declares as json');
170+
});
171+
}
172+
173+
for (const [label, selection, param, field] of [
174+
['a json dimension beside a bogus filter: the json door', { dimensions: ['payload'], measures: ['total'], runtimeFilter: { ghost: 1 } }, 'dimensions', 'payload'],
175+
["a json dimension beside 'region_sum': the json door", { dimensions: ['payload'], measures: ['region_sum'] }, 'dimensions', 'payload'],
176+
["'amount + 1' beside a bogus filter: the where gate", { dimensions: ['region'], measures: ['amount + 1'], runtimeFilter: { ghost: 1 } }, 'where', 'ghost'],
177+
] as const) {
178+
it(`two mistakes, ${label} first, on every face`, async () => {
179+
await refusedAlike(selection, param, field);
180+
});
181+
}
182+
});

‎packages/services/service-analytics/src/analytics-service.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2772,6 +2772,10 @@ export class AnalyticsService implements IAnalyticsService {
27722772
// that names a real field (`amount`) groups by it, here as there.
27732773
this.assertMeasureFields(q, cube, Object.keys(compiled.cube.measures));
27742774
this.assertDimensionFields(q, cube, Object.keys(compiled.cube.dimensions));
2775+
// [#28257] …and the structured-JSON dimension door (#20807): a
2776+
// grouped dimension over a `json` column is refused there, where
2777+
// the evaluator grouped by each row's object value.
2778+
this.assertDimensionsGroupScalarColumns(q, cube);
27752779
// [#21044] …and the aggregate × field-type table: a minted
27762780
// `region_sum` over a text field is refused there, and here it
27772781
// would sum no number at all and read `0`.
@@ -2780,6 +2784,12 @@ export class AnalyticsService implements IAnalyticsService {
27802784
// must not serve the sum across currencies that the published
27812785
// dataset refuses.
27822786
this.assertRecordCurrencyGrouped(q, cube);
2787+
// [#28257] …and the `where` field gate (#5669), last in
2788+
// `ensureCube`: a filter member whose field the object does not
2789+
// have (a `runtimeFilter` typo) is refused `INVALID_FIELD` / 400
2790+
// with `param: 'where'`, where the evaluator matched no row and
2791+
// served `[]` — or, under an `$or`, served the other arm's rows.
2792+
this.assertWhereFields(q, cube, Object.keys(compiled.cube.dimensions));
27832793
// [#21156] `query()` asks the caller-member gate after `ensureCube`.
27842794
this.assertCallerMeasuresResolvable(q, compiled.cube, compiled.cube, compiled.durations, context);
27852795
// [#21120] The body-column refusal rides this preview path too —

0 commit comments

Comments
 (0)