Skip to content

Commit c470581

Browse files
fix(service-analytics): a retired filter operator ($regex, $options) in an analytics filter is refused with the retirement's prescription naming $icontains (#29017)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XWZob23RfrTqvRrnhnhhLrm)_ Fixes #28945 Clause-②: no Metadata-protocol: none ## 维护者速读 - **改了什么**:分析查询(`/analytics/dataset/query` 的 `runtimeFilter`、`/analytics/query`、`/analytics/sql`、草稿预览)遇到已退役的 `$regex` / `$options` 时,仍答 400 `INVALID_FILTER`,但文案改成和数据接口一样:说明该操作符已退役、写 `Write "$icontains" instead.`,再逐字附上退役表里的处方。 - **为什么改**:升级后的仪表盘若还存着 `$regex` 过滤,原来只看到一长串"支持的操作符",找不到一行就能改好的写法;数据接口早已给出 `$icontains`,分析接口应当说同一句话。 - **风险与回滚**:只改报错文字,拒收范围、状态码、错误码都不变,`patch`。新检查放在过滤器归一化的最后一步,已有的拒收都先于它答复,未知操作符仍是原来的通用列表。回滚即还原本 PR。 - **先看**:`packages/services/service-analytics/src/strategies/filter-normalizer.ts`(`retiredFilterOperatorError`、`assertNoRetiredOperators`、`normalizeWhereComparands` 末尾的调用);`src/__tests__/retired-filter-operator-refusal.test.ts`;`packages/qa/dogfood/test/analytics-retired-filter-operator.dogfood.test.ts`。 - **两分钟验证**:`pnpm --filter @objectstack/service-analytics exec vitest run src/__tests__/retired-filter-operator-refusal.test.ts`,预期 18 个全绿,其中各个分析入口对 `$regex` 给出同一句 RETIRED 文案。 - **需要你做的**:无,席位落地。 ## What changes `fieldLeaves` in `filter-normalizer.ts` refused `$regex` / `$options` with the right envelope (`INVALID_FILTER` / 400) but with the generic `Unsupported filter operator "$regex" on "name". Supported: …` list, because it never read `RETIRED_FILTER_OPERATORS`. The draft preview answered with its own `… is not evaluated by the draft-data preview` sentence instead. So the faces disagreed, and none of them named `$icontains`. - **`retiredFilterOperatorError(op, field, siblings)`** builds the refusal the way each data-door face builds it: `[analytics] Filter operator "OP" on field "FIELD" is RETIRED and is no longer evaluated.`, then ` Write "TO" instead.`, then `RETIRED_FILTER_OPERATORS[op].why` word for word, then every other retired key of the same constraint (`{ $regex, $options }` is one mistake with one fix). The lead-in is `driver-sql`'s (the data door on a SQL stack) without its "by this driver". It returns `null` for a non-retired operator. - **`assertNoRetiredOperators(node)`** walks the condition with the shared `forEachWhereFieldEntry` traversal, the same positions the shape, type and flag passes judge: `$and` / `$or` members, `$not`, field entries, and nested relation entries. - **One call for every face.** It is the last pass of `normalizeWhereComparands`, which `lowerAnalyticsWhere` runs for the native execute, the `/analytics/sql` echo and the ObjectQL engine path. The draft preview calls it itself, before its own operator vocabulary. Dataset scope filters and measure filters go through the same door. - **Last, so no existing refusal moves.** A shape, type or non-boolean flag defect anywhere in the same `where` is still answered first. The only answer that changes is the one that used to fall to the unsupported-operator list. An unknown operator such as `$sounds_like` keeps the generic list. - **No path in the lead-in.** A client reads at most 500 characters of a refusal (`CLIENT_MESSAGE_MAX`), and the table's `why` puts everything an author must write in its opening sentences for that budget (#27098). A test pins `Write "$icontains" instead.`, `` `$contains` for a case-sensitive one `` and `match in application code` inside 500 characters. **On the order's "the same message builder".** No shared builder exists on main. The data door's five faces (`driver-sql`, `driver-memory`, `driver-mongodb`, `driver-turso`'s remote transport, `objectql`'s `having`) each compose the lead-in themselves around the shared `to` / `why`, and none of those builders is exported from a package `service-analytics` depends on (a driver dependency is the wrong direction). So this follows the same form from the same table, and no word of the prescription is written in this package. Folding all six into one spec-side builder would be a separate change (spec export, five drivers). It is noted below, not done here. ## Tests Measured at HEAD `49e31fc2` (worktree on `claude/project-thread-jebtpj`, base `135054eb`). - `src/__tests__/retired-filter-operator-refusal.test.ts` (new, 18 tests): - The `where` door: `$regex` and a dangling `$options` each give `code` + `status` + the lead-in + `Write "$icontains" instead.` + the table's `why` verbatim. `{ $regex, $options }` names the other key. The refusal also fires under `$not`, in an `$or` beside a TRUE arm, in an `$and` and on a dotted member, and the client-cap budget holds. Precedence: a type-face defect in the same `where` still answers first. - Every face over a real sql.js engine (native execute, `/analytics/sql` echo, ObjectQL engine path, `AnalyticsService.query` native and ObjectQL, `AnalyticsService.generateSql`, draft preview): each gives the message the door gives, byte for byte, with zero statements and zero engine calls. - Data-door parity: `engine.find` with the same `where` (marked `'author'`, as the SecurityPlugin marks a caller's `where`) refuses with the same tail from ` Write "$icontains" instead.` on. - `queryDataset`: `runtimeFilter`, `runtimeFilter` with `previewDrafts`, the dataset scope filter and a measure filter all give the prescription, and no statement runs. - Controls: `$icontains` is served (`['a1','a2']`) on every live face. `$sounds_like` keeps the generic list, and the draft preview keeps its own sentence for it. The `FilterArray` spelling `['name','regex','acme']` is still refused at `isFilterAST`. - `src/__tests__/filter-normalizer-not-null-safe.test.ts`: the #5325 pin "an unknown operator inside a `$not` still THROWS" used `$regex` as its example of an unknown operator. It now uses `$sounds_like` for the generic list and pins `$regex` beside it as the RETIRED refusal. The pin's intent (fail-closed under `$not`) is unchanged. - `packages/qa/dogfood/test/analytics-retired-filter-operator.dogfood.test.ts` (new, booted stock showcase): `POST /api/v1/analytics/dataset/query` `{ datasetName: "showcase_project_metrics", selection: { measures: ["project_count"], runtimeFilter: { name: { $regex: "acme" } } } }`, and the same with `$options`, answer 400 `INVALID_FILTER` with the RETIRED lead-in, `Write "$icontains" instead.` and the table's opening sentence. The data door `POST /api/v1/data/showcase_project/query` with the same `where` shows the same tail. Controls: `$icontains` → 200, `$sounds_like` → the generic list. <details><summary>Red first on main: unit suite and the live showcase door</summary> The base blob of `filter-normalizer.ts` (`135054eb`) was written over the file, under a trap restore. - Unit: `Tests 11 failed | 7 passed (18)`. The 11 failures are every RETIRED case. The live faces answered `[analytics] Unsupported filter operator "$regex" on "name". Supported: $eq, $ne, …`, and the draft preview answered `[analytics] Filter operator "$regex" on field "name" is not evaluated by the draft-data preview. …`. The 7 that passed are the controls plus the 500-character case, which passed vacuously on a message that held none of the texts. It was then tightened to require each text to be present (so the ablation below reds it). - Live showcase (dogfood file, `service-analytics` rebuilt from the base blob, `ablation-dist-preflight … --absent` confirmed the marker gone from all 6 built files): `Tests 3 failed | 2 passed (5)`. `$regex` and `$options` got `400 INVALID_FILTER` `[analytics] Unsupported filter operator …`, which is the card's reported answer. The parity case failed, and the two controls passed. Restore: blob hash equals HEAD's and `git diff HEAD` is empty. Then a rebuild, and the preflight showed the marker present in 2 built files. </details> <details><summary>Ablation: dropping the retired-table pass reds only the new cases</summary> `node scripts/ablation-replace.mjs --file …/filter-normalizer.ts --anchor ' assertNoRetiredOperators(node, path);\n' --delete -- …`: anchor 1 → 0, blob `34fb0eb6` → `e59a5023`, `calls-left: 0`. The unit file imports the module from `src/` (no dist involved). - `Tests 12 failed | 6 passed (18)`. The 12 failures are every RETIRED case. The 6 that passed are the controls (`$sounds_like` generic list, the `FilterArray` refusal, the `$icontains` leaf and rows, the preview's own sentence for an unknown operator, and type-face precedence). - Restore: blob after restore `34fb0eb6` == HEAD, `git diff HEAD` empty. </details> <details><summary>Package suite, typecheck, dogfood file</summary> All at HEAD `49e31fc2`, through `scripts/pm/os-verify-lock.sh` (`VERDICT command-exit 0`): - `pnpm --filter @objectstack/service-analytics build`, then `ablation-dist-preflight.mjs @objectstack/service-analytics 'is RETIRED and is no longer evaluated.'`: marker present in 2 built files - `pnpm --filter @objectstack/service-analytics test`: `Test Files 205 passed (205)`, `Tests 4907 passed | 282 skipped (5189)` - `pnpm --filter @objectstack/service-analytics typecheck`: exit 0 (`--listFilesOnly` includes both test files) - `pnpm --filter @objectstack/dogfood exec vitest run test/analytics-retired-filter-operator.dogfood.test.ts`: `Tests 5 passed (5)` - `pnpm --filter @objectstack/dogfood typecheck`: exit 0 (`--listFilesOnly` includes the new dogfood file) - Full dogfood battery: not run (no `!` in the title and no `(narrowing)` arm). The dogfood job does not run on PRs; the new file was run locally above. </details> ## Acceptance notes - **Premise** held as stated (triage `6569498837`). It was measured on main through the unit faces and the live showcase door. - **Declared conflict with the order.** "The same message builder" assumes a shared builder. There is none (see What changes). The sentence follows the data door's form from the same table, and no prescription text lives here. - **Observation, not filed: the analytics door serves a refusal message whole, while the data door cuts it at 500 characters with `…`.** Measured on the dogfood showcase: the `$regex` refusal is 1,068 characters on `/analytics/dataset/query` and 500 on `/data/showcase_project/query`. Nothing is wrong in what either door says. Whether analytics should apply `CLIENT_MESSAGE_MAX` is not this card's question. - **Not touched:** the read scope (`read-scope-sql.ts`). It is policy-authored, so its refusal stays `READ_SCOPE_COMPILE_FAILED` / 500 with the message withheld (#5367), and a retired operator there keeps its fail-closed sentence. A top-level `{ $regex: … }` (not a field constraint) keeps "Unsupported top-level filter operator", as the data door's faces have no retired arm there either. NOT MEASURED: what the data door says for a `FilterArray` `['name', 'regex', 'acme']`. This door refuses it at `isFilterAST`, unchanged. - **Possible follow-up (carrier: none):** one spec-side builder for the retired-operator lead-in, so the six faces stop composing it separately. That is a spec export plus five drivers, and none of them is wrong today. - **Overlap.** Open PR #28976 edits `service-analytics/src/strategies/objectql-strategy.ts` (the draft preview's date bucket). No hunk meets this PR's files. No other open PR touches `filter-normalizer.ts` or the two service-analytics test files. All 27 open PRs were checked by file list. - **Precedence change, declared:** none for any existing refusal. A `where` whose only defect is a retired key now gets the RETIRED sentence instead of the generic list. The draft preview now gives the published faces' sentence instead of its own for those two keys. ## Verification Derived gate families: `node scripts/pm/dispatch-gates.mjs --commands` at `49e31fc2` (5 paths vs merge base `135054eb`) gives 64 commands. All 64 ran locally and exited 0. `dispatch-gates --ran`: `64 derived famil(ies) accounted for — 64 run, 0 NOT-MEASURED`. The first run of `check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow clone. After `git fetch --depth=1 origin 621a487…` it exited 0 (48 cases). CI owns the 50 artifact-roster families, the wide-population families, the type-check lanes and `pnpm lint`; their verdicts are on this PR's check-runs. <details><summary>Gate record (COMMAND :: exit N), HEAD 49e31fc</summary> ``` node scripts/check-adr-0087-registration.mjs --base origin/main :: exit 0 node scripts/check-adr-0087-registration.mjs --self-test :: exit 0 node scripts/check-changeset-no-major.mjs --base origin/main :: exit 0 node scripts/check-closing-keyword-parity.mjs :: exit 0 node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0 node scripts/check-comment-mask-adoption.mjs :: exit 0 node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0 node scripts/check-comment-mask-corpus.mjs :: exit 0 node scripts/check-dts-emitted.mjs --self-test :: exit 0 node scripts/check-empty-changeset.mjs --base origin/main :: exit 0 node scripts/check-empty-changeset.mjs --self-test :: exit 0 node scripts/check-issue-citations.mjs :: exit 0 node scripts/check-plugin-teardown-shape.mjs :: exit 0 node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0 node scripts/check-tenant-audit-census.mjs :: exit 0 node scripts/check-tenant-audit-census.mjs --self-test :: exit 0 node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0 node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0 node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0 node scripts/docs-audit/check-affected-docs.mjs :: exit 0 node scripts/docs-audit/check-drift-comment.mjs :: exit 0 node scripts/pm/release-rehearsal-clone.mjs --self-test :: exit 0 node scripts/release-pending-publish.mjs --self-test :: exit 0 pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0 pnpm --filter @objectstack/spec run check:empty-state :: exit 0 pnpm --filter @objectstack/spec run check:liveness :: exit 0 pnpm --filter @objectstack/spec run check:strictness-ledger :: exit 0 pnpm --filter @objectstack/spec run check:variant-docs :: exit 0 pnpm check:changeset-gate-self-tests :: exit 0 pnpm check:ci-filter-parity :: exit 0 pnpm check:closed-ceremony-text :: exit 0 pnpm check:cross-package-test-inputs :: exit 0 pnpm check:doc-authoring :: exit 0 pnpm check:driver-memory-census :: exit 0 pnpm check:dts-closure :: exit 0 pnpm check:dual-build-cjs-loads :: exit 0 pnpm check:engine-double-contract :: exit 0 pnpm check:error-status-conformance :: exit 0 pnpm check:gitlink-declared :: exit 0 pnpm check:issue-citations :: exit 0 pnpm check:keyed-text-bounds :: exit 0 pnpm check:lean-entry-closure :: exit 0 pnpm check:logger-receiver-detach :: exit 0 pnpm check:nul-bytes :: exit 0 pnpm check:objectql-double-limit :: exit 0 pnpm check:org-identifier :: exit 0 pnpm check:page-declaration-shape :: exit 0 pnpm check:platform-object-tenancy-census :: exit 0 pnpm check:pm-changeset-deadline-census :: exit 0 pnpm check:published-files :: exit 0 pnpm check:query-options-erasure :: exit 0 pnpm check:refd-timer-probe :: exit 0 pnpm check:registry-log-declared :: exit 0 pnpm check:rest-log-spy-declared :: exit 0 pnpm check:slot-lookup :: exit 0 pnpm check:sourcemap-no-sources-content :: exit 0 pnpm check:system-context-census :: exit 0 pnpm check:test-source-alias :: exit 0 pnpm check:tier-file-adoption :: exit 0 pnpm check:type-check-coverage :: exit 0 pnpm check:type-check-debt :: exit 0 pnpm check:undeclared-dep-imports :: exit 0 pnpm check:watch-hint-literal :: exit 0 pnpm check:where-matcher :: exit 0 ``` </details> --- _Generated by [Claude Code](https://claude.ai/code/session_01FEkQ5oAC7wmJvoM1VnnCjg)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6b3ba82 commit c470581

5 files changed

Lines changed: 547 additions & 1 deletion

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/service-analytics': patch
3+
---
4+
5+
fix(service-analytics): a retired filter operator (`$regex`, `$options`) in an analytics `where` or dataset `runtimeFilter` is refused with the retirement's prescription naming `$icontains`, as the data door refuses it (#28945)
6+
7+
Clause-②: no
8+
Metadata-protocol: none
9+
10+
The analytics filter already refused `$regex` and `$options` with `INVALID_FILTER` / 400, but its message was the generic "Unsupported filter operator … Supported: …" list. The message now says the operator is RETIRED, says `Write "$icontains" instead.`, and then prints the retirement prescription from `RETIRED_FILTER_OPERATORS` word for word, the way the data door's refusal does. When one field constraint carries both keys, the message names both. Every analytics face answers it from one check: `/analytics/query`, `/analytics/dataset/query` (runtime, dataset and measure filters), `/analytics/sql` and the draft preview. The status, the error code and what is refused are unchanged, and an unknown operator keeps the generic list.
11+
12+
What to do: nothing. A stored widget filter that still uses `$regex` was refused before this change and is still refused. The message now names the fix: write `$icontains`.
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// END-TO-END gate for #28945: a RETIRED filter operator in a dashboard
4+
// widget's `runtimeFilter` is refused on `POST /analytics/dataset/query` with
5+
// the retirement's prescription, the way the data door refuses it — proven on
6+
// the stock showcase, through the real stack (HTTP → REST → analytics plugin →
7+
// the analytics `where` door).
8+
//
9+
// The D3 retirement entry `filter-regex-options-retired` lists stored dashboard
10+
// widget filters as a stored site and promises `$regex` / `$options` are
11+
// "refused everywhere with INVALID_FILTER / 400 naming the replacement". The
12+
// analytics door gave the envelope with a generic "Unsupported filter operator
13+
// … Supported: …" list instead (measured on this file's base). It now answers
14+
// "… is RETIRED … Write "$icontains" instead", then the spec table's
15+
// prescription, word for word what the data door prints after its own lead-in.
16+
//
17+
// Controls: `$icontains` is served (200), and an unknown operator keeps the
18+
// generic list.
19+
20+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
21+
import { RETIRED_FILTER_OPERATORS } from '@objectstack/spec/data';
22+
import { type VerifyStack } from '@objectstack/verify';
23+
import { bootShowcase } from './showcase-boot.js';
24+
25+
const DATASET = 'showcase_project_metrics';
26+
27+
describe('dogfood: a retired filter operator in an analytics runtimeFilter names its replacement (#28945)', () => {
28+
let stack: VerifyStack;
29+
let token: string;
30+
31+
beforeAll(async () => {
32+
stack = await bootShowcase();
33+
token = await stack.signIn();
34+
}, 180_000);
35+
36+
afterAll(async () => {
37+
await stack?.stop?.();
38+
});
39+
40+
async function datasetQuery(runtimeFilter: unknown): Promise<{ status: number; body: Record<string, any> }> {
41+
const res = await stack.apiAs(token, 'POST', '/analytics/dataset/query', {
42+
datasetName: DATASET,
43+
selection: { measures: ['project_count'], runtimeFilter },
44+
});
45+
return { status: res.status, body: (await res.json()) as Record<string, any> };
46+
}
47+
48+
/** The prescription the data door prints after its own lead-in. */
49+
const tail = (message: string): string => message.slice(message.indexOf(' Write "'));
50+
51+
it('CONTROL: $icontains is served', async () => {
52+
const res = await datasetQuery({ name: { $icontains: 'acme' } });
53+
expect(res.status, JSON.stringify(res.body)).toBe(200);
54+
});
55+
56+
for (const op of ['$regex', '$options'] as const) {
57+
it(`${op} → 400 INVALID_FILTER, RETIRED, "Write "$icontains" instead", the table's prescription`, async () => {
58+
const res = await datasetQuery({ name: { [op]: op === '$regex' ? 'acme' : 'i' } });
59+
expect(res.status, JSON.stringify(res.body)).toBe(400);
60+
expect(res.body.code).toBe('INVALID_FILTER');
61+
const message = String(res.body.message);
62+
expect(message).toContain(`Filter operator "${op}" on field "name" is RETIRED`);
63+
expect(message).toContain(`Write "${RETIRED_FILTER_OPERATORS[op].to}" instead.`);
64+
expect(message).not.toContain('Unsupported filter operator');
65+
// The opening of the prescription, inside what a client reads.
66+
expect(message).toContain(RETIRED_FILTER_OPERATORS[op].why.slice(0, 80));
67+
});
68+
}
69+
70+
it('the same prescription the data door gives for the same key', async () => {
71+
const analytics = await datasetQuery({ name: { $regex: 'acme' } });
72+
const data = await stack.apiAs(token, 'POST', '/data/showcase_project/query', { where: { name: { $regex: 'acme' } } });
73+
const dataBody = (await data.json()) as Record<string, any>;
74+
expect(data.status, JSON.stringify(dataBody)).toBe(400);
75+
expect(dataBody.code).toBe('INVALID_FILTER');
76+
const dataMessage = String(dataBody.error);
77+
expect(dataMessage).toContain(' Write "$icontains" instead.');
78+
// The data door cuts its client message at 500 characters with `…`; the
79+
// analytics door serves the message whole. What the data door shows is a
80+
// prefix of what the analytics door shows, from the replacement on.
81+
const shown = tail(dataMessage).replace(/…$/, '');
82+
expect(shown.length).toBeGreaterThan(200);
83+
expect(tail(String(analytics.body.message)).startsWith(shown)).toBe(true);
84+
});
85+
86+
it('CONTROL: an unknown operator keeps the generic list', async () => {
87+
const res = await datasetQuery({ name: { $sounds_like: 'acme' } });
88+
expect(res.status, JSON.stringify(res.body)).toBe(400);
89+
expect(res.body.code).toBe('INVALID_FILTER');
90+
expect(String(res.body.message)).toContain('Unsupported filter operator "$sounds_like"');
91+
});
92+
});

‎packages/services/service-analytics/src/__tests__/filter-normalizer-not-null-safe.test.ts‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -721,7 +721,9 @@ describe('[#5325] analytics `where` — NULL-safe `$not` and the boolean identit
721721
});
722722

723723
it('an unknown operator inside a `$not` still THROWS rather than being guarded', async () => {
724-
await expect(ids({ $not: { stage: { $regex: '.*' } } })).rejects.toThrowError(/Unsupported filter operator/);
724+
await expect(ids({ $not: { stage: { $sounds_like: '.*' } } })).rejects.toThrowError(/Unsupported filter operator/);
725+
// [#28945] A RETIRED operator throws too, with the retirement's prescription.
726+
await expect(ids({ $not: { stage: { $regex: '.*' } } })).rejects.toThrowError(/"\$regex" on field "stage" is RETIRED/);
725727
await expect(ids({ $not: { $nor: [{ stage: 'won' }] } })).rejects.toThrowError(/Unsupported top-level filter operator/);
726728
});
727729

0 commit comments

Comments
 (0)