Repository navigation
Commit c470581
fix(service-analytics): a retired filter operator ($regex, $options) in an analytics filter is refused with the retirement's prescription naming $icontains (#29017)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"}
-->
_Requested by **Leehom** · [project
thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XWZob23RfrTqvRrnhnhhLrm)_
Fixes #28945
Clause-②: no
Metadata-protocol: none
## 维护者速读
- **改了什么**:分析查询(`/analytics/dataset/query` 的
`runtimeFilter`、`/analytics/query`、`/analytics/sql`、草稿预览)遇到已退役的 `$regex`
/ `$options` 时,仍答 400 `INVALID_FILTER`,但文案改成和数据接口一样:说明该操作符已退役、写 `Write
"$icontains" instead.`,再逐字附上退役表里的处方。
- **为什么改**:升级后的仪表盘若还存着 `$regex`
过滤,原来只看到一长串"支持的操作符",找不到一行就能改好的写法;数据接口早已给出 `$icontains`,分析接口应当说同一句话。
-
**风险与回滚**:只改报错文字,拒收范围、状态码、错误码都不变,`patch`。新检查放在过滤器归一化的最后一步,已有的拒收都先于它答复,未知操作符仍是原来的通用列表。回滚即还原本
PR。
-
**先看**:`packages/services/service-analytics/src/strategies/filter-normalizer.ts`(`retiredFilterOperatorError`、`assertNoRetiredOperators`、`normalizeWhereComparands`
末尾的调用);`src/__tests__/retired-filter-operator-refusal.test.ts`;`packages/qa/dogfood/test/analytics-retired-filter-operator.dogfood.test.ts`。
- **两分钟验证**:`pnpm --filter @objectstack/service-analytics exec vitest
run src/__tests__/retired-filter-operator-refusal.test.ts`,预期 18
个全绿,其中各个分析入口对 `$regex` 给出同一句 RETIRED 文案。
- **需要你做的**:无,席位落地。
## What changes
`fieldLeaves` in `filter-normalizer.ts` refused `$regex` / `$options`
with the right envelope (`INVALID_FILTER` / 400) but with the generic
`Unsupported filter operator "$regex" on "name". Supported: …` list,
because it never read `RETIRED_FILTER_OPERATORS`. The draft preview
answered with its own `… is not evaluated by the draft-data preview`
sentence instead. So the faces disagreed, and none of them named
`$icontains`.
- **`retiredFilterOperatorError(op, field, siblings)`** builds the
refusal the way each data-door face builds it: `[analytics] Filter
operator "OP" on field "FIELD" is RETIRED and is no longer evaluated.`,
then ` Write "TO" instead.`, then `RETIRED_FILTER_OPERATORS[op].why`
word for word, then every other retired key of the same constraint (`{
$regex, $options }` is one mistake with one fix). The lead-in is
`driver-sql`'s (the data door on a SQL stack) without its "by this
driver". It returns `null` for a non-retired operator.
- **`assertNoRetiredOperators(node)`** walks the condition with the
shared `forEachWhereFieldEntry` traversal, the same positions the shape,
type and flag passes judge: `$and` / `$or` members, `$not`, field
entries, and nested relation entries.
- **One call for every face.** It is the last pass of
`normalizeWhereComparands`, which `lowerAnalyticsWhere` runs for the
native execute, the `/analytics/sql` echo and the ObjectQL engine path.
The draft preview calls it itself, before its own operator vocabulary.
Dataset scope filters and measure filters go through the same door.
- **Last, so no existing refusal moves.** A shape, type or non-boolean
flag defect anywhere in the same `where` is still answered first. The
only answer that changes is the one that used to fall to the
unsupported-operator list. An unknown operator such as `$sounds_like`
keeps the generic list.
- **No path in the lead-in.** A client reads at most 500 characters of a
refusal (`CLIENT_MESSAGE_MAX`), and the table's `why` puts everything an
author must write in its opening sentences for that budget (#27098). A
test pins `Write "$icontains" instead.`, `` `$contains` for a
case-sensitive one `` and `match in application code` inside 500
characters.
**On the order's "the same message builder".** No shared builder exists
on main. The data door's five faces (`driver-sql`, `driver-memory`,
`driver-mongodb`, `driver-turso`'s remote transport, `objectql`'s
`having`) each compose the lead-in themselves around the shared `to` /
`why`, and none of those builders is exported from a package
`service-analytics` depends on (a driver dependency is the wrong
direction). So this follows the same form from the same table, and no
word of the prescription is written in this package. Folding all six
into one spec-side builder would be a separate change (spec export, five
drivers). It is noted below, not done here.
## Tests
Measured at HEAD `49e31fc2` (worktree on `claude/project-thread-jebtpj`,
base `135054eb`).
- `src/__tests__/retired-filter-operator-refusal.test.ts` (new, 18
tests):
- The `where` door: `$regex` and a dangling `$options` each give `code`
+ `status` + the lead-in + `Write "$icontains" instead.` + the table's
`why` verbatim. `{ $regex, $options }` names the other key. The refusal
also fires under `$not`, in an `$or` beside a TRUE arm, in an `$and` and
on a dotted member, and the client-cap budget holds. Precedence: a
type-face defect in the same `where` still answers first.
- Every face over a real sql.js engine (native execute, `/analytics/sql`
echo, ObjectQL engine path, `AnalyticsService.query` native and
ObjectQL, `AnalyticsService.generateSql`, draft preview): each gives the
message the door gives, byte for byte, with zero statements and zero
engine calls.
- Data-door parity: `engine.find` with the same `where` (marked
`'author'`, as the SecurityPlugin marks a caller's `where`) refuses with
the same tail from ` Write "$icontains" instead.` on.
- `queryDataset`: `runtimeFilter`, `runtimeFilter` with `previewDrafts`,
the dataset scope filter and a measure filter all give the prescription,
and no statement runs.
- Controls: `$icontains` is served (`['a1','a2']`) on every live face.
`$sounds_like` keeps the generic list, and the draft preview keeps its
own sentence for it. The `FilterArray` spelling
`['name','regex','acme']` is still refused at `isFilterAST`.
- `src/__tests__/filter-normalizer-not-null-safe.test.ts`: the #5325 pin
"an unknown operator inside a `$not` still THROWS" used `$regex` as its
example of an unknown operator. It now uses `$sounds_like` for the
generic list and pins `$regex` beside it as the RETIRED refusal. The
pin's intent (fail-closed under `$not`) is unchanged.
-
`packages/qa/dogfood/test/analytics-retired-filter-operator.dogfood.test.ts`
(new, booted stock showcase): `POST /api/v1/analytics/dataset/query` `{
datasetName: "showcase_project_metrics", selection: { measures:
["project_count"], runtimeFilter: { name: { $regex: "acme" } } } }`, and
the same with `$options`, answer 400 `INVALID_FILTER` with the RETIRED
lead-in, `Write "$icontains" instead.` and the table's opening sentence.
The data door `POST /api/v1/data/showcase_project/query` with the same
`where` shows the same tail. Controls: `$icontains` → 200,
`$sounds_like` → the generic list.
<details><summary>Red first on main: unit suite and the live showcase
door</summary>
The base blob of `filter-normalizer.ts` (`135054eb`) was written over
the file, under a trap restore.
- Unit: `Tests 11 failed | 7 passed (18)`. The 11 failures are every
RETIRED case. The live faces answered `[analytics] Unsupported filter
operator "$regex" on "name". Supported: $eq, $ne, …`, and the draft
preview answered `[analytics] Filter operator "$regex" on field "name"
is not evaluated by the draft-data preview. …`. The 7 that passed are
the controls plus the 500-character case, which passed vacuously on a
message that held none of the texts. It was then tightened to require
each text to be present (so the ablation below reds it).
- Live showcase (dogfood file, `service-analytics` rebuilt from the base
blob, `ablation-dist-preflight … --absent` confirmed the marker gone
from all 6 built files): `Tests 3 failed | 2 passed (5)`. `$regex` and
`$options` got `400 INVALID_FILTER` `[analytics] Unsupported filter
operator …`, which is the card's reported answer. The parity case
failed, and the two controls passed.
Restore: blob hash equals HEAD's and `git diff HEAD` is empty. Then a
rebuild, and the preflight showed the marker present in 2 built files.
</details>
<details><summary>Ablation: dropping the retired-table pass reds only
the new cases</summary>
`node scripts/ablation-replace.mjs --file …/filter-normalizer.ts
--anchor ' assertNoRetiredOperators(node, path);\n' --delete -- …`:
anchor 1 → 0, blob `34fb0eb6` → `e59a5023`, `calls-left: 0`. The unit
file imports the module from `src/` (no dist involved).
- `Tests 12 failed | 6 passed (18)`. The 12 failures are every RETIRED
case. The 6 that passed are the controls (`$sounds_like` generic list,
the `FilterArray` refusal, the `$icontains` leaf and rows, the preview's
own sentence for an unknown operator, and type-face precedence).
- Restore: blob after restore `34fb0eb6` == HEAD, `git diff HEAD` empty.
</details>
<details><summary>Package suite, typecheck, dogfood file</summary>
All at HEAD `49e31fc2`, through `scripts/pm/os-verify-lock.sh` (`VERDICT
command-exit 0`):
- `pnpm --filter @objectstack/service-analytics build`, then
`ablation-dist-preflight.mjs @objectstack/service-analytics 'is RETIRED
and is no longer evaluated.'`: marker present in 2 built files
- `pnpm --filter @objectstack/service-analytics test`: `Test Files 205
passed (205)`, `Tests 4907 passed | 282 skipped (5189)`
- `pnpm --filter @objectstack/service-analytics typecheck`: exit 0
(`--listFilesOnly` includes both test files)
- `pnpm --filter @objectstack/dogfood exec vitest run
test/analytics-retired-filter-operator.dogfood.test.ts`: `Tests 5 passed
(5)`
- `pnpm --filter @objectstack/dogfood typecheck`: exit 0
(`--listFilesOnly` includes the new dogfood file)
- Full dogfood battery: not run (no `!` in the title and no
`(narrowing)` arm). The dogfood job does not run on PRs; the new file
was run locally above.
</details>
## Acceptance notes
- **Premise** held as stated (triage `6569498837`). It was measured on
main through the unit faces and the live showcase door.
- **Declared conflict with the order.** "The same message builder"
assumes a shared builder. There is none (see What changes). The sentence
follows the data door's form from the same table, and no prescription
text lives here.
- **Observation, not filed: the analytics door serves a refusal message
whole, while the data door cuts it at 500 characters with `…`.**
Measured on the dogfood showcase: the `$regex` refusal is 1,068
characters on `/analytics/dataset/query` and 500 on
`/data/showcase_project/query`. Nothing is wrong in what either door
says. Whether analytics should apply `CLIENT_MESSAGE_MAX` is not this
card's question.
- **Not touched:** the read scope (`read-scope-sql.ts`). It is
policy-authored, so its refusal stays `READ_SCOPE_COMPILE_FAILED` / 500
with the message withheld (#5367), and a retired operator there keeps
its fail-closed sentence. A top-level `{ $regex: … }` (not a field
constraint) keeps "Unsupported top-level filter operator", as the data
door's faces have no retired arm there either. NOT MEASURED: what the
data door says for a `FilterArray` `['name', 'regex', 'acme']`. This
door refuses it at `isFilterAST`, unchanged.
- **Possible follow-up (carrier: none):** one spec-side builder for the
retired-operator lead-in, so the six faces stop composing it separately.
That is a spec export plus five drivers, and none of them is wrong
today.
- **Overlap.** Open PR #28976 edits
`service-analytics/src/strategies/objectql-strategy.ts` (the draft
preview's date bucket). No hunk meets this PR's files. No other open PR
touches `filter-normalizer.ts` or the two service-analytics test files.
All 27 open PRs were checked by file list.
- **Precedence change, declared:** none for any existing refusal. A
`where` whose only defect is a retired key now gets the RETIRED sentence
instead of the generic list. The draft preview now gives the published
faces' sentence instead of its own for those two keys.
## Verification
Derived gate families: `node scripts/pm/dispatch-gates.mjs --commands`
at `49e31fc2` (5 paths vs merge base `135054eb`) gives 64 commands. All
64 ran locally and exited 0. `dispatch-gates --ran`: `64 derived
famil(ies) accounted for — 64 run, 0 NOT-MEASURED`. The first run of
`check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow
clone. After `git fetch --depth=1 origin 621a487…` it exited 0 (48
cases). CI owns the 50 artifact-roster families, the wide-population
families, the type-check lanes and `pnpm lint`; their verdicts are on
this PR's check-runs.
<details><summary>Gate record (COMMAND :: exit N), HEAD
49e31fc</summary>
```
node scripts/check-adr-0087-registration.mjs --base origin/main :: exit 0
node scripts/check-adr-0087-registration.mjs --self-test :: exit 0
node scripts/check-changeset-no-major.mjs --base origin/main :: exit 0
node scripts/check-closing-keyword-parity.mjs :: exit 0
node scripts/check-closing-keyword-parity.mjs --self-test :: exit 0
node scripts/check-comment-mask-adoption.mjs :: exit 0
node scripts/check-comment-mask-adoption.mjs --self-test :: exit 0
node scripts/check-comment-mask-corpus.mjs :: exit 0
node scripts/check-dts-emitted.mjs --self-test :: exit 0
node scripts/check-empty-changeset.mjs --base origin/main :: exit 0
node scripts/check-empty-changeset.mjs --self-test :: exit 0
node scripts/check-issue-citations.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs :: exit 0
node scripts/check-plugin-teardown-shape.mjs --self-test :: exit 0
node scripts/check-tenant-audit-census.mjs :: exit 0
node scripts/check-tenant-audit-census.mjs --self-test :: exit 0
node scripts/ci/assert-turbo-remote-cache.mjs --self-test :: exit 0
node scripts/ci/plan-test-core-shards.mjs --self-test :: exit 0
node scripts/ci/prune-turbo-cache.mjs --self-test :: exit 0
node scripts/docs-audit/check-affected-docs.mjs :: exit 0
node scripts/docs-audit/check-drift-comment.mjs :: exit 0
node scripts/pm/release-rehearsal-clone.mjs --self-test :: exit 0
node scripts/release-pending-publish.mjs --self-test :: exit 0
pnpm --filter @objectstack/spec run check:duration-unit-keys :: exit 0
pnpm --filter @objectstack/spec run check:empty-state :: exit 0
pnpm --filter @objectstack/spec run check:liveness :: exit 0
pnpm --filter @objectstack/spec run check:strictness-ledger :: exit 0
pnpm --filter @objectstack/spec run check:variant-docs :: exit 0
pnpm check:changeset-gate-self-tests :: exit 0
pnpm check:ci-filter-parity :: exit 0
pnpm check:closed-ceremony-text :: exit 0
pnpm check:cross-package-test-inputs :: exit 0
pnpm check:doc-authoring :: exit 0
pnpm check:driver-memory-census :: exit 0
pnpm check:dts-closure :: exit 0
pnpm check:dual-build-cjs-loads :: exit 0
pnpm check:engine-double-contract :: exit 0
pnpm check:error-status-conformance :: exit 0
pnpm check:gitlink-declared :: exit 0
pnpm check:issue-citations :: exit 0
pnpm check:keyed-text-bounds :: exit 0
pnpm check:lean-entry-closure :: exit 0
pnpm check:logger-receiver-detach :: exit 0
pnpm check:nul-bytes :: exit 0
pnpm check:objectql-double-limit :: exit 0
pnpm check:org-identifier :: exit 0
pnpm check:page-declaration-shape :: exit 0
pnpm check:platform-object-tenancy-census :: exit 0
pnpm check:pm-changeset-deadline-census :: exit 0
pnpm check:published-files :: exit 0
pnpm check:query-options-erasure :: exit 0
pnpm check:refd-timer-probe :: exit 0
pnpm check:registry-log-declared :: exit 0
pnpm check:rest-log-spy-declared :: exit 0
pnpm check:slot-lookup :: exit 0
pnpm check:sourcemap-no-sources-content :: exit 0
pnpm check:system-context-census :: exit 0
pnpm check:test-source-alias :: exit 0
pnpm check:tier-file-adoption :: exit 0
pnpm check:type-check-coverage :: exit 0
pnpm check:type-check-debt :: exit 0
pnpm check:undeclared-dep-imports :: exit 0
pnpm check:watch-hint-literal :: exit 0
pnpm check:where-matcher :: exit 0
```
</details>
---
_Generated by [Claude
Code](https://claude.ai/code/session_01FEkQ5oAC7wmJvoM1VnnCjg)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 6b3ba82 commit c470581
5 files changed
Lines changed: 547 additions & 1 deletion
File tree
- .changeset
- packages
- qa/dogfood/test
- services/service-analytics/src
- __tests__
- strategies
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
721 | 721 | | |
722 | 722 | | |
723 | 723 | | |
724 | | - | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
725 | 727 | | |
726 | 728 | | |
727 | 729 | | |
| |||
0 commit comments