Repository navigation
Commit 55d9dd4
feat(spec,service-analytics): a dataset answer says which records each measure counted, so a drill lists what the number counted (#30596)
<!-- claude-projects-attribution: {"github_login":"os-sales"} -->
_Requested by **Sales** · [project
thread](https://claude.ai/code/project/chan_01D5bWFY3rFzgJcytD5gD4Uu?thread=cmsg_01D5bWFY3rFzgJcytD5gD4Uu6W6tm24ezCZ9jmVrXt12YA)_
domain:spec#2 · os-sales · session_01L6bhgXexAhb1ogpnymn5Jb
Fixes #30526
Clause-②: yes (widening)
Metadata-protocol: none
## 维护者速读
- 改了什么:数据集查询的回答多了一个可选字段
`drillScopes`,按度量说明“这个数字统计的是哪些记录”:数据集自己的筛选,加上度量自己的筛选,再加上被汇总的字段有值。
- 为什么改:点开一个带筛选的数字(例如“已付款发票 2 张”)时,现在的下钻只按分组和组件筛选去列记录,会列出 4
张。客户端读不到数据集和度量的定义,所以只能由服务端在回答里告诉它。
-
风险与回滚:只新增一个可选字段,不读它的客户端不受影响;查询算出的数字不变,唯一的行为改动是日期等占位符改为在查询入口解析一次,算数和下钻范围共用同一个时刻。回滚就是
revert 本 PR。
- 先看:①
`packages/services/service-analytics/src/drill-scope.ts`(什么算范围、什么时候不给);②
`packages/services/service-analytics/src/analytics-service.ts`
里提前解析占位符和写 `result.drillScopes` 的两处;③
`packages/spec/src/api/analytics.zod.ts` 的字段说明。
- 两分钟验证:`pnpm --filter @objectstack/service-analytics exec vitest run
src/__tests__/dataset-drill-scopes.test.ts`,预期 20 个测试全部通过,其中标着 CONTROL
的一条表明:不带范围时列出 4 条记录,而数字是 2。
- 需要你做的:无,席位落地。
## What changes
The precedent is `drillRanges` (#27074), and before it `dimensionFields`
/ `drillRawRows` (#20700). Those keys bound the clicked **bucket**. This
PR adds the fifth drill sidecar, which bounds the clicked **measure**.
It sits on the same answer and follows the same rule (ADR-0021 D2: the
server states the drill bounds, and the client never restates dataset
semantics).
- **`@objectstack/spec`**: `AnalyticsResult.drillScopes`, optional, a
map from measure name to `FilterCondition`, declared on the
`AnalyticsResultResponse` wire schema too. Each selected measure name
maps to the list filter on the answer's `object` that selects the
records the measure aggregates. The filter is built from three parts:
- the dataset's `filter`;
- AND the measure's `filter`;
- AND, for a measure over a field, `{ field: { $null: false } }`, or
both instants for a duration measure.
Tokens are resolved as the query resolved them. The value is `{}` when
no part applies. A measure is **left out** when no list filter can
select its records, and a measure left out offers no drill (the #30080
ruling, option A). That happens in three cases:
- a derived measure;
- a scope naming a relationship path (`account.region` as a key, or a
dotted `$field`);
- a filter that is not a condition object.
A `__compare` column has no entry. `drillScopes` is set on every dataset
answer that has rows. It is absent on a cube query answer and on a
draft-preview answer.
- **`@objectstack/service-analytics`**: `queryDataset` computes the
scope in `drill-scope.ts` (`measureDrillScopes`, a pure function). It
takes the token-resolved filters from **before** the dataset door's
field binding, because the binding's minted member names are
request-local and no record list knows them. `resolveSelectionTokens` is
now exported and called once in `answerDataset`. The executor and the
scope read the same resolved trees, and the executor's own call is then
the identity. `datasetMeasureSource` finds what a measure aggregates the
same way the queries did:
- derived, bare or `CUBE.`-qualified;
- duration, through `durationMeasureIn`;
- declared, through `scopedMeasureKey`;
- caller-minted, through `mintCallerMeasures`.
- **`@object-ui/data-objectstack`**: `queryDataset` hand-picks the keys
it returns, so it dropped any sidecar it did not list. This was measured
before the change: the key never reached a widget. It now passes
`drillScopes` through, typed as the spec slot.
- Generated: `content/docs/references/api/analytics.mdx` (`gen:docs`,
the one artifact `check:generated` proved stale). Ledger:
`packages/spec/dropped-refinements.baseline.json` gains one site; see
the Acceptance notes.
### What each consumer must compose (not in this PR: #30080 and the ui
card the seat files at landing)
A drill of a clicked number on measure `M` lists `{ $and: [bucket,
widgetFilter, drillScopes[M]] }` on `object`, leaving out any part that
is absent:
| Consumer | `bucket` | `widgetFilter` | scope |
|---|---|---|---|
| Report drill (`plugin-report` dataset renderer) |
`buildDatasetDrillFilter` over `dimensionFields` + `drillRawRows[i]` +
`drillRanges[i]` (a totals row: `drillRawTotals[i][j]`) | the report's
`runtimeFilter` | `drillScopes[M]` |
| Chart / table widget drill (`plugin-dashboard` `DatasetWidget`) | the
same builder for the clicked row and series | the widget's
`runtimeFilter` | `drillScopes[M]` of the clicked series |
| Dashboard tile click (#30080) | none (a KPI has no dimensions) | the
tile's `runtimeFilter` | `drillScopes[M]` |
If `drillScopes` has no entry for `M`, the consumer offers no click.
`{}` means "drillable, nothing to add", which is a different thing from
a missing entry.
## Why
A dataset number is computed under the dataset's `filter`, the measure's
`filter` and the widget's `runtimeFilter`. A host that drills the number
holds only the last one, and it has no read of the dataset definition.
So every drill of a filtered measure listed a superset of what the
number counted. On the showcase shape, the bucket listed 4 invoices
behind `paid_count` 2. #30080 (the dashboard tile click, ruled option A)
waits on this answer.
## Tests
-
`packages/services/service-analytics/src/__tests__/dataset-drill-scopes.test.ts`
(new, 20 tests):
- **Unit tier** (mocked ObjectQL face), on the real producer shape
(`examples/app-showcase` revenue-pulse: `paid_count` = count with
`filter: { status: 'paid' }`, and `paid_rate` derived). Pins:
- measure filter only;
- dataset filter only, and both;
- count over a field;
- nested-relation form and a same-record `$field` kept;
- tokens resolved to the caller, and the engine was handed the same
value;
- a cube-qualified name;
- a dimensionless KPI carries the scope, and a zero-row answer carries
none;
- the scope is a deep copy;
- **contract vs wire**: the real service's answer, through
`JSON.stringify`, parsed by `AnalyticsResultResponseSchema`, keeps every
scope.
- **Real-engine tier** (ObjectQL + `SqlDriver` on SQLite, both faces,
with and without a widget filter). Every count's composed drill (`bucket
∧ widget ∧ scope`) lists as many records as the number, and the sum adds
up. Further pins:
- a CONTROL pin: the bucket alone lists 4 against `paid_count` 2;
- a duration measure, where the mean over the listed records equals the
number;
- a nested-relation dataset filter;
- a dotted measure filter and a dotted dataset filter leave the scope
out;
- a CONTROL pin: `engine.find` refuses the dotted key with
`INVALID_FIELD`.
- `packages/spec/src/api/analytics.test.ts`: the response schema keeps
each scope verbatim. It refuses these malformed values at
`data.drillScopes`:
- a string;
- a list in an equality slot;
- `$and` given an object;
- an array map.
- `packages/spec/src/contracts/analytics-service.test.ts`: a typed
`drillScopes` read on the contract, plus a `@ts-expect-error` on a
string scope.
- `ui/packages/data-objectstack/src/queryDataset.test.ts`:
- the passthrough keeps scopes verbatim, `{}` included;
- absent stays `undefined`;
- an array, a string or `null` is dropped;
- a compile-time pin of identity with `AnalyticsResult['drillScopes']`.
## Acceptance notes
- **Shape choices beyond the brief, each deliberate.** Each one is
pinned by a test above.
1. The not-null leg for a measure over a field. Every aggregate skips an
empty value (`COUNT(field)`, #10298), so without the leg, `count of
total` would list rows whose `total` is empty.
2. A duration measure requires both instants.
3. `{}` for a measure with nothing to add, so that "absent" keeps one
meaning: no drill.
4. Tokens are resolved once and shared by the numbers and the scope, so
the two cannot straddle a period boundary. The error path is unchanged:
every token refusal is a declared 400 that the catch rethrows either
way.
5. A caller-minted (undeclared) measure gets a scope too.
- **Relationship paths.** The nested-relation form (`{ account: { tier:
'gold' } }`) is kept, because the record list serves it. A dotted path
is left out, because the list refuses a dotted filter key. A later
change could lower a one-hop dotted path to the nested form; this PR
does not.
- **Ledger growth, a decision for the seat.**
`dropped-refinements.baseline.json` gains `api/AnalyticsResultResponse`
→ `data.drillScopes.valueType.lazy`. This is the same
`FilterConditionSchema` lazy refinement already admitted at
`api/AnalyticsQueryRequest` → `where.lazy`, and the spec build prints
this entry. Header totals move 251 → 252 schemas and 1019 → 1020 sites.
- **Found, not fixed here (in the report):**
- The engine refuses the nested-relation form inside an aggregation's
own `filter` ("the nested-relation form, which the engine serves in
'where' only"). So a dataset **measure** whose `filter` is `{ account: {
tier: 'gold' } }` fails the whole answer, even though the dataset schema
accepts it. This was measured at the service on SQLite, not at the HTTP
door.
- `data-objectstack` also drops `drillRawTotals`. No ui code reads it
today, so the gap is not felt.
- **Branch name.** The branch is the harness-assigned
`claude/project-thread-mg9ioh`, not `claude/issue-30526-*`, as amended
on the claim.
- **Attribution link.** It names this card thread, where the brief named
the seat thread: the platform writes this thread's own link.
- **New lint warnings.** eslint on `data-objectstack` adds 1 + 3
`no-explicit-any` warnings. These are `(data as any)` and test `as any`
casts in the files' existing pattern; there are 0 errors.
## Verification
Head `ba62840b` (branch base `c23eafa8`, `origin/main` `f5aaa0bc` merged
in), under `os-verify-lock`, `NODE_OPTIONS=--max-old-space-size=8192`.
Every count below is a VERDICT `command-exit 0` unless it says
otherwise. Every suite below ran twice, once at `25fc5696` before the
merge and once at `ba62840b` after it, with the same counts both times.
- `pnpm --filter @objectstack/spec build && pnpm --filter
@objectstack/spec check:generated`: 14 of 14 artifacts up to date, after
`gen:docs`, which was the one artifact it proved stale. At the default
4096 MB heap, `check:test-typecheck` stops with "tsc failed without
diagnostics", an out-of-memory error and not a verdict. At 8192 MB it
passes: 52 files, 240 errors, all 131 pinned signatures held.
- `pnpm --filter @objectstack/spec test`: Test Files 815 passed (815),
Tests 25648 passed, 1 todo. `pnpm --filter @objectstack/spec typecheck`:
exit 0.
- `@objectstack/service-analytics`: `test` gives Test Files 211 passed
(211), Tests 4996 passed, 300 skipped. `typecheck`: exit 0.
- `@object-ui/data-objectstack`: `test` gives Test Files 115 passed
(115), Tests 1319 passed. `type-check`: exit 0.
- Consumers that `git grep drillRanges` lists: `@object-ui/core` 233
files passed (4376 tests), `@object-ui/plugin-dashboard` 202 files
passed (1932 tests), `@object-ui/plugin-report` 33 files passed (307
tests).
- eslint on the 10 changed source files: 0 errors (warnings as in
Acceptance notes).
<details><summary>Ablation of the serving half
(scripts/ablation-replace.mjs, restored to the HEAD blob
bca6a05)</summary>
- Mutation: `result.drillScopes = measureDrillScopes({` became `void
measureDrillScopes({` in `analytics-service.ts`. The anchor count went 1
→ 0, and the blob went `bca6a0582f42` → `98dc47836851`.
- `dataset-drill-scopes.test.ts`: **17 failed | 3 passed (20)**.
- Every service pin went red on both tiers, the bucket CONTROL and the
contract-vs-wire pin included.
- The 3 that stayed green are the two direct walk tests and the
record-list CONTROL (`engine.find` refuses a dotted key), none of which
goes through the service.
- Restore: the blob equals HEAD, and `git diff HEAD` is empty. The test
imports the service from `src`, so no rebuild was needed.
</details>
<details><summary>Gates: dispatch-gates --commands, 110 derived at
ba62840, 110 run, 0 NOT-MEASURED</summary>
- `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
--ran ran.list` printed "110 derived famil(ies) accounted for — 110 run,
0 NOT-MEASURED (a DERIVED zero — all 110 recorded an exit code and none
of them is 3)" and exited 0. Every line is `exit 0`.
- On the first pass two gates exited 3 because built output was missing:
- `check:skill-examples` lacked the `client-react` dist.
- `check:dual-build-cjs-loads` lacked the dists of account, setup,
studio, client-react, cloud-connection, connector-imap and
connector-mcp.
- After `turbo run build` of those 7 packages, both exit 0.
- `check-plugin-teardown-shape --self-test` needed its pinned fixture
commit `621a4876` fetched into the shallow clone first. The longest
runs: `check:type-check-debt` 277 s, `check:query-options-erasure` 201
s, `ui` `vitest run scripts/` 192 s.
- No `!` in the title and no `(narrowing)` arm, so there was no full
dogfood run. CI owns the rest of the farm.
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic
(one optional field and its describe; the scope is computed in
`service-analytics`).
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; eslint on the changed files has 0 errors (the
full `pnpm lint` is the CI run).
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: its generated artifacts are
regenerated and committed.
- [x] 7. No hand edits to `content/docs/references/` (regenerated by
`gen:docs`), `packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the 维护者速读 block
excepted, per project convention).
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01L6bhgXexAhb1ogpnymn5Jb
<!-- signed: domain:spec#2 · os-sales · session_01L6bhgVugtCCwBs98FeR27P
-->
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 5b748df commit 55d9dd4
13 files changed
Lines changed: 776 additions & 10 deletions
File tree
- .changeset
- content/docs/references/api
- packages
- services/service-analytics/src
- __tests__
- spec
- src
- api
- contracts
- ui/packages/data-objectstack/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
159 | 159 | | |
160 | 160 | | |
161 | 161 | | |
| 162 | + | |
162 | 163 | | |
163 | 164 | | |
164 | 165 | | |
| |||
0 commit comments