You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a4daa85
Browse filesBrowse the repository at this point in the historyBrowse files
feat(spec): a plugin-driver datasource can declare its row cap, external.rowCap, up to 10,000 (#29854)
<!-- claude-projects-attribution: {"github_login":"os-warren"} -->
_Requested by **Warren** · [project
thread](https://claude.ai/code/project/chan_01U5RdnAAmkSQmZbqh6FtqZ8?thread=cmsg_01U5RdnAAmkSQmZbqh6FtqZ83WAcW3kdc2RHYZ2aeB5krB)_
Part of #29762 — the spec slice. The engine floor reading the key, and
the guide on whole-set views, are the card's second PR, opened after
this one merges.
Clause-②: yes (widening) — one optional, bounded key joins the external
datasource settings; the default keeps today's 1,000
Metadata-protocol: added
Skills-docs: follow-up #29762 — this card's second PR teaches the key in
content/docs/data-modeling/virtual-objects.mdx and odata-datasources.mdx
## 维护者速读
- 改了什么:外部数据源设置新增可选键 `external.rowCap`(1–10,000,不写即 1,000),只在 spec
层声明与校验。
- 为什么改:客户的虚拟对象甘特图约 5,000 行,固定 1,000 行上限直接 422;Warren 裁定 (a)+(c),平台最大值
10,000。
- 风险与回滚:纯增量键,默认值不变,现有数据源零变化;引擎尚未读取(第二个 PR 才接上),回滚即 revert 本 PR。
- 先看:`packages/spec/src/data/datasource.zod.ts`(键、两个常量、内置驱动拒收)→
`packages/spec/liveness/datasource.json`(planned 行)→
`datasource.test.ts`。
- 两分钟验证:`pnpm --filter @objectstack/spec exec vitest run
src/data/datasource.test.ts -t 29762`,预期 7 个用例全绿。
- 需要你做的:无,席位落地。
## What changes
- `ExternalDatasourceSettingsSchema` declares `rowCap`:
`z.number().int().min(1).max(EXTERNAL_ROW_CAP_CEILING).optional()`,
beside `queryTimeoutMs`. No default is stamped onto a parsed block, so a
served or round-tripped datasource is byte-identical to today's; absent
means `EXTERNAL_ROW_CAP_DEFAULT`.
- Two constants exported from `@objectstack/spec/data`:
`EXTERNAL_ROW_CAP_DEFAULT = 1000` (today's fixed value, which
`@objectstack/objectql`'s `EXTERNAL_ROW_CAP` will read from in PR 2) and
`EXTERNAL_ROW_CAP_CEILING = 10_000` (the ruled platform maximum).
- Refusals, each where the key would do nothing: on a built-in driver at
`external.rowCap`, naming the driver (it runs the whole query itself and
is never floored); on a managed datasource through the existing
managed-block refusal, which lists `rowCap` among the keys to remove.
Above the ceiling, zero, negative and fractional values are refused at
parse. `maxRows` / `rowLimit` / `maxRowCount` fold onto `rowCap` in the
suggestion.
- Liveness: `external.rowCap` graded `planned`, with the reader
(`external-driver-floor.ts`) named and the flip to `live` owed by PR 2.
- Regenerated: `content/docs/references/data/datasource.mdx`
(`gen:docs`) and `packages/spec/authorable-surface/data.json`
(`gen:schema`). `authorable-surface.base.json` is not touched.
Precedent for the shape: `LoopConfigSchema.maxIterations` is bounded by
`LOOP_MAX_ITERATIONS_CEILING` the same way
(`packages/spec/src/automation/control-flow.zod.ts`). Name chosen:
`rowCap`, the customer's own spelling and the `EXTERNAL_ROW_CAP*`
family's.
## Tests
`packages/spec/src/data/datasource.test.ts`, describe `[#29762]
external.rowCap`: the ruled numbers; accepted on a plugin driver from 1
to the ceiling in `external` and `validate-only`; no default stamped on
parse; refused above the ceiling, at 0, -1 and 2500.5; the alias fold;
refused on `sqlite` naming the driver; refused on a managed datasource
through the managed-block refusal.
## Acceptance notes
- Measurement before building: issue comment 6595439927. Ruling: (a) +
(c), ceiling 10,000; (b) stays with #27261; the 2,000-row renderer
ceiling (objectui#7210 ruling a′) was not ruled on.
- The engine contract `EngineDatasourceDef.external` gains `rowCap` in
PR 2, with the engine that reads it, so this PR adds no surface ahead of
a reader beyond the authored key itself.
## Verification
All readings on head `d7048915`.
- `pnpm --filter @objectstack/spec exec vitest run
src/data/datasource.test.ts -t 29762`: 7 passed. Whole file: 69 passed.
- `pnpm --filter @objectstack/spec typecheck && pnpm --filter
@objectstack/spec test` under the verify lock: `VERDICT command-exit 0`,
Test Files 806 passed, Tests 23981 passed, 1 todo.
- `pnpm lint` (`eslint . --no-inline-config`): no problems printed.
- `node scripts/pm/dispatch-gates.mjs --commands` derived 103 families;
all 103 run. `--ran` reconciliation: "103 derived famil(ies) accounted
for — 101 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)".
<details><summary>Gate readings</summary>
- First pass red on three generated artifacts, as expected for two new
exports and one new liveness row: `check:api-surface`,
`check:export-origins`, `check:generated` and `check:liveness` (the
`state-counts/datasource.md` count). Regenerated with `gen:api-surface`,
`gen:export-origins`, `gen:liveness-counts` from a freshly built
`packages/spec` (commit `d7048915`); all four re-run green.
- `check:skill-examples` first refused with exit 3 (client-react not
built); after building `@objectstack/client-react...` it re-ran green.
- NOT MEASURED, exit 3 (prerequisite): `node
scripts/check-plugin-teardown-shape.mjs --self-test` (its positive
control is pinned to a commit outside this shallow clone; checker
health, not a verdict on this diff) and `pnpm
check:dual-build-cjs-loads` (needs every package built). Both run in CI.
- Every other derived family: exit 0.
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes.
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset.
- [x] 6. `packages/spec` touched: its generated artifacts are
regenerated and committed.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English.
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01JQwxbZdyXCxmKHk7PbwmfZ
---
_Generated by [Claude
Code](https://claude.ai/code/session_01JQwxbZdyXCxmKHk7PbwmfZ)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
feat(spec): a plugin-driver datasource can declare its row cap, `external.rowCap`, up to a platform maximum of 10,000 (#29762)
6
+
7
+
Clause-②: yes (widening) — one optional, bounded key joins the external datasource settings; the default keeps today's 1,000
8
+
Metadata-protocol: added
9
+
10
+
`ExternalDatasourceSettingsSchema` declares one optional key, `rowCap`: a whole number from 1 to `EXTERNAL_ROW_CAP_CEILING` (10,000). It is the most rows the engine reads from a plugin driver for one request, and so the most rows one read on such a datasource may match before it is refused with `EXTERNAL_ROW_CAP_EXCEEDED` (422). Absent, the cap is `EXTERNAL_ROW_CAP_DEFAULT` (1,000), today's fixed value, so no existing datasource changes. Both constants are exported from `@objectstack/spec/data`.
11
+
12
+
-**Refused where it would do nothing.** On a built-in driver (which runs the whole query itself and is never capped) the key is refused at `external.rowCap`, naming the driver. On a managed datasource it is refused by the existing managed-block refusal, which names it among the keys to remove. A value above the ceiling, zero, a negative or a fraction is refused at parse. `maxRows` and `rowLimit` are folded onto `rowCap` in the refusal's suggestion.
13
+
-**Not acted on yet.** This is the contract half. The engine's floor over a plugin driver reads the key in a following change; until then a datasource that sets it still reads at 1,000 rows. The liveness ledger grades it `planned` until that reader lands.
Copy file name to clipboardExpand all lines: content/docs/references/data/datasource.mdx
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,6 +77,7 @@ const result = DatasourceSchema.parse(data);
77
77
|**credentialsRef**|`string`| optional | Reference into the secrets store; never inline credentials. Valid in every schemaMode — the one `external` key a managed datasource may carry. |
78
78
|**connector**|`string`| optional | Name of a declared connector instance (a `connectors` entry with a `provider`) whose credential this API-backed datasource borrows — whichever kind the instance's `auth` declares (`bearer`, `api-key` or `basic`; ADR-0143 D2 adds the grant-bearing kinds), resolved the way the connector resolves it. One Connect serves both the connector's actions and this datasource's records. Exclusive with `credentialsRef`; refused on a built-in database driver and on a managed datasource. |
79
79
|**queryTimeoutMs**|`number`| optional (default: `30000`) | Hard cap on per-query execution time. |
80
+
|**rowCap**|`integer`| optional | Plugin-driver datasources only (a non-managed datasource whose `driver` is not a built-in id): the most rows the engine reads from the driver for one request, and so the most rows one read may match before it is refused with EXTERNAL_ROW_CAP_EXCEEDED (422). Absent: 1000. At most 10000. Each list page reads every match up to this cap from the remote, twice (the rows and the total), so raise it only as far as the widest read needs. |
80
81
81
82
82
83
---
@@ -119,6 +120,7 @@ External datasource settings: federation policy (schemaMode != "managed") plus t
119
120
|**credentialsRef**|`string`| optional | Reference into the secrets store; never inline credentials. Valid in every schemaMode — the one `external` key a managed datasource may carry. |
120
121
|**connector**|`string`| optional | Name of a declared connector instance (a `connectors` entry with a `provider`) whose credential this API-backed datasource borrows — whichever kind the instance's `auth` declares (`bearer`, `api-key` or `basic`; ADR-0143 D2 adds the grant-bearing kinds), resolved the way the connector resolves it. One Connect serves both the connector's actions and this datasource's records. Exclusive with `credentialsRef`; refused on a built-in database driver and on a managed datasource. |
121
122
|**queryTimeoutMs**|`number`| optional (default: `30000`) | Hard cap on per-query execution time. |
123
+
|**rowCap**|`integer`| optional | Plugin-driver datasources only (a non-managed datasource whose `driver` is not a built-in id): the most rows the engine reads from the driver for one request, and so the most rows one read may match before it is refused with EXTERNAL_ROW_CAP_EXCEEDED (422). Absent: 1000. At most 10000. Each list page reads every match up to this cap from the remote, twice (the rows and the total), so raise it only as far as the widest read needs. |
Copy file name to clipboardExpand all lines: packages/spec/liveness/datasource.json
+5Lines changed: 5 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -155,6 +155,11 @@
155
155
"verifiedAt": "2026-08-28",
156
156
"evidence": "packages/services/service-datasource/src/datasource-admin-service.ts#testConnection (`const queryTimeoutMs = (input.external as …)?.queryTimeoutMs`, spread onto the probe options as `timeoutMs` only when it is a number)",
157
157
"note": "carried into the external-datasource probe options as `timeoutMs`. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) and REPOINTED — `:220` had rotted onto a BLANK LINE, which is the most legible form of this rot: in range, resolvable, and pointing at nothing at all. Re-closed by hand against c459da6bc; DATED for the first time."
158
+
},
159
+
"rowCap": {
160
+
"status": "planned",
161
+
"verifiedAt": "2026-11-03",
162
+
"note": "PLANNED (#29762, maintainer ruling: a per-datasource cap bounded by a platform maximum of 10,000). The plugin-driver row cap: the most rows the engine's correctness floor reads from a plugin driver for one request, and so the most one read may match before EXTERNAL_ROW_CAP_EXCEEDED (422). Declared in the spec slice (packages/spec/src/data/datasource.zod.ts, beside `EXTERNAL_ROW_CAP_DEFAULT` and `EXTERNAL_ROW_CAP_CEILING`); refused at parse above the ceiling, on a built-in driver, and (through the managed-block refusal) on a managed datasource. ACTED ON by nothing yet: the reader is the floor in packages/objectql/src/external-driver-floor.ts, which lands in the card's second PR. Flip to `live` on the commit that lands that reader, citing it."
158
163
}
159
164
},
160
165
"note": "5 of 7 live. The two dead ones are opposite in risk: `external.label` is cosmetic, `external.requirePermission` is security-shaped — it reads as an access gate and is not one."
0 commit comments