fix(objectql,rest,metadata-core)!: a datasource definition's stored row is served only to datasource administrators (#27884)

.changeset/27884-datasource-stored-row-read-admission.md

@@ -0,0 +1,21 @@
+---
+'@objectstack/metadata-core': patch
+'@objectstack/objectql': patch
+'@objectstack/rest': patch
+---
+
+fix(objectql,rest,metadata-core)!: a datasource definition's stored row is served only to datasource administrators, whichever exit reads it (#27884)
+
+Clause-②: no (narrowing for readers below datasource administration)
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no metadata an author writes changes meaning: a stored datasource definition is written, stored and served to its administrators exactly as before. What narrows is which readers a stored row of a capability-gated type is served to. There is nothing for a migration to rewrite and no tombstone; a reader who needs the definition is granted the capability. -->
+
+**BREAKING** read-set narrowing for callers below datasource administration. The metadata read door already served a `datasource` or `external_catalog` definition only to holders of `manage_platform_settings`, the capability the datasource family's own door requires. The stored row each definition lives in now admits the same readers, so one document admits one set of callers at every exit.
+
+- **One row.** The type-to-capability table moves to `@objectstack/metadata-core` as `STORED_METADATA_TYPE_READ_CAPABILITIES`, with `storedMetadataTypesWithheldFrom(caller)` and `isCapabilityGatedStoredMetadataType(type)`. `@objectstack/rest` keeps exporting `META_TYPE_READ_CAPABILITIES`, now that same frozen object, and the metadata read door answers exactly as before.
+- **Now.** A caller-context read of a stored metadata row of a listed type, by a caller without the type's capability, is served no row: a list or count leaves it out, and a read of its id answers as a read of a row that does not exist. Holders of the capability, platform administrators among them, are served everything as before. System reads are unchanged.
+- **Events.** A record-change event of such a row carries no metadata body to any subscriber; the row's other columns are unchanged.
+- **Who sees it.** A reader below datasource administration is no longer served stored datasource or federation-catalog rows.
+
+**The remedy.** Grant `manage_platform_settings` to a caller that needs a datasource definition, or read it through the datasource admin door (`GET /api/v1/datasources/:name`) as such a caller.

content/docs/permissions/system-context.mdx

@@ -122,6 +122,7 @@ bugs — it is the protection or the side effect that silently does not happen.
 | 18 | REST anonymous-deny seam satisfied | rest | Get: `enforceAuth` passes with no `userId`. Not reachable from the wire — `isSystem` is never set on an inbound request | `enforceAuth` |
 | 18b | Settings namespace read scope not applied | service-settings | Get: a read of `sys_setting`, `sys_setting_audit` or `sys_platform_setting` across every namespace, whatever each namespace's `readPermission` names — the settings service's own reads of its stores take this path. Lose: the per-namespace read gate the generic data API applies to every other caller, the same rule the settings routes enforce | `settingsReadDoorMiddleware` |
 | 18c | **Credential-column filter / sort refusal not applied** — a `where`, `orderBy`, per-aggregation `filter` or `expand` entry naming a `password` / `secret` column is evaluated | objectql | Get: the server's own match on a stored credential value, a lookup that never returns it. Every other caller, administrators included, is refused `400 INVALID_FIELD` for anything but a has-a-value check (ADR-0100). Lose: nothing a wire caller reaches — `isSystem` is never set on an inbound request | `rejectCredentialEvaluation` |
+| 18d | **Stored-metadata type read gate not applied** — a read of `sys_metadata` / `sys_metadata_history` is not narrowed by the capability its row's type requires | objectql | Get: every row of the family on `find` / `findOne` / `count` / `aggregate` — the metadata loader, the `/meta` doors' own store reads, boot (a read with no execution context is not narrowed either). Lose: the narrowing that serves a stored row of a capability-gated type (`datasource`, `external_catalog`) only to holders of the capability the `/meta` type gate names for it | `storedMetadataTypeReadFilter` |
 
 ### 2. Write pipeline and data integrity
 
@@ -543,6 +544,7 @@ and CI refuses one that differs from what the tree produces.
 | `packages/objectql/src/engine.ts#insert` | 3 | 21, 22 |
 | `packages/objectql/src/engine.ts#isSystem` | 1 | 64 |
 | `packages/objectql/src/engine.ts#update` | 4 | 19, 20 |
+| `packages/objectql/src/stored-metadata-type-read-gate.ts#storedMetadataTypeReadFilter` | 1 | 18d |
 | `packages/observability/src/perf-timing.ts#isPerfDisclosurePrincipal` | 1 | 13 |
 | `packages/plugins/organizations/src/organizations-plugin.ts#start` | 1 | 62 |
 | `packages/plugins/plugin-approvals/src/approval-service.ts#comment` | 1 | 43 |

packages/metadata-core/src/index.ts

@@ -101,6 +101,11 @@ export * from './item-key-discriminators.js';
 // axis. Sunk here because the doors live in `@objectstack/runtime` and
 // `@objectstack/rest`, which share no other common home.
 export * from './meta-write-capability.js';
+// [#27884] The read-side twin: which readers a stored metadata document of a
+// capability-gated type (datasource, external_catalog) is served to, asked by the
+// `/meta` type gate (`@objectstack/rest`) and the engine's stored-row read gate
+// (`@objectstack/objectql`) alike.
+export * from './stored-metadata-type-read-admission.js';
 
 // [commit 1408fe385 / #10101] The shared platform-row organization resolver — sunk here
 // from `@objectstack/plugin-audit` per the maintainer ruling recorded on

packages/metadata-core/src/stored-metadata-type-read-admission.ts

@@ -0,0 +1,80 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#27884] Which READERS a stored metadata document of a capability-gated type
+ * is served to — the ONE row both exits that serve the document ask.
+ *
+ * ## The two exits, one row
+ *
+ * A datasource document (a driver, its connection configuration and the handle
+ * of its bound secret) is stored as a row of the stored-metadata family
+ * (`sys_metadata`, and its version snapshots in `sys_metadata_history`), typed
+ * by the row's `type` column. It is served through two exits:
+ *
+ *  - the `/meta` read doors, whose type-level read admission
+ *    (`metaTypeReadRefusal`, `@objectstack/rest`) refuses a caller without the
+ *    row's capability before any store read; and
+ *  - every caller-context engine read of the family's rows, whichever door
+ *    issues it, narrowed by
+ *    {@link storedMetadataTypesWithheldFrom}, which the engine composes into
+ *    the read's predicate (`stored-metadata-type-read-gate.ts`,
+ *    `@objectstack/objectql`).
+ *
+ * One operation reached through two mounted routes cannot admit two different
+ * sets of callers, so both read THIS table. It lives here because the `/meta`
+ * gate lives in `@objectstack/rest` and the engine gate in
+ * `@objectstack/objectql`, neither of which depends on the other, and this
+ * package is the one both already depend on. `@objectstack/rest` re-exports it
+ * as `META_TYPE_READ_CAPABILITIES`, the same frozen object.
+ *
+ * ## The rows, and why each capability
+ *
+ *  - `datasource` → `manage_platform_settings`. The datasource family's own
+ *    door — `GET /api/v1/datasources[/:name]` (`DATASOURCE_ADMIN_CAPABILITY`,
+ *    `@objectstack/service-datasource`) — serves exactly that configuration and
+ *    admits only this capability.
+ *  - `external_catalog` → `manage_platform_settings`. The cached remote-schema
+ *    snapshot of a federated datasource: the remote tables and columns the
+ *    federation family's read door introspects live and admits only this
+ *    capability for.
+ *
+ * ⛔ A row names the capability the type's OWN door already requires —
+ * matched, never minted; a type with no such door has no row. ⛔ Never a
+ * second copy of this table at either exit.
+ */
+export const STORED_METADATA_TYPE_READ_CAPABILITIES: Readonly<Record<string, string>> = Object.freeze({
+    datasource: 'manage_platform_settings',
+    external_catalog: 'manage_platform_settings',
+});
+
+/**
+ * [#27884] The stored metadata types a caller is NOT served, read from
+ * {@link STORED_METADATA_TYPE_READ_CAPABILITIES}: every type whose capability
+ * is absent from the caller's resolved `systemPermissions` (the capability set
+ * the request's authorization resolver stamps on the execution context).
+ *
+ * Empty for a holder of every row's capability. Every listed type for a caller
+ * that carries no capability set at all, an anonymous one included: absence is
+ * never a grant. The caller's `isSystem` is not read here — an elevated read is
+ * the platform's own, and the engine gate admits it before asking.
+ */
+export function storedMetadataTypesWithheldFrom(caller: unknown): string[] {
+    const held = caller && typeof caller === 'object'
+        ? (caller as { systemPermissions?: unknown }).systemPermissions
+        : undefined;
+    const holds = Array.isArray(held) ? held : [];
+    return Object.entries(STORED_METADATA_TYPE_READ_CAPABILITIES)
+        .filter(([, capability]) => !holds.includes(capability))
+        .map(([type]) => type);
+}
+
+/**
+ * [#27884] Whether stored documents of `type` are served only to holders of a
+ * capability — the question a caller-blind exit asks (a realtime event, whose
+ * subscribers are unknown at publish, withholds such a document's body from
+ * everyone).
+ */
+export function isCapabilityGatedStoredMetadataType(type: unknown): boolean {
+    return typeof type === 'string'
+        && Object.prototype.hasOwnProperty.call(STORED_METADATA_TYPE_READ_CAPABILITIES, type);
+}

packages/objectql/src/engine-realtime-publish.ts

@@ -63,6 +63,9 @@ import {
   omitInternalFieldsFromWriteResponse,
 } from '@objectstack/core';
 import { TenantLayer0VerdictSchema } from '@objectstack/spec/security';
+// [#27884] The one row of capability-gated stored metadata types — the row the
+// `/meta` type gate and the engine's stored-row read gate read.
+import { isCapabilityGatedStoredMetadataType } from '@objectstack/metadata-core';
 import { resolveTenantFieldName, carriesOrganization } from './tenancy/system-write-organization.js';
 import type { ObjectQL } from './engine.js';
 
@@ -131,7 +134,9 @@ function eventRecordBody(value: unknown): Record<string, unknown> | undefined {
  *
  * `fullRow` is the whole written record (`input.after`), read for the `type`
  * column that selects the redactor — a `changes` patch need not carry it. A
- * body the redactor cannot judge is DROPPED from the event (fail-closed): the
+ * body of a capability-gated type (`isCapabilityGatedStoredMetadataType`,
+ * #27884) is DROPPED for every subscriber. A body the redactor cannot judge is
+ * DROPPED from the event (fail-closed): the
  * key is omitted rather than published raw. A non-record body, and every object
  * outside the set, pass through unchanged. Pure: the event body is a fresh
  * object, never the stored row.
@@ -149,7 +154,13 @@ function redactEventMetadataBody(
       : (fullRow && typeof fullRow === 'object' && !Array.isArray(fullRow)
           ? (fullRow as Record<string, unknown>)[STORED_METADATA_TYPE_COLUMN]
           : undefined);
-  const outcome = redactStoredMetadataBody(type, body[STORED_METADATA_BODY_COLUMN]);
+  // [#27884] A document of a capability-gated type (a datasource's connection
+  // configuration and secret handle) is served only to holders of the type's
+  // read capability, and the event's subscribers are not known here — so its
+  // body is withheld from every subscriber, the fail-closed arm below.
+  const outcome = isCapabilityGatedStoredMetadataType(type)
+    ? ({ ok: false } as const)
+    : redactStoredMetadataBody(type, body[STORED_METADATA_BODY_COLUMN]);
   if (outcome.ok) {
     if (outcome.body === body[STORED_METADATA_BODY_COLUMN]) return body;
     return { ...body, [STORED_METADATA_BODY_COLUMN]: outcome.body };

packages/objectql/src/engine-realtime-stored-metadata-body.test.ts

@@ -109,16 +109,18 @@ describe('[#21120] realtime data.record.* events redact a stored metadata body',
     expect(published).toHaveLength(1);
     const payload = published[0].payload as Record<string, any>;
     expect(JSON.stringify(payload)).not.toContain(CRED);
-    // The event still describes the row — type survives, the body is still present but projected.
+    // The event still describes the row — type survives. [#27884] A datasource
+    // body is withheld whole (the connection configuration with it): the
+    // type's documents are served only to holders of its read capability, and
+    // an event's subscribers are unknown at publish.
     expect(payload.after.type).toBe('datasource');
-    expect(payload.after.metadata).toBeDefined();
-    expect(JSON.parse(payload.after.metadata).config.authToken).toBeUndefined();
-    expect(JSON.parse(payload.after.metadata).config.url).toBe('libsql://db.turso.io');
+    expect(payload.after).not.toHaveProperty('metadata');
+    expect(JSON.stringify(payload)).not.toContain('libsql://db.turso.io');
   });
 
-  it('a credential-free sys_metadata body rides the event unchanged', async () => {
-    const clean = JSON.stringify({ name: 'v', driver: 'postgres', config: { host: 'h', database: 'd' } });
-    await engine.insert('sys_metadata', { id: 'm2', type: 'datasource', name: 'v', metadata: clean });
+  it('a credential-free sys_metadata body of an ungated type rides the event unchanged', async () => {
+    const clean = JSON.stringify({ name: 'v', label: 'V', type: 'grid' });
+    await engine.insert('sys_metadata', { id: 'm2', type: 'view', name: 'v', metadata: clean });
     const payload = published[0].payload as Record<string, any>;
     expect(payload.after.metadata).toBe(clean);
   });

packages/objectql/src/engine.ts

@@ -158,6 +158,8 @@ import { buildDriverOptions, driverReadScope, driverWriteTarget, engineLayer0Ver
 import { announceOpenMigrationGates, isOrganizationOwnershipCertified, isValueShapesMigrationVerified, noteAdmittedValueShapeViolation, readFileReferencesFlagRow, readMigrationFlagVerified, supplyTenantKeyCertifiedResolver, wasDatastoreCreatedFromEmpty } from './engine-migration-flags.js';
 import { ExecutionContext, ExecutionContextSchema, type InstalledPackageSource } from '@objectstack/spec/kernel';
 import { isStoredMetadataBodyObject } from '@objectstack/spec/kernel';
+// [#27884] The stored-row read gate, registered on the family's tables below.
+import { STORED_METADATA_TYPE_READ_GATE_OBJECTS, storedMetadataTypeReadGate } from './stored-metadata-type-read-gate.js';
 import type { FlowFunctionEffect } from '@objectstack/spec/automation';
 // Imported from spec directly rather than through `@objectstack/core`'s
 // re-export block: that block is labelled backward-compatibility, and this
@@ -5125,6 +5127,14 @@ export class ObjectQL implements IObjectQLEngine {
     if (process?.env?.OBJECTQL_WARN_LEGACY_HANDLER === '1') {
       (this as any)._warnLegacyHandler = true;
     }
+    // [#27884] The stored-row read gate: a caller-context read of a
+    // stored-metadata table serves a capability-gated type's documents only to
+    // holders of that capability — the row the `/meta` type gate reads. In the
+    // constructor, so no engine serves those tables without it. See
+    // `stored-metadata-type-read-gate.ts`.
+    for (const object of STORED_METADATA_TYPE_READ_GATE_OBJECTS) {
+      this.registerMiddleware(storedMetadataTypeReadGate, { object });
+    }
     this.logger.info('ObjectQL Engine Instance Created');
   }