Skip to content

Commit 1d6b7d0

Browse files
committed
fix(metadata-protocol)!: refuse a $searchFields entry naming a secret / password field at the data door
The door resolved the allowed search set from the object's declared searchableFields, which kept a credential entry, so the request passed and the engine's credential withhold (ADR-0100) then searched the rest of the declared set: a narrower search answered with a wider one. The door now refuses such an entry 400 INVALID_FIELD, judged by isCredentialFieldType, the predicate the engine's withhold reads. The engine's withhold stays as the fail-closed floor. Claude-Session: https://claude.ai/code/session_01L6bhgUCizoYWQVJWjhdR6F Co-authored-by: Claude <noreply@anthropic.com>
1 parent 637cfc6 commit 1d6b7d0

4 files changed

Lines changed: 198 additions & 1 deletion

File tree

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
'@objectstack/metadata-protocol': patch
3+
---
4+
5+
fix(metadata-protocol)!: a `$searchFields` entry naming a `secret` / `password` field is refused 400 `INVALID_FIELD` at the data door, even when the object's `searchableFields` declares it, instead of searching the rest of the declared set (#27829)
6+
7+
Clause-②: no (narrowing) — the data doors (`GET /data/:object`, `POST /data/:object/query`, and every other caller of `findData`) refuse a search-field override naming a credential field, which they answered 200 over the rest of the declared search set before. Nothing is widened.
8+
Metadata-protocol: none
9+
10+
<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes: a query value naming a credential field is refused where the engine silently dropped it. The fix for a caller is to search another field, which the refusal says; there is nothing for a migration to rewrite and no tombstone. -->
11+
12+
**BREAKING** accept-set narrowing at the query door (`@objectstack/metadata-protocol`), served by every Data API door.
13+
14+
- **Before.** On an object whose `searchableFields` declares a `secret` or `password` field, `?$search=TERM&$searchFields=api_key` passed the door, the engine withheld the credential column from the scan (ADR-0100), and its tolerant intersection then searched the rest of the declared set: 200 with rows matched in `name`. `$searchFields=name,api_key` silently dropped `api_key`. The credential itself was never searched.
15+
- **Now.** `400 INVALID_FIELD` naming the field (`Field 'api_key' on object 'OBJ' is type 'secret', a credential, and cannot be searched`), before any row is read — the same answer the auto-default path already gave a credential name. It covers the standalone `searchFields` / `$searchFields` slot and the object-form `search.fields`. Judged by `isCredentialFieldType` (`@objectstack/spec/data`), the predicate the engine's withhold reads.
16+
- **Unchanged.** A request that names no credential field is served as before; with no `$searchFields`, the declared set is searched and the declared credential entry is never scanned. The engine's withhold stays as the floor for direct engine callers.
17+
- **Who sees it.** An API caller naming a credential field in `$searchFields`, typically a client echoing the object's `searchableFields` verbatim on an object that declares one (the linter has flagged that declaration since `validate-searchable-fields` learned the credential check).
18+
19+
**The remedy.** Remove the credential field from the object's `searchableFields` and from the request; search another column.

‎content/docs/api/data-api.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@ corrupts something the earlier axes do not:
156156
| `?search=alpha&searchFields=no_such_field` | `400 INVALID_FIELD` |
157157
| `?search=alpha&searchFields=` (or `=,`, or `[]` in a body) | `400 VALIDATION_ERROR` — names no field, the answer `$select=` gets; leave the parameter out to scan the default set |
158158
| `?search=alpha&searchFields=amount` | `400 INVALID_FIELD` — real field, but not searchable |
159+
| `?search=alpha&searchFields=api_key` (a `secret` or `password` field, even one `searchableFields` declares) | `400 INVALID_FIELD` — a credential is never searched; remove it from `searchableFields` too |
159160
| `?search=alpha&searchFields=project_id.name` | `400 INVALID_FIELD` — search scans this object's own columns; mirror the related title instead (see below) |
160161
| `?search=alpha` on an object with no searchable field (two lookups and a `formula` title, say) | `400 INVALID_FIELD` — nothing to scan; declare a stored field searchable or mirror the title (see below) |
161162
| `groupBy: ["status"]` | one bucket per status value |

‎packages/metadata-protocol/src/protocol-query-validation.ts‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ import type { ObjectStackProtocolImplementation } from './protocol.js';
4646
import { FILTER_LOGICAL_KEYS, ODATA_SPELLING, QUERY_PARAM_NEAR_MISS, collectFilterReads, invalidSortError, isObjectOfDisabledPackage, isPlainRecord, nearMissKey, objectPackageDisabledError, quotedQueryValue, suggestFieldName } from './protocol.js';
4747
import { objectNotFoundError, relationConditionSites } from '@objectstack/core';
4848
import { collectInternalWriteResponseFields } from './write-response-internal-fields.js';
49-
import { AggregationFunction, DateGranularity, REFERENCE_VALUE_TYPES, SEARCHABLE_ENUM_TYPES, SEARCHABLE_TEXTUAL_TYPES, SEARCH_AUTO_EXCLUDED_FIELDS, apiExposureDenialReason, canServeApiOperation, classifyDottedFilterHead, effectiveOperationsArray, isFilterAST, isVirtualSearchField, parseFilterAST, referenceTargetOf, resolveEffectiveApiMethods, resolveSearchFieldResolution } from '@objectstack/spec/data';
49+
import { AggregationFunction, DateGranularity, REFERENCE_VALUE_TYPES, SEARCHABLE_ENUM_TYPES, SEARCHABLE_TEXTUAL_TYPES, SEARCH_AUTO_EXCLUDED_FIELDS, apiExposureDenialReason, canServeApiOperation, classifyDottedFilterHead, effectiveOperationsArray, isCredentialFieldType, isFilterAST, isVirtualSearchField, parseFilterAST, referenceTargetOf, resolveEffectiveApiMethods, resolveSearchFieldResolution } from '@objectstack/spec/data';
5050

5151
/**
5252
* [#20802] The nested-relation spelling of a dotted relation path, as the
@@ -959,10 +959,22 @@ export function assertSearchFieldsAreSearchable(
959959
// types have a column.
960960
const virtual = names.filter((n) => !allowedSet.has(n) && gate.known.has(n) && isVirtualSearchField(gate.fields[n]));
961961
const unsearchable = names.filter((n) => !allowedSet.has(n) && gate.known.has(n) && !isVirtualSearchField(gate.fields[n]));
962+
// [#27829] A CREDENTIAL field (`secret` / `password`) the allowed set kept —
963+
// only a declared `searchableFields` keeps one; the auto default never does,
964+
// and refuses it below by its type — is refused here too. The engine
965+
// withholds every credential column from the scan (ADR-0100, the
966+
// fail-closed floor), so admitting the name let the engine's tolerant
967+
// intersection search the REST of the declared set: a narrower search
968+
// answered with a wider one, the #4254 posture this gate exists for.
969+
// Judged by the predicate the engine's withhold reads
970+
// (`isCredentialFieldType`, `@objectstack/spec/data`), so the door and the
971+
// engine cannot disagree about which fields are credentials.
972+
const credential = names.filter((n) => allowedSet.has(n) && isCredentialFieldType(gate.fields[n]?.type));
962973
const [offenders, reason] =
963974
unknown.length > 0 ? [unknown, 'unknown' as const]
964975
: staleDeclared.length > 0 ? [staleDeclared, 'stale-declared' as const]
965976
: virtual.length > 0 ? [virtual, 'virtual' as const]
977+
: credential.length > 0 ? [credential, 'credential' as const]
966978
: [unsearchable, 'unsearchable' as const];
967979
if (offenders.length === 0) return;
968980
const first = offenders[0];
@@ -985,6 +997,18 @@ export function assertSearchFieldsAreSearchable(
985997
+ 'look like coverage; remove it there as well.'
986998
: '')
987999
+ ` Mirror the computed value onto a stored text field on '${object}' and search that instead.`;
1000+
} else if (reason === 'credential') {
1001+
detail = `Field '${first}' on object '${object}' is type '${gate.fields[first]?.type}', a credential, `
1002+
+ 'and cannot be searched'
1003+
+ (offenders.length > 1 ? ` (also: ${offenders.slice(1).join(', ')})` : '')
1004+
+ `. 'search' never scans a credential column, so '${param}' naming it cannot narrow `
1005+
+ 'anything — the engine used to drop it and scan the rest of the declared columns instead, '
1006+
+ 'answering a NARROWER search with a WIDER one.'
1007+
+ (declaredSet.has(first)
1008+
? " The object's 'searchableFields' declares it, which is what made the entry look searchable; "
1009+
+ 'remove it there as well.'
1010+
: '')
1011+
+ ' Search another column instead.';
9881012
} else if (reason === 'unknown') {
9891013
// A dotted path is a special unknown: plausible vocabulary from the
9901014
// select/sort axes, but search scans this object's own columns.
Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#27829] A `$searchFields` entry naming a CREDENTIAL field (`secret` /
5+
* `password`) is refused `400 INVALID_FIELD` at the Data API door, naming the
6+
* field, even when the object's own `searchableFields` declares it — over a
7+
* real `SqlDriver` (SQLite), the harness of `data-empty-search-fields-door.test.ts`.
8+
*
9+
* Before: the door (`assertSearchFieldsAreSearchable`) resolved the allowed set
10+
* from the declaration, which kept the credential entry, so the request passed;
11+
* the engine then withheld the credential column from the scan (ADR-0100) and,
12+
* by its tolerant intersection, searched the REST of the declared set.
13+
* `?$search=…&$searchFields=api_key` answered 200 with rows matched in `name` —
14+
* a narrower search answered with a wider one. The engine's withhold stays as
15+
* the fail-closed floor; this file pins the door's half: the status, `code`,
16+
* named field, that the driver was never asked, and the controls unchanged.
17+
*/
18+
19+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
20+
import { ObjectQL } from '@objectstack/objectql';
21+
import { SqlDriver } from '@objectstack/driver-sql';
22+
import { ObjectStackProtocolImplementation } from '@objectstack/metadata-protocol';
23+
import { RestServer } from './rest-server';
24+
import { installDefaultOrganization } from './default-organization.testkit.js';
25+
26+
/** object name · the credential type its `api_key` field carries. */
27+
const OBJECTS = [
28+
['rest_search_credential_secret_27829', 'secret'],
29+
['rest_search_credential_password_27829', 'password'],
30+
] as const;
31+
32+
const NAMES = ['qq7-a', 'qq7-b', 'other'];
33+
34+
/** The term every case searches: in two rows' `name`. */
35+
const TERM = 'qq7';
36+
37+
function createMockServer() {
38+
const noop = () => {};
39+
return { get: noop, post: noop, put: noop, delete: noop, patch: noop, use: noop, listen: async () => {}, close: async () => {} };
40+
}
41+
42+
function makeRes() {
43+
const res: any = {
44+
write: () => true, end: () => {},
45+
header: () => res,
46+
status: (code: number) => { res._status = code; return res; },
47+
json: (body: any) => { res._json = body; return res; },
48+
};
49+
return res;
50+
}
51+
52+
describe('[#27829] $searchFields naming a declared credential field on the Data API door — sqlite', () => {
53+
let engine: ObjectQL;
54+
let driver: any;
55+
const reads = { n: 0 };
56+
let list: (object: string, query: Record<string, string>) => Promise<{ status: number; body: any }>;
57+
let query: (object: string, body: Record<string, unknown>) => Promise<{ status: number; body: any }>;
58+
59+
beforeAll(async () => {
60+
driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true } as any);
61+
engine = new ObjectQL();
62+
engine.registerDriver(driver, true);
63+
await engine.init();
64+
for (const [name, ctype] of OBJECTS) {
65+
engine.registry.registerObject({
66+
name,
67+
label: `Credential ${ctype} 27829`,
68+
searchableFields: ['name', 'api_key'],
69+
fields: {
70+
name: { name: 'name', type: 'text' as const },
71+
api_key: { name: 'api_key', type: ctype },
72+
},
73+
} as any);
74+
}
75+
await engine.syncSchemas();
76+
// A real `single` install's Default Organization (ADR-0131 D9, D1).
77+
await installDefaultOrganization(engine);
78+
for (const [object] of OBJECTS) {
79+
for (const [i, name] of NAMES.entries()) await engine.insert(object, { id: `r${i}`, name } as any);
80+
}
81+
82+
const watched = new Set<string>(OBJECTS.map(([o]) => o));
83+
for (const verb of ['find', 'findOne', 'count', 'aggregate'] as const) {
84+
const real = driver[verb].bind(driver);
85+
driver[verb] = (o: string, ...rest: unknown[]) => { if (watched.has(o)) reads.n += 1; return real(o, ...rest); };
86+
}
87+
88+
const protocol = new ObjectStackProtocolImplementation(engine as any);
89+
const rest = new RestServer(createMockServer() as any, protocol as any, { api: { requireAuth: false } } as any);
90+
(rest as any).resolveExecCtx = async () => ({ userId: 'test-user' });
91+
rest.registerRoutes();
92+
const getRoute = rest.getRoutes().find((r: any) => r.method === 'GET' && r.path === '/api/v1/data/:object');
93+
expect(getRoute, 'GET /api/v1/data/:object').toBeDefined();
94+
list = async (object, q) => {
95+
const res = makeRes();
96+
// What a querystring carries: strings.
97+
await getRoute!.handler({ params: { object }, query: { $orderby: 'name', $search: TERM, ...q }, headers: {} } as any, res);
98+
return { status: res._status ?? 200, body: res._json };
99+
};
100+
const postRoute = rest.getRoutes().find((r: any) => r.method === 'POST' && r.path === '/api/v1/data/:object/query');
101+
expect(postRoute, 'POST /api/v1/data/:object/query').toBeDefined();
102+
query = async (object, body) => {
103+
const res = makeRes();
104+
// What the wire carries: JSON.
105+
await postRoute!.handler({ params: { object }, body: JSON.parse(JSON.stringify(body)), query: {}, headers: {} } as any, res);
106+
return { status: res._status ?? 200, body: res._json };
107+
};
108+
});
109+
110+
afterAll(async () => {
111+
try { await engine?.destroy(); } catch { /* noop */ }
112+
});
113+
114+
for (const [object, ctype] of OBJECTS) {
115+
it(`${ctype}: $searchFields=api_key and $searchFields=name,api_key are 400 INVALID_FIELD naming api_key — no read`, async () => {
116+
const before = reads.n;
117+
for (const searchFields of ['api_key', 'name,api_key']) {
118+
const res = await list(object, { $searchFields: searchFields });
119+
const label = `$searchFields=${searchFields}: ${JSON.stringify(res.body)}`;
120+
expect(res.status, label).toBe(400);
121+
expect(res.body.code, label).toBe('INVALID_FIELD');
122+
expect(res.body.error, label).toContain(`Field 'api_key' on object '${object}' is type '${ctype}'`);
123+
}
124+
expect(reads.n - before, 'no read of the object — the refusal precedes the driver').toBe(0);
125+
});
126+
127+
it(`${ctype}: POST query — searchFields and the object-form search.fields naming api_key are the same refusal — no read`, async () => {
128+
const before = reads.n;
129+
for (const body of [{ search: TERM, searchFields: ['api_key'] }, { search: { query: TERM, fields: ['name', 'api_key'] } }]) {
130+
const res = await query(object, body);
131+
const label = `${JSON.stringify(body)}: ${JSON.stringify(res.body)}`;
132+
expect(res.status, label).toBe(400);
133+
expect(res.body.code, label).toBe('INVALID_FIELD');
134+
expect(res.body.error, label).toContain(`Field 'api_key' on object '${object}' is type '${ctype}'`);
135+
}
136+
expect(reads.n - before, 'no read of the object — the refusal precedes the driver').toBe(0);
137+
});
138+
139+
it(`${ctype}: controls — $searchFields=name narrows, no $searchFields searches the declared set, as before`, async () => {
140+
const narrowed = await list(object, { $searchFields: 'name' });
141+
expect(narrowed.status, JSON.stringify(narrowed.body)).toBe(200);
142+
expect(narrowed.body.records.map((r: any) => r.name)).toEqual(['qq7-a', 'qq7-b']);
143+
144+
const all = await list(object, {});
145+
expect(all.status, JSON.stringify(all.body)).toBe(200);
146+
expect(all.body.records.map((r: any) => r.name)).toEqual(['qq7-a', 'qq7-b']);
147+
148+
const posted = await query(object, { search: TERM, searchFields: ['name'] });
149+
expect(posted.status, JSON.stringify(posted.body)).toBe(200);
150+
expect(posted.body.records).toHaveLength(2);
151+
});
152+
}
153+
});

0 commit comments

Comments
 (0)