Skip to content

Commit 4cd19ec

Browse files
committed
fix(metadata-protocol): an offset with no limit reports the filtered total
`findData` treated a request as paged only when it carried a limit. Without one it kept `total = records.length`, which is the match count minus the offset once an offset is given (`?$skip=3&$count=true` over 14 rows answered `total: 11`, and an offset past the end answered `total: 0`). An offset with no limit now takes the same count path as a paged request, `$count=false` arm included. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FEkQ5pNK9fm2KtGj8ijVqC
1 parent 310cb96 commit 4cd19ec

2 files changed

Lines changed: 205 additions & 3 deletions

File tree

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* #28547 — a list's `total` is the filtered count whatever the offset, also
5+
* when no limit is given.
6+
*
7+
* ## What was wrong
8+
*
9+
* `findData` decided "is this a paged request?" from the limit alone. Without
10+
* one it kept `total = records.length` on the reasoning that "the full result
11+
* set is returned". With an offset that reasoning is false: the read returns
12+
* every match AFTER the offset, so the total short-counted by exactly the
13+
* offset. Measured on the stock showcase (`showcase_account`, 14 rows):
14+
*
15+
* GET ?$skip=3&$count=true 11 rows, total 11 ← the defect
16+
* GET ?$skip=3&$top=100&$count=true 11 rows, total 14
17+
* POST {"offset":3,"count":true} total 11 ← the defect
18+
* POST {"offset":999,"count":true} total 0 ← the defect
19+
*
20+
* ## What this suite pins
21+
*
22+
* 1. An offset with no limit reports the filtered count, on both the GET
23+
* dialect (`$skip`) and the POST body form (`offset`), and an offset past
24+
* the end still reports it (so "add the offset back" is not a fix: it
25+
* would answer 999 there).
26+
* 2. CONTROL — a filter and a search move the count like any other paged
27+
* request (#22790: the COUNT carries what the rows were read with).
28+
* 3. The neighbouring arms keep their meaning: no offset and no limit issues
29+
* no COUNT; a limit is unchanged; `$count=false` with an offset issues no
30+
* COUNT and omits `total` (#10757).
31+
*/
32+
33+
import { describe, it, expect, vi } from 'vitest';
34+
import { ObjectStackProtocolImplementation } from './protocol.js';
35+
36+
const SCHEMA = {
37+
name: 'mz_account',
38+
nameField: 'name',
39+
fields: {
40+
name: { name: 'name', type: 'text' },
41+
tier: { name: 'tier', type: 'text' },
42+
},
43+
};
44+
45+
// 14 rows, like the showcase's seeded `showcase_account`; 4 of them in tier `b`.
46+
const ROWS = Array.from({ length: 14 }, (_, i) => ({
47+
id: `a${i}`,
48+
name: `a${i}`,
49+
tier: i % 4 === 3 || i === 0 ? 'b' : 'a',
50+
}));
51+
const TIER_B = ROWS.filter((r) => r.tier === 'b').length;
52+
53+
/**
54+
* Equality-only `where`, the shape the implicit field filter `?tier=b` lowers
55+
* to. Anything else is REFUSED rather than answered silently wrong.
56+
*/
57+
function matches(row: Record<string, unknown>, where: unknown): boolean {
58+
if (where == null) return true;
59+
if (typeof where !== 'object' || Array.isArray(where)) {
60+
throw new Error(`test double: unexpected where shape ${JSON.stringify(where)}`);
61+
}
62+
return Object.entries(where as Record<string, unknown>).every(([k, v]) => {
63+
if (k.startsWith('$')) throw new Error(`test double: combinator '${k}' is not implemented`);
64+
if (v !== null && typeof v === 'object') throw new Error(`test double: operator on '${k}' is not implemented`);
65+
return row[k] === v;
66+
});
67+
}
68+
69+
/**
70+
* An engine double that honours `offset` and `limit` as written, and counts
71+
* over the same `where`.
72+
*/
73+
function makeProtocol() {
74+
const find = vi.fn(async (_object: string, opts: any = {}) => {
75+
const hits = ROWS.filter((r) => matches(r, opts.where));
76+
const offset = typeof opts.offset === 'number' ? opts.offset : 0;
77+
const sliced = hits.slice(offset);
78+
return opts.limit !== undefined ? sliced.slice(0, opts.limit) : sliced;
79+
});
80+
const count = vi.fn(async (_object: string, opts: any = {}) => ROWS.filter((r) => matches(r, opts.where)).length);
81+
const engine = {
82+
registry: { getObject: (n: string) => (n === SCHEMA.name ? SCHEMA : undefined) },
83+
find,
84+
count,
85+
aggregate: vi.fn(async () => [] as unknown[]),
86+
};
87+
return { p: new ObjectStackProtocolImplementation(engine as any), find, count };
88+
}
89+
90+
async function list(query: Record<string, unknown>) {
91+
const h = makeProtocol();
92+
const result: any = await h.p.findData({ object: SCHEMA.name, query } as never);
93+
return { result, ...h };
94+
}
95+
96+
describe('[#28547] an offset with no limit reports the filtered total', () => {
97+
it('GET ?$skip=3&$count=true answers the match count, not the count after the offset', async () => {
98+
const { result, count } = await list({ $skip: '3', $count: 'true' });
99+
100+
expect(result.records).toHaveLength(11);
101+
expect(result.total).toBe(14);
102+
expect(result.hasMore).toBe(false);
103+
expect(count).toHaveBeenCalledTimes(1);
104+
});
105+
106+
it('POST {"offset":3,"count":true} answers the same', async () => {
107+
const { result } = await list({ offset: 3, count: true });
108+
109+
expect(result.records).toHaveLength(11);
110+
expect(result.total).toBe(14);
111+
expect(result.hasMore).toBe(false);
112+
});
113+
114+
it('an offset past the end returns no rows and still reports the match count', async () => {
115+
const { result } = await list({ offset: 999, count: true });
116+
117+
expect(result.records).toEqual([]);
118+
expect(result.total).toBe(14);
119+
expect(result.hasMore).toBe(false);
120+
});
121+
122+
it('agrees with the same offset under a limit', async () => {
123+
const withoutLimit = await list({ $skip: '3', $count: 'true' });
124+
const withLimit = await list({ $skip: '3', $top: '100', $count: 'true' });
125+
126+
expect(withoutLimit.result.total).toBe(withLimit.result.total);
127+
expect(withoutLimit.result.records).toEqual(withLimit.result.records);
128+
});
129+
130+
it('an absent $count reports the same total as $count=true', async () => {
131+
const { result } = await list({ skip: '3' });
132+
133+
expect(result.total).toBe(14);
134+
});
135+
});
136+
137+
describe('[#28547] CONTROL — the count is the filtered count', () => {
138+
it('a filter moves the total, and the rows are read past the offset', async () => {
139+
const { result, count } = await list({ $skip: '1', tier: 'b' });
140+
141+
expect(result.records).toHaveLength(TIER_B - 1);
142+
expect(result.total).toBe(TIER_B);
143+
expect((count.mock.calls[0] as unknown[])[1]).toMatchObject({ where: { tier: 'b' } });
144+
});
145+
146+
it('a search is counted with the search the rows were read with (#22790)', async () => {
147+
const { count, find } = await list({ $skip: '2', $search: 'a1' });
148+
149+
expect(count).toHaveBeenCalledTimes(1);
150+
expect((count.mock.calls[0] as unknown[])[1]).toMatchObject({ search: 'a1' });
151+
expect((find.mock.calls[0] as unknown[])[1]).toMatchObject({ search: 'a1' });
152+
});
153+
});
154+
155+
describe('[#28547] the neighbouring arms keep their meaning', () => {
156+
it('no offset and no limit: no COUNT, total is the set length', async () => {
157+
const { result, count } = await list({ $count: 'true' });
158+
159+
expect(count).not.toHaveBeenCalled();
160+
expect(result.total).toBe(14);
161+
expect(result.hasMore).toBe(false);
162+
});
163+
164+
it('an explicit offset of 0 is no offset: no COUNT', async () => {
165+
const { result, count } = await list({ offset: 0, count: true });
166+
167+
expect(count).not.toHaveBeenCalled();
168+
expect(result.total).toBe(14);
169+
});
170+
171+
it('a limit is unchanged: one COUNT, total is the match count, hasMore from the page', async () => {
172+
const { result, count } = await list({ offset: 3, limit: 5, count: true });
173+
174+
expect(count).toHaveBeenCalledTimes(1);
175+
expect(result.records).toHaveLength(5);
176+
expect(result.total).toBe(14);
177+
expect(result.hasMore).toBe(true);
178+
});
179+
180+
it('$count=false with an offset issues no COUNT and omits total (#10757)', async () => {
181+
for (const query of [{ $skip: '3', $count: 'false' }, { offset: 3, count: false }]) {
182+
const { result, count } = await list(query);
183+
184+
expect(count, JSON.stringify(query)).not.toHaveBeenCalled();
185+
expect('total' in result, JSON.stringify(query)).toBe(false);
186+
expect(result.records, JSON.stringify(query)).toHaveLength(11);
187+
expect(result.hasMore, JSON.stringify(query)).toBe(false);
188+
}
189+
});
190+
});

‎packages/metadata-protocol/src/protocol.ts‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13999,7 +13999,17 @@ export class ObjectStackProtocolImplementation implements
1399913999
// page, so `records.length` is the page size — NOT the match total. Run a
1400014000
// count over the same `where` so the client can render total pages and know
1400114001
// whether more pages remain (true server-side pagination). Without a limit
14002-
// the full result set is returned, so its length already IS the total.
14002+
// AND without an offset the full result set is returned, so its length
14003+
// already IS the total.
14004+
//
14005+
// [#28547] An offset with no limit is a paged request too. The read
14006+
// returns every match AFTER the offset, so `records.length` is the
14007+
// match count MINUS the offset: `?$skip=3&$count=true` over 14 rows
14008+
// answered `total: 11` while `?$skip=3&$top=100&$count=true` answered
14009+
// 14, and an offset past the end answered `total: 0`. Adding the offset
14010+
// back is not the fix (it says 999 for `offset: 999` over 14 rows), so
14011+
// such a request takes the one count path below, `$count=false` arm
14012+
// included: `total` is the filtered count whatever the offset.
1400314013
//
1400414014
// [#22790] The count takes the query's `search` / `searchFields` too: the
1400514015
// engine expands them through the one ADR-0061 expander `find` uses, so
@@ -14022,7 +14032,7 @@ export class ObjectStackProtocolImplementation implements
1402214032
const pageOffset = typeof options.offset === 'number' && options.offset > 0 ? options.offset : 0;
1402314033
let total: number | undefined = records.length;
1402414034
let hasMore = false;
14025-
if (pageLimit !== undefined) {
14035+
if (pageLimit !== undefined || pageOffset > 0) {
1402614036
// `distinct` used to suppress the count here too — #4286 finding 2:
1402714037
// the flag's ONLY observable effect platform-wide, on a capability
1402814038
// that never deduplicated a row. Removed with `query.distinct`
@@ -14045,7 +14055,9 @@ export class ObjectStackProtocolImplementation implements
1404514055
// `hasMore` is still answered, from the page alone: a FULL page
1404614056
// means there may be more. It never over-reports the data — it
1404714057
// can only say "maybe more" on an exactly-full last page.
14048-
hasMore = records.length === pageLimit;
14058+
// With an offset and no limit, every row after the offset came
14059+
// back, so there is no further page.
14060+
hasMore = pageLimit !== undefined && records.length === pageLimit;
1404914061
total = undefined;
1405014062
} else {
1405114063
// [#10757] `counted` is a separate, always-assigned local so

0 commit comments

Comments
 (0)