Skip to content

Commit d062675

Browse files
committed
fix(pm): board-snapshot — a blank listing page is not a reading
The Board Snapshot check went red on this PR (and on every scheduled run since 2026-10-21, #24351) with a count-check SHORTFALL of the whole board: the open walk's first listing page answered `200 []`, the tool read it as the short page, declared the open set complete at zero records, and compared zero with the board's own count. Measured 2026-10-22: the issues listing answers blank to more than half of the requests made to it (39 of 68 probes), each with its own request id and its own count against the rate limit, so no cache is involved; cache directives, a cache-busting query, sort, direction, page size, state and since make no difference, and objectui and hotcrm answer the same way. `/repos/{repo}`, the pulls listing and the per-number comment pages never blanked. So a blank listing page is asked again, same URL and page size, up to `BLANK_PAGE_ASKS` (6) times, each ask a budgeted request and never a sleep. A blank that survives the bound is acted on by where the walk is (`blankPageStep`): past an anchor it is believed, because the cursor does not advance and the next run re-reads from it; from the beginning it is believed only if the board's own `open_issues_count`, read once as the witness, is zero — otherwise the walk stops with its cursor (`stopped_by: blank-listing`), nothing is marked complete, the count check stays pending and the run exits 0 as a planned stop, which the freshness row turns red if it becomes a streak. The manifest counts the re-asks in `run.blank_pages_reasked`; the summary prints the count and the stop. A new header section records the measurement. Self-test: a new battery of 13 cases (the pure verdicts, the measured failure healed on a fake board that blanks once, the bounded stop, the anchored blank believed with its cursor unmoved), floor 12 -> 13, 169 cases. The PR leg's own arguments (`--dry-run --limit=5 --max-requests=25`) ran six times against the live board: six exits 0, one of them a blank-listing stop and two after re-asks. Claude-Session: https://claude.ai/code/session_01VCLHrrmPbwQHZcc2yqdFHD Co-authored-by: Claude <noreply@anthropic.com>
1 parent c5c1735 commit d062675

1 file changed

Lines changed: 146 additions & 4 deletions

File tree

‎scripts/pm/board-snapshot.mjs‎

Lines changed: 146 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,48 @@
184184
* never sleeps and never retries the request — a loop against a zero budget is
185185
* how one repository's automation starves every other caller of that budget.
186186
*
187+
* ## A blank page is not a reading — measured 2026-10-22
188+
*
189+
* Since 2026-10-21 the issues listing of this board answers `200 []` to more
190+
* than half of the requests made to it: 39 of 68 probes on 2026-10-22, each
191+
* with its own `x-github-request-id` and its own count against the rate limit,
192+
* so no cache was involved; `cache-control: no-cache`, a cache-busting query,
193+
* `direction`, `sort`, `per_page`, `state` and a `since` made no difference,
194+
* and objectui and hotcrm answered the same way. `/repos/{repo}`, the pulls
195+
* listing and the per-number comment pages never did (#24351). This tool read
196+
* a blank first page as the short page, declared the open set complete at ZERO
197+
* records, and the count check reported a SHORTFALL of the whole board — the
198+
* PR leg's 12 requests were exactly a blank open listing followed by the
199+
* history walk.
200+
*
201+
* So a listing page that answers EMPTY is not believed on sight. It is asked
202+
* again — the same URL, the same page size, no sleep, each ask a request of
203+
* the budget — up to `BLANK_PAGE_ASKS` times, and a blank that survives the
204+
* bound is acted on by WHERE the walk is (`blankPageStep`):
205+
*
206+
* past an anchor (`since` set, or an offset page) it is believed. The
207+
* cursor does not advance on a blank page, so a blank
208+
* that was false costs one run and nothing permanent:
209+
* the next run re-reads from the same inclusive cursor.
210+
* from the beginning (no `since`, page 1) it is believed only if the board
211+
* itself reports no open item — `open_issues_count` of
212+
* `/repos/{repo}`, read once as the witness when the
213+
* first page is blank. A board that reports open items
214+
* cannot answer a blank first page, so the walk STOPS
215+
* with its cursor (`stopped_by: blank-listing`): nothing
216+
* is marked complete, the count check stays `pending`,
217+
* and the run exits 0 as a planned stop. The next run
218+
* asks again, and a streak of such runs is what the
219+
* freshness row turns red. Believing it would end a
220+
* phase at zero records — the measured failure.
221+
*
222+
* ⛔ This is not the retry the section above forbids: it re-asks a `200` that
223+
* carried no rows, never a refusal; it is bounded by the constant and by the
224+
* budget; and it never sleeps. `run.blank_pages_reasked` counts the re-asks
225+
* and the summary prints the count when it is not zero. The per-number reads
226+
* (comments, reviews) are not re-asked: an empty page is their ordinary
227+
* answer, and they were not seen to blank.
228+
*
187229
* ## The count check, and why a mismatch is red rather than a warning
188230
*
189231
* `rest-channel.md`: an enumeration without a count check is not a reading. So
@@ -562,6 +604,29 @@ export function nextWalkStep({ batchLength, lastUpdatedAt, since, page, perPage
562604
return { done: false, since, page: page + 1, reanchored: false };
563605
}
564606

607+
/** How many times one listing page is asked before a blank answer is acted on. */
608+
export const BLANK_PAGE_ASKS = 6;
609+
610+
/**
611+
* What the walk does with a listing page that answered EMPTY (§ A blank page
612+
* is not a reading). `asks` counts the asks of THIS page so far, blank every
613+
* time; `boardOpenItems` is the board's own `open_issues_count` (issues and
614+
* pull requests), or null when it was not read.
615+
*
616+
* ask-again below the bound: the same page once more.
617+
* believe at the bound, where a blank CAN be the board's answer — past an
618+
* anchor, where the cursor stays put and the next run re-reads.
619+
* stop at the bound, from the BEGINNING of a board that reports open
620+
* items (or whose count is unknown): believing it would end a
621+
* phase at zero records.
622+
*/
623+
export function blankPageStep({ asks, since, page, boardOpenItems, bound = BLANK_PAGE_ASKS }) {
624+
if (asks < bound) return { action: 'ask-again' };
625+
const fromTheBeginning = since === null && page === 1;
626+
if (fromTheBeginning && boardOpenItems !== 0) return { action: 'stop' };
627+
return { action: 'believe' };
628+
}
629+
565630
export const WALK_PHASES = Object.freeze(['open', 'history', 'incremental']);
566631

567632
/**
@@ -1238,6 +1303,7 @@ async function snapshot(repo, options) {
12381303
// the history walk at a delta cursor.
12391304
let stoppedIn = null;
12401305
let walkComplete = false;
1306+
let blankPagesReasked = 0;
12411307
let board = { open_issues_count: null, open_pull_requests: null, read_at: null };
12421308

12431309
/**
@@ -1248,10 +1314,32 @@ async function snapshot(repo, options) {
12481314
async function walk(key, state, walkBudget = budget) {
12491315
let since = cursors[key];
12501316
let page = 1;
1317+
let blankAsks = 0;
1318+
let witness;
12511319
for (;;) {
12521320
walkBudget.spend();
12531321
const batch = await rest(listingPath(repo, { since, page, state }));
12541322
const rows = Array.isArray(batch) ? batch : [];
1323+
if (rows.length === 0) {
1324+
// § A blank page is not a reading. The witness — the board's own open
1325+
// count — is read once per walk, and only when a first page is blank.
1326+
blankAsks += 1;
1327+
if (since === null && page === 1 && witness === undefined) {
1328+
walkBudget.spend();
1329+
witness = (await rest(`/repos/${repo}`))?.open_issues_count ?? null;
1330+
}
1331+
const verdict = blankPageStep({ asks: blankAsks, since, page, boardOpenItems: witness ?? null });
1332+
if (verdict.action === 'ask-again') {
1333+
blankPagesReasked += 1;
1334+
continue;
1335+
}
1336+
if (verdict.action === 'stop') {
1337+
stopped = { kind: 'blank-listing', reason: `the ${key} listing answered EMPTY ${blankAsks} times from the beginning while the board reports ${witness ?? 'an unknown number of'} open item(s)` };
1338+
return false;
1339+
}
1340+
} else {
1341+
blankAsks = 0;
1342+
}
12551343
for (const raw of rows) {
12561344
if (seen.has(raw.number)) continue;
12571345
if (options.limit && seen.size >= options.limit) {
@@ -1439,6 +1527,7 @@ async function snapshot(repo, options) {
14391527
delta_ran: deltaRan,
14401528
delta_completed_here: deltaCompletedHere,
14411529
delta_requests: deltaRequests,
1530+
blank_pages_reasked: blankPagesReasked,
14421531
},
14431532
});
14441533

@@ -1482,7 +1571,7 @@ export function renderRun(result, options) {
14821571
` open set ${openLine}`,
14831572
` delta ${deltaLine}`,
14841573
` since ${m.since ?? '(from the beginning)'} — ${m.since_reason}`,
1485-
` read ${m.run.numbers_read} number(s) in ${m.requests} request(s); walk ${m.run.walk_complete ? 'complete' : 'INCOMPLETE'}`,
1574+
` read ${m.run.numbers_read} number(s) in ${m.requests} request(s); walk ${m.run.walk_complete ? 'complete' : 'INCOMPLETE'}${m.run.blank_pages_reasked ? `; ${m.run.blank_pages_reasked} blank page(s) asked again` : ''}`,
14861575
` written ${result.written.length} file(s)${result.manifestMoved ? ' + manifest' : ''}`,
14871576
` archive ${m.counts.records} record(s): ${m.counts.issues_open} open / ${m.counts.issues_closed} closed issue(s), ${m.counts.pulls_open} open / ${m.counts.pulls_closed} closed pull request(s)`,
14881577
];
@@ -1519,6 +1608,12 @@ export function renderRun(result, options) {
15191608
` STOPPED rate limit: ${result.stopped.reason}`,
15201609
` Nothing was retried. The next run resumes the ${m.resume.phase} walk at ${m.resume.since}.`,
15211610
);
1611+
} else if (result.stopped?.kind === 'blank-listing') {
1612+
lines.push(
1613+
` STOPPED ${result.stopped.reason} — a blank page is not a reading.`,
1614+
' Nothing was believed and nothing was marked complete; the count check stays pending. The next run asks again,',
1615+
` ${m.resume.since ? `resuming the ${m.resume.phase} walk at ${m.resume.since}` : `from the beginning of the ${m.resume.phase} walk`}.`,
1616+
);
15221617
} else if (result.stopped) {
15231618
lines.push(
15241619
` paused ${result.stopped.reason} — this is a planned stop, not a failure.`,
@@ -1816,8 +1911,11 @@ function rearmThroughProxy(args) {
18161911
* because those are what the walk is steering by — a stub that ignored them
18171912
* would agree with any walk order at all.
18181913
*/
1819-
function fakeBoard({ rows, openPulls = [] }) {
1914+
function fakeBoard({ rows, openPulls = [], blankListings = 0 }) {
18201915
const calls = [];
1916+
// The blank reply measured 2026-10-22, reproduced: the first N listing GETs
1917+
// answer `[]` whatever the board holds. The witness and the pulls do not.
1918+
let blanksLeft = blankListings;
18211919
const serve = (path) => {
18221920
if (path === '/repos/o/r') return { open_issues_count: rows.filter((r) => r.state === 'open').length };
18231921
const url = new URL(`https://board${path}`);
@@ -1829,6 +1927,10 @@ function fakeBoard({ rows, openPulls = [] }) {
18291927
if (thread) return page === 1 ? (rows.find((r) => r.number === Number(thread[1]))?.thread ?? []) : [];
18301928
if (/^\/repos\/o\/r\/pulls\/\d+\/(reviews|comments)$/.test(url.pathname)) return [];
18311929
if (url.pathname !== '/repos/o/r/issues') return [];
1930+
if (blanksLeft > 0) {
1931+
blanksLeft -= 1;
1932+
return [];
1933+
}
18321934
const sinceAt = q.get('since') ? Date.parse(q.get('since')) : null;
18331935
const matching = rows
18341936
.filter((r) => (q.get('state') === 'open' ? r.state === 'open' : true))
@@ -1907,8 +2009,9 @@ const SELF_TEST_BATTERIES = Object.freeze({
19072009
'the rate-limit stop, and the two properties that are structural': 8,
19082010
'the delta walk, driven: the archive catches up with the live board': 19,
19092011
'the freshness row: a stale archive is never a green run': 18,
2012+
'the blank page: asked again, then believed or stopped by where the walk is': 13,
19102013
});
1911-
const SELF_TEST_BATTERY_FLOOR = 12;
2014+
const SELF_TEST_BATTERY_FLOOR = 13;
19122015
const UNATTRIBUTED_BATTERY = '(unattributed)';
19132016

19142017
let selfTestReachedVerdict = false;
@@ -2244,6 +2347,9 @@ export async function selfTest() {
22442347
const idemDir = mkdtempSync(join(tmpdir(), 'board-snapshot-idem-'));
22452348
const quietDir = mkdtempSync(join(tmpdir(), 'board-snapshot-quiet-'));
22462349
const staleDir = mkdtempSync(join(tmpdir(), 'board-snapshot-stale-'));
2350+
const blankDir = mkdtempSync(join(tmpdir(), 'board-snapshot-blank-'));
2351+
const blankStopDir = mkdtempSync(join(tmpdir(), 'board-snapshot-blank-stop-'));
2352+
const plainDir = mkdtempSync(join(tmpdir(), 'board-snapshot-plain-'));
22472353
try {
22482354
// THE REGRESSION, driven from the manifest board-archive actually carried.
22492355
//
@@ -2480,8 +2586,44 @@ export async function selfTest() {
24802586
t('…while the streak stamp it carries forward is the one the previous manifest opened, never this run\'s',
24812587
staleRun.manifest.freshness.pending_since === readJsonFile(join(staleDir, MANIFEST_NAME)).freshness.pending_since
24822588
&& Date.parse(staleRun.manifest.freshness.pending_since) < Date.parse(staleRun.manifest.generated_at) - STALE_AFTER_MS);
2589+
2590+
// -- the blank page ------------------------------------------------------
2591+
battery('the blank page: asked again, then believed or stopped by where the walk is');
2592+
const B = BLANK_PAGE_ASKS;
2593+
t('below the bound a blank page is asked again — from the beginning of a board that reports open items', blankPageStep({ asks: 1, since: null, page: 1, boardOpenItems: 118 }).action === 'ask-again');
2594+
t('…and past an anchor alike', blankPageStep({ asks: B - 1, since: '2026-10-21T00:00:00Z', page: 1, boardOpenItems: null }).action === 'ask-again');
2595+
t('at the bound, past an anchor, it is believed — the cursor stays put and the next run re-reads from it', blankPageStep({ asks: B, since: '2026-10-21T00:00:00Z', page: 1, boardOpenItems: 118 }).action === 'believe');
2596+
t('…and on an offset page alike', blankPageStep({ asks: B, since: null, page: 3, boardOpenItems: 118 }).action === 'believe');
2597+
t('at the bound, from the BEGINNING of a board that reports open items, the walk STOPS — believing it would end a phase at zero records', blankPageStep({ asks: B, since: null, page: 1, boardOpenItems: 118 }).action === 'stop');
2598+
t('…and so does a board whose count is unknown: a blank is believed only where it can be true', blankPageStep({ asks: B, since: null, page: 1, boardOpenItems: null }).action === 'stop');
2599+
t('…while a board that reports no open item can answer a blank first page, and is believed', blankPageStep({ asks: B, since: null, page: 1, boardOpenItems: 0 }).action === 'believe');
2600+
const smallOpen = Array.from({ length: 3 }, (_, i) => RAW({ number: 30000 + i, state: 'open', comments: 0, updated_at: `2026-10-2${i}T08:00:00Z` }));
2601+
const smallRows = [...smallOpen, oldClosed];
2602+
const blankOnce = fakeBoard({ rows: smallRows, blankListings: 1 });
2603+
const healed = await driveSnapshot({ dir: blankDir, board: blankOnce });
2604+
t('THE MEASURED FAILURE, HEALED: a first listing that answers [] no longer ends the open phase at zero — the second ask sees the rows, the open set completes with every one of them and the count check reads OK',
2605+
healed.manifest.walk.open_set.complete === true && healed.manifest.counts.issues_open === smallOpen.length && healed.manifest.count_check.verdict === 'ok' && healed.manifest.run.blank_pages_reasked === 1);
2606+
t('…the witness was read between the two asks, and the second ask is the SAME request: same URL, same page size',
2607+
blankOnce.calls[1] === '/repos/o/r' && blankOnce.calls[2] === blankOnce.calls[0] && blankOnce.calls[0].includes(`per_page=${PER_PAGE}`));
2608+
const blankAlways = fakeBoard({ rows: smallRows, blankListings: 99 });
2609+
const halted = await driveSnapshot({ dir: blankStopDir, board: blankAlways });
2610+
t('a listing that stays blank from the beginning STOPS the open walk: nothing is marked complete and the count check stays pending',
2611+
halted.stopped?.kind === 'blank-listing' && halted.manifest.walk.open_set.complete === false && halted.manifest.count_check.verdict === 'pending' && halted.manifest.resume?.stopped_by === 'blank-listing' && halted.manifest.resume?.phase === 'open');
2612+
t('…bounded to the request: exactly BLANK_PAGE_ASKS listings and one witness read, never a loop',
2613+
blankAlways.calls.filter((c) => c.startsWith('/repos/o/r/issues?')).length === B && halted.manifest.requests === B + 1);
2614+
const haltedRendered = renderRun(halted, { out: blankStopDir, dryRun: true });
2615+
t('…and the run says so and exits 0 — a planned stop the next run resumes, which the freshness row turns red if it becomes a streak',
2616+
haltedRendered.exitCode === EXIT_OK && /STOPPED the open listing answered EMPTY/.test(haltedRendered.text) && /a blank page is not a reading/.test(haltedRendered.text));
2617+
const plain = fakeBoard({ rows: smallRows });
2618+
const unpaid = await driveSnapshot({ dir: plainDir, board: plain });
2619+
const quietBlank = fakeBoard({ rows: smallRows, blankListings: 99 });
2620+
const believed = await driveSnapshot({ dir: plainDir, board: quietBlank });
2621+
t('a board that answers rows pays nothing, and past an anchor a blank that survives the bound is believed: the incremental walk completes with its cursor UNMOVED, so the next run re-reads from it',
2622+
unpaid.manifest.run.blank_pages_reasked === 0 && !/asked again/.test(renderRun(unpaid, { out: plainDir, dryRun: true }).text)
2623+
&& believed.stopped === null && believed.manifest.walk_phase === 'incremental' && believed.manifest.run.walk_complete === true
2624+
&& believed.manifest.run.blank_pages_reasked === B - 1 && believed.manifest.next_since === unpaid.manifest.next_since);
24832625
} finally {
2484-
for (const d of [liveDir, busyDir, stopDir, idemDir, quietDir, staleDir]) rmSync(d, { recursive: true, force: true });
2626+
for (const d of [liveDir, busyDir, stopDir, idemDir, quietDir, staleDir, blankDir, blankStopDir, plainDir]) rmSync(d, { recursive: true, force: true });
24852627
}
24862628

24872629
const declared = Object.keys(SELF_TEST_BATTERIES);

0 commit comments

Comments
 (0)