Skip to content

test(history): assert bounded work at ranked yield boundaries - #1252

Merged
pascalandr merged 1 commit into
devfrom
fix/history-yield-budget
Nov 1, 2026
Merged

pascalandr merged 1 commit into
devfrom
fix/history-yield-budget

Conversation

@pascalandr

Copy link
Copy Markdown
Contributor

Summary

Fix the recurrent ranked-history CI measurement, without changing production pruning/search code, thresholds, retries, input/security bounds, or suite coverage. The regression now checks real bounded work and event-loop progress rather than treating a loaded runner's wall clock as a synchronous-work measurement.

Based on origin/dev at fb44655cd, in registered worktree history-yield-budget; npm ci completed.

Root cause and production trace

  • plugin.ts's historyRanked RPC and the native session_search tool both enter queryBoundRankedHistory → withHistoryDatabase → queryRankedHistory. The workspace HTTP broker uses that same fixed RPC. Other direct iterator callers are tests/the local benchmark.
  • Prose-only rows above 64 KiB pass through shared history-content.ts's SQLite textProjection before the ranked scalar. SQLite still synchronously validates/traverses the stored JSON, including ignored metadata, but JavaScript receives only readable prose. Candidate body reads use the same projection; technical search intentionally decodes the bounded native body. Literal history shares the projection and remains untouched.
  • Collection streams descending rowid ranges, yielding at the expired 40 ms slice and between chunks. Lookahead yields between ranges; body judging yields at expired slices. Fresh directory validation and the final physical root/horizon fences remain intact. A running SQLite step/JSON parse finishes first, as the documented contract states.
  • The old test queued one immediate before searching and asserted its wall-clock latency was below 300 ms for a single 12,689,000-byte row. It neither isolated CPU scheduling nor asserted progress between successive rows. That deadline conflated bounded synchronous work with process descheduling.

Observed CI evidence

All three failures decoded only 50 bytes in prose-only JavaScript parsing:

PR / run / attempt First callback, prose-only Largest JSON.parse input
#1240 / 41246810971 / 1 363.26 ms 50 bytes
#1244 / 41276069250 / 1 309.17 ms 50 bytes
#1244 / 41282429738 / 2 367.55 ms 50 bytes

The affected production pruning sources match current dev. The latest logged runner was Ubuntu 26.04.1 / Node 24.20.0. pr-build.yml runs the entire server glob (370 files at this baseline), with no explicit test concurrency. Node 24 defaults to process isolation and max(availableParallelism() - 1, 1) concurrent test files; individual tests also create children. The job logs do not record CPU topology/utilization, so they cannot establish the historical contention level. Controlled Linux contention below demonstrates the measurement flaw without inferring unrecorded telemetry.

Replacement regression

  • Keep the same adversarial metadata and exact stored-row size, for two rows in one native rowid chunk, with both technical settings.
  • Keep real SQLite, the real ranked scalar/body reader and real setImmediate turns. Only performance.now is controlled: each completed classification/body read advances it by one expired slice.
  • Require scan → turn → scan → turn → body → turn → body → turn. This detects eager chunk evaluation and missing within-chunk/body yields, even when chunk-end or ownership-validation yields survive.
  • Require both rows to be scanned/judged/returned. Retain the prose-only <64 KiB decode assertion; require technical parsing to retain the full 12,689,000-byte body.
  • Document that responsiveness means bounded admitted work followed by event-loop progress, with a single synchronous SQLite/JSON operation allowed to finish. Existing independent latency, cancellation, oversized-input, ranking, ownership and native-contract checks remain in coverage.

Controlled validation

WSL2 Linux x86_64, Node 24.20.0, i9-14900KF. Baseline/fixed tests were bundled with the same installed esbuild for dependency-independent execution; production source is identical. Both were run pinned to CPU 0, idle and with four bounded busy Node workers inheriting that affinity. The workers were terminated after each run.

Unchanged dev, original test Prose-only callback wall time Process CPU until callback Largest decode Result
Idle 113.70 ms 112.33 ms 50 bytes pass
Four competitors / same CPU 592.83 ms 117.71 ms 50 bytes fails old 300 ms assertion

The new cases pass idle 2/2 and under identical contention 2/2, while observing the actual turns between all four row operations. This is not a claim of a universal per-row latency SLA or a production speedup: SQLite's admitted single-row work remains synchronous and hardware/load-dependent.

Negative controls (disposable bundles; no production file edits)

Deliberately broken implementation New regression
Replace shared text projection with raw m.data fails prose-only decode bound: 12,689,000 bytes entered JSON.parse
Disable collection's within-slice yield, retaining chunk-end yields fails both technical cases' turn order
Disable body's expired-slice yield, retaining validation yields fails both technical cases' turn order
Change candidate rows.iterate to eager rows.all fails both technical cases' turn order

Source-suite checks

  • Linux Node 24.20.0, real source via tsx: 291/291 pruning/history and pruning/history-route tests pass (no skips).
  • Windows Node 25.2.1: 91/91 history and history-route tests pass (no skips).
  • npm run typecheck --workspace @neuralnomads/codenomad passes.
  • git diff --check passes.
node --import tsx --test packages/server/src/opencode/session-pruning/*.test.ts packages/server/src/server/routes/session-pruning.test.ts 'packages/server/src/server/routes/session-history*.test.ts'
npm run typecheck --workspace @neuralnomads/codenomad

The Linux source run used a separate Linux esbuild binary via ESBUILD_BINARY_PATH alongside this Windows checkout's npm-ci dependencies. Tests use synthetic in-memory databases only. No shared daemon/user database was opened or stopped; no new native-runtime/desktop end-to-end claim is made. Public help is unchanged because runtime behavior and public contracts are unchanged.

Repeated Linux failures measured 309-368 ms until a scheduled immediate even
though prose-only search decoded just 50 bytes. The adversarial row still
requires a synchronous SQLite projection, and elapsed callback latency also
includes time the test process is descheduled. The native contract permits a
bounded SQLite step or JSON parse to finish before the 40 ms slice yields.

Replace the machine-dependent deadline with two real 12,689,000-byte rows and
a controlled slice clock advanced after each real classification/body read.
Require a real event-loop turn before the next row in both collection and
judging, retain the prose-only decode bound, and verify technical search still
decodes its bounded native body. Keep the test in the ordinary server suite.
Document precisely what its responsiveness check guarantees. Production
queries, ownership/cancellation fences, input bounds and native RPC contracts
are unchanged.

On Linux Node 24.20.0, unchanged dev took 113.7 ms wall/112.3 ms CPU idle and
592.8 ms wall/117.7 ms CPU with four competitors pinned to one CPU; both
decoded 50 bytes. The deterministic cases pass under the same contention.
Disposable bundle mutations fail when projection, scan yields, body yields or
incremental chunk streaming are broken. Linux pruning/history/routes: 291/291;
Windows history/routes: 91/91; server typecheck and diff checks pass.

@pascalandr pascalandr left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: PASS (COMMENT). P0: 0 | P1: 0 | P2: 0 | P3: 0.

Reviewed exact head 9a4134d81065929181ede6c94eaf1fd0c9d5ea85 against fb44655cdde57efc767ebe5e068c0e024b1d2ae3. No actionable findings. This is a justified regression-harness correction, not a production performance fix or a universal first-yield latency guarantee.

Contract and test fidelity

  • history-ranked-gatekeeper.test.ts:58–113 retains the exact 12,689,000-byte adversary (793,054 nested metadata records), exercises both technical settings, and now processes two rows within the same native 128-position chunk. Real SQLite, the production scalar, body reader, extraction/judge and real setImmediate callbacks execute. Only the slice clock is synthetic; the instrumentation invokes the original operations before advancing it. The hooks schedule callbacks rather than yielding themselves. The exact event order therefore distinguishes an event-loop turn from an await that only drains microtasks.
  • The original #1182 performance defect was candidate-side whole-row JavaScript decoding of ignored metadata. The retained <64 KiB assertion catches that specific regression independently of runtime scheduling. Technical coverage is strengthened by requiring a full 12,689,000-byte decode, and both rows must be scanned, judged and returned. No sleeps, retries, dropped technical cases or increased deadlines were introduced.
  • Production is byte-for-byte unchanged. The 16 MiB message admission guard, 64 KiB prefilter/projection handling, 40 ms cooperative slice, scan/judge limits, cancellation, authenticated cursors, directory ownership and final horizon/root fences remain in place. The two changed files are 115 and 352 lines; no oversized files were touched.

Independently reproduced Linux evidence

WSL2 Linux x86_64, Node 24.20.0, i9-14900KF; baseline/fixed bundles independently built from this checkout using its installed esbuild. Runs were pinned to CPU 0. Controlled contention used four bounded, readiness-confirmed child workers on that same CPU, all stopped afterward.

Original test on unchanged base First callback wall Process CPU to callback Largest JS decode Result
Idle 122.16 ms 119.65 ms 50 bytes pass
Four competitors 552.97 ms 110.87 ms 50 bytes fails the original 300 ms assertion

The replacement passes 2/2 both idle and under contention. This independently demonstrates that unchanged bounded work can fail the old wall-clock gate through descheduling; it does not establish the historical CI runners' unrecorded CPU load.

The remaining synchronous work is real: history-ranked-store.ts:88–93,190–209 still performs byte-size/cast checks and, for admitted prose-only large rows, SQLite JSON validation/traversal and readable projection before JavaScript receives the tiny result. Candidate and body reads each pay projection work. Technical mode transfers and decodes the bounded raw body, including its object graph; extraction and single-message judging also finish synchronously. The controlled clock does not measure or cap any of those individual costs.

A separately instrumented run around actual iterator.next() / body get() measured prose scan steps at 109.05 / 54.99 ms wall (108.51 / 54.98 ms process CPU), and body SQL reads at 115.01 / 111.15 ms wall. Under contention those prose scan steps took 532.84 / 279.04 ms wall with approximately 106.78 / 55.24 ms CPU. Technical scan steps took 217.88 / 308.92 ms wall idle. These are observed costs, not maximums. Thus the qualified contract is bounded admitted work followed by cooperative progress after an expired slice; neither a hard 40 ms operation limit nor the deleted 300/1800 ms fixture deadlines is established. The new documentation accurately makes this distinction. The independent existing large-readable-projection and indexed-streaming latency checks remain.

Red/green checks on real implementation variants

All mutations were applied only while building disposable scratch bundles; repository files were untouched.

Deliberate regression Observed failure
Revert candidate-only SQL projection, retaining the body projection prose case fails: 12,689,000 bytes entered JSON.parse
Replace shared text projection with raw data prose decode-bound failure
Remove collection's expired-slice yield, retaining chunk-end yields both cases fail inter-row turn order
Remove body's expired-slice yield, retaining ownership-validation yields both cases fail body turn order
Use eager rows.all() both cases fail streaming order
Force a real SQLite temporary sort with ORDER BY (m.rowid+0) both cases fail streaming order

Validation and limits

  • Windows Node 25.2.1: pruning/history and corresponding route source tests 291/291, zero skips.
  • Linux Node 24.20.0: the same source tests through tsx with a separate Linux esbuild executable 291/291, zero skips.
  • npm run typecheck --workspace @neuralnomads/codenomad: pass. git diff --check: pass. Worktree remains clean at the reviewed head.
  • Test command: node --import tsx --test packages/server/src/opencode/session-pruning/*.test.ts packages/server/src/server/routes/session-pruning.test.ts packages/server/src/server/routes/session-history*.test.ts.
  • These checks use synthetic in-memory databases. No shared daemon/user database, native desktop runtime, CI controls or repository files were changed. Worst-case single-row SQLite/JSON CPU cost across arbitrary shapes/hardware remains unqualified; the replacement intentionally has no direct elapsed-time regression gate for this metadata shape. No production speedup is claimed.

@pascalandr
pascalandr merged commit 72942a9 into dev Nov 1, 2026
27 checks passed
@pascalandr
pascalandr deleted the fix/history-yield-budget branch November 1, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant