perf: batch the prediction fan-out into one kernel call (35% faster planning) by springfall2008 · Pull Request #4540 · springfall2008/batpred

springfall2008 · 2026-08-15T21:19:09Z

What

Replaces the per-scenario prediction fan-out with a lazily-flushed batch that reaches the C++ kernel in a single pk_run_batch call, and removes the multiprocessing pool it supersedes.

launch_run_prediction_* no longer runs anything: it prepares the trial inputs, queues a BatchJob on the Prediction, and returns a BatchHandle. The first .get() flushes the whole pending batch — prediction-cache hits resolved in Python, intra-batch duplicates collapsed, the remainder marshalled once and run in one call — then shapes each job's result into exactly what its thread_run_prediction_* returned before. Anything the kernel cannot take (no binary, debug_enable, a per-job status failure) falls back to the Python engine per job.

The call sites did not have to change: every fan-out in plan.py was already written launch-all-then-collect-all for the old pool.

Measured against current `main`

Each side running its own Python and its own kernel, interleaved, best of 3, same machine. All 20 scenarios identical on metric, cost, both PV futures, final SoC, battery cycles and carbon — 0 mismatches out of 140 fields compared.

20-scenario suite	`main`	this PR
serial (`threads: 0`)	41.70s	26.26s	37.0% faster
default (`threads: auto`)	39.53s	24.82s	37.2% faster

Window start/end arrays are now marshalled once per fan-out instead of once per scenario, and the ctypes boundary is crossed once instead of thousands of times. The win holds with parallelism off, so it is the batching rather than a threading effect.

Also fixes a crash

a3254924 made PkScenario::soc_out optional — Python passes NULL for every cached run, which is the common path — but left PK_ABI_VERSION at 3. An ABI 3 binary writes to soc_out unconditionally, so the loader would accept a stale binary and Predbat would segfault on its first prediction instead of falling back.

Reproduced by building the kernel from main and pointing PREDBAT_KERNEL_SO at it: model_kernel exits 139. With the bump to ABI 4 the loader reports stale binary (abi 3 revision 5, expected abi 4 revision 5) - using Python engine, which is what the check exists to do. Anyone with a locally built .so from an older checkout, or an install whose per-architecture binary was not replaced, would have hit this.

`threads` changes meaning

It used to size a pool of worker processes; it now sizes the C++ kernel's thread pool. Results are identical at any setting, and 0 and 1 both mean serial. docs/apps-yaml.md updated.

Incidentally the old pool was broken outside the __main__ entry point: its workers rebuild their Prediction from a module global populated only in the parent, which needs fork start semantics that hass.py sets only under if __name__ == "__main__". On any spawn-default platform (macOS since 3.8) every worker started empty and every scenario died with KeyError: 'dict' — the benchmark above needed fork forced to get numbers out of main at all. Removing the pool removes that failure mode.

Risk

The design rests on one invariant: no caller may mutate a list or window dict it passed to launch_run_prediction_* before reading the matching handle, because the read is now deferred to flush time. All four fan-out sites were verified against it, one pre-existing violation was fixed (thread_run_prediction_export used to write the trial start into the caller's window dict, safe only because each pool worker mutated its own unpickled copy), and the invariant is stated on all eight queue/launch docstrings.

Testing

20-scenario byte-identical gate re-run at every step, including with threading enabled.
pk_run_batch pinned bit-identical to a loop of pk_run across 60 scenarios at 1, 2, 4 and 8 threads, and across heterogeneous window lists.
New prediction_batch suite: queued-vs-direct equivalence for all trial shapes, laziness, dedup, cache hits, both fallback routes, past-window SoC ranges, coarse steps, and result-to-job routing.
Full suite green (211 tests), pre-commit clean.

Three defects were found by deliberately breaking the code and checking the suite still passed: a kernel_supported gate whose test compared values the two engines agree on to 1e-6, a job-to-result routing path nothing constrained, and a replacement test whose trials produced identical costs. All three are now pinned by tests that demonstrably fail when their target breaks.

🤖 Generated with Claude Code

Two changes to the kernel, one that pays off immediately and one that lays the groundwork for running a fan-out of scenarios in parallel. 1. Do not compute the per-minute SoC series when nobody reads it (-13%) run_prediction_kernel allocated an n_steps buffer for every scenario and told the kernel to fill it, but a cached run - which is 19,209 of the 25,727 predictions in a plan - discards it immediately (see the `if not cache` block). Filling it cost a round_py per step, and round_py is snprintf+strtod at ~96ns, by far the most expensive thing in the hot loop. soc is never itself rounded, the result only ever fed soc_out, so skipping it cannot change the simulation. 52.8us -> 28.5us per scenario (1.85x), and the 20 scenario benchmark 42.5s -> 36.9s (-13.1%), byte-identical on all 20 and kernel_parity green. 2. pk_run_batch, so a fan-out can be run in one call Adds PkBatchJob/PkBatchResult and pk_run_batch(handle, jobs, n, results, n_threads), sharing pk_run_one with pk_run so there is a single copy of the simulation. Additive: pk_run keeps its signature and PkScenario keeps its layout, and Python probes for the batch symbol (KERNEL_HAS_BATCH), so a binary built before this still loads and runs every scenario the old way. Batched jobs pass soc_out = null. The one thing a caller still wants from the series is the SoC range across a charge window, so the kernel tracks that inline over an optional step range instead, mirroring the predict_soc scan in thread_run_prediction_charge_min_max - materialising a buffer per job would have cost ~84MB on a large batch. Verified bit-identical against a loop of pk_run over 120 scenarios, every field. Threading is present but NOT yet usable, and n_threads should stay 1: the remaining round_py(soc, 1) at the charge-rate curve lookup is load bearing, and snprintf/strtod serialise on the C library's global locale. With constant work per thread it degrades 17x at 8 threads, which is why 4 threads currently measures 0.6x of serial. A per-thread locale (newlocale + uselocale) takes that from 17.3x to 2.2x in isolation and is the next step before the batch path is wired up in Python. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ent buckets round_py is snprintf+strtod, ~96ns, and was the most expensive thing in the simulation loop. It survives in the loop for one reason: prediction.py rounds SoC to 1dp before the charge/discharge rate curve lookup. That rounding is not a modelling decision. It exists so utils.get_charge_rate_curve_cached's lru_cache key quantises and the cache hits - a Python memoisation device. The kernel has no such cache and gains nothing from it, but rounding before the percent is taken occasionally moves the resulting integer, so it is observable and parity forced the kernel to reproduce it. Everything downstream only wants an integer 0-100: round(soc,1) -> calc_percent_limit -> curve[percent]. That map is monotone onto 101 buckets, so the boundaries can be found once and the loop reduced to a binary search over them. The boundaries are located by bisecting the double's bit pattern (non-negative doubles order the same as their bits read as integers), so each one is the true boundary to the last bit rather than an approximation. This is exact, not approximate: pk_verify_soc_percent_table sweeps 2,000,000 SoC values plus five probes around all 101 boundaries - where a disagreement would actually hide - for ten battery sizes from 2 to 100 kWh. Zero disagreements. Per scenario, on a 1200 scenario batch: 52.8us original 28.5us after skipping the discarded SoC series 10.9us now 4.8x on serial and it unblocks parallelism, because the contention was libc's global locale inside snprintf/strtod, not the simulation: 1 thread 13.0 ms 2 threads 6.7 ms 1.9x 4 threads 3.5 ms 3.8x 8 threads 2.4 ms 5.4x 16 threads 1.9 ms 6.7x bit-identical at every thread count The 20 scenario benchmark is 36.9s -> 31.6s even before the Python side batches anything, byte-identical on all 20, and kernel_parity passes. No ABI change: the bucket table lives on ContextStore, which never crosses the Python boundary, so PkContext keeps its layout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ries Adds run_soc_percent_table_tests to the kernel parity suite. It drives the kernel's own pk_verify_soc_percent_table over eleven battery sizes from 0.5 to 100 kWh, sweeping the SoC range and - the part that matters - probing either side of all 101 bucket boundaries, since a boundary is the only place the table could disagree with the round_py path it replaced. Checked that the test actually bites: nudging a single boundary by one ULP makes it fail on five of the eleven capacities, so it detects the smallest corruption the table can have rather than only catching gross breakage. Also rebuilds all six shipped binaries, which were built before the batch entry point and the bucket table existed. Verified the new symbols are present in every one (dynamic symbols for the four stripped Linux ELFs, nm for the two Mach-O), and loaded the cross-built darwin_arm64 directly to confirm it reports the batch entry point and agrees with round_py on the table. Without this a non-macOS install would silently fall back to the Python engine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… for pk_run_one clamped soc_range_min/max whenever the out pointers were non-null, but the batch path always passes them. A job that asks for no range (all_n, where Python skips the scan) would therefore have come back with (soc_max, soc_max) where thread_run_prediction_charge_min_max returns (soc_max, 0) - Python's clamping lines sit inside the `if not all_n` branch, so they never run for that case either. Latent rather than live: nothing calls pk_run_batch from Python yet. Fixed before wiring it up, since it would have surfaced as a plan difference confined to all_n charge windows and been thoroughly unpleasant to track down. pk_run is unaffected - it passes nullptr for both. Full suite green including the slow tests (debug_cases, annual_integration, load_ml), 101.22s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Working notes so Phase B can be picked up cold: what the kernel work already landed and what it measured, why Python is now 78% of plan time, the lazy-batch design, the traps that cost time here (the export window mutation, round_py's locale contention, test-fixture isolation), the verification protocol, and the three approaches already measured and rejected so they are not tried again. Delete before this branch merges. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Working plan for Phase B of the kernel batch work: launch_run_prediction_* queues jobs that a single pk_run_batch call runs per fan-out, which is both where the remaining Python overhead goes and the only shape that lets the kernel use threads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…utating the caller's The trial start was written straight into the caller's window dict, which was only safe because each pool worker mutated its own unpickled copy. A batched fan-out shares one window list across every job in the batch, so this has to be local. Ported from perf/threadpool-prototype. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The hand-off notes landed without passing cspell or markdownlint, so pre-commit has been red on this branch since. Adds the five technical words the notes use and tags their two code fences. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Adds BatchJob, run_prediction_kernel_batch and the shared result assembly, and gives pk_run_batch's ctypes binding the n_threads argument it was missing. Window start/end arrays are marshalled once per distinct window list rather than once per job, which is where the batching win comes from. Also adds Prediction.scan_soc_range and routes thread_run_prediction_charge_min_max through it so the scan logic is not duplicated ahead of the batch runner landing in a later task. Bit-identical to a loop of pk_run across 60 scenarios at 1, 2, 4 and 8 threads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…he key Closes two review findings on the batch entry point: the argtypes fix had no regression test (ctypes silently accepts extra call arguments, and a garbage n_threads only changes work splitting, not results, so the bug was invisible to the batch parity sweep), and the window-array memo was only ever exercised with one shared list per side, never with genuinely distinct lists in the same batch. Adds a direct argtypes assertion to run_marshalling_tests, and a new run_batch_window_variant_tests that cycles jobs across several distinct charge/export window lists and checks the batch against the same per-scenario pk_run reference. Extracted build_batch_reference and check_batch_results out of run_batch_parity_tests so the new test reuses them instead of duplicating the comparison, which also brings run_batch_parity_tests back under the repo's flake8 complexity limit. Also rewords a comment on the buffers keepalive list that overstated why it's needed (ctypes' own container keepalive already covers it; the list is belt-and-braces, not load-bearing) - comment only, no code change. Both new assertions verified to actually catch their target bugs via temporary mutation, reverted before committing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ediction_* Each thread_run_prediction_* built the trial inputs, ran the prediction and shaped the result. The batch path needs the first and third but not the second, so they are split rather than copied - one definition of what a trial input is, for both paths. No-op: 20 random scenarios byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

launch_run_prediction_* no longer runs anything: it prepares the trial inputs, queues a job and returns a handle whose first get() flushes the whole batch through a single pk_run_batch call. Window start/end arrays are marshalled once per fan-out rather than once per scenario, cache hits and intra-batch duplicates are resolved in Python first, and anything the kernel refuses falls back to the Python engine per job. Serial for now. 20 random scenarios byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The debug_enable case of test_batch_fallbacks asserted a result value, which the two engines agree on to ~1e-6 with every field rounded to 4dp - so it passed whether or not the kernel_supported() gate in flush_batch existed, leaving the one guard that keeps debug runs on the Python engine unpinned. It now spies on run_prediction_kernel_batch and asserts the kernel was never reached, keeping the value assertion alongside. Also: make "a min/max job is never cached" structural rather than a property of the single caller that passes cache=False, since a cache hit on a range job would answer (0.0, 0.0) and silently collapse the optimiser's SoC pruning envelope; cover a range window that starts before minutes_now (the max(...,0) clamp and the wholly-past negative end step were never executed); cover a single prediction at the levels optimiser's coarse step; and fix the garbled module docstring. 20 random scenarios still byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

args[threads] now sizes the kernel's thread pool rather than a pool of worker processes. The GIL is released for the whole pk_run_batch call, so unlike a Python ThreadPool these are real cores. Also corrects a stale comment in annual.py that still described calculate_plan() as spinning up a multiprocessing Pool; it no longer does, and forcing threads=0 there is now about skipping the C++ kernel's per-call std::thread pool creation on hundreds of tiny per-day plans, not fork safety. Identical results at 1, 2 and 8 threads; 20 random scenarios byte-identical with threading actually on (16 threads, this machine's CPU count). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Nothing has launched work into it since the fan-out started batching, so PRED_GLOBAL, the four wrapped_run_prediction_* trampolines, DummyThread and the pool lifecycle all go with it. The pool was measured at 1.11x on 16 cores, with the parent's per-call pickling as the serial bottleneck. Also cleans up the code and comments left pointing at the removed pool: cleanup_pool() and its call sites in predbat.py, the stale "pool" entry in userinterface.py's debug-dump exclude list, and pool-specific reasoning in comments in hass.py and load_ml_component.py. Two tests that used DummyThread purely as a generic result-handle double (not to exercise pool behaviour) now use a small local FakeHandle in test_pv90.py; test_infra.py's simple_scenario() calls Prediction.thread_run_prediction_single directly instead of the deleted wrapper. 20 random scenarios byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…esult routing Three things the batch was relying on nobody getting wrong. Nothing pinned flush_batch's job->result mapping. Injecting a reversed mapping (batched[len(kernel_jobs) - 1 - index]) passed the prediction_batch, kernel_parity and model_kernel suites in full: every trial in a fan-out would silently take another trial's cost. The hole was that no test compared values across a multi-job batch - test_batch_is_lazy only checked results were not None, and the threading test compared the batch against itself at different thread counts, which cannot see a thread-count-independent Python defect because n_threads goes straight to C. That test is now test_batch_results_match_their_own_job and uses an external oracle: ten distinct try_soc values queued in one batch, each handle compared against thread_run_prediction_charge for the same SoC with the prediction cache cleared between the two runs. The SoCs are asserted to give ten different costs first, so a misrouted result cannot match by coincidence - the previous scenario saturated and gave nine identical costs, hence the trial spread across every charge window and the different scenario seed. With the reversed mapping applied, all ten jobs fail at all three thread counts. A saving run is the only one that populates predict_soc_best and friends. If a batch were left pending across it, the next handle read would flush it and reset_kernel_run_state would blank exactly those attributes - the published plan and the debug HTML would come back empty (verified: 288 entries -> 0 with the guard removed) - and a job flushed after its inputs were mutated would be cached under a key hashed from the old ones. All four fan-outs drain their handles first so neither is reachable, but run_prediction now drains any pending batch when save is set. The guard is a single test of a local on the ~250k non-save calls per plan. pending_batch and batch_threads were initialised only inside __init__'s `if base:` branch, so a Prediction built without a base raised AttributeError on its first enqueue. Nothing constructs one today - the wrapped_* trampolines that did went with the process pool - which is what made it worth closing. Also replaces two kernel_parity assertions that could not fail: the SoC series check now requires the single-scenario reference to have materialised the series the batch omits, and car_charging_soc_next (recorded at minute 0, so identical for every job in a fan-out) is backed by an assertion that the kernel fills it at all. The four thread_run_prediction_* docstrings no longer claim to run in a thread; they are the direct synchronous path the batch falls back to. The four queue_run_prediction_* docstrings now state that inputs are read at flush time and must not be mutated before get(). 20-scenario random-compare: +0.0000 on metric, cost and all three PV futures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The process pool is gone, but several places still document it as how predbat works - and one of them is read by AI coding assistants as ground truth. .github/copilot-instructions.md documented self.pool, wrapped_run_prediction_*, PRED_GLOBAL and DummyThread as current architecture, with code samples an assistant would happily regenerate. Replaced with the batch model: queue through launch_run_prediction_*, first get() flushes the fan-out through one pk_run_batch call, kernel threads rather than worker processes, plus the no-mutation invariant. double_array()'s docstring justified its from_buffer view with "each pool worker is a separate process (multiprocessing with fork), so no buffer is ever shared between workers". That was the stated reason an unsafe-looking construct was safe, so it needed a real replacement rather than deletion: the views are filled on the calling thread before the kernel is entered and only ever read afterwards, so even a buffer deliberately shared between jobs (the window memo) is only read concurrently, and the one thread that could change it is blocked inside the call. Same staleness fixed on the module-level loader state and load_kernel's cache note. plan.py's module and class docstrings still described multi-threaded prediction runs; the planner is strictly serial Python now and the threading lives inside pk_run_batch. The four launch_run_prediction_* docstrings now carry the no-mutation-before-get() invariant, which was documented only in prediction_batch.py's module docstring - a file a contributor adding a fifth fan-out would never open. The window memo comment called window reuse "the bulk of the batching win", true for the charge fan-outs but not the export one, where _prepare_export builds a fresh export_window per job and every one misses the memo. BatchJob's docstring listed step as a field the kernel reads, when run_prediction_kernel_batch overrides it with PREDICT_STEP and only the Python fallback reads it. BatchHandle was described as the drop-in for the pool's AsyncResult. The threads doc said '0' disables threads, when it is clamped up to one kernel thread. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Fix two comment-only inaccuracies describing the batched prediction system that replaced the multiprocessing pool: 1. copilot-instructions.md: replace outdated bullets describing deleted wrapped_run_prediction_* and PRED_GLOBAL with description of current launch_run_prediction_* batching mechanism. 2. prediction_kernel.py: reword window memoisation comment to accurately describe export window sharing, which depends on the all_n parameter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Keeps the numbers with the plan now the hand-off notes are gone: the serial win is real (31.6s -> 26.3s), threading is flat (~0.8%, inside noise), and the census that explains why - 25,091 batch calls with a median of 6 jobs each, so a fresh std::thread pool is built per call to cover ~180us of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The 31.6s in the hand-off notes did not survive re-measurement. Against the fork point, kernel built from each commit's own source and confirmed active on both sides: 41.74s -> 26.22s serial, 40.02s -> 25.81s at each side's default. Neither side's parallelism is worth much; the win is the batching. Also records that the old process pool died with KeyError outside the __main__ entry point on any spawn-default platform, which removing it fixes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR refactors Predbat’s planning hot path by batching prediction fan-outs into a single C++ kernel call (pk_run_batch) per fan-out, removing the legacy multiprocessing pool and updating documentation/tests accordingly. This fits the codebase’s architecture by keeping the optimiser logic in plan.py (still serial Python) while moving scalable parallelism into the kernel where the GIL is released.

Changes:

Introduces lazy, handle-based batching (PredictionBatch/BatchHandle) so launch sites queue jobs and the first .get() flushes the entire batch.
Adds a ctypes batch entry point and C++ kernel support for batched execution, plus parity/behaviour tests for correctness and routing.
Removes the process pool and its associated globals/trampolines, and updates docs/comments to reflect the new “threads = kernel threads” meaning.

Reviewed changes

Copilot reviewed 22 out of 28 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
docs/superpowers/plans/2026-08-15-phase-b-batch-prediction.md	Adds/records the implementation plan and rationale for batched prediction fan-out.
docs/apps-yaml.md	Updates `threads` documentation to reflect kernel-thread semantics and batching behaviour.
apps/predbat/userinterface.py	Removes `pool` from debug exclusion list now that the pool is removed.
apps/predbat/unit_test.py	Registers the new `prediction_batch` test suite in the test registry.
apps/predbat/tests/test_pv90.py	Removes `DummyThread` dependency and replaces it with a minimal local handle double for the test.
apps/predbat/tests/test_prediction_batch.py	Adds comprehensive behavioural tests for batching (laziness, equivalence, cache/dedup, fallbacks, routing, invariants).
apps/predbat/tests/test_kernel_parity.py	Adds batch parity tests, window-variant tests, and additional kernel verification hooks.
apps/predbat/tests/test_infra.py	Removes pool-era wrapper usage; routes single-scenario runs through the direct prediction API.
apps/predbat/tests/test_export_commitment.py	Removes `my_predbat.pool = None` setup now that pool lifecycle is removed.
apps/predbat/tests/test_download.py	Removes dummy `pool` attribute from test doubles after pool removal.
apps/predbat/prediction.py	Adds batching mixin + queue entry points, removes pool wrapper globals, and centralises trial-input preparation helpers.
apps/predbat/prediction_kernel.py	Adds ctypes structs/bindings for batch calls, shared result assembly, and kernel batch execution support.
apps/predbat/prediction_kernel.cpp	Implements `pk_run_batch`, SoC percent-table optimisation, and batch execution threading in the kernel.
apps/predbat/prediction_batch.py	New module implementing lazy batching, cache-key sharing, dedup/cache-hit resolution, and kernel/Python fallback behaviour.
apps/predbat/predbat.py	Removes pool lifecycle management and pool-related reset/termination logic.
apps/predbat/plan.py	Switches prediction “launch” APIs to queue handles; removes multiprocessing pool creation/teardown; wires `threads` to kernel batch threads.
apps/predbat/marginal.py	Removes pool-era comment about `PRED_GLOBAL` side effects.
apps/predbat/load_ml_component.py	Updates comment: LoadML no longer gates multiprocessing fork safety; it’s now just a status flag.
apps/predbat/hass.py	Updates log-rotation comment to remove pool/pickling rationale.
apps/predbat/annual.py	Updates rationale for forcing `threads: 0` to avoid per-batch kernel thread-pool overhead in many small per-day runs.
.github/copilot-instructions.md	Updates repository guidance to describe the new batching model instead of the process pool model.
.cspell/custom-dictionary-workspace.txt	Adds new technical terms introduced by the refactor to the spell-check dictionary.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

… loaded a325492 made PkScenario::soc_out optional - Python passes no SoC buffer for a cached run, which is the common case - but left PK_ABI_VERSION at 3. An ABI 3 binary writes to soc_out unconditionally, so loading one against this Python is a null write on the first prediction: predbat segfaults instead of falling back to the Python engine. Reproduced by building the kernel from main and pointing PREDBAT_KERNEL_SO at it: model_kernel exits 139. With the bump the loader reports 'stale binary (abi 3 revision 5, expected abi 4 revision 5)' and uses the Python engine, which is what the version check exists to do. Anyone with a locally built .so from an older checkout, or an install whose per-architecture binary was not replaced, would have hit this. All six shipped binaries rebuilt at ABI 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… faster) (#4546) * docs: design for a persistent scratch and thread pool in the kernel Batching made planning 35% faster but threading on top of it bought 1.6%. A census showed why: 25,091 batch calls with a median of 6 jobs, against which a fresh thread pool and a fresh 70KB scratch are built per call. Staged so the low-risk half (per-thread scratch, which also helps the serial default) is measured before the concurrency half. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(kernel): give each thread one scratch instead of one per call PARKED - measured as a no-op, do not merge on its own. Replaces the three per-call PkScratch constructions with a thread_local accessor, so a thread grows its ~70KB of buffers once rather than once per pk_run_batch call (~25,000 of them per plan). Measured on the serial benchmark, interleaved, best of 3: 26.291s before, 26.344s after - no difference against ~1.8% run-to-run spread. The hypothesis was wrong: the kernel's comment about the allocator being the bottleneck was recorded under four threads contending, and one thread repeatedly allocating and freeing the same-sized block never leaves the allocator's thread cache. Kept on this branch because the persistent-worker design needs per-worker scratch anyway, and the two tests are worth having either way: one pins that the buffers stop being rebuilt per call (it fails before this change), the other that a reused scratch cannot leak between contexts of different n_steps (verified by mutation - relaxing the assign to skip shrinking makes it fail). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(kernel): keep a persistent worker pool instead of threads per batch pk_run_batch built and joined a fresh std::vector<std::thread> on every call. Measured over one benchmark plan that was 201,545 thread creations costing 1.5s against 3.1s of simulation, which is why threading was worth only 1.6% - the creation was cancelling what the parallelism won. Workers now park on a condition variable for the life of the process, and each has its own flag and condvar so a batch wakes exactly the lanes it needs. Broadcasting instead cost most of the win: a six-job batch woke all fifteen workers, ~250,000 wasted wakeups per plan, and measured 1.3% against 3.7% for targeted wakes. The calling thread takes lane 0 itself, so a six-job batch wakes five. Also gives each thread one scratch instead of one per call. That is a no-op on its own (26.291s vs 26.344s, inside noise - the allocator never left its thread cache) but the parked workers need it. 20-scenario benchmark, best of 3: 26.33s serial, 24.71s at 6 threads - 6.1%, on top of the batching work. All 20 scenarios byte-identical at every thread count, in both the normal regime and one with each job made eight times dearer. threads: auto stays at the core count. A cap looked right on a fast machine, where the curve peaks below it, but in the kernel-heavy regime the curve stops turning over - capping at 4 would cost 10.7% there against 1.3% for no cap at all. resolve_batch_threads carries the numbers. No ABI or parity bump: results are unchanged, so an older binary still loads and runs correctly. All six shipped binaries rebuilt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(kernel): skip the pool tests when the binary predates pk_run_batch The batch and pool tests call run_prediction_kernel_batch, which returns None when the loaded binary has no pk_run_batch. Four of them treated that as a failure rather than a skip, so running against an older binary reported errors instead of degrading the way the product does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update prediction kernel binaries for all platforms * fix(kernel): include <system_error> explicitly The pool catches std::system_error when the system refuses a thread. libc++ pulls that declaration in through <thread>, libstdc++ does not, so the file built on macOS and failed on GCC with a parse error at the catch clause. Caught by CI, which builds natively with g++; neither the local clang build nor the zig cross-build sees it, since both use libc++. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update prediction kernel binaries for all platforms --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

springfall2008 and others added 20 commits August 15, 2026 17:39

Copilot AI lite review requested due to automatic review settings August 15, 2026 21:19

Copilot started reviewing on behalf of springfall2008 August 15, 2026 21:19

View session

Copilot AI reviewed Aug 15, 2026

View reviewed changes

chore: update prediction kernel binaries for all platforms

b633d4f

springfall2008 mentioned this pull request Aug 16, 2026

perf(kernel): persistent worker pool instead of threads per batch (6% faster) #4546

Merged

springfall2008 and others added 3 commits August 16, 2026 10:01

chore: update prediction kernel binaries for all platforms

9546ffe

springfall2008 merged commit 67154a7 into main Aug 16, 2026
2 checks passed

springfall2008 deleted the perf/kernel-batch-abi branch August 16, 2026 15:36

This was referenced Aug 16, 2026

docs: report on the planning performance work #4547

Merged

perf: cut ~23% off planning by removing redundant scans and copies #4549

Merged

perf: cache window bounds and replace deepcopy on window lists (-21% planning) #4536

Merged

dandwhelan mentioned this pull request Aug 17, 2026

Merge upstream v8.48.4 into the fork; keep FIT, fast round_py and the dashboard dandwhelan/batpred#44

Merged

Navigation Menu

Repository navigation

perf: batch the prediction fan-out into one kernel call (35% faster planning) - #4540

perf: batch the prediction fan-out into one kernel call (35% faster planning)#4540
springfall2008 merged 24 commits into
mainfrom
perf/kernel-batch-abi

springfall2008 commented Aug 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

springfall2008 commented Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Measured against current main

Also fixes a crash

threads changes meaning

Risk

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

springfall2008 commented Aug 15, 2026 •

edited

Loading

Measured against current `main`

`threads` changes meaning