Skip to content

Commit 1b5305b

Browse files
docs: record what the batching refactor measured
Keeps the numbers with the plan now the hand-off notes are gone: the serial win is real (31.6s -> 26.3s), threading is flat (~0.8%, inside noise), and the census that explains why - 25,091 batch calls with a median of 6 jobs each, so a fresh std::thread pool is built per call to cover ~180us of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 8905bb5 commit 1b5305b

1 file changed

Lines changed: 43 additions & 0 deletions

File tree

‎docs/superpowers/plans/2026-08-15-phase-b-batch-prediction.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1581,6 +1581,49 @@ Required: 20 scenarios unchanged with `+0.0000` on metric, cost and all three PV
15811581

15821582
Then report, best of 3 each: serial batch vs threaded batch vs the 31.6s pre-Phase-B baseline.
15831583

1584+
## What it actually measured, once built
1585+
1586+
The plan was executed and these are the results, kept here because the hand-off notes this plan was
1587+
written from are deleted by the last task.
1588+
1589+
**The serial win is the payoff.** 20-scenario benchmark, best of 3: **26.3s** against the 31.6s
1590+
recorded before this work (that baseline was inherited, not re-measured on the same machine). It
1591+
comes from paying the ctypes boundary and the window marshalling once per fan-out.
1592+
1593+
**Threading buys nothing measurable: 26.27s serial vs 26.05s at 16 threads**, inside the 2-6%
1594+
run-to-run noise band. Results are bit-identical either way.
1595+
1596+
**Why**, from a `n_jobs` census over one benchmark run:
1597+
1598+
```text
1599+
25,091 pk_run_batch calls carrying 253,899 jobs
1600+
mean 10.12 jobs/call, median 6, p95 24, max 363
1601+
97.4% of calls carry fewer than 32 jobs
1602+
```
1603+
1604+
This plan's own profile predicted fan-outs of ~1176, ~900 and ~628 jobs. The gap is mostly not a
1605+
batching failure: the large fan-out is scoped to one `(loop_price, coarse)` iteration and its ~1176
1606+
launches become ~363 kernel jobs once the ~44% prediction-cache hit rate and the intra-batch dedup
1607+
have taken their share. The median of 6 is `optimise_charge_limit`'s min/max pre-pass, which is
1608+
genuinely that small. So the batch is collecting whole fan-outs — the fan-outs are just small.
1609+
1610+
At ~6 jobs of ~30us each, `pk_run_batch` builds and joins six `std::thread`s to cover ~180us of work.
1611+
Thread setup dominates by construction.
1612+
1613+
**Where the next change goes**, in order:
1614+
1615+
1. **`pk_run_batch` in `prediction_kernel.cpp`** — replace the per-call `std::vector<std::thread>`
1616+
with a pool created once at `pk_context_create` and woken per batch. This is the smallest change
1617+
with the largest effect, and it makes threading pay at today's batch sizes without touching Python.
1618+
2. **`plan.py`, not `prediction_batch.py`**, if larger batches are wanted. The ceiling is
1619+
`optimise_charge_limit`'s internal serialisation: the min/max pre-pass computes the SoC-pruning
1620+
envelope that decides which `try_socs` are launched at all. Breaking that read-then-decide
1621+
dependency means either launching the unpruned SoC set speculatively or running the pre-pass for
1622+
every window before any trial — both change optimiser semantics and risk moving plans.
1623+
`enqueue_prediction`/`flush_batch` already accept arbitrarily large batches; they need no change.
1624+
3. **Not** a batch-size threshold before threading in `run_prediction_kernel_batch` — the C++ already
1625+
clamps threads to job count, and a Python-side knob would only hide (1).
1626+
15841627
## Notes for whoever picks this up
15851628

15861629
- **`PR #4536` (`perf/window-bounds-cache`) is in flight and independent**, worth -21% on its own. Batching partly subsumes it — a batch ships window bounds once per fan-out rather than per call — but the `sim_hash` half still matters, and `prediction_cache_key()` in `prediction_batch.py` is exactly where that half would land. If both branches merge, re-profile before assuming what is left.

0 commit comments

Comments
 (0)