22
33Branch: ` poc/parallel-lpt-buckets ` (local only, ** not committed** — no tests yet)
44
5- Opt-in flag: ` --experimental-runner ` . Without it nothing changes.
5+ Opt-in flag: ` --lpt ` . Without it nothing changes.
66
77Source: https://github.com/rectorphp/rector-src/issues/8494#issuecomment-5664864017 (@samsonasik )
88
@@ -23,8 +23,8 @@ Faithful to 1 + 3, with one deliberate deviation on 2.
2323| | StructArmed | this PoC |
2424| --- | --- | --- |
2525| bucketing | LPT by filesize | LPT by filesize (same greedy) |
26- | assignment | static, 1 bucket per worker | static, 1 bucket per worker |
27- | dispatch | whole bucket in one shot | bucket sub-chunked into 16-file jobs, ** only to its own worker ** |
26+ | assignment | static, 1 bucket per worker | static, ** plus stealing once a worker runs dry ** |
27+ | dispatch | whole bucket in one shot | bucket sub-chunked into ` jobSize ` jobs |
2828| worker lifetime | one process per bucket, never respawned | one process per bucket, never respawned |
2929| transport | ` proc_open ` + stdout polling | unchanged (ReactPHP + NDJSON over TCP) |
3030
@@ -44,8 +44,8 @@ Sub-chunking inside a statically-assigned bucket keeps the heartbeat while prese
4444win: ** the worker process is never recycled** , so its warm in-memory caches (PHPStan reflection,
4545parsed AST, node scope resolver) survive across the whole bucket.
4646
47- Round -trip cost of sub-chunking: ~ 820 extra NDJSON round-trips on a 13 k file project spread over
48- 16 workers — sub-second, negligible next to a container boot .
47+ The round -trip cost of sub-chunking turned out to be far cheaper than expected — small chunks win
48+ outright, see the ` jobSize ` section below .
4949
5050### The other half of the change: no forced respawn
5151
@@ -67,8 +67,8 @@ New, all isolated under `src/Parallel/Experimental/`:
6767
6868Existing files, minimal edits only:
6969
70- - ` src/Configuration/Option.php ` — ` EXPERIMENTAL_RUNNER ` constant
71- - ` src/Console/ProcessConfigureDecorator.php ` — registers ` --experimental-runner `
70+ - ` src/Configuration/Option.php ` — ` LPT ` constant
71+ - ` src/Console/ProcessConfigureDecorator.php ` — registers ` --lpt `
7272- ` src/Application/ApplicationFileProcessor.php ` — two constructor deps + one branch in ` runParallel() `
7373
7474Nothing on the default path changed, so a run without the flag is byte-identical to ` main ` .
@@ -94,8 +94,11 @@ same 2 errors. The scheduler does not change the outcome.
9494
9595"busy cores" = ` (user + sys) / real ` , i.e. how many of the 14 were actually kept busy.
9696
97- Headline: best legacy 152.2 s → best experimental 113.2 s, ** -26 %** . At equal worker count (both on
98- the default 14) it is 152.2 s → 122.1 s, ** -20 %** .
97+ Every row above is at ` jobSize: 16 ` , which was the hardcoded chunk at the time. ` jobSize ` turned out
98+ to matter more than anything in this table — see its own section below.
99+
100+ Headline: best legacy ** 152.2 s** → best ` --lpt ` ** 90.7 s** (14 workers, ` jobSize: 2 ` ), i.e. ** -40 %** .
101+ At the same worker count and the same ` jobSize: 16 ` , it is 152.2 s → 113.4 s, ** -25 %** .
99102
100103### Work stealing is the load-bearing part, not LPT
101104
@@ -113,16 +116,75 @@ This also sharpens item 1 in "what is left": a timing-based cost model would mak
113116buckets right, but stealing is what makes the schedule robust when the model is wrong. Both, not
114117either.
115118
116- ### Worker count interacts with the scheduler — do not tune it standalone
119+ ### Worker count: a red herring once the chunk size is right
120+
121+ Measured at ` jobSize: 16 ` , dropping from 14 to 10 workers looked like a win (122.1 → 113.2), and
122+ legacy got worse at 10 (152.2 → 164.3). That suggested tuning the worker count, and that
123+ ` CpuCoreCountProvider ` counting logical cores was a problem on big.LITTLE hardware.
124+
125+ ** That was an artefact of the coarse chunk.** At ` jobSize: 4 ` the ordering flips: 14 workers give
126+ 99.2 s, 12 workers give 107.6 s. With fine-grained stealing an efficiency core simply takes fewer
127+ chunks and contributes instead of straggling, so there is nothing left to compensate for by starving
128+ the pool.
129+
130+ Conclusion: fix the granularity and the worker-count question mostly disappears. ` hw.ncpu ` is a fine
131+ default. Do not tune the worker count before tuning ` jobSize ` — you will tune against a symptom.
132+
133+ ### jobSize: was inert, now it is the most important knob
134+
135+ Originally this PoC hardcoded the chunk at 16 files and let ` jobSize ` feed only the worker-count
136+ formula ` min(ceil(files / jobSize), cpuCores, maxProcesses) ` . That makes it inert on any real
137+ project: once ` ceil(files / jobSize) >= cpuCores ` — from ~ 224 files at 14 cores and ` jobSize: 16 ` —
138+ it stops changing anything. On a 13 k file project it did nothing below ` jobSize ` ~ 936.
139+
140+ ` jobSize ` is now the chunk a worker is handed per request (it still implies at least one whole job
141+ per worker). It therefore controls three things at once: ** the granularity at which an idle worker
142+ can steal work** , the heartbeat that advances the progress bar, and the window the per-job timeout
143+ measures.
144+
145+ 14 workers, LPT + stealing, ` vendor/ ` . Run-to-run spread across batches is ~ 5 %, so the two figures
146+ for ` jobSize: 4 ` are the same measurement taken twice.
117147
118- Dropping from 14 to 10 workers ** helps** the bucket scheduler (122.1 → 113.2) and ** hurts** legacy
119- (152.2 → 164.3). There is no worker count that is right independent of how work is handed out, so
120- ` CpuCoreCountProvider ` returning logical cores is not a standalone bug to fix — it only looks like
121- one under the bucket scheduler.
148+ | jobSize | wall | diffs | |
149+ | --: | --: | --: | --- |
150+ | 1 | 98.8 s | 3 338 | round-trip overhead takes over |
151+ | 2 | ** 90.7 / 91.4 s** | 3 338 | optimum |
152+ | 4 | 96.7 / 99.2 s | 3 338 | |
153+ | 8 | 104.6 s | 3 338 | |
154+ | 16 | 113.4 s | 3 338 | today's default |
155+ | 50 | 118.1 s | 3 338 | |
156+ | 150 | 127.5 s | ** 3 237** | ** timed out, results truncated** |
122157
123- ` user ` time also drops as workers drop (1089 → 931 under stealing). Careful with that number: work
124- done on an efficiency core accumulates more CPU-seconds for the same result, so part of the "extra
125- CPU" at 14 workers is just E-cores being slow, not waste.
158+ The curve is U-shaped with a clear floor at ** ` jobSize: 2 ` ** — reproduced twice at 90.7 s and 91.4 s,
159+ so the ~ 6 % gap to ` jobSize: 4 ` is outside run-to-run noise. Below that, at ` jobSize: 1 ` , the round
160+ trips finally cost more than the balance they buy (and ` user ` time peaks at 1 173 s, the highest of
161+ any run).
162+
163+ This ** inverts the recommendation in #8494 ** . There, raising ` jobSize ` was the only lever that
164+ reduced respawn churn, which is why 150 and 300 looked good. With respawns gone, a large ` jobSize `
165+ buys nothing and only coarsens stealing — and stealing is what carries the win.
166+
167+ Caveat before generalising: the optimum is a balance between round-trip cost and stealing
168+ granularity, and both scale with the corpus. A heavier rule set makes each file cost more, which
169+ shifts the floor upward. Treat "2" as the answer for this corpus on this machine, and re-measure
170+ the 2-8 range elsewhere rather than hardcoding it.
171+
172+ ### A large jobSize both fails the build and truncates the result
173+
174+ ` jobSize: 150 ` hit ` Child process timed out after 120 seconds ` . That calls ` handleErrorCallable ` →
175+ ` quitAll() ` , and the run finished ** reporting 3 237 diffs instead of 3 338** — 101 missing.
176+
177+ The build does go red: ` ProcessCommand ` returns ` ExitCode::FAILURE ` whenever there are system errors
178+ (` src/Console/Command/ProcessCommand.php:232 ` ), which matches the ` exit 1 ` reported in #8494 . The
179+ extra hazard is that the * output* is silently short by 101 diffs on top of that, so anyone who
180+ tolerates or filters system errors in CI loses findings without a signal.
181+
182+ Worth reporting upstream separately: the message printed in that case is
183+ ` Reached system errors count limit of 50, exiting... ` even though only 4 errors occurred, because
184+ ` ParallelFileProcessor::$handleErrorCallable ` sets ` $reachedSystemErrorsCountLimit = true `
185+ unconditionally on * any* error (` src/Parallel/Application/ParallelFileProcessor.php:134 ` ). This is a
186+ pre-existing bug on ` main ` , faithfully copied into the experimental processor. It hides the real
187+ cause behind a wrong one — exactly the kind of thing that makes a timeout hard to diagnose.
126188
127189### Memory is the consistent cost
128190
@@ -136,7 +198,7 @@ noise. At that size a worker never reaches the `MAX_CHUNKS_PER_WORKER × jobSize
136198threshold, so both schedulers do exactly the same thing. ** The gain appears precisely where the
137199respawns do** , which supports reading #8494 's table as respawn churn rather than round-trip overhead.
138200
139- sellero at ~ 13 k files sits far past that threshold.
201+ Any project past ~ 2 000 files sits beyond that threshold on a 14-core machine .
140202
141203### Why work stealing matters here
142204
@@ -147,8 +209,8 @@ bucketing throws that correction away. Stealing puts it back while keeping the n
147209hence it beats both.
148210
149211Consequence for benchmarking: ** the split between static and stealing depends on the hardware.** On a
150- homogeneous CI runner static buckets should lose much less. Worth measuring on sellero's CI too, not
151- only on a laptop. ` RECTOR_EXPERIMENTAL_NO_STEAL =1` turns stealing off for that A/B (temporary knob).
212+ homogeneous CI runner static buckets should lose much less. Worth measuring on the target CI too, not
213+ only on a laptop. ` RECTOR_LPT_NO_STEAL =1` turns stealing off for that A/B (temporary knob).
152214
153215### What is left on the table
154216
@@ -174,7 +236,7 @@ only on a laptop. `RECTOR_EXPERIMENTAL_NO_STEAL=1` turns stealing off for that A
174236 and full-JSON runs differ little, so the main loop is not saturated.
1752376 . ** Worker boot.** 14 container boots, concurrent, ~ 1 s out of 113. Not worth chasing.
176238
177- ## How to A/B benchmark on sellero
239+ ## How to A/B benchmark on a real project
178240
179241The flag is opt-in, so baseline and experiment differ by one argument — no branch switching:
180242
@@ -183,22 +245,28 @@ vendor/bin/rector process --dry-run --clear-cache --no-progress-bar
183245```
184246
185247``` bash
186- vendor/bin/rector process --dry-run --clear-cache --no-progress-bar --experimental-runner
248+ vendor/bin/rector process --dry-run --clear-cache --no-progress-bar --lpt
187249```
188250
189251Run them sequentially, never in parallel, and keep ` --clear-cache ` on both (or off on both).
190252
191253Suggested matrix:
192254
193- | run | flag | ` jobSize ` |
194- | --- | --- | --- |
195- | baseline | — | current |
196- | baseline tuned | — | 150 |
197- | experimental | ` --experimental-runner ` | 16 |
198- | experimental, fewer workers | ` --experimental-runner ` | 100 |
255+ | run | flag | ` jobSize ` | why |
256+ | --- | --- | --: | --- |
257+ | baseline | — | 16 | today's default |
258+ | baseline tuned | — | 150 | what #8494 recommends raising it to |
259+ | lpt | ` --lpt ` | 16 | same knob as the default, scheduler is the only variable |
260+ | lpt, fine chunks | ` --lpt ` | 4 | fastest here; expected best |
261+ | lpt, coarse chunks | ` --lpt ` | 150 | expected worst, and watch for the timeout |
262+
263+ ** ` jobSize ` means something different under ` --lpt ` .** On the default path it is the chunk any worker
264+ may pull from a shared pool. Under ` --lpt ` it is the chunk handed to a worker from its own bucket,
265+ which makes it the granularity at which an idle worker can steal from a busy one — the setting that
266+ turned out to matter most. It still implies at least one whole job per worker, so it also caps the
267+ worker count on small file sets, exactly as before.
199268
200- ` jobSize ` keeps its old meaning under the flag — it is the number of files that makes a worker worth
201- starting, so it still controls the worker count; it no longer controls how work is split.
269+ Do not carry a tuned ` jobSize ` across the two paths: the optimum moves in opposite directions.
202270
203271Watch, besides wall time:
204272
@@ -254,8 +322,23 @@ Nothing below is covered by a test yet — the user asked for a PoC first.
254322
255323### Cleanup before this could become a PR
256324
257- - [ ] drop the ` RECTOR_PARALLEL_SCHEDULER ` env var, or promote it to a real ` Option `
258- - [ ] ` MAX_CHUNKS_PER_WORKER ` becomes dead in bucket mode — decide its fate
259- - [ ] revisit the ` jobSize ` default (#8494 argues 16 is too low); with warm workers the optimum
260- likely moves again — re-measure before changing it
325+ - [ ] ** Give the bucket path its own chunk-size default, separate from ` PARALLEL_JOB_SIZE ` .**
326+ Right now ` --lpt ` reuses the shared ` jobSize ` , whose default is 16
327+ (` src/Config/RectorConfig.php:137 ` and ` src/Configuration/RectorConfigBuilder.php:106 ` ),
328+ untouched by this PoC. Two consequences:
329+ - ` --lpt ` alone gives 113.4 s, not the 90.7 s headline. Reaching the best number needs an
330+ explicit ` withParallel(jobSize: 2) ` .
331+ - The optimum moves in ** opposite directions** on the two paths — down to ~ 2 under ` --lpt ` ,
332+ up towards 100-300 on the default path per #8494 — so one shared default cannot serve both,
333+ and lowering the shared one would hurt every run without the flag.
334+
335+ A dedicated default for the bucket path lets each side sit at its own optimum. Pick it by
336+ measuring on more than one corpus first: "2" comes from a single repo on a single machine, and
337+ a heavier rule set raises the per-file cost and shifts the floor upward.
338+ - [ ] drop the ` --lpt ` flag and ` RECTOR_LPT_NO_STEAL ` env var, or promote them to real options
339+ - [ ] ` MAX_CHUNKS_PER_WORKER ` is dead in bucket mode — decide its fate (see the memory-driven
340+ recycle item above, which would give it a real job again)
341+ - [ ] fold ` ExperimentalParallelFileProcessor ` back into ` ParallelFileProcessor ` — it is a near-copy,
342+ kept separate only so the default path stays provably untouched, and it will trip the
343+ duplicate-code gate
261344- [ ] ` composer check-cs ` , ` composer phpstan ` , ` vendor/bin/phpunit `
0 commit comments