Repository navigation
Commit 4dcd1de
feat: parallelize rlm_query_batched subcalls using ThreadPoolExecutor
When rlm_query_batched fans out multiple independent child RLM calls,
they now run concurrently via a bounded thread pool instead of
sequentially. This reduces wall time from N*T to ~T for N I/O-bound
subcalls.
Key changes:
- Use ThreadPoolExecutor in _rlm_query_batched for 2+ prompts
- Add max_concurrent_subcalls parameter (default 4) with semaphore-style
concurrency control at each recursion level
- Propagate setting from RLM -> LocalREPL -> child RLMs
- Thread-safe completion collection with deterministic ordering
- Skip thread pool for 0-1 prompts (no overhead)
Closes #135
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1 parent 2e5764b commit 4dcd1de
3 files changed
Lines changed: 175 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| 73 | + | |
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
| |||
102 | 103 | | |
103 | 104 | | |
104 | 105 | | |
| 106 | + | |
| 107 | + | |
105 | 108 | | |
106 | 109 | | |
107 | 110 | | |
| |||
132 | 135 | | |
133 | 136 | | |
134 | 137 | | |
| 138 | + | |
135 | 139 | | |
136 | 140 | | |
137 | 141 | | |
| |||
238 | 242 | | |
239 | 243 | | |
240 | 244 | | |
| 245 | + | |
| 246 | + | |
241 | 247 | | |
242 | 248 | | |
243 | 249 | | |
| |||
765 | 771 | | |
766 | 772 | | |
767 | 773 | | |
| 774 | + | |
| 775 | + | |
768 | 776 | | |
769 | 777 | | |
770 | 778 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
135 | 136 | | |
136 | 137 | | |
137 | 138 | | |
| 139 | + | |
138 | 140 | | |
139 | 141 | | |
140 | 142 | | |
141 | 143 | | |
142 | 144 | | |
143 | 145 | | |
| 146 | + | |
144 | 147 | | |
145 | 148 | | |
146 | 149 | | |
| |||
316 | 319 | | |
317 | 320 | | |
318 | 321 | | |
319 | | - | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
320 | 328 | | |
321 | | - | |
322 | 329 | | |
323 | 330 | | |
324 | 331 | | |
| |||
329 | 336 | | |
330 | 337 | | |
331 | 338 | | |
332 | | - | |
333 | | - | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
334 | 359 | | |
335 | 360 | | |
336 | | - | |
337 | | - | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
338 | 364 | | |
339 | | - | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
340 | 381 | | |
341 | 382 | | |
342 | 383 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
3 | 5 | | |
4 | 6 | | |
5 | 7 | | |
| |||
186 | 188 | | |
187 | 189 | | |
188 | 190 | | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
189 | 308 | | |
190 | 309 | | |
191 | 310 | | |
| |||
0 commit comments