Skip to content

feat: parallelize rlm_query_batched subcalls for faster execution - #139

Merged
alexzhang13 merged 2 commits into
alexzhang13:mainfrom
dhananjai1729:feat/parallel-rlm-query-batched
Mar 19, 2026
Merged

alexzhang13 merged 2 commits into
alexzhang13:mainfrom
dhananjai1729:feat/parallel-rlm-query-batched

Conversation

@dhananjai1729

Copy link
Copy Markdown
Contributor

Summary

Addresses #135 — rlm_query_batched now runs independent child RLM subcalls in parallel using ThreadPoolExecutor, reducing wall time from N × T to ~T for N I/O-bound subcalls.

  • Parallel execution via bounded thread pool for 2+ prompts in _rlm_query_batched
  • New max_concurrent_subcalls parameter (default: 4) on both RLM and LocalREPL, propagated to child RLMs at each recursion level
  • Thread-safe completion collection with a local lock; results and _pending_llm_calls maintain original prompt order for deterministic metadata
  • Zero overhead for single prompts — skips ThreadPoolExecutor when len(prompts) <= 1
  • Partial failure isolation — exceptions in one thread don't affect others

Files changed

File Change
rlm/environments/local_repl.py Rewrote _rlm_query_batched with ThreadPoolExecutor; added max_concurrent_subcalls param
rlm/core/rlm.py Added max_concurrent_subcalls param; propagated to environment and child RLMs
tests/test_rlm_query.py Added 6 new tests for parallel execution

Test plan

  • All 263 existing tests pass (10 skipped due to missing API keys)
  • New TestRlmQueryBatchedParallel test class with 6 tests:
    • test_batched_runs_in_parallel — verifies wall time is significantly less than sequential
    • test_batched_respects_max_concurrent — verifies concurrency cap via active thread tracking
    • test_batched_preserves_order_parallel — verifies result order matches input order with random delays
    • test_batched_parallel_partial_failure — verifies failures in one thread don't affect others
    • test_batched_pending_calls_ordered — verifies _pending_llm_calls are in prompt order
    • test_single_prompt_skips_thread_pool — verifies no thread pool overhead for single prompt

cc @alexzhang13 @alt-glitch @lakshyaag @rawwerks — would appreciate your review and feedback!

🤖 Generated with Claude Code

dhananjai1729 and others added 2 commits March 19, 2026 21:09
When rlm_query_batched fans out multiple independent child RLM calls,
they now run concurrently via a bounded thread pool instead of
sequentially. This reduces wall time from N*T to ~T for N I/O-bound
subcalls.

Key changes:
- Use ThreadPoolExecutor in _rlm_query_batched for 2+ prompts
- Add max_concurrent_subcalls parameter (default 4) with semaphore-style
  concurrency control at each recursion level
- Propagate setting from RLM -> LocalREPL -> child RLMs
- Thread-safe completion collection with deterministic ordering
- Skip thread pool for 0-1 prompts (no overhead)

Closes alexzhang13#135

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alexzhang13
alexzhang13 merged commit 5995713 into alexzhang13:main Mar 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants