Skip to content

Commit 680b852

Browse files
committed
fix(core): instantiate each sub-client exactly once in _spawn_completion_context
`_spawn_completion_context` previously constructed the first sub-client TWICE — once at line 206 as `other_backend_client`, again at line 213 inside the registration loop with identical args. Two independent BaseLM instances pointed at the same sub-model. Impact: - For backends that hold expensive underlying state (HTTP connection pools, async clients, OS sockets), the duplicate carried real cost. We observed a 3-minute hang at iter 2 of every multi-backend trajectory on macOS with anthropic backends — fully recovered by de-duplicating the instantiation. - Usage tracking merged via dict.update at lm_handler.py:198-205, so the duplicate's summary over-wrote one of the two instances' usage data — silently under-counted cost in `get_usage_summary()`. Fix: build a single list of sub-clients from `zip(self.other_backends, self.other_backend_kwargs)` exactly once. The first entry serves as `other_backend_client` (depth=1 default routing when LMHandler.get_client receives no `model` arg) AND is registered by its `model_name` for explicit-model routing — same instance, both paths. Routing semantics are preserved: `get_client(model, depth)` at lm_handler.py:131 still returns `clients[model]` first, then falls back to `other_backend_client` for `depth==1`. Both now point to the same object for the first sub-model. Verified end-to-end downstream in delta360-rlm harness PR #6: - pre-fix: trajectory hung at iter 2 for 3+ minutes (0% CPU, 300s proxy/socket timeout suspected) - post-fix: 47.3s wall, $0.058 actual cost, 2 sub-calls, depth_reached=1 No behavior change for single-backend (no-sub_model) callers — the loop body is unchanged for them. Tests in delta360-rlm (`tests/test_multibackend_patch.py`) verify get_client call counts and instance-identity invariants for: (a) single sub-model, (b) no sub-models, (c) multiple sub-models. 5/5 pass.
1 parent 03a1774 commit 680b852

1 file changed

Lines changed: 29 additions & 8 deletions

File tree

‎rlm/core/rlm.py‎

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -200,18 +200,39 @@ def _spawn_completion_context(self, prompt: str | dict[str, Any]):
200200
# Create client and wrap in handler
201201
client: BaseLM = get_client(self.backend, self.backend_kwargs)
202202

203-
# Create other_backend_client if provided (for depth=1 routing)
204-
other_backend_client: BaseLM | None = None
203+
# Create one BaseLM instance per (backend, kwargs) pair. Each entry in
204+
# `other_backends` / `other_backend_kwargs` is constructed exactly
205+
# once. The first sub-client serves both as `other_backend_client`
206+
# (depth=1 default routing when LMHandler.get_client receives no
207+
# `model` arg) AND is registered by its `model_name` for explicit-
208+
# model routing — same instance, both paths.
209+
#
210+
# Previous behavior constructed the first sub-client TWICE: once at
211+
# line 206 (for `other_backend_client`) and again inside the loop at
212+
# line 213 (for `register_client`). With backends that hold expensive
213+
# underlying state (HTTP connection pools, async clients, OS sockets),
214+
# the duplicate carried real cost; with multi-backend Anthropic on
215+
# macOS we observed a 3-min hang at iter 2 of every multi-backend
216+
# trajectory, fully recovered by de-dup. Usage tracking also merged
217+
# via dict.update at lm_handler.py:198-205, so the duplicate
218+
# over-wrote one of the two instances' usage summaries.
219+
other_clients: list[BaseLM] = []
205220
if self.other_backends and self.other_backend_kwargs:
206-
other_backend_client = get_client(self.other_backends[0], self.other_backend_kwargs[0])
221+
for backend, kwargs in zip(
222+
self.other_backends, self.other_backend_kwargs, strict=True,
223+
):
224+
other_clients.append(get_client(backend, kwargs))
225+
226+
other_backend_client: BaseLM | None = (
227+
other_clients[0] if other_clients else None
228+
)
207229

208230
lm_handler = LMHandler(client, other_backend_client=other_backend_client)
209231

210-
# Register other clients to be available as sub-call options (by model name)
211-
if self.other_backends and self.other_backend_kwargs:
212-
for backend, kwargs in zip(self.other_backends, self.other_backend_kwargs, strict=True):
213-
other_client: BaseLM = get_client(backend, kwargs)
214-
lm_handler.register_client(other_client.model_name, other_client)
232+
# Register every sub-client (including `other_backend_client`) by its
233+
# `model_name` — the SAME instance, NOT a new one.
234+
for sub_client in other_clients:
235+
lm_handler.register_client(sub_client.model_name, sub_client)
215236

216237
lm_handler.start()
217238

0 commit comments

Comments
 (0)