Skip to content

Commit db1e06b

Browse files
committed
fix(core): instantiate each sub-client exactly once in _spawn_completion_context
Previously the first sub-client was constructed twice — once for `other_backend_client` and again inside the registration loop. With backends that hold expensive underlying state (HTTP connection pools, async clients, OS sockets), the duplicate carried real cost; with multi-backend Anthropic on macOS this showed up as a ~3-min hang at iter 2 of every multi-backend trajectory. Usage tracking also merged via dict.update, so the duplicate overwrote one of the two instances' usage summaries. Reuse `other_backend_client` for the first registration and only construct the remaining sub-clients in the loop.
1 parent a740331 commit db1e06b

1 file changed

Lines changed: 10 additions & 3 deletions

File tree

‎rlm/core/rlm.py‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -206,9 +206,16 @@ def _spawn_completion_context(self, prompt: str | dict[str, Any]):
206206

207207
lm_handler = LMHandler(client, other_backend_client=other_backend_client)
208208

209-
# Register other clients to be available as sub-call options (by model name)
210-
if self.other_backends and self.other_backend_kwargs:
211-
for backend, kwargs in zip(self.other_backends, self.other_backend_kwargs, strict=True):
209+
# Register other clients to be available as sub-call options (by model name).
210+
# Reuse other_backend_client for the first entry so each (backend, kwargs)
211+
# pair is instantiated exactly once.
212+
if other_backend_client is not None:
213+
lm_handler.register_client(other_backend_client.model_name, other_backend_client)
214+
for backend, kwargs in zip(
215+
self.other_backends[1:],
216+
self.other_backend_kwargs[1:],
217+
strict=True,
218+
):
212219
other_client: BaseLM = get_client(backend, kwargs)
213220
lm_handler.register_client(other_client.model_name, other_client)
214221

0 commit comments

Comments
 (0)