Repository navigation
Commit 680b852
committed
fix(core): instantiate each sub-client exactly once in _spawn_completion_context
`_spawn_completion_context` previously constructed the first sub-client
TWICE — once at line 206 as `other_backend_client`, again at line 213
inside the registration loop with identical args. Two independent
BaseLM instances pointed at the same sub-model.
Impact:
- For backends that hold expensive underlying state (HTTP connection
pools, async clients, OS sockets), the duplicate carried real cost.
We observed a 3-minute hang at iter 2 of every multi-backend
trajectory on macOS with anthropic backends — fully recovered by
de-duplicating the instantiation.
- Usage tracking merged via dict.update at lm_handler.py:198-205, so
the duplicate's summary over-wrote one of the two instances' usage
data — silently under-counted cost in `get_usage_summary()`.
Fix: build a single list of sub-clients from
`zip(self.other_backends, self.other_backend_kwargs)` exactly once.
The first entry serves as `other_backend_client` (depth=1 default
routing when LMHandler.get_client receives no `model` arg) AND is
registered by its `model_name` for explicit-model routing — same
instance, both paths.
Routing semantics are preserved: `get_client(model, depth)` at
lm_handler.py:131 still returns `clients[model]` first, then falls
back to `other_backend_client` for `depth==1`. Both now point to the
same object for the first sub-model.
Verified end-to-end downstream in delta360-rlm harness PR #6:
- pre-fix: trajectory hung at iter 2 for 3+ minutes (0% CPU, 300s
proxy/socket timeout suspected)
- post-fix: 47.3s wall, $0.058 actual cost, 2 sub-calls, depth_reached=1
No behavior change for single-backend (no-sub_model) callers — the
loop body is unchanged for them. Tests in delta360-rlm
(`tests/test_multibackend_patch.py`) verify get_client call counts
and instance-identity invariants for: (a) single sub-model, (b) no
sub-models, (c) multiple sub-models. 5/5 pass.1 parent 03a1774 commit 680b852
1 file changed
Lines changed: 29 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
200 | 200 | | |
201 | 201 | | |
202 | 202 | | |
203 | | - | |
204 | | - | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
205 | 220 | | |
206 | | - | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
207 | 229 | | |
208 | 230 | | |
209 | 231 | | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
215 | 236 | | |
216 | 237 | | |
217 | 238 | | |
| |||
0 commit comments