Skip to content

Commit 1e507dc

Browse files
committed
docs: update for depth>1, add e2e test
1 parent 51cb725 commit 1e507dc

3 files changed

Lines changed: 96 additions & 3 deletions

File tree

‎docs/api/rlm.md‎

Lines changed: 56 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,20 @@ RLM(
4545
depth: int = 0,
4646
max_depth: int = 1,
4747
max_iterations: int = 30,
48+
max_budget: float | None = None,
49+
max_timeout: float | None = None,
50+
max_tokens: int | None = None,
51+
max_errors: int | None = None,
4852
custom_system_prompt: str | None = None,
4953
other_backends: list[str] | None = None,
5054
other_backend_kwargs: list[dict] | None = None,
5155
logger: RLMLogger | None = None,
5256
verbose: bool = False,
57+
persistent: bool = False,
58+
on_subcall_start: Callable[[int, str, str], None] | None = None,
59+
on_subcall_complete: Callable[[int, str, float, str | None], None] | None = None,
60+
on_iteration_start: Callable[[int, int], None] | None = None,
61+
on_iteration_complete: Callable[[int, int, float], None] | None = None,
5362
)
5463
```
5564

@@ -158,7 +167,7 @@ environment_kwargs = {
158167
**Type:** `int`
159168
**Default:** `1`
160169

161-
Maximum recursion depth for nested RLM calls. Currently only depth 1 is fully supported.
170+
Maximum recursion depth for nested RLM calls. Depth>1 recursive subcalls are supported in the local environment via `llm_query()`.
162171

163172
When `depth >= max_depth`, the RLM falls back to a regular LM completion.
164173

@@ -187,6 +196,46 @@ rlm = RLM(
187196

188197
---
189198

199+
#### `max_budget`
200+
{: .no_toc }
201+
202+
**Type:** `float | None`
203+
**Default:** `None`
204+
205+
Maximum total USD cost for a completion. If exceeded, RLM raises `BudgetExceededError`. This requires a backend that reports cost.
206+
207+
---
208+
209+
#### `max_timeout`
210+
{: .no_toc }
211+
212+
**Type:** `float | None`
213+
**Default:** `None`
214+
215+
Maximum wall-clock seconds for a completion. If exceeded, RLM raises `TimeoutExceededError`.
216+
217+
---
218+
219+
#### `max_tokens`
220+
{: .no_toc }
221+
222+
**Type:** `int | None`
223+
**Default:** `None`
224+
225+
Maximum total tokens (input + output) for a completion. If exceeded, RLM raises `TokenLimitExceededError`.
226+
227+
---
228+
229+
#### `max_errors`
230+
{: .no_toc }
231+
232+
**Type:** `int | None`
233+
**Default:** `None`
234+
235+
Maximum consecutive REPL errors before aborting. If exceeded, RLM raises `ErrorThresholdExceededError`.
236+
237+
---
238+
190239
#### `custom_system_prompt`
191240
{: .no_toc }
192241

@@ -401,6 +450,12 @@ rlm = RLM(backend="unknown")
401450

402451
If the RLM exhausts `max_iterations` without finding a `FINAL()` answer, it prompts the LM one more time to provide a final answer based on the conversation history.
403452

453+
RLM also raises explicit exceptions when limits are exceeded:
454+
- `BudgetExceededError`
455+
- `TimeoutExceededError`
456+
- `TokenLimitExceededError`
457+
- `ErrorThresholdExceededError`
458+
404459
---
405460

406461
## Thread Safety
@@ -458,4 +513,3 @@ result = rlm.completion(
458513
root_prompt="Summarize the key findings"
459514
)
460515
```
461-

‎docs/getting-started.md‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,10 @@ This will display:
133133
| `environment_kwargs` | `dict` | `None` | Environment configuration |
134134
| `max_depth` | `int` | `1` | Maximum recursion depth |
135135
| `max_iterations` | `int` | `30` | Max REPL iterations per call |
136+
| `max_budget` | `float` | `None` | Max total USD cost (if provider reports cost) |
137+
| `max_timeout` | `float` | `None` | Max wall-clock seconds per completion |
138+
| `max_tokens` | `int` | `None` | Max total tokens (input + output) per completion |
139+
| `max_errors` | `int` | `None` | Max consecutive REPL errors before abort |
136140
| `custom_system_prompt` | `str` | `None` | Override default system prompt |
137141
| `other_backends` | `list` | `None` | Additional backends for sub-calls |
138142
| `other_backend_kwargs` | `list` | `None` | Configs for additional backends |
@@ -159,6 +163,18 @@ result = rlm.completion(
159163
- `root_model`: Model name used
160164
- `prompt`: Original input
161165

166+
### Depth>1 Recursion
167+
168+
Depth>1 recursive subcalls are supported. When code inside the REPL calls `llm_query()` and `max_depth` allows it, RLM spawns a child RLM with its own REPL. When `depth >= max_depth`, it falls back to a plain LM completion.
169+
170+
### Limits and Exceptions
171+
172+
RLM raises explicit exceptions when limits are exceeded:
173+
- `BudgetExceededError`
174+
- `TimeoutExceededError`
175+
- `TokenLimitExceededError`
176+
- `ErrorThresholdExceededError`
177+
162178
---
163179

164180
## Choosing an Environment
@@ -340,4 +356,3 @@ Upload `.jsonl` log files to visualize:
340356
- [API Reference](api/rlm.md) - Complete RLM class documentation
341357
- [Environments](environments/) - Deep dive into each environment
342358
- [Backends](backends.md) - Detailed backend configuration
343-

‎tests/test_e2e_depth.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
"""E2E test for depth>1 with real LLM via OpenRouter."""
2+
import os
3+
4+
import pytest
5+
6+
from rlm import RLM
7+
8+
9+
@pytest.mark.skipif(
10+
not os.environ.get("OPENROUTER_API_KEY"),
11+
reason="OPENROUTER_API_KEY not set",
12+
)
13+
def test_depth_2_real_llm():
14+
"""Test depth=2 recursion with google/gemini-3-flash-preview."""
15+
rlm = RLM(
16+
backend="openrouter",
17+
backend_kwargs={"model_name": "google/gemini-3-flash-preview"},
18+
max_iterations=2,
19+
max_depth=2,
20+
)
21+
result = rlm.completion("What is 2+2? Answer with just the number.")
22+
assert result.response is not None
23+
assert len(result.response) > 0
24+
print(f"Response: {result.response}")

0 commit comments

Comments
 (0)