Skip to content

Commit 51c25b0

Browse files
committed
new dict-based final answer format
1 parent 03a1774 commit 51c25b0

30 files changed

Lines changed: 336 additions & 548 deletions

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ Environments must provide these globals to executed code:
173173
- `llm_query_batched(prompts, model=None)`: Batched plain LM completions
174174
- `rlm_query(prompt, model=None)`: Recursive child RLM call (own REPL + iteration). Falls back to `llm_query` at max depth.
175175
- `rlm_query_batched(prompts, model=None)`: Batched recursive child RLM calls
176-
- `FINAL_VAR(variable_name)`: For returning final answers
176+
- `answer`: A dict (`{"content": "", "ready": False}`); the model sets `answer["content"]` and `answer["ready"] = True` to return a final answer. The env surfaces the content on `REPLResult.final_answer` once `ready` flips truthy.
177177
- `SHOW_VARS()`: For listing available variables
178178

179179
### Example Structure
@@ -208,7 +208,7 @@ class MyEnvironment(NonIsolatedEnv):
208208
- Environment works with basic RLM completion calls
209209
- `cleanup()` properly releases all resources
210210
- Sub-LM calls work via `llm_query()` and `rlm_query()`
211-
- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `FINAL_VAR`, `SHOW_VARS`) are restored after each execution
211+
- Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`) are restored after each execution
212212

213213
## Architecture: Environment ↔ LM Handler Communication
214214

‎docs/api/rlm.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -255,12 +255,15 @@ Override the default RLM system prompt. The default prompt instructs the LM on:
255255
- How to use the `context` variable
256256
- How to call `llm_query()` / `llm_query_batched()` for plain LM calls
257257
- How to call `rlm_query()` / `rlm_query_batched()` for recursive sub-calls
258-
- How to signal completion with `FINAL()` or `FINAL_VAR()`
258+
- How to signal completion by setting `answer["content"]` and `answer["ready"] = True`
259259

260260
```python
261261
custom_prompt = """You are a data analysis expert.
262262
Use the REPL to analyze the context variable.
263-
When done, output FINAL(your answer)."""
263+
When done, run a ```repl``` block that does:
264+
answer["content"] = "your final answer here"
265+
answer["ready"] = True
266+
"""
264267

265268
rlm = RLM(..., custom_system_prompt=custom_prompt)
266269
```
@@ -372,7 +375,7 @@ custom_tools = {
372375
}
373376
```
374377

375-
Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `FINAL_VAR`, `SHOW_VARS`, and their batched variants) cannot be used as tool names.
378+
Reserved names (`llm_query`, `rlm_query`, `context`, `history`, `answer`, `SHOW_VARS`, and their batched variants) cannot be used as tool names.
376379

377380
---
378381

@@ -550,7 +553,7 @@ The following functions are available to model-generated code inside the REPL:
550553
| `llm_query_batched(prompts, model=None)` | Multiple plain LM completions concurrently. |
551554
| `rlm_query(prompt, model=None)` | Spawn a child RLM with its own REPL for deeper thinking. Falls back to `llm_query` at max depth. |
552555
| `rlm_query_batched(prompts, model=None)` | Spawn multiple child RLMs. Falls back to `llm_query_batched` at max depth. |
553-
| `FINAL_VAR(variable_name)` | Return a REPL variable as the final answer. |
556+
| `answer` | A dict (`{"content": "", "ready": False}`). Set `answer["content"]` to your final answer and `answer["ready"] = True` to terminate the run. |
554557
| `SHOW_VARS()` | List all user-created variables in the REPL. |
555558
| `print(...)` | Print output visible to the model in the next iteration. |
556559

@@ -570,7 +573,7 @@ rlm = RLM(backend="unknown")
570573
# Raises: ValueError: Unknown backend: unknown
571574
```
572575

573-
If the RLM exhausts `max_iterations` without finding a `FINAL()` / `FINAL_VAR()` answer, it prompts the LM one more time to provide a final answer based on the conversation history.
576+
If the RLM exhausts `max_iterations` without the model setting `answer["ready"] = True`, it prompts the LM one more time to provide a final answer based on the conversation history.
574577

575578
RLM raises explicit exceptions when limits are exceeded:
576579

‎docs/architecture.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ An RLM completion involves three cooperating pieces:
3737
│ b. Extract ```repl``` code blocks from response │
3838
│ c. Execute code in LocalREPL │
3939
│ d. Append stdout/stderr to message history │
40-
│ e. Repeat until FINAL_VAR / FINAL or limits exceeded │
40+
│ e. Repeat until answer["ready"] is True or limits exceeded │
4141
│ 4. Tear down handler and environment │
4242
└────────────────────────────────────────────────────────────────┘
4343
```
@@ -139,7 +139,6 @@ globals (shared across all executions):
139139
├── llm_query_batched() → batched plain LM calls
140140
├── rlm_query() → recursive RLM sub-call (or fallback to llm_query)
141141
├── rlm_query_batched() → batched recursive sub-calls
142-
├── FINAL_VAR() → mark a variable as the final answer
143142
├── SHOW_VARS() → list user-created variables
144143
└── <custom_tools> → user-provided callable tools
145144
@@ -148,6 +147,7 @@ locals (accumulates user variables):
148147
├── context_0 → first context payload
149148
├── context_1, … → additional contexts (persistent mode)
150149
├── history → conversation history (persistent/compaction mode)
150+
├── answer → {"content": "", "ready": False}; set ready=True to finish
151151
└── <user variables> → anything created by model code
152152
```
153153

@@ -333,13 +333,13 @@ RLM (depth=0)
333333
│ │ │ │
334334
│ │ │ └─ llm_query() → TCP to Handler #2 → LM API → response
335335
│ │ │
336-
│ │ ├─ Child iteration 2: LM calls FINAL_VAR(result)
336+
│ │ ├─ Child iteration 2: LM sets answer["content"]=result, answer["ready"]=True
337337
│ │ │
338338
│ │ └─ Returns RLMChatCompletion to parent
339339
│ │
340-
│ └─ answer = child_completion.response
340+
│ └─ child_response = child_completion.response
341341
│
342-
├─ Iteration 2: LM uses answer, calls FINAL_VAR(final)
342+
├─ Iteration 2: LM uses child_response, sets answer["content"]=final, answer["ready"]=True
343343
│
344344
├─ LMHandler #1 stops
345345
└─ Returns RLMChatCompletion to user

‎examples/compaction_example.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@
5656
"Step 4: Compute and print the mean, median, min, max, and standard deviation "
5757
"of `data_b`. Store the mean as `mean_b`.\n\n"
5858
"Step 5: You previously computed `mean_a` and `mean_b`. "
59-
"Compute final_answer = round(mean_a + mean_b, 2) and call FINAL_VAR(final_answer)."
59+
"Compute final_value = round(mean_a + mean_b, 2), then set answer['content'] = final_value and answer['ready'] = True."
6060
)
6161

6262
result = rlm.completion(prompt, root_prompt=prompt)

‎examples/compaction_history_retrieval_example.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,8 @@
5252
"Step 6: Compute the standard deviation of each group. Print all four. "
5353
"Store as `s1`, `s2`, `s3`, `s4`.\n\n"
5454
"Step 7: You previously computed four means and four standard deviations. "
55-
"Compute final_answer = round((m1 + m2 + m3 + m4) / (s1 + s2 + s3 + s4), 4). "
56-
"Print the full equation with values and call FINAL_VAR(final_answer)."
55+
"Compute final_value = round((m1 + m2 + m3 + m4) / (s1 + s2 + s3 + s4), 4). "
56+
"Print the full equation with values, then set answer['content'] = final_value and answer['ready'] = True."
5757
)
5858

5959
result = rlm.completion(prompt, root_prompt=prompt)

‎examples/daytona_repl_example.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,9 +58,8 @@ def main():
5858
print("Executed: print(x * 2)")
5959
print(f"Stdout: {result.stdout.strip()}")
6060

61-
result = repl.execute_code("answer = 42")
62-
result = repl.execute_code('print(FINAL_VAR("answer"))')
63-
print(f"FINAL_VAR('answer'): {result.stdout.strip()}")
61+
result = repl.execute_code('answer["content"] = 42\nanswer["ready"] = True')
62+
print(f"answer-dict final answer: {result.final_answer!r}")
6463

6564
# Example 2: With LLM handler
6665
print("\n[2] Code execution with LLM handler")

‎examples/depth_metadata_example.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ def main():
107107
prompt = (
108108
"Use rlm_query() to ask the sub-model: 'What are the first 5 prime numbers? "
109109
"Reply with just the numbers separated by commas.' "
110-
"Store the response in a variable called 'primes', then return it with FINAL_VAR('primes')."
110+
"Store the response in a variable called 'primes', then set answer['content'] = primes and answer['ready'] = True."
111111
)
112112

113113
print("Prompt:", prompt)

‎examples/logger_example.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
verbose=True,
3030
)
3131

32-
prompt = "Compute the sum 1 + 2 + 3 + 4 + 5 and print the result using a REPL block, then return it with FINAL_VAR."
32+
prompt = "Compute the sum 1 + 2 + 3 + 4 + 5 and print the result using a REPL block, then set answer['content'] to the sum and answer['ready'] = True."
3333
result = rlm.completion(prompt)
3434

3535
print("Response:", result.response)

‎examples/modal_repl_example.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,8 @@ def main():
5353
print("Executed: print(x * 2)")
5454
print(f"Stdout: {result.stdout.strip()}")
5555

56-
result = repl.execute_code("answer = 42")
57-
result = repl.execute_code('print(FINAL_VAR("answer"))')
58-
print(f"FINAL_VAR('answer'): {result.stdout.strip()}")
56+
result = repl.execute_code('answer["content"] = 42\nanswer["ready"] = True')
57+
print(f"answer-dict final answer: {result.final_answer!r}")
5958

6059
# Example 2: With LLM handler
6160
print("\n[2] Code execution with LLM handler")

‎examples/rlm_query_batched_example.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,8 @@ def main():
106106
" 1. 'What are the first 5 prime numbers? Reply with just the numbers.'\n"
107107
" 2. 'What are the first 5 even numbers? Reply with just the numbers.'\n"
108108
" 3. 'What are the first 5 square numbers? Reply with just the numbers.'\n"
109-
"Store the list of responses in a variable called 'answers', "
110-
"then return it with FINAL_VAR(answers)."
109+
"Store the list of responses in a variable called 'responses', "
110+
"then set answer['content'] = responses and answer['ready'] = True."
111111
)
112112

113113
print("Prompt:", prompt)

0 commit comments

Comments
 (0)