|
9 | 9 |
|
10 | 10 | The REPL environment is initialized with: |
11 | 11 | 1. A `context` variable that contains extremely important information about your query. You should check the content of the `context` variable to understand what you are working with. Make sure you look through it sufficiently as you answer your query. |
12 | | -2. A `llm_query` function that allows you to query an LLM (that can handle around 500K chars) inside your REPL environment. |
13 | | -3. A `llm_query_batched` function that allows you to query multiple prompts concurrently: `llm_query_batched(prompts: List[str]) -> List[str]`. This is much faster than sequential `llm_query` calls when you have multiple independent queries. Results are returned in the same order as the input prompts. |
14 | | -4. A `SHOW_VARS()` function that returns all variables you have created in the REPL. Use this to check what variables exist before using FINAL_VAR. |
15 | | -5. The ability to use `print()` statements to view the output of your REPL code and continue your reasoning. |
| 12 | +2. A `llm_query(prompt, model=None)` function that makes a single LLM completion call (no REPL, no iteration). Fast and lightweight -- use this for simple extraction, summarization, or Q&A over a chunk of text. The sub-LLM can handle around 500K chars. |
| 13 | +3. A `llm_query_batched(prompts, model=None)` function that runs multiple `llm_query` calls concurrently: returns `List[str]` in the same order as input prompts. Much faster than sequential `llm_query` calls for independent queries. |
| 14 | +4. A `rlm_query(prompt, model=None)` function that spawns a **recursive RLM sub-call** for deeper thinking subtasks. The child gets its own REPL environment and can reason iteratively over the prompt, just like you. Use this when a subtask requires multi-step reasoning, code execution, or its own iterative problem-solving -- not just a simple one-shot answer. Falls back to `llm_query` if recursion is not available. |
| 15 | +5. A `rlm_query_batched(prompts, model=None)` function that spawns multiple recursive RLM sub-calls. Each prompt gets its own child RLM. Falls back to `llm_query_batched` if recursion is not available. |
| 16 | +6. A `SHOW_VARS()` function that returns all variables you have created in the REPL. Use this to check what variables exist before using FINAL_VAR. |
| 17 | +7. The ability to use `print()` statements to view the output of your REPL code and continue your reasoning. |
16 | 18 | {custom_tools_section} |
17 | 19 |
|
| 20 | +**When to use `llm_query` vs `rlm_query`:** |
| 21 | +- Use `llm_query` for simple, one-shot tasks: extracting info from a chunk, summarizing text, answering a factual question, classifying content. These are fast single LLM calls. |
| 22 | +- Use `rlm_query` when the subtask itself requires deeper thinking: multi-step reasoning, solving a sub-problem that needs its own REPL and iteration, or tasks where a single LLM call might not be enough. The child RLM can write and run code, query further sub-LLMs, and iterate to find the answer. |
| 23 | +
|
| 24 | +**Breaking down problems:** You must break problems into more digestible components—whether that means chunking or summarizing a large context, or decomposing a hard task into easier sub-problems and delegating them via `llm_query` / `rlm_query`. Use the REPL to write a **programmatic strategy** that uses these LLM calls to solve the problem, as if you were building an agent: plan steps, branch on results, combine answers in code. |
| 25 | +
|
| 26 | +**REPL for computation:** You can also use the REPL to compute programmatic steps (e.g. `math.sin(x)`, distances, physics formulas) and then chain those results into an LLM call. For complex math or physics, compute intermediate quantities in code and pass the numbers to the LM for interpretation or the final answer. Example: data describes an electron in a magnetic field undergoing helical motion; task is to find the entry angle. |
| 27 | +```repl |
| 28 | +import math |
| 29 | +# Suppose the context or an earlier LM call gave us: B, m, q, pitch, R (radius). Extract or set them. |
| 30 | +# Helical motion: v_parallel = pitch * (q*B)/(2*pi*m), v_perp = R * (q*B)/m. Entry angle theta: tan(theta) = v_perp/v_parallel. |
| 31 | +v_parallel = pitch * (q * B) / (2 * math.pi * m) |
| 32 | +v_perp = R * (q * B) / m |
| 33 | +theta_rad = math.atan2(v_perp, v_parallel) |
| 34 | +theta_deg = math.degrees(theta_rad) |
| 35 | +final_answer = llm_query(f"An electron entered a B field and underwent helical motion. Computed entry angle: {{theta_deg:.2f}} deg. State the answer clearly for the user.") |
| 36 | +``` |
18 | 37 | You will only be able to see truncated outputs from the REPL environment, so you should use the query LLM function on variables you want to analyze. You will find this function especially useful when you have to analyze the semantics of the context. Use these variables as buffers to build up your final answer. |
19 | | -Make sure to explicitly look through the entire context in REPL before answering your query. An example strategy is to first look at the context and figure out a chunking strategy, then break up the context into smart chunks, and query an LLM per chunk with a particular question and save the answers to a buffer, then query an LLM with all the buffers to produce your final answer. |
| 38 | +Make sure to explicitly look through the entire context in REPL before answering your query. Break the context and the problem into digestible pieces: e.g. figure out a chunking strategy, break up the context into smart chunks, query an LLM per chunk and save answers to a buffer, then query an LLM over the buffers to produce your final answer. |
20 | 39 |
|
21 | 40 | You can use the REPL environment to help you understand your context, especially if it is huge. Remember that your sub LLMs are powerful -- they can fit around 500K characters in their context window, so don't be afraid to put a lot of context into them. For example, a viable strategy is to feed 10 documents per sub-LLM query. Analyze your input data and see if it is sufficient to just fit it in a few sub-LLM calls! |
22 | 41 |
|
|
60 | 79 | final_answer = llm_query(f"Aggregating all the answers per chunk, answer the original query about total number of jobs: {{query}}\\n\\nAnswers:\\n" + "\\n".join(answers)) |
61 | 80 | ``` |
62 | 81 |
|
63 | | -As a final example, after analyzing the context and realizing its separated by Markdown headers, we can maintain state through buffers by chunking the context by headers, and iteratively querying an LLM over it: |
| 82 | +For subtasks that require deeper reasoning (e.g. solving a complex sub-problem), use `rlm_query` instead. The child gets its own REPL to iterate; you can then use the result in parent logic: |
64 | 83 | ```repl |
65 | | -# After finding out the context is separated by Markdown headers, we can chunk, summarize, and answer |
66 | | -import re |
67 | | -sections = re.split(r'### (.+)', context["content"]) |
68 | | -buffers = [] |
69 | | -for i in range(1, len(sections), 2): |
70 | | - header = sections[i] |
71 | | - info = sections[i+1] |
72 | | - summary = llm_query(f"Summarize this {{header}} section: {{info}}") |
73 | | - buffers.append(f"{{header}}: {{summary}}") |
74 | | -final_answer = llm_query(f"Based on these summaries, answer the original query: {{query}}\\n\\nSummaries:\\n" + "\\n".join(buffers)) |
| 84 | +# Child RLM solves the sub-problem in its own REPL; we use the result in code |
| 85 | +trend = rlm_query(f"Analyze this dataset and conclude with one word: up, down, or stable: {{data}}") |
| 86 | +if "up" in trend.lower(): |
| 87 | + recommendation = "Consider increasing exposure." |
| 88 | +elif "down" in trend.lower(): |
| 89 | + recommendation = "Consider hedging." |
| 90 | +else: |
| 91 | + recommendation = "Hold position." |
| 92 | +final_answer = llm_query(f"Given trend={{trend}} and recommendation={{recommendation}}, one-sentence summary for the user.") |
75 | 93 | ``` |
76 | | -In the next step, we can return FINAL_VAR(final_answer). |
| 94 | +
|
| 95 | +As a final example, implement the solution as a **program**: try one approach via `rlm_query`; inspect the result and branch. If it suffices, use it. If not, break into one easier subproblem and delegate that only. More branches, one path runs—don't load the model. Example: prove √2 irrational. |
| 96 | +```repl |
| 97 | +r = rlm_query("Prove √2 irrational. Give a 1-2 sentence proof, or reply only: USE_LEMMA or USE_CONTRADICTION.") |
| 98 | +if "USE_LEMMA" in r.upper(): |
| 99 | + final_answer = rlm_query("Prove 'n^2 even => n even' then use it to show √2 irrational. Two sentences.") |
77 | 100 |
|
78 | 101 | IMPORTANT: When you are done with the iterative process, you MUST provide a final answer inside a FINAL function when you have completed your task, NOT in code. Do not use these tags unless you have completed your task. You have two options: |
79 | 102 | 1. Use FINAL(your final answer here) to provide the answer directly |
@@ -136,7 +159,7 @@ def build_rlm_system_prompt( |
136 | 159 |
|
137 | 160 | return [ |
138 | 161 | {"role": "system", "content": final_system_prompt}, |
139 | | - {"role": "assistant", "content": metadata_prompt}, |
| 162 | + {"role": "user", "content": metadata_prompt}, |
140 | 163 | ] |
141 | 164 |
|
142 | 165 |
|
|
0 commit comments