Repository navigation
Commit 5c29380
committed
feat: Add LLM call memoization/caching for llm_query (Issue #82)
This PR implements memoization for llm_query and llm_query_batched calls
to avoid redundant API requests during recursive RLM execution.
Key features:
- Thread-safe LRU cache with configurable max_size
- Optional TTL for time-based expiration
- Cache statistics for monitoring (hits, misses, evictions)
- SHA-256 based cache keys for (prompt, model) pairs
- Zero-configuration opt-in via cache parameter on RLM class
Files added:
- rlm/utils/cache.py: Core cache implementation (~400 lines)
- tests/test_cache.py: 39 comprehensive tests
Files modified:
- rlm/core/rlm.py: Added cache parameter to __init__
- rlm/environments/local_repl.py: Integrated caching in _llm_query
- rlm/utils/__init__.py: Export cache classes
Usage:
from rlm import RLM
from rlm.utils import LLMCallCache
cache = LLMCallCache(max_size=1000, ttl_seconds=3600)
rlm = RLM(backend='openai', cache=cache)
Addresses: #821 parent beb0603 commit 5c29380
5 files changed
Lines changed: 1010 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
41 | 44 | | |
42 | 45 | | |
43 | 46 | | |
| |||
70 | 73 | | |
71 | 74 | | |
72 | 75 | | |
| 76 | + | |
73 | 77 | | |
74 | 78 | | |
75 | 79 | | |
| |||
102 | 106 | | |
103 | 107 | | |
104 | 108 | | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
105 | 112 | | |
106 | 113 | | |
107 | 114 | | |
| |||
133 | 140 | | |
134 | 141 | | |
135 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
136 | 146 | | |
137 | 147 | | |
138 | 148 | | |
| |||
238 | 248 | | |
239 | 249 | | |
240 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
241 | 254 | | |
242 | 255 | | |
243 | 256 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
24 | 27 | | |
25 | 28 | | |
26 | 29 | | |
| |||
135 | 138 | | |
136 | 139 | | |
137 | 140 | | |
| 141 | + | |
138 | 142 | | |
139 | 143 | | |
140 | 144 | | |
141 | 145 | | |
142 | 146 | | |
143 | 147 | | |
| 148 | + | |
144 | 149 | | |
145 | 150 | | |
146 | 151 | | |
| |||
242 | 247 | | |
243 | 248 | | |
244 | 249 | | |
| 250 | + | |
245 | 251 | | |
246 | 252 | | |
247 | 253 | | |
| |||
250 | 256 | | |
251 | 257 | | |
252 | 258 | | |
253 | | - | |
| 259 | + | |
| 260 | + | |
254 | 261 | | |
255 | 262 | | |
256 | 263 | | |
257 | 264 | | |
258 | | - | |
| 265 | + | |
259 | 266 | | |
260 | 267 | | |
261 | 268 | | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
262 | 277 | | |
263 | 278 | | |
264 | 279 | | |
265 | 280 | | |
266 | 281 | | |
267 | 282 | | |
268 | 283 | | |
| 284 | + | |
269 | 285 | | |
270 | 286 | | |
271 | 287 | | |
| |||
276 | 292 | | |
277 | 293 | | |
278 | 294 | | |
| 295 | + | |
279 | 296 | | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
280 | 332 | | |
281 | 333 | | |
282 | 334 | | |
283 | 335 | | |
284 | | - | |
| 336 | + | |
285 | 337 | | |
286 | 338 | | |
287 | | - | |
| 339 | + | |
288 | 340 | | |
289 | 341 | | |
290 | | - | |
| 342 | + | |
291 | 343 | | |
292 | | - | |
| 344 | + | |
293 | 345 | | |
294 | 346 | | |
295 | 347 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
0 commit comments