You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rebased on main (post-#63). Adds RetryConfig.__post_init__ validation for negative field values, uses the named _TRANSIENT_STATUS_RE constant instead of an inline regex, keeps all tests from both sides including the ConnectionError/TimeoutError subclass tests from #63.
> Automatic repair for failing Python code, powered by any LLM.
9
9
10
-
self-heal demo
10
+

11
11
12
12
`self-heal` catches failures, proposes an LLM-guided fix with memory of prior attempts, verifies it, and retries. Works with Claude, OpenAI, Gemini, and 100+ other providers. Sync and async. One decorator.
13
13
@@ -32,23 +32,19 @@ Two suites, both run against Gemini 2.5 Flash, 3 max attempts, v0.4 harness.
Reproduce: `self-heal bench --proposer gemini --model gemini-2.5-flash` (default) or `--suite quixbugs`. Full numbers, historical rows, and how to contribute your own in `[benchmarks/RESULTS.md](benchmarks/RESULTS.md)`. Task source in `[benchmarks/tasks.py](benchmarks/tasks.py)`.
47
+
Reproduce: `self-heal bench --proposer gemini --model gemini-2.5-flash` (default) or `--suite quixbugs`. Full numbers, historical rows, and how to contribute your own in [`benchmarks/RESULTS.md`](benchmarks/RESULTS.md). Task source in [`benchmarks/tasks.py`](benchmarks/tasks.py).
52
48
53
49
The +2 tasks on each suite share a pattern: the first proposed fix handles one edge case but misses another. Memory of the failed attempt plus test feedback lets the second proposal cover both. Roughly 20% more LLM calls for the additional wins. As frontier models keep improving the naive floor rises and this delta compresses; earlier runs against Gemini 2.5 Flash had naive at 68% instead of 84%, which is honest signal not cherry-picked.
Every proposal sees the history of *prior failed attempts* so the LLM can't repeat the same mistake. This is the single biggest quality win over naive retry.
94
87
95
88
### Verifiers: `verify=callable`
96
-
97
89
Catch bad *return values*, not just exceptions:
98
90
99
91
```python
@@ -104,7 +96,6 @@ def extract_price(text): ...
104
96
If the predicate returns `False` or raises, self-heal treats it as a failure and repairs.
105
97
106
98
### Test-driven repair: `tests=[...]`
107
-
108
99
Give self-heal a test suite; it repairs until every test passes:
109
100
110
101
```python
@@ -116,7 +107,6 @@ def extract_price(text): ...
116
107
```
117
108
118
109
### Async-native
119
-
120
110
The decorator auto-detects `async def` and awaits correctly; the LLM call runs in a thread pool so your event loop stays free.
Append domain-specific instructions to every repair prompt. Useful for "always handle None inputs" or "use only the standard library."
130
119
131
120
### Bring your own LLM
132
-
133
121
Implement the `LLMProposer` Protocol (`def propose(self, system: str, user: str) -> str`) and pass it in.
134
122
135
123
### Repair cache: skip the LLM when you've seen it before
136
-
137
124
```python
138
125
from self_heal import repair
139
126
140
127
@repair(cache_path=".self_heal_cache.db")
141
128
defmy_fn(...): ...
142
129
```
143
-
144
130
First repair hits the LLM. Subsequent identical failures are served from SQLite (zero latency, zero cost). Keyed on source hash + failure signature with whitespace and memory-address normalization.
`moderate` rejects proposals that call `eval` / `exec` / `os.system`, import `subprocess` / `socket` / `pickle` / `ctypes`, or touch `__globals__` / `__class__` / other escape hatches. `strict` additionally forbids any non-whitelisted import. The subprocess sandbox adds a real process boundary: args and return values are pickled over stdin/stdout, and the child inherits none of the caller's globals (proposals must be self-contained). See [Safety](#safety) for the full trust model.
163
147
164
148
> **Sandbox + imports.** When `sandbox="subprocess"` is active, the child runs with `python -I` in a fresh namespace. **The repaired function must import every module it uses at the top of the definition.**`import math` at the caller module scope does NOT reach the sandbox, so a proposal that references `math.sqrt` without a local `import math` raises `NameError` on the first call. `self-heal` already hints at this in the LLM prompt when sandbox is active, but if you're writing a proposer by hand the same rule applies.
Hooks fire on attempt start, failure, propose start/complete, install, cache hit/miss, safety violation, verify, and repair completion. Perfect for agent UIs and observability pipelines.
179
161
180
162
### Token streaming
181
-
182
163
When a callback is registered, self-heal streams LLM tokens through `propose_chunk` events as they arrive:
All four built-in proposers stream natively via their SDKs. Custom proposers can implement `propose_stream(system, user) -> Iterator[str]` (and `apropose_stream` for async) to participate; those without streaming fall back to a single completion. See `[examples/streaming_progress.py](examples/streaming_progress.py)`.
175
+
All four built-in proposers stream natively via their SDKs. Custom proposers can implement `propose_stream(system, user) -> Iterator[str]` (and `apropose_stream` for async) to participate; those without streaming fall back to a single completion. See [`examples/streaming_progress.py`](examples/streaming_progress.py).
196
176
197
177
### Native async proposers
198
-
199
178
`arun` prefers each SDK's native async client when the proposer provides `apropose`, falling back to `asyncio.to_thread(propose)` otherwise. All four built-in adapters ship with native async; custom proposers work either way.
200
179
201
180
### Resilience: retry on transient provider errors
202
-
203
181
Rate limits (429), service blips (502/503/504), and timeouts are common with LLM APIs. Pass a `RetryConfig` and self-heal will retry the proposer call with exponential backoff + jitter before giving up. Auth and validation errors are not retried.
204
182
205
183
```python
@@ -212,7 +190,6 @@ def my_fn(...): ...
212
190
Defaults: 3 retries, 1s base delay, 2x backoff, ±25% jitter, capped at 30s. Retries do not count against `max_attempts` and each one fires a `transient_retry` event with `retry_attempt` and `retry_delay`. Set `retry_config=None` (the default) to disable retries.
213
191
214
192
### pytest plugin: `pytest --heal`
215
-
216
193
Mark any test with `@pytest.mark.heal(target="mymod.my_fn")`. When it fails with `--heal`, self-heal loads the target, repairs it using the test as verification, and prints the proposed diff at the end of the session.
217
194
218
195
```python
@@ -223,23 +200,19 @@ from mymod import extract_price
pytest --heal-apply # write the fix back to disk (creates a .py.heal-backup)
230
206
pytest --heal-apply-force # also allow modification of git-dirty files
231
207
```
232
-
233
208
`--heal-apply` uses libcst for AST-faithful replacement when installed, falling back to textual replacement. It refuses to modify files with uncommitted git changes unless `--heal-apply-force` is given.
234
209
235
210
### CLI: heal a function from the command line
236
-
237
211
```bash
238
212
self-heal heal mymod.py::extract_price \
239
213
--test tests/test_mymod.py::test_rupees \
240
214
--apply
241
215
```
242
-
243
216
Loads the function, runs self-heal with your pytest-style test as verification, prints a unified diff, and (with `--apply`) writes the fix back to the file.
244
217
245
218
## Why this exists
@@ -293,14 +266,12 @@ result = await loop.arun(my_async_fn, args=(...))
- v0.4.0: streaming token events (`propose_chunk`), native async proposers (`apropose`) for all four adapters
431
-
-**v0.4.1: sandbox preserves custom exceptions from proposals; `is_git_dirty` fails closed on timeout; Claude Agent SDK and LangChain/LangGraph first-class integrations**
-[x]v0.4.0: streaming token events (`propose_chunk`), native async proposers (`apropose`) for all four adapters
398
+
-[x]**v0.4.1: sandbox preserves custom exceptions from proposals; `is_git_dirty` fails closed on timeout; Claude Agent SDK and LangChain/LangGraph first-class integrations**
-[ ]v1.0: stable API + extended benchmark suite (HumanEval-Fix, Refactory)
434
401
435
402
## Deeper docs
436
403
437
-
-`[docs/sandbox-threat-model.md](docs/sandbox-threat-model.md)`: what the subprocess sandbox protects against and what it does not. Read before running against untrusted inputs.
438
-
-`[docs/custom-proposer.md](docs/custom-proposer.md)`: implementing the `LLMProposer` Protocol for an unsupported provider.
-[`docs/sandbox-threat-model.md`](docs/sandbox-threat-model.md): what the subprocess sandbox protects against and what it does not. Read before running against untrusted inputs.
405
+
-[`docs/custom-proposer.md`](docs/custom-proposer.md): implementing the `LLMProposer` Protocol for an unsupported provider.
See `[CONTRIBUTING.md](CONTRIBUTING.md)` for the full guide: dev setup, everyday commands, how to add a new LLM proposer or benchmark task, and the PR checklist. Good first issues are tagged [here](https://github.com/Johin2/self-heal/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
410
+
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the full guide: dev setup, everyday commands, how to add a new LLM proposer or benchmark task, and the PR checklist. Good first issues are tagged [here](https://github.com/Johin2/self-heal/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
444
411
445
412
## Development (quick start)
446
413
@@ -455,7 +422,6 @@ ruff check .
455
422
```
456
423
457
424
Run the benchmark locally:
458
-
459
425
```bash
460
426
python benchmarks/run.py --proposer claude # uses ANTHROPIC_API_KEY
0 commit comments