Skip to content

Commit afb7686

Browse files
docs(readme): note the up-to-94% peak in the hero line (DietrichGebert#165)
The hero showed only ~54% (the mean); the rigorous agentic run also reaches 94% on the over-build tasks (the date picker), so the headline now reads "~54% (up to 94%)". The sub-line is reworded so 80-94% reads as the per-task ceiling against a fair baseline, not the old single-shot figure, which would otherwise contradict the hero. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent aeeaa38 commit afb7686

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@
1919
</p>
2020

2121
<p align="center">
22-
<strong>~54% less code &middot; ~20% cheaper &middot; ~27% faster &middot; 100% safe</strong><br>
23-
<sub>Measured on real Claude Code sessions editing a real open-source repo (FastAPI + React), against the same agent with no skill. Mean across 12 feature tasks (Haiku 4.5, n=4). ponytail keeps every safety guard while a bare "write one-liners" prompt drops one. (An older single-shot test showed a larger 80-94% gap, but that counted a chatty model's prose; this is the honest multi-turn number.) <a href="benchmarks/results/2026-06-18-agentic.md">Full writeup</a> &middot; <a href="benchmarks/">reproduce it</a>.</sub>
22+
<strong>~54% less code (up to 94%) &middot; ~20% cheaper &middot; ~27% faster &middot; 100% safe</strong><br>
23+
<sub>Measured on real Claude Code sessions editing a real open-source repo (FastAPI + React), against the same agent with no skill. ~54% is the mean across 12 feature tasks (Haiku 4.5, n=4); it reaches 94% where an agent over-builds (a date picker) and is near zero where the code is already minimal. ponytail keeps every safety guard while a bare "write one-liners" prompt drops one. (The earlier single-shot benchmark reported 80-94% as a flat figure; against a fair agentic baseline that is the per-task ceiling, not the average.) <a href="benchmarks/results/2026-06-18-agentic.md">Full writeup</a> &middot; <a href="benchmarks/">reproduce it</a>.</sub>
2424
</p>
2525

2626
---

0 commit comments

Comments
 (0)