Skip to content

Commit 8d5037d

Browse files
docs(readme): add the agentic benchmark chart (#160)
Grouped bars of LOC, tokens, cost and time as a % of the no-skill baseline (lower is leaner/cheaper/faster), plus a separate safety strip (baseline, caveman and ponytail 100%; yagni-oneliner 95%). System-gray palette so it reads on both GitHub themes. The chart commits landed after #158 had already squash-merged, so this brings the chart onto main. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent b8d6aa7 commit 8d5037d

2 files changed

Lines changed: 66 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ More survivors in [examples/](examples/).
4646

4747
The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5.
4848

49+
<p align="center">
50+
<img src="assets/benchmark-agentic.svg" width="860" alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on every metric (LOC 46%, tokens 78%, cost 80%, time 73%); caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%. Safety, separate adversarial tier: baseline, caveman and ponytail 100%, yagni-oneliner 95%.">
51+
</p>
52+
4953
| vs no-skill baseline | LOC | tokens | cost | time | safe |
5054
|---|--:|--:|--:|--:|--:|
5155
| **ponytail** | **-54%** | **-22%** | **-20%** | **-27%** | **100%** |

‎assets/benchmark-agentic.svg‎

Lines changed: 62 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)