Skip to content

Commit 57458ea

Browse files
GeanCartierclaude
authored andcommitted
docs: align README benchmark claims with the agentic results
- 'only arm that cuts every metric' contradicts the table: the YAGNI one-liner prompt also cuts all four and is slightly cheaper and faster; state what does set ponytail apart (biggest code cut, fully safe) - qualify the headline: 94% is one task, safety is 20/20 adversarial runs - fix the chart alt text the same way - drop the hook count for Codex (the shared hook file defines three) - list Kimi Code and Goose among hosts with commands, per INSTALL.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 552acd5 commit 57458ea

2 files changed

Lines changed: 7 additions & 7 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
</p>
3131

3232
<p align="center">
33-
<strong>~54% less code (up to 94%) &middot; ~20% cheaper &middot; ~27% faster &middot; 100% safe</strong><br>
33+
<strong>~54% less code (up to 94% on one task) &middot; ~20% cheaper &middot; ~27% faster &middot; safe in 20/20 adversarial runs</strong><br>
3434
<sub>Real Claude Code sessions editing a real FastAPI + React repo, the same agent with and without the skill (12 feature tasks, Haiku 4.5, n=4). <a href="#numbers">Details</a>.</sub>
3535
</p>
3636

@@ -81,7 +81,7 @@ codex plugin marketplace add DietrichGebert/ponytail
8181
codex plugin add ponytail@ponytail
8282
```
8383

84-
Then open `/hooks` in Codex, trust its two lifecycle hooks, and start a new thread.
84+
Then open `/hooks` in Codex, trust its lifecycle hooks, and start a new thread.
8585

8686
**Any other agent:** copy [`AGENTS.md`](AGENTS.md) into your project, or ask your agent to install [`skills/ponytail/SKILL.md`](skills/ponytail/SKILL.md) as a skill. Step by step for Copilot, Cursor, OpenCode, Gemini and the rest: **[INSTALL.md](INSTALL.md)**.
8787

@@ -133,14 +133,14 @@ Lazy, not negligent: trust-boundary validation, data-loss handling, security, an
133133
| `/ponytail-gain` | Show the measured impact scoreboard (less code, less cost, more speed) from the benchmark. |
134134
| `/ponytail-help` | Quick reference for the commands above. |
135135

136-
Commands need a skill-capable host (Claude Code, Codex, Devin CLI, OpenCode, Gemini, pi, Hermes Agent, Qoder, Grok Build). In Codex CLI and the IDE extension they're skills under the plugin's namespace; invoke with `$ponytail:ponytail-review`. Cursor with the [hooks](INSTALL.md#cursor) gets `/ponytail` level switching only, typed as a plain message. The instruction-only adapters (Cursor's rule file, Windsurf, Cline, Copilot, Kiro, Antigravity) load the always-on ruleset without the commands.
136+
Commands need a skill-capable host (Claude Code, Codex, Devin CLI, OpenCode, Gemini, pi, Hermes Agent, Qoder, Grok Build, Kimi Code, Goose). In Codex CLI and the IDE extension they're skills under the plugin's namespace; invoke with `$ponytail:ponytail-review`. Cursor with the [hooks](INSTALL.md#cursor) gets `/ponytail` level switching only, typed as a plain message. The instruction-only adapters (Cursor's rule file, Windsurf, Cline, Copilot, Kiro, Antigravity) load the always-on ruleset without the commands.
137137

138138
## Numbers
139139

140140
The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5.
141141

142142
<p align="center">
143-
<img src="assets/benchmark-agentic.svg" width="860" alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on every metric (LOC 46%, tokens 78%, cost 80%, time 73%); caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%. Safety, separate adversarial tier: baseline, caveman and ponytail 100%, yagni-oneliner 95%.">
143+
<img src="assets/benchmark-agentic.svg" width="860" alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on LOC and tokens (LOC 46%, tokens 78%, cost 80%, time 73%), the YAGNI one-liner prompt slightly lower on cost and time; caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%. Safety, separate adversarial tier: baseline, caveman and ponytail 100%, yagni-oneliner 95%.">
144144
</p>
145145

146146
| vs no-skill baseline | LOC | tokens | cost | time | safe |
@@ -149,7 +149,7 @@ The honest measurement is a real agent doing real work: a headless Claude Code s
149149
| caveman (terse-prose control) | -20% | +7% | +3% | +2% | 100% |
150150
| "YAGNI + one-liners" prompt | -33% | -14% | -21% | -30% | 95% |
151151

152-
ponytail is the only arm that cuts every metric, and the only one that stays fully safe while doing it. The cut is biggest where there is a real over-build trap (date picker 404 to 23 lines, color picker 287 to 23, because it reaches for a native `<input>` instead of a component) and near zero on code that is already minimal. Full method, per-task tables, and limitations: [benchmarks/results/2026-06-18-agentic.md](benchmarks/results/2026-06-18-agentic.md).
152+
ponytail makes by far the biggest code cut, and it is the only arm that cuts every metric while staying fully safe: the "YAGNI + one-liners" prompt is slightly cheaper and faster, but cuts less code and dropped a path-traversal guard in 1 of 20 safety runs. The cut is biggest where there is a real over-build trap (date picker 404 to 23 lines, color picker 287 to 23, because it reaches for a native `<input>` instead of a component) and near zero on code that is already minimal. Full method, per-task tables, and limitations: [benchmarks/results/2026-06-18-agentic.md](benchmarks/results/2026-06-18-agentic.md).
153153

154154
<details>
155155
<summary><strong>Older single-shot numbers (isolated generation)</strong></summary>

‎benchmarks/results/2026-06-18-agentic.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -173,8 +173,8 @@ Negative is less code / cheaper / faster.
173173

174174
Reading it:
175175

176-
- **ponytail is the only arm that cuts every metric** on the feature tasks, and the only large code
177-
cut (−54%). caveman writes less code but spends *more* tokens (+7%), terse output, same
176+
- **ponytail cuts every metric** on the feature tasks and makes the only large code cut (−54%);
177+
yagni-oneliner also cuts every metric, slightly more on cost and time. caveman writes less code but spends *more* tokens (+7%), terse output, same
178178
deliberation, so it is not cheaper. yagni-oneliner is cheap and fast but cuts less code than
179179
ponytail and is the one arm that dropped a safety guard.
180180
- The **−54% LOC is the across-task aggregate**; per task it runs from ~0% (irreducible backend

0 commit comments

Comments
 (0)