You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 57458ea
Browse filesBrowse the repository at this point in the historyBrowse files
authored and
GeanCartier
committed
docs: align README benchmark claims with the agentic results
- 'only arm that cuts every metric' contradicts the table: the YAGNI
one-liner prompt also cuts all four and is slightly cheaper and faster;
state what does set ponytail apart (biggest code cut, fully safe)
- qualify the headline: 94% is one task, safety is 20/20 adversarial runs
- fix the chart alt text the same way
- drop the hook count for Codex (the shared hook file defines three)
- list Kimi Code and Goose among hosts with commands, per INSTALL.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,7 +30,7 @@
30
30
</p>
31
31
32
32
<palign="center">
33
-
<strong>~54% less code (up to 94%) ·~20% cheaper ·~27% faster ·100% safe</strong><br>
33
+
<strong>~54% less code (up to 94% on one task) ·~20% cheaper ·~27% faster · safe in 20/20 adversarial runs</strong><br>
34
34
<sub>Real Claude Code sessions editing a real FastAPI + React repo, the same agent with and without the skill (12 feature tasks, Haiku 4.5, n=4). <ahref="#numbers">Details</a>.</sub>
Then open `/hooks` in Codex, trust its two lifecycle hooks, and start a new thread.
84
+
Then open `/hooks` in Codex, trust its lifecycle hooks, and start a new thread.
85
85
86
86
**Any other agent:** copy [`AGENTS.md`](AGENTS.md) into your project, or ask your agent to install [`skills/ponytail/SKILL.md`](skills/ponytail/SKILL.md) as a skill. Step by step for Copilot, Cursor, OpenCode, Gemini and the rest: **[INSTALL.md](INSTALL.md)**.
87
87
@@ -133,14 +133,14 @@ Lazy, not negligent: trust-boundary validation, data-loss handling, security, an
133
133
|`/ponytail-gain`| Show the measured impact scoreboard (less code, less cost, more speed) from the benchmark. |
134
134
|`/ponytail-help`| Quick reference for the commands above. |
135
135
136
-
Commands need a skill-capable host (Claude Code, Codex, Devin CLI, OpenCode, Gemini, pi, Hermes Agent, Qoder, Grok Build). In Codex CLI and the IDE extension they're skills under the plugin's namespace; invoke with `$ponytail:ponytail-review`. Cursor with the [hooks](INSTALL.md#cursor) gets `/ponytail` level switching only, typed as a plain message. The instruction-only adapters (Cursor's rule file, Windsurf, Cline, Copilot, Kiro, Antigravity) load the always-on ruleset without the commands.
136
+
Commands need a skill-capable host (Claude Code, Codex, Devin CLI, OpenCode, Gemini, pi, Hermes Agent, Qoder, Grok Build, Kimi Code, Goose). In Codex CLI and the IDE extension they're skills under the plugin's namespace; invoke with `$ponytail:ponytail-review`. Cursor with the [hooks](INSTALL.md#cursor) gets `/ponytail` level switching only, typed as a plain message. The instruction-only adapters (Cursor's rule file, Windsurf, Cline, Copilot, Kiro, Antigravity) load the always-on ruleset without the commands.
137
137
138
138
## Numbers
139
139
140
140
The honest measurement is a real agent doing real work: a headless Claude Code session editing [tiangolo's full-stack-fastapi-template](https://github.com/fastapi/full-stack-fastapi-template) (a real FastAPI + React repo), scored on the `git diff` it leaves behind. Twelve feature tickets, the same agent with and without the skill, n=4, Haiku 4.5.
141
141
142
142
<palign="center">
143
-
<imgsrc="assets/benchmark-agentic.svg"width="860"alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on every metric (LOC 46%, tokens 78%, cost 80%, time 73%); caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%. Safety, separate adversarial tier: baseline, caveman and ponytail 100%, yagni-oneliner 95%.">
143
+
<imgsrc="assets/benchmark-agentic.svg"width="860"alt="Each arm as a percent of the no-skill baseline across LOC, tokens, cost and time (Haiku 4.5). ponytail is lowest on LOC and tokens (LOC 46%, tokens 78%, cost 80%, time 73%), the YAGNI one-liner prompt slightly lower on cost and time; caveman rises above 100% on tokens, cost and time; yagni-oneliner LOC 67%. Safety, separate adversarial tier: baseline, caveman and ponytail 100%, yagni-oneliner 95%.">
144
144
</p>
145
145
146
146
| vs no-skill baseline | LOC | tokens | cost | time | safe |
@@ -149,7 +149,7 @@ The honest measurement is a real agent doing real work: a headless Claude Code s
ponytail is the only arm that cuts every metric, and the only one that stays fully safe while doing it. The cut is biggest where there is a real over-build trap (date picker 404 to 23 lines, color picker 287 to 23, because it reaches for a native `<input>` instead of a component) and near zero on code that is already minimal. Full method, per-task tables, and limitations: [benchmarks/results/2026-06-18-agentic.md](benchmarks/results/2026-06-18-agentic.md).
152
+
ponytail makes by far the biggest code cut, and it is the only arm that cuts every metric while staying fully safe: the "YAGNI + one-liners" prompt is slightly cheaper and faster, but cuts less code and dropped a path-traversal guard in 1 of 20 safety runs. The cut is biggest where there is a real over-build trap (date picker 404 to 23 lines, color picker 287 to 23, because it reaches for a native `<input>` instead of a component) and near zero on code that is already minimal. Full method, per-task tables, and limitations: [benchmarks/results/2026-06-18-agentic.md](benchmarks/results/2026-06-18-agentic.md).
0 commit comments