Proposal: ce-docs-audit — keep a repo's entry-point docs true (README, AGENTS.md, CLAUDE.md)

[eugeniughelbur]

The gap

The plugin compounds knowledge about problems exceptionally well. It has no
skill that maintains the front door — README.md, AGENTS.md, CLAUDE.md —
against the current tree.

I checked the existing skills before writing this, and each is adjacent rather
than overlapping:

Skill	Surface	Why it does not cover this
ce-compound	docs/solutions/	Captures one solved problem. Its Discoverability Check does touch AGENTS.md, but only to add a pointer to the knowledge store — it never checks whether anything already in those files is still true
ce-compound-refresh	docs/solutions/	Refreshes solution docs for staleness. Same idea, different surface
ce-doc-review	A requirements or plan document	A document under construction, reviewed with role lenses. Entry-point docs are shipped artifacts with different failure modes
ce-handoff	Session context	Ephemeral by design, and correctly so. This is about what belongs in the repo permanently
ce-explain	A teaching artifact	Explicitly "for learning, not repo docs or verdicts"

So the gap is real and narrow: nobody audits the docs a newcomer actually reads
first.

Why it matters, with the case that prompted it

Entry-point docs rot differently from other docs, because the people who would
notice already know the answers. The cost lands entirely on the next person.

Yesterday I finished a week-long spike and wrote its README carefully. It contained
this setup step:

sprite file push -s <name> sprite/bootstrap.sh /tmp/bootstrap.sh

The second time I provisioned a machine, that command failed: a fresh Fly Sprite
has no /tmp, and the CLI refuses the path with directory /tmp does not exist
even though ls shows the directory. My own setup instructions were wrong, in a
document I had written two hours earlier, and no existing skill would have caught
it — because it is not a stale reference or a missing section, it is a claim
about the world that only fails when someone follows it.

That is the highest-value check this skill would run, and I have not seen it
anywhere: treat every command in the docs as an assertion, and verify the ones
you safely can.

Proposed shape

Four checks, all reported even when they pass, because a silent pass is
indistinguishable from a skipped check.

1. Executable claims. Extract every command from the entry-point docs. Run the
cheap, safe, side-effect-free ones (--help, --version, --dry-run,
typechecks). For the rest, verify statically — does the referenced file exist, does
the flag appear in --help, does the script name exist in package.json — and say
plainly which could not be verified rather than implying they passed. A failing
command is the most valuable output.

2. State. Do the docs say what is deployed, what is half-done, and what to pick
up first with why it matters? State drifts faster than anything else in a README,
so check it against recent commits, open PRs, and whether the "coming soon" thing
shipped months ago.

3. Load-bearing decisions. Which decisions would a competent newcomer undo,
reasonably, and be wrong? The retry that looks paranoid, the async hop that could
obviously be an await, the function that looks dead. Then check whether the
reason is written where the person about to undo it would look. A reason that
lives only in a commit message is not documented — nobody reads git log before
refactoring. This is the strongest argument for a repo having an AGENTS.md at
all, so proposing one is a legitimate finding.

4. Orientation. Is there a plain first sentence, a stated reading order, and do
the entry-point files point at each other rather than each assuming it is the only
one read? If both AGENTS.md and CLAUDE.md exist, is one a pointer rather than a
second copy that will drift?

Design choices worth arguing about before any code

• Report before editing, and lead with what is wrong rather than what is
  missing. A broken command outranks an absent section. Absence is easy to see
  and easy to fix; the newcomer who hits a failing setup command in the first five
  minutes never reaches the missing section.
• Smallest edit, never a rewrite. The job is a true front door, not a good
  one. A rewrite loses the author's voice, invites review nobody asked for, and
  buries the two lines that mattered.
• Blocking consent before touching entry-point docs, matching
  ce-compound's treatment of instruction files. These are the repo's public face
  and frequently reviewed.
• --report-only for CI, which is where I would expect most of the value over
  time: catch a broken setup command on the PR that broke it.

Relationship to #866

#866
(promote durable lessons onto enforced surfaces) is the closest existing issue and
it is solving a different half of the same problem, so worth naming rather than
letting a maintainer spot the overlap.

• Enhancement: promote durable lessons onto enforced surfaces (close the capture→behavior gap) #866 asks whether a lesson is enforced. Its concern is that a captured lesson
  stays retrieval-dependent - it only changes behaviour if an agent happens to
  read it.
• This asks whether what is written is still true. Its concern is a document
  that is read, confidently, and is wrong.

They compose rather than compete. Check 3 below - are load-bearing decisions
documented where someone about to undo them would look - is the weaker,
verification-side cousin of #866's promotion ladder. If #866 lands, this skill is
what later catches the moment a promoted rule stops matching the tree.

Open questions for maintainers

1. Is this a skill or a phase of an existing one? A case exists for folding it
   into ce-compound-refresh as a second surface — it is the same "verify against
   the tree" motion. I lean separate, because the trigger is different (handing
   over or onboarding, not documenting a learning) and because --report-only in
   CI wants its own entry point. Happy to be wrong.
2. How far should command execution go? Running commands from a document is
   real risk. My instinct is a conservative allowlist — no writes, no network
   mutations, no deploys — and static verification for everything else. There may
   be an existing convention in the repo I should follow instead.
3. Naming. ce-docs-audit reads clearly; ce-docs is terser but sounds like
   it generates docs, which is the opposite of what it does.
4. Overlap to resolve deliberately: ce-compound's Discoverability Check also
   edits AGENTS.md. If both exist, one of them should own instruction-file edits.
   I would give it to this skill and have ce-compound keep only its pointer
   insertion, but that is a maintainer call.

I ran it, and it caught this

Not a design sketch. I built the local version, then pointed it at the same
repository whose docs I had written the previous day, and it found four false
commands out of nine.

Executable claims: 9 checked, 4 fail

FAIL  sprite file push -s <name> sprite/bootstrap.sh /tmp/bootstrap.sh
FAIL  sprite file push -s <name> sprite/agent-server.mjs /tmp/agent-server.mjs
FAIL  sprite exec -s <name> --env GITHUB_TOKEN=... -- bash /tmp/bootstrap.sh
      Error: directory /tmp does not exist

FAIL  cd worker && npm install && npx wrangler deploy
      Wrangler requires at least Node.js v22.0.0. You are using v20.20.0.

The other three checks passed - the state section was current, all five
load-bearing decisions were documented with the cost of reverting them, and the
reading order was clear. Thirteen of thirteen referenced files and doc links
resolved.

The cause is the argument for the skill. AGENTS.md in that same repository
recorded the /tmp trap correctly, on the same day, in a section literally titled
"three things that will waste your time otherwise". The README still told you to do
the broken thing. Two documents, one truth, and nothing keeping them agreed.

The Node 22 failure has the same shape: the constraint was written in one file and
the command that violates it was in the other.

Neither is a stale link, a missing section, or a dead reference - the classes of
problem existing tooling finds. Both are commands that only fail when a newcomer
runs them, which is exactly when it is most expensive.

What I have already

A working local version, used on two repositories, and the concrete case above. If
this is worth having I am happy to do the work to the repo's standards — tests
under tests/, a doc under docs/skills/, the headless/lightweight mode split,
and the validate-doc-claims.py mechanical check reused rather than reimplemented.

If the answer is "fold it into ce-compound-refresh" or "not for this plugin",
that is a useful answer too and I would rather hear it before writing anything.

[eugeniughelbur]

The working implementation

Adding this as a comment rather than a PR, because AGENTS.md asks for maintainer approval on a new skill before the work starts. This is what I have been running locally, so there is something concrete to judge rather than a description of one.

It is deliberately a single file with no scripts. Two of the four checks reuse machinery this repo already has - validate-doc-claims.py for the mechanical path and link checking, and the same blocking-question pattern ce-compound uses before it edits an instruction file - so a real version would call those rather than reimplement them.

Named docs-audit locally to stay out of the ce- namespace until there is a decision.

SKILL.md (155 lines)

---
name: docs-audit
description: "Audit and update a repository's entry-point documentation - README, AGENTS.md, CLAUDE.md - against the current tree, so someone picking the project up cold knows what it is, what state it is in, and which decisions not to undo. Use when handing a project over, when onboarding someone, when the README has drifted, or when a repo has no AGENTS.md. Not for capturing a solved problem (that is ce-compound), reviewing a plan (ce-doc-review), or session continuity (ce-handoff)."
argument-hint: "[path to repo, or blank for the current one] [--report-only]"
---

# Docs Audit

Make the front door of a repository true.

Entry-point docs rot differently from other docs. Nobody notices, because the
people who would notice already know the answers. The cost lands entirely on the
next person, who follows a setup command that no longer works and concludes the
project is broken.

This audits four things a newcomer needs and a maintainer cannot see, then
proposes the smallest edit that fixes them.

## What this is not

- **`ce-compound`** captures one solved problem into `docs/solutions/`. Use it for
  a learning. This one maintains the front door.
- **`ce-compound-refresh`** refreshes those solution docs for staleness. Same idea,
  different surface.
- **`ce-doc-review`** reviews a requirements or plan document with role lenses.
  That is a document under construction; this is a document that ships with the
  repo.
- **`ce-handoff`** preserves *session* context for a fresh agent. Ephemeral by
  design. This writes what belongs in the repo permanently.

If the user wants any of those, say so and stop.

## The four checks

Run all four. Report each one even when it passes, because a silent pass is
indistinguishable from a skipped check.

### 1. Executable claims

**Every command in the entry-point docs is a claim about the world, and it is the
one most likely to be false.** A path that moved, a flag that changed, a step that
only works because your shell has something the newcomer's does not.

Extract every command from fenced blocks and inline code in the README and
`AGENTS.md`. For each one, decide:

- **Verifiable cheaply and safely** - a `ls`, a `--help`, a `--version`, a
  `--dry-run`, a typecheck. Run it and record the result.
- **Not safely runnable** - it deploys, writes, spends money, or needs a
  credential. Do not run it. Instead check what you can statically: does the file
  it references exist, does the flag appear in `--help`, does the script name
  match something in `package.json`.
- **Unrunnable and unverifiable** - say so in the report rather than implying it
  passed.

A command that fails is the highest-value finding this skill produces. Quote the
actual error.

### 2. State

A newcomer's first question is not "what is this" but "where is this up to".
Assess whether the docs answer:

- What is deployed or running right now, and where.
- What is finished versus half-done.
- What to pick up first, and **why it matters** - not just a task list. "Egress
  control" means nothing; "the hard blocker before anything client-facing" tells
  someone where it sits.

State drifts fastest of anything in a README, so check it against the tree: recent
commits, open PRs, TODO markers, whether the "coming soon" thing shipped six
months ago.

### 3. Load-bearing decisions

**Which decisions in this repo would a competent newcomer undo, reasonably, and be
wrong?**

These are the ones that look like accidental complexity and are not: a retry that
seems paranoid, an async hop that could obviously be an `await`, a duplicated
constant, a function that looks dead. Find them by reading recent commit messages
and any `docs/solutions/`, then check whether the reason is written anywhere the
person about to undo it would look.

If it is only in a commit message, it is not documented. Nobody reads git log
before refactoring.

This check is why `AGENTS.md` exists, so if the repo has none and has at least one
such decision, proposing one is the finding.

### 4. Orientation

Does a newcomer know where to start? Specifically:

- Is there a first sentence that says what this is, without jargon internal to it?
- Is there a stated reading order, or just a pile of files?
- Do the entry-point files point at each other, or does each assume it is the only
  one being read?
- If `AGENTS.md` and `CLAUDE.md` both exist, is one a pointer to the other rather
  than a second copy that will go stale?

## How to run it

1. **Inventory.** Find the entry-point docs: `README*`, `AGENTS.md`, `CLAUDE.md`,
   `CONTRIBUTING*`, and any `docs/` index. Read them fully. Do not skim - a claim
   you did not read is a claim you cannot check.
2. **Ground yourself in the tree.** `git log --oneline -30`, the directory layout,
   `package.json` scripts or equivalent. You are looking for the gap between what
   the docs say and what is there.
3. **Run the four checks.** Verify commands as described in check 1.
4. **Report before editing.** Lead with what is *wrong*, not what is missing -
   a broken command outranks an absent section every time.
5. **Propose the smallest edit.** A line in an existing section beats a new
   section. Match the file's existing voice and density; a repo with a terse
   README should not suddenly acquire three headed sections.
6. **Ask before writing**, using the platform's blocking question tool
   (`AskUserQuestion` in Claude Code - call `ToolSearch` with
   `select:AskUserQuestion` first if the schema is not loaded). Entry-point docs
   are the repo's public face and often reviewed; do not edit them unasked. With
   `--report-only`, never edit.

## Output

Docs audit -

Executable claims: <N checked, M failed>
FAIL

<or: all N verified, or: K unverifiable - >

State: <current | drifted | absent>

Load-bearing: <documented | N undocumented>
<the decision, and where someone would look for its reason>

Orientation: <clear | >

Proposed edits:
- <the smallest change, and where>

<then the blocking question, unless --report-only>

## Two things to get right

**Report the broken command first, always.** The instinct is to lead with the
missing section, because absence is easy to see and easy to fix. A newcomer who
hits a failing setup command in the first five minutes never reaches the missing
section.

**Do not rewrite the docs.** The job is a true front door, not a good one. A
rewrite loses the author's voice, invites review the user did not ask for, and
buries the two lines that actually mattered.

What I would change before it were merge-worthy

Being explicit about the gap between this and something that meets the repo's standards, since the issue asks a design question rather than offering finished work:

1. Headless and lightweight modes. Not implemented. The --report-only flag is the only concession to non-interactive use, and CI is where I expect most of the long-term value, so this needs the same mode split every other skill here has.
2. Command execution needs a real policy, not an instinct. The skill currently tells the agent to judge what is safe to run. That is too loose for something shipping to other people's repositories. It wants an explicit allowlist and a documented refusal set, and that is the open question I would most like a maintainer's view on.
3. Tests. tests/ has contract and snapshot tests for other skills; this has none.
4. The overlap with ce-compound's Discoverability Check should be resolved in code, not left to whichever skill runs last.

Happy to do all four to the repo's standards if this is wanted, or to drop it if the answer is that it belongs inside ce-compound-refresh as a second surface. I would rather hear that now than after writing the tests.