Verify the skills-symlink premise with a canary, not a self-report

The standing item to make `.claude/skills/` available outside this repo rested
on a claim nobody had tested here: that Claude also reads `~/.claude/skills/`.
Untested because that directory has never existed on this machine, so the plan's
foundation was an assumption.

Asking a model whether it can see a file is the weakest available evidence, so:
a canary skill at /tmp/canary-skill/SKILL.md carrying two different random
tokens -- one in the frontmatter `description`, one in the body -- installed the
way the plan proposes, as a symlink
`~/.claude/skills/canary-probe -> /tmp/canary-skill`. `claude -p` run from /tmp,
a directory with no `.claude/skills` of its own, returned both. Two tokens
because they answer different questions: the description token proves the skill
was registered, the body token proves the file was read.

So it holds, and symlinks are followed -- the second half, and the half that
could have failed silently.

The negative control is what makes it evidence. The symlink was removed while
the file was left on disk inside the working directory, and the same prompt
returned NO-CANARY-SKILL. That excludes the confound a positive result alone
leaves open: the file simply sitting near the session's cwd.

Corroborating, and ranked as such rather than as proof: the client binary
carries 14 references to `~/.claude/skills/`, including a `synced` subdirectory
named in help text about a skills-sync setting. String evidence shows the path
is known; only the canary shows it is loaded.

Both artefacts were removed and `~/.claude/skills/` left absent, so todo.md
still describes the machine as it stands.

The symlink itself is still not done and still needs a decision: it writes
outside the repo, and the tradeoff is unchanged -- a symlink points at a path,
so if this repo moves the skills stop working silently, while a copy trades
that for two versions that drift.

Generalised into `evidence-discipline` as "verify a mechanism with a canary,
not with a self-report".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: claude

.claude/skills/evidence-discipline/SKILL.md

@@ -103,6 +103,35 @@ number you write down. It usually costs minutes. Acting on a stale one costs a d
 something that is already there — or, worse, produces a confident write-up of a fix for a problem
 that no longer exists.
 
+## Verify a mechanism with a canary, not with a self-report
+
+When the question is "does this system read that file", asking the system is the weakest available
+evidence: it will answer plausibly either way, and a plausible answer to a question about your own
+plumbing is indistinguishable from a correct one.
+
+Plant a **canary** with a value nothing could produce by inference — a random token — and ask for
+the token back. One project needed to know whether a tool loaded skills from a user-level directory,
+a premise a planned change rested on and which had never been tested because the directory had
+never existed there. The check was a file carrying **two different tokens, one in the metadata and
+one in the body**, because they answer different questions: the metadata token proves the thing was
+*registered*, the body token proves the file was *read*. Both came back.
+
+**Then remove the pointer and leave the file.** That is the control that turns it into evidence. The
+confound is that the file sat inside the working directory, so a positive result alone is also
+consistent with it being found some entirely different way. Deleting only the symlink — file still
+on disk, same prompt — returned the "no such thing" answer, which excludes that and nothing else
+would have.
+
+Where the mechanism has an artefact you can read directly, read it, and rank it honestly. Grepping
+the tool's binary found fourteen references to the path, which establishes that the path is *known*
+to it. It does not establish that anything *loads* from it. String evidence and behavioural evidence
+answer different questions, and the weaker one is worth recording as corroboration rather than
+proof.
+
+Two habits that go with it: make the canary's value random, so a lucky guess is not a pass; and
+**remove the artefacts afterwards and say so**, because a test that leaves a directory behind has
+quietly falsified the note you are about to write about the machine's state.
+
 ## A seed is not a reproduction
 
 Randomised test orderings are seeded, and a seed is only a permutation **of the list of files that

docs/changelog.md

@@ -351,6 +351,7 @@ zero across these commits; several were pre-existing bugs the old markup had bee
 | `166cda001` | **A live Bootstrap button survived the entire migration, because the class audit read `app/views` and never `app/helpers`.** Found while chasing the `responsive-audit` flicker, which is the only reason anyone was looking at `/organization` at 320px: a **14×20** button carrying `btn btn-outline-primary btn-xs` — three classes the stylesheet has not defined since ADR 0011 removed Bootstrap — named only by `title`, with `alt` on a `<button>`, where that attribute means nothing. `undefined-classes.py` had reported **0 orphans throughout**, and helpers building markup is the whole reason. **Widening the glob was not enough, and that is the part worth keeping**: the "is this a deliberate hook?" test greps `*.rb` under `app/`, so once helpers were read the helper's own `class:` counted as a reference to itself and every helper-only token was filed as a hook. That test had always excluded `.erb` for precisely this reason — a view writing `class="foo"` is not evidence that `foo` means anything — and helpers write class attributes in a file type it does read. The scanned files are now subtracted from its matches. **The run in between reported clean and looked like proof**, which is the general lesson: widening what an audit reads can require widening what it discounts. **With both halves in it found four undefined tokens in two helpers.** `reinvite_user_link` is now `essentials_action_button`, measured in the browser before and after: **14×20 → 28×28**, accessible name moved from `title` to `aria-label`, `bi-envelope` and the POST preserved. Three unreferenced `PartnersHelper` methods emitting `label label-teal`, `badge badge-pill bg-primary` and `col-sm-3 col-3` are deleted. **The line between deleting and keeping**: an unused helper that renders design system output is inert and stays on the documented list — `UiHelper#status_label` has no callers after this and stays, because a future caller gets a correct pill — while an unused helper that renders classes nothing defines hands that caller invisible markup. Controls run both ways: a class planted in a helper is now reported, and one planted in a view still is. **`state.rb` carried the same false-clean shape and is fixed too**: it counted the audit's indented output lines, so a crashed audit and a clean one both answered `0`. It reads the printed count and says `n/a (audit produced no count)` when there is none — proven against clean, 2-finding and crashed output. **Rollback #8 landed in the middle of this**, and is why the first negative control appeared to fail: the `grep` that showed the widened script and the `sed` that showed the original were reading either side of it, so the "failure" was never a measurement. Disk had gone back to `548db78f6` with **368 commits of content missing** while `HEAD` stayed put and matched the remote exactly; `bin/workspace-restore` put it back and the two uncommitted edits were redone. 100 examples 0 failures across the helper, request and system specs. |
 | `c855ff0ec` | **The `responsive-audit` flicker, fixed at the fifth attempt — and it had been hiding two real defects.** Findings alternated 8/9, always `/items/inventory`. **A target's own ancestor is not a neighbour**: WCAG 2.5.8's spacing exception excuses an undersized target when a 24px circle on it reaches no *other* target's hit area, and "other" was implemented as `o.el !== t.el` — which is true of the target's own parent, whose box necessarily contains the child's centre. So any control inside a focusable container could never pass. `clipped_text_controller` gives a truncated `<td>` `tabindex="0"` so a keyboard user can reach its tooltip, and on that page the marked cell **wraps** each row's disclosure button: fifty buttons reported, at every width, and whether 320 joined them varied per run. One predicate — `!a.contains(b) && !b.contains(a)` — and three full runs are **byte-identical**. Verified the honest way: the underlying population still swings (`allTargets` 133 or 183 between runs) and the output no longer moves with it, which is better than suppressing the swing because the audit no longer depends on it. Four isolated controls, one of which corrected **my expectation** rather than the code — a focusable *wrapper* that is itself under 24px is a real target and is still reported. **Why four attempts missed it**: they compared counts, then positions. The number that moved was neither the sizes (`undersized` sat at 52 throughout) nor the geometry but the size of the *comparison set*, and nothing printed that. **Then 8 findings became 2, which is where a fix turns into a suppression unless you read what vanished** — every removed line was the same page and the same message, and two genuine defects were underneath. **The button really was 20×28**, a hand-written near-copy of `ROW_ICON_CLASSES` missing `size-7`, against design.md's rule that every actions-column control is 28×28; the audit had said so at every width for as long as it had flickered and it read as more flicker. *A permanently noisy check is a check whose true findings get filed as noise.* **And the clipped-cell scan counted `sr-only` text**: `textContent` includes it, so any cell whose only text was a screen-reader label qualified — **55 cells marked at 320px, 50 holding nothing but "Show storage locations for …"**, each a tab stop raising a bubble repeating a deliberately hidden string over a cell showing a chevron, which is the tab-stop-per-row that controller's own comment exists to prevent, reached from the other side. After: 5 marked, 0 unreadable, button 28×28. Both pinned by `item_system_spec.rb` and **both specs watched failing first** (`got: [20, 28]`; 2 spurious tab stops). 13 + 31 examples 0 failures, audit-selftest 11 controls 0 wrong, erb_lint and rubocop clean. Left open deliberately and written down: that 133/183 swing is the clipped scan reacting to `resize` while stacking labels arrive on a `matchMedia` change — no audit result and no user symptom depends on it now. |
 | `daba48850` | **`responsive-audit` reports zero across the whole app, and the last two findings were the same class of false positive as the flicker.** Both remaining findings were "fixed/sticky chrome covers 186px of a 360px viewport", on `/admin/base_items` and `/admin/partners`. The check sums the vertical bands of everything `fixed` or `sticky`; **a `sticky` element with `top` and `bottom` both `auto` is pinned to a horizontal edge**, and its band scrolls away with the content rather than occluding a fixed strip. The frozen actions column is precisely that — measured on `/admin/partners` at 740×360, eight `td.cell-actions` at `top: auto, bottom: auto, right: 0px` in consecutive bands (216..269, 269..322, 322..375), unioned to 186 of 360 and over the 50% threshold, for a column that eats **no height at all**. Same family as the ancestor bug fixed an hour earlier: a geometric rule reading an element whose geometry does not mean what the rule assumes. **Five controls, all correct**: a right-pinned column over eight rows scores 0, a sticky topbar 64, a fixed bottom bar 56, both together 120 (unioned, not summed), and a topbar beside a pinned column scores only the topbar's 64. Three full runs byte-identical, **0 findings across 155 routes**. **Then the interesting part, which is a correction of my own.** Having removed the column, a four-page spot check showed nothing pinned and I concluded — and wrote into the code and the to-do — that the check had gone inert on this app. So the run now prints what it considered, per the suite's own "count what you examined" rule, and that line **falsified me in one run**: it considers **30 elements across 146 page visits**, identical on three runs, every one of them `.table-rail`, the fixed 24px scroll rail on a wide table, at 24px of 360 — about 7%. Live input, negative verdict. Both claims corrected in place. What is genuinely unexercised is the *reporting* path above 50%, which no real screen reaches; the five controls live nowhere yet because the check is inline rather than exported, and `docs/todo.md` names the extraction. **Two doc corrections in the same commit.** `design-decisions.md` still said `/broadcast_announcements/1/edit` and `/partners/children/new` "still 404 and are still skipped by every audit" — fixed the next day by `3403c72b7`; marked superseded rather than rewritten, because a dated entry is a record and not a status page, and re-verified at 155 routes with an empty not-reached list. And `todo.md`'s skills figures said 17 files/1,383 lines against an actual **18/1,599**. The other two numbers in that file were re-measured and both still hold: 11 of `UiHelper`'s 27 methods live, and 9 users whose `display_name` is the placeholder. |
+| `TBD` | **The skills-symlink premise, verified — it holds, and symlinks are followed.** The standing item to make `.claude/skills/` available outside this repo rested on a claim nobody had tested here: that Claude also reads `~/.claude/skills/`. It had never been tested because that directory has never existed on this machine, so the plan's foundation was an assumption. **Asking a model whether it can see a file is the weakest available evidence**, so it was checked with a canary: a skill at `/tmp/canary-skill/SKILL.md` carrying **two different random tokens**, one in the frontmatter `description` and one in the body, installed exactly as the plan proposes — as a symlink, `~/.claude/skills/canary-probe -> /tmp/canary-skill`. `claude -p` run from `/tmp`, a directory with no `.claude/skills` of its own, returned **both**. Two tokens because they answer different questions: the description token proves *registration*, the body token proves the file was *read*. **The negative control is what makes it evidence** — the symlink was removed while the file was left on disk inside the working directory, and the same prompt returned `NO-CANARY-SKILL`, excluding the confound that a positive result alone leaves open (the file simply being near the session's cwd). Corroborating but insufficient on its own, and ranked as such: the client binary carries **14** references to `~/.claude/skills/`, including a `synced` subdirectory named in help text — string evidence shows the path is *known*, only the canary shows it is *loaded*. Both artefacts were removed and `~/.claude/skills/` left absent, so `todo.md` still describes the machine as it stands. **The symlink itself is still not done** and still needs a decision: it writes outside the repo, and the one real tradeoff is unchanged — a symlink points at a path, so if this repo moves the skills stop working silently, while a copy trades that for two versions that drift. Generalised into `evidence-discipline` as "verify a mechanism with a canary, not with a self-report". |
 
 
 

docs/todo.md

@@ -92,11 +92,34 @@ symlink points at a path, so if this repo moves or is deleted the skills silentl
 copy avoids that and introduces two versions that drift — which is the worse failure, and the one
 this project has spent a week finding in its own documents.
 
-**Still unverified**, and it is the one thing worth checking before relying on it: whether this
-client reads `~/.claude/skills/` on this setup at all. Confirm by running the loop, opening a
-different project, and seeing whether the skills are offered. If they are not, the answer is a copy
-into whatever directory this client does read, and the drift problem comes back and needs a
-different solution.
+**Verified 2026-09-08, and it was the one thing worth checking before relying on it.** The premise
+had never been tested here — the directory has never existed on this machine, so "Claude also reads
+`~/.claude/skills/`" was an assumption the whole plan rested on. It holds, and **symlinks are
+followed**, which was the second half and the half that could have failed silently.
+
+How it was checked, because a self-report from a model is weak evidence on its own. A canary skill
+was written to `/tmp/canary-skill/SKILL.md` carrying **two different random tokens** — one in the
+frontmatter `description`, one in the body — and installed the way the plan proposes, as a symlink:
+`~/.claude/skills/canary-probe -> /tmp/canary-skill`. Then `claude -p` was run from `/tmp`, a
+directory with no `.claude/skills` of its own, asking for both tokens. Both came back. Two tokens
+rather than one because they answer different questions: the description token proves the skill was
+*registered*, the body token proves the file was actually *read*.
+
+**The negative control is what makes it evidence.** The symlink was removed while
+`/tmp/canary-skill/SKILL.md` was left on disk inside the working directory, and the same prompt
+returned `NO-CANARY-SKILL`. So discovery came through `~/.claude/skills/` and not from the file
+happening to sit near the session's cwd — which is exactly the confound a positive result alone
+would not have excluded.
+
+Corroborating, though not sufficient by itself: the client binary
+(`~/.local/share/claude/versions/2.1.250`) contains 14 references to `~/.claude/skills/`, including
+operational ones like a `synced` subdirectory named in help text about a skills-sync setting. String
+evidence shows the path is *known*; only the canary shows it is *loaded*.
+
+Both test artefacts were removed afterwards and `~/.claude/skills/` was left absent, so this entry
+still describes the machine as it stands. **The remaining tradeoff is unchanged** and is the only
+open question: a symlink points at a path, so if this repo moves or is deleted the skills stop
+working silently.
 
 ## `responsive-audit` flicker: closed, and it was hiding two real defects