chore: sync catalog counts and sharpen stocktake verdicts (#18)

User-facing copy still advertised 59 skills (43 model-invoked), the
v1.8.0 numbers, while the tree and vendor policy have been 62 (46
model, 16 manual) since 1.8.1. Reconcile from the measured tree in
README, docs/skills.md, and the architecture diagram. Historical
changelog entries keep their original numbers.

skill-stocktake becomes a verdict auditor: every item resolves to
exactly one of KEEP, COMPRESS, UPDATE, MERGE -> <target>, or RETIRE,
gated by an existence pass, a currency pass, and cited evidence. It
reports only; it does not rewrite bodies or delete files.

eval-harness gains an A/B utility gate (references/skill-utility.md).
Run the same dataset without the skill and with it; quality parity at
lower token or latency cost passes, and A ~= B on the skill's own
claimed task is evidence for RETIRE or COMPRESS. Deterministic
assertions first, no external harness, no API key.

tests/test_skills.py now reconciles policy against the actual skills/,
manual-skills/, and commands/ trees instead of trusting declared
counts, plus frontmatter conformance, model/manual collision, a
lexical description-overlap gate (warn 50%, fail 75% Jaccard), and a
~/.claude runtime path assertion. Each gate was verified red-capable
by temporary mutation.

License authority stays vendor/license-audit.json, which states that a
missing frontmatter license is not a grant and already maps all 62
skills to MIT or Apache-2.0 with evidence. The test asserts against
that file and only forbids frontmatter from contradicting it, rather
than forcing a license key into 28 SKILL.md files.

Warehouse: DEFER no longer promotes to NEW without an existence pass
and a no-skill baseline. Capability drift justifies COMPRESS/RETIRE,
never a twin skill. Wave 1-3 counts unchanged.

No new skills, no new MCP, no new dependencies. VERSION stays 1.8.2.

Author: kuker24

CHANGELOG.md

@@ -2,7 +2,14 @@
 
 ## Unreleased
 
-None.
+Catalog hygiene patch. No new skills, no new MCP, no foreign harness runtime. Totals stay **62** (46 model-invoked, 16 manual slash commands).
+
+- Sync stale user-facing catalog copy (`59` / `43 model-invoked`) to the measured tree in `README.md`, `docs/skills.md`, and `docs/architecture.md`. Historical release entries keep their original numbers.
+- `skill-stocktake`: replace the schema-only sweep with a verdict protocol. Every catalog item resolves to exactly one of `KEEP`, `COMPRESS`, `UPDATE`, `MERGE → <target>`, or `RETIRE`, backed by an existence pass, a currency pass, and cited evidence. The skill never rewrites or deletes another skill; `COMPRESS`/`UPDATE` hand off to `writing-for-agents` or `prompt-optimizer`.
+- `eval-harness`: add `references/skill-utility.md`, an A/B utility gate that scores a task with the skill (Run B) against the same task without it (Run A). Quality parity with lower token or latency passes; `A ≈ B` on the skill's own claimed task is evidence for `RETIRE` or `COMPRESS`. Deterministic assertions first, no external harness, no API key.
+- `tests/test_skills.py`: reconcile `vendor/skill-policy.json` against the actual `skills/`, `manual-skills/`, and `commands/` trees instead of trusting declared counts alone. Add frontmatter conformance (`name`, `description`, `compatibility: opencode`, `license`), model/manual name collision detection, a lexical description-overlap gate (warn at 50%, fail at 75% Jaccard), and a `~/.claude` runtime path assertion. Network-free and key-free.
+- `docs/warehouse-inventory.md`: `DEFER` no longer promotes to `NEW` without an existence pass and a no-skill baseline. Capability drift is grounds for `COMPRESS`/`RETIRE`, never a twin skill. Wave 1–3 counts are unchanged.
+- Document NVIDIA SkillEvaluator as `FOREIGN_ON_DEMAND`: never installed, absent is not a doctor failure, and a malformed MCP entry still fails closed.
 
 ## 1.8.2 — 2026-09-10
 

README.md

@@ -1,14 +1,14 @@
 # OpenCodeBestFriend
 
 Production-ready capability layer for OpenCode:
-59 routed skills (core + Wave 2/3 warehouse specialists), MCP, Codebase Memory,
+62 routed skills (core + Wave 2/3 warehouse specialists), MCP, Codebase Memory,
 Design Bank, Design Intelligence, SmartDoc, browser and verification tooling.
 
 OpenCodeBestFriend is an installer and runtime overlay for [OpenCode](https://opencode.ai). It is **not** Claude Code, **not** a model provider, and **not** a dump of a developer home directory.
 
 ## What it is
 
-- 59 skills: 43 model-invoked, 16 manual slash commands
+- 62 skills: 46 model-invoked, 16 manual slash commands
 - A thin `AGENTS.md` router (lazy, one primary specialist)
 - Core MCP: Codebase Memory, Context7, shadcn
 - Design Bank discovery or download (media is **not** in git)
@@ -61,7 +61,7 @@ Restart OpenCode after install. Config is not hot-reloaded.
         ┌───────────────────┼────────────────────┐
         ▼                   ▼                    ▼
       Skills               MCP                 Rules
-     43 automatic       Codebase Memory        Verification
+     46 automatic       Codebase Memory        Verification
      16 manual          Context7              Engineering
                        shadcn
         │
@@ -127,6 +127,8 @@ Optional:
 - `serena` — host binary may exist; MCP is **not** registered unless you run `opencode-bf serena enable`
 - `exa` — `FOREIGN_ON_DEMAND`; installer never adds, removes, or overwrites it
 
+NVIDIA SkillEvaluator is `FOREIGN_ON_DEMAND` in the same sense: a maintainer may run it externally for embedding-based overlap scoring or live catalog evaluation. The installer never adds it, `doctor` does not fail when it is absent, and a malformed MCP entry for it fails closed like any other schema violation.
+
 The installer merges only owned MCP keys. Provider, model, permissions, plugins, and foreign MCP stay yours.
 
 ## Design Bank

docs/architecture.md

@@ -10,7 +10,7 @@
         ┌───────────────────┼────────────────────┐
         ▼                   ▼                    ▼
       Skills               MCP                 Rules
-     43 automatic       Codebase Memory        Verification
+     46 automatic       Codebase Memory        Verification
      16 manual          Context7              Engineering
                        shadcn
         │

docs/skills.md

@@ -2,7 +2,7 @@
 
 Policy: `vendor/skill-policy.json` plus `vendor/skill-allowlist.txt`.
 
-- 43 model-invoked skills live under `skills/` and install to `~/.config/opencode/skills/` (core + Wave 2/3 warehouse specialists)
+- 46 model-invoked skills live under `skills/` and install to `~/.config/opencode/skills/` (core + Wave 2/3 warehouse specialists)
 - 16 manual skills live under `manual-skills/` and install to `~/.config/opencode/bestfriend/skills/` plus `commands/`
 
 `smartdoc` is per-job document intelligence. `smartbook-ingest` compiles reusable local knowledge. `humanizer` cleans user-facing prose tells (`/unslop` is its manual alias). `academic` manages scholarly research, writing, and peer review. `hyperframes` handles deterministic HTML-to-MP4 video composition. `diagram-design` crafts editorial HTML/SVG diagrams. Warehouse diagnostics include `agent-architecture-audit` (agent stack layers), `cost-aware-llm-pipeline` (token budgeting), `eval-harness` (benchmarks), `prompt-optimizer` (prompt refinement), and `skill-stocktake` (catalog hygiene). Wave 3 adds `api-design`, `contract-first`, `automation-audit-ops`, `code-tour`, and `click-path-audit`. Handwriting is a SmartDoc renderer, not a skill.

docs/warehouse-inventory.md

@@ -12,6 +12,8 @@ This inventory establishes the contract for warehouse skills across the five ana
 - Batch `2a`: Rows marked `decision=NEW` with `batch=2a` were ported in Wave 2.
 - Batch `3a`: Only remaining `DEFER` rows marked `decision=NEW` with `batch=3a` are ported in Wave 3. Do not reopen `REJECT`. Do not re-port `DONE`/`MERGE`.
 - External harness runtimes, auto-mutations, and CC-BY-NC text are strictly REJECTED.
+- `DEFER` does not promote to `NEW` without an existence pass and a no-skill baseline. Route the existence pass through `skill-stocktake` and the baseline through the `eval-harness` utility gate. A procedure that a current model already performs unprompted stays `DEFER` or becomes `REJECT`.
+- Capability drift is a legitimate reason to `COMPRESS` or `RETIRE` an existing specialist. It is never a reason to add a twin skill beside it.
 
 ## Summary Counts
 

skills/eval-harness/SKILL.md

@@ -20,6 +20,7 @@ This skill establishes objective quality gates using Eval-Driven Development (ED
 | Application build verification profiles (FAST, STANDARD, RELEASE) | `rules/01-verification.md` |
 | Deep architectural failure diagnosis across agent layers | `agent-architecture-audit` |
 | Auditing cost tradeoffs across model tiers | `cost-aware-llm-pipeline` |
+| Catalog verdicts and skill retirement decisions | `skill-stocktake` |
 | **Designing benchmarks, evaluation datasets, rubrics, and regression suites** | **`eval-harness`** |
 
 ## Core Methodology
@@ -42,3 +43,8 @@ Consult [references/methodology.md](references/methodology.md) for detailed scor
    - Establish a baseline score on the current stable prompt or agent setup.
    - Run the eval suite on proposed changes and calculate the score delta.
    - Reject any prompt change that introduces regressions on previously passing test cases.
+
+5. **Skill Utility Gate (A/B):**
+   - Consult [references/skill-utility.md](references/skill-utility.md) when the question is whether a skill beats its own absence.
+   - Run the same dataset without the skill (Run A) and with it (Run B); quality parity at lower token or latency cost still passes.
+   - `A ≈ B` on the skill's claimed task is evidence for `RETIRE` or `COMPRESS`; route that verdict to `skill-stocktake`.

skills/eval-harness/references/skill-utility.md

@@ -0,0 +1,60 @@
+# Skill Utility Gate (A/B)
+
+Decide whether a skill earns its always-loaded description cost. This measures the skill against its own absence, not against a competitor.
+
+Runs locally. No network, no API key, no external harness runtime, no container image.
+
+## 1. Scope the Claim
+
+Write the claim in one sentence, taken from the skill's own description: what job does it say it wins?
+
+The gate only tests that claim. A skill that wins some unrelated task still fails if it does not beat no-skill on its own claimed job.
+
+## 2. Build the Dataset
+
+Minimum three bands, small on purpose:
+
+- **Happy path** — the canonical case the description promises.
+- **Boundary** — same domain, awkward input: missing field, ambiguous intent, conflicting constraint.
+- **Adjacent-negative** — a task the description explicitly excludes. Correct behavior is declining or handing off.
+
+Five to ten cases per band is enough to expose a dead skill. Record inputs and expected output properties before running anything.
+
+## 3. Run Both Arms
+
+- **Run A (baseline):** the task with the skill absent from context.
+- **Run B (treatment):** the identical task with the skill loaded.
+
+Same model, same temperature, same tooling, same ordering. Change one variable only.
+
+## 4. Score
+
+Deterministic assertions first, per the tiers in [methodology.md](methodology.md). Reach for a model-graded rubric only when the property cannot be checked mechanically.
+
+Record per arm:
+
+- Quality score against the rubric
+- `pass@1`, and `pass@k` when the workflow is stochastic
+- Total tokens, prompt and completion separately
+- Wall-clock latency
+
+Keep `pass@1` and `pass@k` as named metrics. Do not collapse them into a generic 1-5 rating.
+
+## 5. Verdict Thresholds
+
+| Result | Reading |
+|---|---|
+| B beats A on quality | Utility pass |
+| Quality equal, B uses fewer tokens or less latency | Utility pass |
+| Quality equal, B costs more | Utility fail; the body is overhead |
+| `A ≈ B` on the skill's own claimed task | Evidence for `RETIRE` |
+| B wins only on a subset, and the rest of the body is inert | Evidence for `COMPRESS` |
+| B fails the adjacent-negative band where A passed | Fence is too broad; fix the description before re-testing |
+
+Quality parity means identical verdicts on every gold case, not a close average.
+
+## 6. Hand Off
+
+Report the numbers and route the decision. `RETIRE` and `COMPRESS` proposals go to `skill-stocktake`, which owns the verdict table and the catalog view. This gate supplies the baseline; it does not delete skills, edit `SKILL.md`, or change the allowlist.
+
+Project unit tests stay with `tdd`. Browser flows stay with `playwright-qa`. Security findings stay with `full-audit-keamanan`.

skills/skill-stocktake/SKILL.md

@@ -17,32 +17,66 @@ This skill inspects installed skills (`~/.config/opencode/skills/`), manual comm
 |---|---|
 | Reviewing application source code quality or standards | `matt-code-review` |
 | Auditing code security, secrets, permissions, or supply chain | `full-audit-keamanan` |
-| Authoring new SKILL.md files or adapting new skills | `writing-for-agents` |
+| Authoring or rewriting SKILL.md bodies and descriptions | `writing-for-agents` |
 | Optimizing prompt wording and instructional clarity | `prompt-optimizer` |
+| Measuring whether a skill actually beats no-skill | `eval-harness` |
 | **Auditing skill catalog hygiene, boundaries, and schema conformance** | **`skill-stocktake`** |
 
+This skill audits and reports. It does not rewrite another skill's body, and it does not delete files.
+
+## Verdict Protocol
+
+Every audited skill or command resolves to exactly one verdict. No item is left unjudged, and no item carries two verdicts.
+
+| Verdict | Meaning |
+|---|---|
+| `KEEP` | Unique trigger, current artifacts, defensible cost asymmetry, existence pass clears |
+| `COMPRESS` | Still needed, but the body is longer than the judgment it carries. Scripts and verification gates survive; prose shrinks |
+| `UPDATE` | Artifacts, versions, paths, or flags are stale. Requires cited evidence |
+| `MERGE → <target>` | Overlaps a sibling. Name the single target and the residue that moves |
+| `RETIRE` | Existence pass fails, or the no-skill baseline matches the with-skill run, or the trigger was always covered by runtime, rules, Context7, or the model itself |
+
 ## Audit Methodology
 
-Consult [references/checklist.md](references/checklist.md) for the 10-point inspection protocol:
+Consult [references/checklist.md](references/checklist.md) for the full inspection protocol.
+
+1. **Existence pass (ask explicitly, per item):**
+   - If this file disappeared, would the user lose a job that no other skill, rule, core MCP, or cheap on-demand generation already covers?
+   - Does an independent trigger plus reusable judgment justify the selection cost, drift risk, and maintenance burden?
+   - A skill that only restates what a current model does unprompted fails this pass.
 
-1. **Frontmatter Schema Validation:**
+2. **Currency pass:**
+   - Paths resolve under `~/.config/opencode/...`, not foreign agent-host directories.
+   - Flag references to foreign platforms and stale CLI flags, versions, or pinned artifacts.
+   - Preserve greppable terms, self-enforcing prohibitions, and numeric thresholds verbatim. Do not abstract them into softer prose.
+
+3. **Frontmatter schema validation:**
    - Mandatory keys: `name`, `description`, `compatibility: opencode`, `license`.
-   - Description length and trigger specificity: description must define unambiguous triggers and explicit negative boundaries ("Use when... Not for...").
+   - `name` matches the directory name.
+   - Description carries unambiguous positive triggers and explicit negative boundaries ("Use when... Not for...").
+
+4. **Trigger and routing discipline:**
+   - Detect overlapping or competing trigger phrases across skills.
+   - Verify model-invoked skills do not shadow manual slash commands.
+   - Confirm each high-traffic intent has one owner.
+
+5. **Provenance and policy parity:**
+   - Non-first-party skills carry a compliant `NOTICE.md` under MIT or Apache-2.0. Unknown license is not a grant.
+   - Name is registered in `vendor/skill-policy.json` and listed alphabetically in `vendor/skill-allowlist.txt`.
+   - `tests/test_skills.py` assertions match the measured tree, not a remembered count.
+
+## Evidence Requirement
+
+`UPDATE`, `MERGE`, and `RETIRE` are not opinions. Cite at least one of: a path listing, a `--help` or version probe, a checksum, an upstream doc reference, or a no-skill baseline from `eval-harness`. An unproven claim downgrades to `KEEP` with a follow-up note.
+
+## Output
 
-2. **Trigger & Routing Discipline:**
-   - Detect overlapping or competing trigger phrases across different skills.
-   - Verify that model-invoked skills do not shadow manual slash commands.
-   - Ensure high-traffic intents have distinct single-specialist ownership.
+Report a single table, one row per audited item, plus a short list of follow-ups.
 
-3. **Path & Environment Hygiene:**
-   - Verify all path references target OpenCode paths (`~/.config/opencode/skills/...`).
-   - Flag stale references to foreign platforms (Claude Code, Cursor, Codex).
+```text
+| Skill | Verdict | Evidence | Handoff |
+```
 
-4. **Provenance & Attribution:**
-   - Confirm every non-first-party skill contains a compliant `NOTICE.md`.
-   - Ensure licenses match approved policies (MIT or Apache-2.0).
-   - Verify zero uncredited third-party verbatim copying.
+Handoffs: `COMPRESS` and `UPDATE` go to `writing-for-agents` (structure, description, pointers) or `prompt-optimizer` (instruction phrasing). Security findings go to `full-audit-keamanan`. Application code style goes to `matt-code-review`. Utility measurement goes to `eval-harness`.
 
-5. **Test & Policy Parity:**
-   - Confirm the skill name is registered in `vendor/skill-policy.json` and `vendor/skill-allowlist.txt`.
-   - Verify that test assertions in `tests/test_skills.py` reflect the current catalog count.
+Never auto-delete a file, never auto-edit another skill's body, and never mutate a catalog from a learning log.