World-class freshness pass for prompts, docs, evals, and skill control plane

.github/workflows/publish.yml

@@ -8,9 +8,14 @@ permissions:
   contents: read
   id-token: write
 
+concurrency:
+  group: publish-${{ github.event.release.tag_name }}
+  cancel-in-progress: false
+
 jobs:
   publish-npm:
     runs-on: ubuntu-latest
+    timeout-minutes: 30
     steps:
       - uses: actions/checkout@v7
         with:
@@ -52,9 +57,21 @@ jobs:
       - name: Install dependencies
         run: pnpm install --frozen-lockfile
 
+      - name: Verify generated runtime assets
+        run: pnpm check:generated
+
       - name: Run tests
         run: pnpm test
 
+      - name: Run benchmark quality gate
+        run: pnpm benchmark:gate
+
+      - name: Run runtime preflight
+        run: node scripts/run-bug-hunter.cjs preflight --skill-dir .
+
+      - name: Inspect package inventory
+        run: pnpm verify:package
+
       - name: Verify source provenance
         run: node scripts/prepublish-guard.cjs
 

CHANGELOG.md

@@ -7,18 +7,101 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+The current source package version is `3.2.0`. The public `v3.2.0` GitHub
+release/tag has not yet been published, so npm/GitHub release availability may
+lag the source tree. Do not infer publication from `package.json` alone.
+
 ### Added
 
-- Factory Droid CLI support: `install --agent droid` and `doctor --agent droid`
-  target `~/.factory/skills/bug-hunter`, and the docs cover Droid's
-  project-scoped `.factory/skills` and legacy `~/.agents/skills` locations.
+- Measurable hidden-label benchmark scoring for precision, recall, F1,
+  severity-weighted recall, false positives per KLOC, calibration, repeat
+  stability, token/cost efficiency, latency, and time to first true positive.
+- Permanent benchmark quality gate and `pnpm quality:world-class` repository
+  validation across generated assets, tests, benchmark thresholds, preflight,
+  and package inventory.
+- Adaptive `fast`, `balanced`, and `assurance` execution profiles with `auto`
+  selection from deterministic risk and available benchmark evidence.
+- Hypothesis-driven retrieval planning with mandatory evidence plus bounded
+  symbol, dependency, dependent, cross-reference, and trust-boundary context.
+- Exact content-addressed evidence caching keyed to protocol, role, relevant
+  configuration, hypothesis identity, and current source hashes.
+- Bounded hybrid verification for tests, type checks, static checks, builds,
+  reproduction commands, fuzzing, and security-static checks.
+- Canonical `adaptive-plan.json`, `retrieval-plan.json`,
+  `verification-report.json`, and `benchmark-report.json` schemas/artifacts.
+- Precision and measurable protocol documentation under
+  `docs/precision-protocol.md` and `docs/world-class-protocol.md`.
+- Factory Droid CLI support through the `droid` install/doctor target, including
+  personal, project-scoped, and legacy shared skill locations.
+- Protocol-aligned evaluation coverage for adaptive routing, retrieval budgets,
+  exact-cache invalidation, required verification, source mutation, worker
+  scope, truthful coverage, Fixer scope, and benchmark degradation.
+- Freshness regression tests that reject stale task prompts, historical artifact
+  names, old loop-default claims, and missing measurable-protocol guidance.
 
 ### Changed
 
-- Skill description, preflight directory probe, and orchestration backend notes
-  list Factory Droid CLI alongside the other supported coding agents.
+- `SKILL.md` is now a compact control-plane entrypoint: scan-only and
+  single-pass remain explicit defaults, loop coverage is opt-in, calibration
+  examples load progressively, and detailed protocol/mode guidance is loaded on
+  demand instead of duplicated into every context.
+- Triage/indexing use the same maintained source classifier and preserve
+  risk-prioritized order through state, delta selection, and expansion.
+- Chunk sizing uses the combined estimated tokens of actual assigned files;
+  oversized single files are isolated rather than hidden in oversized mixed
+  chunks.
+- Hunter/Skeptic calibration examples are conditional rather than mandatory
+  context for every assignment.
+- Agent metadata, `llms.txt`, `llms-full.txt`, dispatch wrapper, onboarding,
+  installation, usage, CLI, architecture, troubleshooting, contributor, skill,
+  and security guidance now describe the same v3.2 protocol and canonical
+  artifacts.
+- Published-package and current-GitHub-source installation paths are explicitly
+  separated so a stale global CLI cannot be mistaken for current source.
+- Package metadata now describes measurable precision-first review, adaptive
+  retrieval, hybrid verification, and explicit mutation authorization.
+
+### Fixed
+
+- Risk order no longer collapses back to alphabetical ordering in indexed/delta
+  execution.
+- Mixed-size chunks no longer exceed the configured source-token budget because
+  of average/percentile file-size assumptions.
+- Workers cannot report findings for valid repository files outside their exact
+  assigned chunk.
+- Source mutation, deletion, unreadability, and repository-escape conditions
+  fail the affected chunk before findings or completion state are committed.
+- Resume keeps the original source baseline and rejects drift before another
+  worker launches.
+- Coverage cannot derive a file's completion solely from a parent chunk marked
+  done.
+- Duplicate observations retain strongest evidence while preserving useful
+  cross-references and security metadata.
+- Large-file hashing uses content digests rather than size/mtime identity.
+- Fixer scope is limited to executable canary/rollout entries and rejects
+  realpath/symlink escapes.
+- Skeptic guidance no longer auto-dismisses concrete credential stuffing,
+  OTP/reset abuse, lockout bypass, measurable amplification, or attacker-driven
+  expensive operations merely because they involve rate limiting.
+- Legacy task-specific frontmatter prompts and historical `findings.json` /
+  loop-default expectations have been removed from current agent-facing docs
+  and evals.
+
+### Security
+
+- Required hybrid-verification failure or unavailability blocks Fixer
+  authorization instead of being downgraded to a clean result.
+- Hybrid commands execute as inert argv with repository containment,
+  `shell:false`, secret stripping, sensitive-environment rejection, output
+  redaction, and bounded time/output budgets.
+- Exact evidence-cache reuse rejects changed source/protocol/options and
+  symlink-escaped cache shards.
+- Source hashes are verified before dispatch and before findings/completion are
+  committed, preserving fail-closed scan integrity.
+- Immutable Fixer authorization binds repository root, base commit, approved bug
+  IDs, and approved files; report-only/manual-review work remains non-writable.
 
-## [3.1.1] - 2026-08-04
+## 3.1.1 - 2026-08-04 (source version; not tagged)
 
 ### Added
 
@@ -372,8 +455,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Coverage enforcement - partial audits produce explicit warnings
 - Large codebase strategy with domain-first tiered scanning
 
-[Unreleased]: https://github.com/codexstar69/bug-hunter/compare/v3.1.1...HEAD
-[3.1.1]: https://github.com/codexstar69/bug-hunter/compare/v3.1.0...v3.1.1
+[Unreleased]: https://github.com/codexstar69/bug-hunter/compare/v3.1.0...HEAD
 [3.1.0]: https://github.com/codexstar69/bug-hunter/compare/v3.0.10...v3.1.0
 [3.0.10]: https://github.com/codexstar69/bug-hunter/compare/v3.0.9...v3.0.10
 [3.0.9]: https://github.com/codexstar69/bug-hunter/compare/v3.0.8...v3.0.9

CONTRIBUTING.md

@@ -1,72 +1,127 @@
 ---
 title: Contributing to Bug Hunter
 description: >
-  Set up the source checkout, change runtime or prompt contracts, and validate
-  a pull request.
+  Set up the source checkout, change runtime or role contracts, and validate a
+  pull request against the measurable precision-first quality gate.
 prompt: |
-  Keep contributor setup aligned with @package.json,
-  @.github/workflows/ci.yml, @scripts/tests/, and @SKILL.md.
+  Keep contributor changes aligned with package scripts, generated artifacts,
+  canonical role skills, benchmark gates, source-integrity invariants, and the
+  current Bug Hunter orchestration contract.
 ---
 
 # Contributing to Bug Hunter
 
-Thanks for your interest in contributing. Bug Hunter is an open-source adversarial code auditing skill for AI coding agents.
+Thanks for contributing to Bug Hunter, an open-source adversarial code-audit
+skill for AI coding agents.
 
-## Ways to Contribute
+## Ways to contribute
 
-- **Report bugs** — open an issue with reproduction steps
-- **Improve role skills** — the agent instructions in `skills/` are the core of Bug Hunter's accuracy; PRs that reduce false positives or catch more real bugs are highly valued
-- **Add calibration examples** — `skills/hunter/examples.md` and `skills/skeptic/examples.md` tune agent behavior; more real-world examples improve precision
-- **Improve scripts** — the Node.js helpers in `scripts/` handle triage, state, and orchestration; performance and reliability improvements welcome
-- **Documentation** — fix typos, clarify instructions, add usage examples
+- **Report bugs** with reproducible behavior and affected version/commit.
+- **Improve role skills** under `skills/` when evidence shows a false positive
+  or missed real bug.
+- **Improve deterministic scope/retrieval** without weakening source-integrity
+  or coverage truthfulness.
+- **Improve verification** with bounded, argv-only checks and clear required vs
+  optional semantics.
+- **Improve benchmarks** with stronger blinded fixtures, metrics, and baselines.
+- **Improve scripts** for reliability, performance, portability, and safety.
+- **Improve documentation** while keeping package/source and public/internal
+  interfaces accurate.
 
-## Development Setup
+## Development setup
 
 ```bash
 git clone https://github.com/codexstar69/bug-hunter.git
 cd bug-hunter
 pnpm install --frozen-lockfile
 
-# Verify generated runtime files and run the suite
+# Full repository quality gate
+pnpm quality:world-class
+```
+
+`quality:world-class` currently covers:
+
+- generated schema validators and compatibility prompts;
+- the complete Node test suite;
+- the deterministic benchmark quality gate;
+- runtime preflight;
+- npm package inventory verification.
+
+Individual commands remain useful while iterating:
+
+```bash
 pnpm check:generated
 pnpm test
-
-# Run the runtime preflight
+pnpm benchmark:gate
 node scripts/run-bug-hunter.cjs preflight --skill-dir .
-
-# Inspect the npm package allowlist
 pnpm verify:package
+```
+
+Optional Context Hub CLI for documentation-verification development:
 
-# Optional: install Context Hub CLI for doc verification testing
+```bash
 npm install -g @aisuite/chub
 ```
 
-## Pull Request Guidelines
+## Pull request guidelines
+
+1. Keep the change focused and explain the behavioral contract it changes.
+2. Run `pnpm quality:world-class` before requesting review.
+3. If changing role behavior, describe the real false positive/missed bug that
+   motivates it and the expected precision/recall tradeoff.
+4. If changing adaptive policy, retrieval, caching, or verification, add a
+   deterministic regression that proves the safety/cost invariant.
+5. If changing artifact shape, update the schema, generated validator, docs,
+   examples, and consumers together.
+6. Update `CHANGELOG.md` for user-visible behavior.
+7. Do not claim “world-class” from the bundled fixture alone; external claims
+   require unseen/blinded evaluation as described in
+   `docs/world-class-protocol.md`.
+
+## Code style
+
+- Runtime scripts use CommonJS (`.cjs`) for broad agent-runtime compatibility.
+- Prefer Node.js built-ins for runtime paths; development-only generation and
+  validation may use declared dev dependencies.
+- Pass untrusted values as argv/data, not interpolated shell strings.
+- Preserve repository containment, exact source identity, output/time bounds,
+  and fail-closed behavior in security-sensitive helpers.
+- Keep canonical JSON machine-readable; derive Markdown views from it.
+
+## Role and prompt changes
+
+Canonical role instructions live in `skills/*/SKILL.md`. Compatibility prompt
+files under `prompts/` are generated copies and must not be edited directly.
+After changing a canonical role skill:
+
+```bash
+pnpm generate:compat-prompts
+pnpm check:generated
+```
 
-1. Keep PRs focused — one concern per PR
-2. Test your changes against the `test-fixture/` directory
-3. If modifying agent prompts, explain the reasoning and expected impact on false positive / true positive rates
-4. Run `pnpm check:generated` and `pnpm test`
-5. Run `node scripts/run-bug-hunter.cjs preflight --skill-dir .` to verify preflight checks
-6. Update `CHANGELOG.md` with your changes
+When submitting role changes:
 
-## Code Style
+- show the prior false positive or missed real bug;
+- show why the new rule does not suppress an adjacent real bug class;
+- consider Hunter, Skeptic, and Referee together as one adversarial system;
+- keep examples progressive/conditional rather than loading calibration text
+  into every assignment;
+- preserve immutable scope and Referee-only fix authorization.
 
-- Scripts use CommonJS (`.cjs`) for maximum compatibility across agent runtimes
-- Runtime scripts use Node.js built-ins; generation and validation commands
-  use the development dependencies declared in `package.json`
-- Prompts are markdown — keep them concise and structured
+## Benchmark and protocol changes
 
-## Prompt Changes
+The bundled benchmark is calibration/regression data, not an external ranking.
+Protocol changes should keep measurements reproducible and disclose any changed
+thresholds or cost assumptions.
 
-Changes to role skills (`skills/*/SKILL.md`) have outsized impact. The files in
-`prompts/` are generated compatibility copies and must not be edited directly.
-When submitting role-skill changes:
+Read:
 
-- Describe the false positive or missed bug that motivated the change
-- Show before/after behavior if possible
-- Consider impact on all three agents (Hunter, Skeptic, Referee) — they form an adversarial system
+- `docs/precision-protocol.md` for evidence/scope invariants;
+- `docs/world-class-protocol.md` for metrics, adaptive policy, retrieval,
+  evidence cache, and hybrid verification;
+- `modes/dispatch.md` for backend-neutral delegation boundaries.
 
 ## License
 
-By contributing, you agree that your contributions will be licensed under the MIT License.
+By contributing, you agree that your contributions are licensed under the MIT
+License.