Bundle Ferm DoE skill references

Author: jvogan

scripts/check_markdown_links.py

@@ -72,10 +72,20 @@ def iter_markdown_files(paths: list[Path], excluded_dirs: set[str]) -> list[Path
         for child in path.rglob("*.md"):
             if any(part in excluded_dirs for part in child.parts):
                 continue
+            if is_bundled_skill_reference(child):
+                continue
             files.append(child)
     return sorted({path.resolve() for path in files})
 
 
+def is_bundled_skill_reference(path: Path) -> bool:
+    parts = path.parts
+    for index, part in enumerate(parts):
+        if part == "skills" and len(parts) > index + 3 and parts[index + 2] == "references":
+            return True
+    return False
+
+
 def normalize_target(raw: str) -> str:
     target = raw.split("#", 1)[0].strip()
     if target.startswith("<") and target.endswith(">"):

skills/biosymphony-ferm-doe/SKILL.md

@@ -0,0 +1,58 @@
+# Adapter Map
+
+Capability-centric map of optional extras. The README's [Optional extras](../README.md#optional-extras) table is install-centric; this one starts from "I want to do X" and tells you which extra activates it.
+
+The CLI is stdlib-only at runtime. Every adapter degrades to a `not_available` report when its extra is missing, so the demos and the closed-loop path run on a clean install. Install the extra only when a campaign needs the capability.
+
+## By capability
+
+| I want to... | Extra | Install | CLI surface that activates it |
+|---|---|---|---|
+| Get Student-t p-values in first-batch analysis | `scipy` | `pip install "biosymphony-ferm-doe[scipy]"` | `ferm-doe analyze` (auto) |
+| Get t-quantile in DoE power | `scipy` | `pip install "biosymphony-ferm-doe[scipy]"` | `ferm-doe doe-power --sigma S` (auto) |
+| Generate Box-Behnken with k ≥ 5 | `pydoe3` | `pip install "biosymphony-ferm-doe[pydoe3]"` | `ferm-doe generate-design` with `doe.family = box_behnken` and 5+ numeric factors |
+| Generate maximin Latin Hypercube | `pydoe3` | `pip install "biosymphony-ferm-doe[pydoe3]"` | `ferm-doe generate-design` with `doe.family = latin_hypercube` |
+| Run follow-up Bayesian optimization with a Gaussian-process surrogate | `botorch` | `pip install "biosymphony-ferm-doe[botorch]"` | `ferm-doe plan-wave2 --backend botorch --acquisition qei` or `qucb` (see [`WAVE2_BOTORCH.md`](WAVE2_BOTORCH.md)) |
+| Route constrained DoE through BoFire (linear, total-mass, NChooseK) | `bofire` | `pip install "biosymphony-ferm-doe[bofire]"` | `ferm-doe plan-wave2 --backend bofire` or auto-routing when the manifest declares non-box constraints (see [`BOFIRE_POSITIONING.md`](BOFIRE_POSITIONING.md)) |
+| Route NChooseK DoE through BoFire main | `adaptive-nchoosek-doe` | `pip install "biosymphony-ferm-doe[adaptive-nchoosek-doe]"` | Use when a first-batch DoE screen has a load-bearing NChooseK `min_count`; see [`BOFIRE_CONSTRAINT_PATTERNS.md`](BOFIRE_CONSTRAINT_PATTERNS.md) |
+| Route multi-fidelity scale-bridge planning through BoFire | `bofire` | `pip install "biosymphony-ferm-doe[bofire]"` | `ferm-doe scale-recipe` with `MultiFidelityVarianceBasedStrategy` declared in the manifest |
+| Run NChooseK Bayesian optimization (cardinality is load-bearing in BO) | `entmoot` | `pip install "biosymphony-ferm-doe[entmoot]"` | ENTMOOT v2 adapter; the documented swap for BoFire's `SoboStrategy + NChooseK` stall (see [`ENTMOOT_SWAP_DESIGN.md`](ENTMOOT_SWAP_DESIGN.md)) |
+| Run MIP-optimized surrogate planning over linear and NChooseK constraints | `omlt` | `pip install "biosymphony-ferm-doe[omlt]"` | OMLT adapter at `adapters/omlt_strategy.py`; activates from the planner when MIP routing fits the constraint shape |
+| Use a token-gated foundation-model surrogate for low-data sequential planning | `tabpfn` | `pip install "biosymphony-ferm-doe[tabpfn]"` | TabPFN adapter at `adapters/tabpfn_strategy.py`; inactive unless `TABPFN_TOKEN` is set at runtime |
+| Compare follow-up candidate-generators (BayBE, Ax against the in-repo BoTorch route) | `backend-eval` | `pip install "biosymphony-ferm-doe[backend-eval]"` | `examples/adaptive-backend-eval/` fixtures; see [`BIOMANUFACTURING_ADAPTIVE_BACKENDS.md`](BIOMANUFACTURING_ADAPTIVE_BACKENDS.md) |
+| Run SALib Sobol / Morris sensitivity over first-batch result rows | `sensitivity` | `pip install "biosymphony-ferm-doe[sensitivity]"` | SALib adapter at `adapters/salib_sensitivity.py` |
+| Render Plotly figures in the BoFire HTML report | `report` | `pip install "biosymphony-ferm-doe[report]"` | `reporters/bofire_html.py` |
+| Frictionless-validate table contracts (run ledger, evidence, design, results) | `contracts` | `pip install "biosymphony-ferm-doe[contracts]"` | Validators that read `schemas/tables/*.yaml` |
+
+## Catch-all install
+
+```bash
+pip install "biosymphony-ferm-doe[all]"
+```
+
+Installs everything that does not require a paid or token-gated service. ENTMOOT, OMLT, and TabPFN are not in the catch-all because they each pull in solver stacks (`pyomo` + `highspy` + `lightgbm`) or foundation-model weights. Install those individually when a campaign uses them.
+
+## Routing rules
+
+The planner picks an adapter route based on what the manifest declares plus optional CLI flags:
+
+- **Box constraints only, n < 4 usable first-batch rows, or categorical-heavy**: stdlib closed-loop path. No extra needed.
+- **Numeric factors, n ≥ 4 usable rows, primary response declared**: `botorch` route is available; pass `--backend botorch`.
+- **Linear constraints, mixture sums, total-mass**: `bofire` route fits; routes automatically when the manifest declares non-box constraints. Pass `--backend bofire` to force.
+- **NChooseK cardinality matters in the BO loop**: route to `entmoot`. BoFire's `SoboStrategy + NChooseK` stalls on upstream issue #450; the ENTMOOT swap is documented at [`ENTMOOT_SWAP_DESIGN.md`](ENTMOOT_SWAP_DESIGN.md).
+- **Hard MIP constraints over a learned surrogate**: `omlt` route.
+- **Low-data prediction with a foundation model**: `tabpfn` route, token-gated.
+
+When the requested route's extra is missing, the adapter writes a `not_available` report and the planner falls back to the stdlib path. The orchestrator can surface the short-circuit reason to the user.
+
+## Health check
+
+```bash
+ferm-doe doctor
+```
+
+Reports which extras are installed and which adapters are active. Useful for debugging "why did this campaign route through stdlib when I expected BoFire?".
+
+## Adapter status from the tool registry
+
+See [`TOOL_REGISTRY.md`](TOOL_REGISTRY.md) and [`tool-registry.json`](tool-registry.json) for the curated 47-tool surface that includes each adapter's routing rationale, current signal, fit, and risks.

skills/biosymphony-ferm-doe/references/docs/ADAPTER_DESIGN_NOTES.md

@@ -0,0 +1,112 @@
+# Adaptive Follow-Up Planning
+
+Adaptive follow-up planning turns completed first-batch result rows into auditable next-step artifacts. The stable CLI and file identifiers still use `wave2` (`ferm-doe plan-wave2`, `planned_wave2_design`) because they are part of the repo contract. That label means "the next planning checkpoint after first-batch results," not a predetermined second experiment. Outputs are deliberately conservative: `planned_wave2_design`, not optimized, validated, production-ready, or scale-transfer proven.
+
+## What It Does
+
+`ferm-doe plan-wave2` reads:
+
+- `campaign_manifest.json`
+- a first-batch results CSV
+- optional selected-design CSV presence check
+
+It writes:
+
+- `adaptive_wave2_plan.json`
+- `result_ingestion_report.json`
+- `assay_power_results.json`
+- `wave2_recommendation.json`
+- `wave2_recommendation.md`
+- `locked_prior_runs.csv`
+- `augment_design.csv`
+- `adaptive_trace.json`
+- `negative_result_memory.json`
+- `learning_ledger.csv`
+- `hiccup_review.md`
+- `wave2_manifest.patch.json`
+- `bofire_strategy_report.json` when the BoFire routing rule fires
+
+## Recommendation Actions
+
+The public planner can recommend the next action:
+
+- `confirm`: repeat or confirm a modest winner.
+- `narrow`: plan local candidate rows near a strong non-boundary winner.
+- `expand`: the best row is on a factor boundary, so factor-space review comes before local narrowing.
+- `pause`: results are missing, low-trust, nonnumeric, cross-arm pooled without an active arm, or bridge eligibility blocks the requested move.
+- `stop`: the primary response is flat enough that more runs may not be useful under the current objective.
+- `scale_or_downscale`: only when explicitly requested and bridge eligibility passes. This still means planning next-arm candidates, not validated transfer.
+
+## Result CSV Shape
+
+Minimum columns:
+
+```csv
+design_run_id,arm_id,qc_status,inclusion_status,trust_score,primary_response
+R001,plate,pass,include,0.95,12.4
+```
+
+Recommended columns:
+
+- `design_run_id` or `run_id`
+- `arm_id`
+- `qc_status`
+- `inclusion_status`
+- `trust_score`
+- one column per measured response
+- factor columns from the selected design, when available
+
+Rows with failed QC, explicit exclusion, or trust score below `0.6` do not drive recommendations.
+
+## Manifest Slot
+
+```json
+{
+  "adaptive_wave2": {
+    "claim_level": "planned_wave2_design",
+    "primary_response_id": "product_titer_g_l",
+    "active_arm_id": "shake_flask",
+    "allowed_actions": ["confirm", "narrow", "expand", "pause", "stop"],
+    "require_assay_power": true,
+    "self_learning": {
+      "enabled": true,
+      "learning_ledger_path": "wave2/learning_ledger.csv",
+      "hiccup_review_path": "wave2/hiccup_review.md",
+      "negative_memory_scope": "arm"
+    }
+  }
+}
+```
+
+Use `active_arm_id` when multiple arms are present. Without it, the planner will not pool incompatible plate, flask, and reactor rows into one narrowing decision.
+
+## Optional BoFire Route
+
+`plan-wave2` can route through `adapters/bofire_strategy.py` when the campaign declares non-box constraints, multiple optimized responses, scale fidelity, or `--backend bofire`.
+
+BoFire remains optional. If `bofire[optimization]` is missing, or if the adapter cannot safely translate a declared constraint, the packet records `bofire_strategy_report.json` and falls back to the stdlib augmentation path. BoFire-backed rows keep the same `planned_wave2_design` claim boundary.
+
+## CLI
+
+```bash
+ferm-doe assay-power examples/demo-xylanase-public
+
+ferm-doe plan-wave2 examples/demo-pb-screening-public \
+  --results examples/demo-pb-screening-public/inputs/wave1_results.csv \
+  --out-dir wave2_public_plan \
+  --selected-design examples/demo-pb-screening-public/expected/selected_wave_1_design.csv \
+  --remaining-budget 3
+```
+
+## Non-Claims
+
+The public planner does not:
+
+- generate commercial-grade optimal designs
+- validate assay data
+- validate scale transfer
+- approve lab execution
+- replace statistical review
+- write GxP batch records
+
+It creates a deterministic planning packet so an agent or scientist can review the next move without losing the evidence trail.

skills/biosymphony-ferm-doe/references/docs/ADAPTER_MAP.md

@@ -0,0 +1,87 @@
+# Agent Harnesses
+
+How to plug `biosymphony-ferm-doe` into the runtimes long-running agents actually use. The skill is runtime-agnostic; this doc is the bridge.
+
+## Common ground
+
+- **The campaign manifest is durable state.** `<campaign_dir>/campaign_manifest.json` is the single source of truth. Updating it across turns is the agent's job.
+- **The CLI is stdlib-only at runtime.** No deps to manage, no virtualenv to ship; just `python3 -m biosymphony_ferm_doe.cli`.
+- **JSON output is machine-readable.** `--summary` returns ~10 fields; full output returns the per-check list. Pipe it, parse it, post it.
+- **Hand-off lives in `expected/AGENTS.md`.** Every session ends by updating it.
+
+## State persistence model
+
+| Layer | Where it lives | Who owns it |
+|---|---|---|
+| Manifest (`campaign_manifest.json`) | repo / workspace | the skill (you write to it; the validator reads it) |
+| Inputs (`inputs/*.csv`) | repo / workspace | the agent / scientist |
+| Expected artifacts (`expected/*`) | repo / workspace | the agent |
+| Hand-off (`expected/AGENTS.md`) | repo / workspace | the agent that's pausing |
+| Validator output | stdout / `--out FILE` | ephemeral |
+| Issue tracker comments / labels | external system | the agent's harness |
+
+Resume = read the manifest, read `expected/AGENTS.md`, run `validate --summary`, continue.
+
+## Claude Code
+
+`agents/claude.md` is the starting reference. Patterns specific to Claude Code:
+
+- Use the Bash tool for `ferm-doe list-campaigns`, `ferm-doe inspect-campaign`, `ferm-doe agent-brief`, `ferm-doe validate`, and `audit`. The output is small and parseable.
+- Use the Read tool for the manifest, then the Edit tool for incremental updates. Avoid Write; it overwrites the whole file and risks losing state another agent might be holding.
+- Use TaskCreate to plan profile-by-profile work. Mark tasks completed as you fix warnings.
+
+## OpenAI Agents SDK / Codex CLI
+
+`agents/openai.yaml` is the starting reference. Patterns:
+
+- Wrap `ferm-doe list-campaigns`, `ferm-doe inspect-campaign`, `ferm-doe agent-brief`, and `ferm-doe validate --summary` as JSON-schema-typed tools; parse them directly into the agent's reasoning loop.
+- For Codex CLI: pass the campaign directory as an argument; let Codex own the read/write of the manifest file.
+
+## Long-horizon orchestrators (generic)
+
+`agents/generic.md` covers the baseline. The pattern that always works:
+
+1. Persist `<campaign_dir>` somewhere durable (repo, S3, local filesystem with backup).
+2. The orchestrator's worker calls `ferm-doe list-campaigns`, reads the selected manifest, calls `ferm-doe inspect-campaign`, `ferm-doe agent-brief`, and `ferm-doe validate --summary`, then decides the next action.
+3. The worker writes the updated manifest back atomically (write to `<path>.tmp`, then rename).
+4. The orchestrator re-dispatches when new evidence arrives (lab-execution results, instrument calibration logs, additional literature).
+
+## Tracker-driven runners
+
+For trackers like Linear, Jira, or GitHub Issues:
+
+- Tracker issue ↔ campaign
+- Tracker sub-issue ↔ wave
+- Tracker comment ↔ readiness summary
+- Tracker label ↔ profile + worst_axis
+- Tracker status ↔ readiness verdict (RED / YELLOW / GREEN)
+
+## What the skill does *not* do
+
+- Schedule, dispatch, or orchestrate workers. That work belongs to the harness.
+- Authenticate to external systems. The harness brings credentials.
+- Track time, cost, or budget. Record those as `constraints[]` if needed; the validator will not model them.
+- Execute physical lab experiments. BioSymphony Ferm DoE is a pre-experiment planning system, not an execution system; the lab team owns physical execution and the batch record built on top of the DOE plan.
+
+## Anti-patterns
+
+- **Two agents updating the same manifest in parallel.** Use a lock or a worktree per agent. The validator is fast enough that serial updates are not a bottleneck.
+- **Agent writes a "summary" to the manifest's free-text fields instead of `assumptions[]`.** Use the structured slot. Free-text drifts.
+- **Agent commits a campaign manifest with private process data to a public repo.** Use a private workspace; keep the public examples synthetic.
+- **Agent ignores `stop_rules[]`.** They exist so a paused campaign cannot be silently resumed by a different agent.
+
+## Testing your harness against the skill
+
+```bash
+# 1. fresh checkout
+git clone https://github.com/BioSymphony/ferm-doe.git
+cd ferm-doe
+
+# 2. validate your harness can call the CLI and parse the output
+PYTHONPATH=src python3 -m biosymphony_ferm_doe.cli validate examples/demo-warnings-walkthrough-public --summary
+
+# 3. confirm the diagnostic warnings round-trip through your harness's tool plumbing
+# 4. wire stop-rule firing into your harness's escalation path
+```
+
+If your harness can't surface the eight warnings from the diagnostic demo to a human, fix the harness before pointing it at a real campaign.