Skip to content

ROADMAP.md

Latest commit

 

History

History
217 lines (162 loc) · 9.68 KB

ROADMAP.md

File metadata and controls

217 lines (162 loc) · 9.68 KB
Raw

TiterBench Roadmap — Predictive Throughput Platform

Inspired by baCta's Predictive Throughput is the new high (Oct 2025). Goal: test 100 strains, see 10+ with industrial potential — not thousands.

Architecture overview

┌──────────────────────────────────────────────────────────────────────┐
│  DBTL Loop  (Design → Build → Test → Learn)                          │
│                                                                       │
│  ① Generate          ② Screen           ③ Order          ④ Learn     │
│  LLM/EA proposes  →  Ensemble scores  → Shortlist CSV  → Wet-lab     │
│  N sequences         rank by value +     dispatched to   results      │
│  or conditions       diversity +         lab robotics    auto-paired  │
│                      robustness                          → retrain    │
└──────────────────────────────────────────────────────────────────────┘

Phase 0 — Multi-objective virtual screening ✅ DONE

Already implemented (this session).

  • A Greedy maximin diverse shortlist (diversity_weight slider)
  • B Robustness score — re-score each candidate at ±2°C / ±0.3 pH perturbations; lower σ = more robust
  • C Mahalanobis distance — distance to training set centroid; flags extrapolation
  • D Confidence band — Q90 − Q10 interval width from quantile models
  • E Constraint locks per dimension (organism, promoter, RPM, …)

Files: backend/app/virtual_screen.py, backend/app/schemas.py

Phase 1 — Autonomous DBTL feedback loop ✅ DONE (this session)

1.1 Commit order form

  • POST /api/virtual-screen/commit-order — writes each shortlisted candidate to metric_history.jsonl with status=planned
  • Returns a structured CSV (organism, promoter, RBS, conditions, predicted_value, design_id)

1.2 Auto-pair measured results

  • POST /api/workspace/{id}/upload-results — accepts CSV with design_id + measured_value
  • Looks up planned experiment, auto-populates predicted_value + metadata, appends to feedback_log.jsonl
  • Immediately runs should_auto_retrain() check

1.3 DBTL state endpoint + dashboard

  • GET /api/workspace/{id}/dbtl-state — returns: designs_in_flight, awaiting_measurement, feedback_rows, last_retrain_ts, model_version
  • Overview tab: pill row showing DBTL loop progress

1.4 Auto-retrain on feedback threshold (E2)

  • Already wired via should_auto_retrain + /auto-retrain
  • Dashboard shows "Model updated" badge when triggered

Files: backend/app/workspace.py, backend/app/main.py, frontend/index.html, frontend/assets/app.js

Phase 2 — DNA / sequence generation ✅ DONE

Tool choices

Task Tool Model RAM Notes
Promoter/RBS variant generation (default, no download) Evolutionary algorithm (GA/CMA-ES) with GBM fitness our trained ensemble <100 MB Works offline, instant
Biological plausibility scoring NucleotideTransformer-v2-50M InstaDeepAI/nucleotide-transformer-v2-50m-multi-species ~1 GB HF download, CPU-ok
High-quality generation Evo-2 (optional) arcinstitute/evo2_7b_base ~14 GB Needs CUDA or Apple Silicon MPS
LoRA fine-tuning of NT on workspace data HuggingFace peft + bitsandbytes NT-v2-50M ~2 GB 4-bit quant available
NOT used llama.cpp / Ollama — — These are text-LLM runtimes; no DNA support

Architecture

backend/finetune/dna_generate.py
  ├─ generate_variants_ea(sequences, n, fitness_fn)    # evolutionary, always available
  ├─ score_with_nt(sequences, workspace_dir)           # NucleotideTransformer scoring
  ├─ generate_with_evo2(prompt_seq, n, workspace_dir)  # Evo-2 causal generation
  ├─ finetune_lora_nt(workspace_dir, epochs, lr)       # LoRA adapter on NT-v2-50M
  └─ generate_and_screen(engine, req)                  # orchestrate all phases

EA-based generation (no model required)

  1. Start from known sequences in workspace (promoters, CDS segments)
  2. Apply: point mutations (rate ~0.05), crossover with similar sequences, length-preserving indels
  3. Score each variant with our GBM ensemble (predict_strain)
  4. Select top 20% as next generation parents (tournament selection)
  5. Repeat for n_generations (default 50); return Pareto front (titer, GC content, predicted robustness)

LoRA fine-tuning strategy

  • Base: InstaDeepAI/nucleotide-transformer-v2-50m-multi-species (~200M tokens context)
  • Task: regression head predicting log(titer) from promoter sequence
  • LoRA rank: 8, alpha: 16, target modules: query, value
  • Training data: workspace promoter sequences + measured titers (minimum 10 pairs)
  • ~5 min on CPU for 20 epochs with 50 sequences

Files: backend/finetune/dna_generate.py, backend/app/main.py (/api/generate/*)

Model download (Phase 2)

  • GET /api/models — cache status for esm2, nt, evo2
  • POST /api/models/download/stream — NDJSON progress (HF settings UI)
  • POST /api/models/download — batch download
  • CLI: python scripts/download_models.py --model nt evo2
  • Evo-2 uses official evo2 package + evo2_7b_base.pt (not transformers AutoTokenizer)

Phase 3 — Scale-up and industrial features ✅ DONE (this session)

3.1 Scale-dependent fermentation features

Add optional columns to FermentationRun:

  • working_volume_l — batch size
  • impeller_diameter_m + fluid_density → P/V (W/m³)
  • do_setpoint_pct — dissolved oxygen control
  • feed_rate_g_h — fed-batch substrate feed
  • rq — respiratory quotient (CO₂/O₂ ratio, proxy for metabolic state)
  • batch_mode — batch | fed-batch | continuous

Features auto-degrade to zeros when absent (current behaviour).

3.2 Product cost proxy

Derived feature added to fermentation scoring:

cost_proxy = substrate_g_l × temperature_c × (horizon_h / predicted_titer)

Exposed as production_cost_proxy on DesignCandidate.

3.3 Downstream processability score

For recombinant proteins: hydrophobicity index + predicted aggregation propensity (BioPython or simple heuristic). Surface as processability_score in DesignCandidate.

3.4 Add screened strain to project

  • POST /api/workspace/{id}/strains/from-screen — per-row or bulk Add to project on Screen tab

Files: backend/app/features.py, backend/app/schemas.py, backend/app/virtual_screen.py, backend/app/ensemble.py, backend/app/import_data.py, frontend/assets/app.js

Note: Retrain bioproduction models after upgrade — fermentation feature vector grew from 21 → 27 dims.

Phase 4 — Reinforcement learning over DNA-LLM (future)

Once Phase 2 LoRA adapters are trained and a meaningful feedback dataset exists (>100 data-points):

  1. Use trained ensemble as reward model
  2. Run PPO/DPO over Evo-2 or NT fine-tuned model
  3. RL loop generates sequences → ensemble scores → reward signal → policy update
  4. Result: model learns to sample from the high-titer region of sequence space

Prerequisite: ≥100 (sequence, titer) pairs from wet lab.

Testing strategy

Suite File Runs Coverage
Unit tests tests/test_unit.py python -m pytest tests/test_unit.py -v Features, CV, ensemble, workspace, catalog
Virtual screen E2E tests/e2e_virtual_screen.py python tests/e2e_virtual_screen.py Generate, score, shortlist, multi-objective, DBTL
Full pipeline E2E tests/e2e_astaxanthin_pipeline.py python tests/e2e_astaxanthin_pipeline.py Train→finetune→validate→predict
DNA generation tests/test_unit.py::TestDnaGenerate included in unit suite EA generation, diversity, fitness

Running all tests

# Unit tests (fast, <10s, no server needed)
python -m pytest tests/test_unit.py -v

# Virtual screen E2E (medium, <30s, no server needed)
python tests/e2e_virtual_screen.py

# Full pipeline E2E (slow, ~2 min, no server needed)
python tests/e2e_astaxanthin_pipeline.py

# All at once
python -m pytest tests/ -v --timeout=120

Install requirements

# Core (already installed)
pip install scikit-learn torch numpy fastapi

# For DNA generation — Phase 2
pip install transformers peft

# For 4-bit quantization (optional, GPU only)
pip install bitsandbytes

# For Evo-2 (optional, large model)
pip install evo2  # or: pip install git+https://github.com/ArcInstitute/evo2

Status board

Feature Status Phase
Ensemble (GBM + LSTM + Ridge + quantile) ✅ Done —
Virtual screening (generate + score + shortlist) ✅ Done 0
Multi-objective shortlist (robustness + Mahalanobis + confidence) ✅ Done 0
DBTL state dashboard ✅ Done 1
Commit order form → planned experiments ✅ Done 1
Auto-pair upload results → feedback log ✅ Done 1
Auto-retrain on feedback threshold ✅ Done 1
EA-based sequence generation ✅ Done 2
NucleotideTransformer scoring scaffold ✅ Done 2
LoRA fine-tune NT on workspace data ✅ Scaffold done 2
Evo-2 generation ✅ Done (evo2 pkg + download) 2
ML model download API + HF UI ✅ Done 2
Scale-up features (P/V, RQ, DO) ✅ Done 3
Production cost proxy (screen) ✅ Done 3
Processability score (screen) ✅ Done 3
Add screen strain → project ✅ Done 3
RL over DNA-LLM 🔲 Future 4