diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 237ee21..471b197 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -111,7 +111,7 @@ artifact, not bolted onto this.) **To build (Track B / product — the team work):** - ONNX export for the taste models (`skl2onnx`) - ASP.NET Core API + ONNX Runtime in-process serving -- Aroma: get the OpenPOM model trained, then ONNX-export or sidecar +- Aroma: get the OpenPOM model trained, then ONNX-export or sidecar — **deferred until clean fuel exists; see [`AROMA.md`](AROMA.md)** for the data/licensing situation and the OpenPOM commercial scope - React workbench against the fixed JSON contract - Dockerfiles + `docker-compose.yml` (single-box) - Auth + per-seat (pilot-stage) diff --git a/docs/AROMA.md b/docs/AROMA.md index fcd8d79..62bd672 100644 --- a/docs/AROMA.md +++ b/docs/AROMA.md @@ -23,6 +23,33 @@ reserved"), AromaDB (CSIR), Dravnieks (ASTM ©), sharma_2021 (ACS ©), snitz_201 (CC-BY-NC). The **only** commercially-clean odor-descriptor set is **`keller_2016`** (Keller & Vosshall 2016, *BMC Neuroscience*, CC-BY-4.0; ~480 molecules, 20 descriptors). +### Can we use OpenPOM commercially? (the precise scope) + +Yes for the **engine**, no for the **fuel it ships with** — and that distinction is +the whole answer: + +| Component | License | Commercial use | +|---|---|---| +| OpenPOM **code** (the GNN / message-passing model) | MIT | ✅ **Yes** — use the architecture freely | +| Bundled **data** (`curated_GS_LF_merged_4983.csv`) | GS-LF = Leffingwell/GoodScents, **NonCommercial** | ❌ **No** — the MIT repo does *not* relicense third-party data it doesn't own (same rule as Pyrfume/SweetenersDB: a repo's license only covers what the uploader authored) | +| Any **pretrained weights** trained on that data | derivative of NonCommercial data | ❌ **Murky/risky** — a model trained on NC data and used commercially is legally unsettled; not something to bet a product on | + +So **OpenPOM does *not* "come with commercially-allowed data"** — it comes with +*NonCommercial* data. The code was never the blocker; the data always was. + +**What this means for us:** we keep only OpenPOM's *code* — the MIT architecture and +training recipe — and we **never use its GS-LF-trained weights**. When clean fuel +exists we **train our own weights from scratch** on it: either a **licensed copy** +(Leffingwell PMP 2001, ~$2,775, ideally licensed by the customer and run on-prem) or +**the customer's own odor data**. There is no shortcut around this — a usable aroma +model *must* be our own, trained on data we can legally use. That's exactly the taste +pipeline over again: structures + labels in, model out. (For a small set, plain +RandomForest suffices — see the taste heads; OpenPOM's GNN only earns its keep once +the labeled set is large, thousands of expert-labeled molecules like PMP 2001 — so +"keep OpenPOM" really means *keep the option of its architecture*, not any pre-baked +model.) The only requirements: the labels are **expert sensory descriptors** (GC-MS +identifies the molecules but carries no smell labels) and the data is ours/licensed. + ## The empirical result We aggregated `keller_2016` ([`training/build_aroma_dataset.py`](../training/build_aroma_dataset.py)) diff --git a/docs/BUILD-STATUS.md b/docs/BUILD-STATUS.md new file mode 100644 index 0000000..5f90b83 --- /dev/null +++ b/docs/BUILD-STATUS.md @@ -0,0 +1,94 @@ +# Build Status + +A living inventory: what's **actually built**, what's **built-but-needs-validation**, +what's **stubbed/data-gated**, and what's **deliberately deferred**. Updated as +milestones land. For *why* things are the way they are, see `ARCHITECTURE.md`, +`CAPABILITIES.md`, and `AROMA.md`. + +## Legend +- ✅ **Built & validated** — trained/run, output sanity-checked +- 🟡 **Built, needs validation** — code exists, not yet under an end-to-end test suite +- 🔩 **Stub / data-gated** — interface exists, returns an honest "not available" until fueled +- 🧱 **Team milestone (Track B)** — scaffold only, on the roadmap +- ⏸️ **Deferred** — intentionally parked, with a documented reason + +--- + +## Track A — Python core (`training/`, runs on the R620) + +### Taste engine ✅ +- sweet / bitter / umami classifiers (RandomForest on Morgan fingerprints) — AUROC 0.95 / 0.95 / 0.99 +- sweetness-**intensity** regressor (SweetenersDB v2.0) — R² 0.82 +- sour: trained **indicative** head (AUROC 0.90) **+** acidity SMARTS rule (recall 0.93) +- salty: cation-aware rule **+** verified dataset-label override +- ranked `taste_profile` + `multitaste` flag +- ONNX export with roundtrip self-validation (< 1e-6) + +### Behaviour / chemistry packs (`predict.py`) 🟡 *(spot-validated 2026-06-27)* +- `physchem` — MW, logP, TPSA, H-bonds, solubility, volatility, pKa (computed, RDKit) +- `stability` — oxidation / hydrolysis / photo watch-flags (rule) +- `chemesthesis` — cooling / pungent / astringent (rule) +- `safety` — disclaimer, structural alerts, GRAS cross-check, TTC hint, EU allergen labeling +- `check_mixture` — documented dangerous-pair screen +- `analyze_balance` — OAV dosing balance (qualitative) + +All return chemically-sane output on spot checks (vanillin correctly flags its +phenol + aldehyde as oxidation-prone; citric acid; glucose). **Validation gap:** +these aren't yet under a regression-test suite — that's the work to move them to ✅. + +### Substitution search ✅ *(new — issue #22 core)* +- `substitute()` — Tanimoto / Morgan nearest-neighbor over the labeled molecule set, + each neighbor returned with its known tastes +- demonstrated: glucose → ribose / sugars (sim 0.94); vanillin → ethyl ferulate / + creosol / anisaldehyde (vanillin-adjacent aromatics) +- this is the **Track-A** implementation; Track B mirrors it as a pgvector ANN query + over the same fingerprints (M6) + +### Track-A demo ✅ +- FastAPI `app.py` (`/api/predict`, `/api/neighbors`) + `workbench.html` + +### Data-gated / stubs 🔩 +- `retention_index` (Kovats RI) — needs a trained RI QSPR / NIST data +- `analyze_balance` **quantitative** dosing — needs odor-threshold tables +- GRAS cross-check — needs the FEMA/FDA GRAS list loaded (`gras_reference.parquet`) +- aroma — see **Deferred** below + +--- + +## Track B — Product (the team builds) +- 🧱 `api/` (.NET) — buildable skeleton + `/health`; endpoints / rule-port / in-process + ONNX serving = **M2** (Aaron) +- 🧱 `frontend/` (React workbench) — greenfield = **M3** (Jamie) +- 🧱 `infra/` (Docker Compose, Postgres + pgvector) = **M5** (Ty) +- 🧱 substitution **at scale** (pgvector ANN), auth / per-seat = **M6** + +--- + +## Deferred +- ⏸️ **Aroma model.** No commercially-clean *public* data yields a working model + (`keller_2016` scored CV-R² ≤ 0 across all 20 descriptors; the rich set, GS-LF, is + NonCommercial). The **engine** (OpenPOM — MIT *code*) is kept and wired; it's + unlocked by **licensed PMP 2001** or **customer data**. Full record: `AROMA.md`. + +--- + +## Product vision — staging (near-term → long-horizon) + +What we sell, and when: + +1. **Now — clean public data.** Taste + behaviour + safety read on any structure, + plus the **substitution library** (swap an ingredient for a close analogue). + This is the public demo and the open-door. + +2. **Paid pilot — their data.** An **aroma model** trained on the customer's + licensed/owned odor data (GC-MS to identify the molecules **+** their sensory + panel's descriptors as labels — both are required; GC-MS alone has no smell + labels to learn). Plus **blend-ratio suggestions** tuned on their formulation + archive. The pitch line is literal: *"aroma comes with your data."* + +3. **Long-horizon — R&D + a regulatory wall.** **Novel-molecule generation** + (propose new flavor molecules) and **retrosynthesis** (synthesis routes). + Technically far harder, and gated by food-safety approval — a newly-generated + molecule isn't GRAS and can't be sold as a flavor without a regulatory path. + A research direction, not a near-term feature; documented here so the staging + is explicit and doesn't get over-promised in a pitch. diff --git a/docs/README.md b/docs/README.md index 7df3982..ab7faac 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,6 +8,7 @@ Architecture, capability catalogue, and design decisions. - `SOURCES.md` — data sources, libraries, research, and license attribution - `API-CONTRACT.md` — the fixed JSON contract the .NET API exposes and the React UI consumes - `DATA-SOURCES.md` — data acquisition tracker: what each source unlocks and how to get it -- `AROMA.md` — why the aroma model is deferred (clean-data audit + the empirical evaluation) +- `AROMA.md` — why the aroma model is deferred (clean-data audit + the empirical evaluation + OpenPOM's commercial scope) +- `BUILD-STATUS.md` — living inventory: what's built / needs validation / stubbed / deferred, and the product-vision staging These document *what we're building and why*, so decisions don't get re-argued. diff --git a/training/predict.py b/training/predict.py index fd8e57b..4892b24 100644 --- a/training/predict.py +++ b/training/predict.py @@ -656,6 +656,61 @@ def _taste_profile(out): return ranked +# --- Substitution search (issue #22) ------------------------------------------- +# "Find me a molecule that behaves like X." Nearest-neighbor search over our +# labeled molecules by Morgan/Tanimoto similarity — the reformulation / cost-down +# tool (swap an expensive or supply-constrained ingredient for a close analogue, +# with its known tastes shown). This is the clean Track-A core; the product +# (Track B, #22) mirrors it as a pgvector ANN query over the same fingerprints. +_SUB_INDEX = None # lazily built: (list[bitvect], list[smiles], list[known_tastes]) + + +def _build_sub_index(): + global _SUB_INDEX + fps, smis, tastes = [], [], [] + if _MASTER.exists(): + import pandas as pd # noqa: F811 + m = pd.read_parquet(_MASTER) + basic = [t for t in ("sweet", "bitter", "umami", "sour", "salty") if t in m.columns] + for _, r in m.iterrows(): + mol = Chem.MolFromSmiles(str(r["smiles"])) + if mol is None: + continue + fps.append(_MORGAN.GetFingerprint(mol)) + smis.append(Chem.MolToSmiles(mol)) + tastes.append([t for t in basic if r.get(t) == 1]) + _SUB_INDEX = (fps, smis, tastes) + + +def substitute(smiles: str, k: int = 8, min_similarity: float = 0.0) -> dict: + """Nearest-neighbor substitution: the k labeled molecules most structurally + similar to the query (Tanimoto over Morgan fingerprints), each with its known + tastes. The reformulation / cost-down tool — swap an ingredient for a close + analogue. Returns {'neighbors': [...]} ranked by similarity (self excluded).""" + mol = Chem.MolFromSmiles(smiles) + if mol is None: + return {"error": f"unparseable SMILES: {smiles}"} + if _SUB_INDEX is None: + _build_sub_index() + fps, smis, tastes = _SUB_INDEX + if not fps: + return {"neighbors": [], "note": "no reference set loaded (taste_master.parquet absent)"} + q = _MORGAN.GetFingerprint(mol) + self_smi = Chem.MolToSmiles(mol) + sims = DataStructs.BulkTanimotoSimilarity(q, fps) + order = sorted(range(len(sims)), key=lambda i: sims[i], reverse=True) + neighbors = [] + for i in order: + if smis[i] == self_smi or sims[i] < min_similarity: + continue + neighbors.append({"smiles": smis[i], "similarity": round(float(sims[i]), 3), + "known_tastes": tastes[i]}) + if len(neighbors) >= k: + break + return {"query": self_smi, "neighbors": neighbors, + "basis": "Tanimoto / Morgan r2 2048-bit over labeled molecules"} + + def predict(smiles: str, include_aroma: bool = False) -> dict: mol = Chem.MolFromSmiles(smiles) if mol is None: