diff --git a/README.md b/README.md index d4729d8..e134d5e 100644 --- a/README.md +++ b/README.md @@ -41,12 +41,12 @@ reference table), and **qualitative** (a class/flag, not a number). Python trains the models offline; a .NET application serves them at runtime. The training language is a build-time detail — nothing at runtime depends on Python -(aside from an optional aroma sidecar if the GNN won't export cleanly). +(aside from a possible aroma sidecar later — aroma is deferred; see docs/AROMA.md). ``` data sources ─► Python training (build-time) ─► ONNX (taste) ──┐ - RDKit · scikit-learn · OpenPOM aroma model ─┐ │ - ▼ ▼ + RDKit · scikit-learn │ + ▼ React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pgvector │ ▼ @@ -55,7 +55,7 @@ React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pg | Layer | Technology | |-------|-----------| -| Model training (build-time) | Python · RDKit · scikit-learn · OpenPOM/DeepChem | +| Model training (build-time) | Python · RDKit · scikit-learn · skl2onnx | | Model handoff | ONNX | | App / API | ASP.NET Core (C#) | | ML serving | ONNX Runtime, in-process in .NET | @@ -67,7 +67,6 @@ React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pg ``` training/ Python — dataset build + model training (build-time) -aroma-sidecar/ Python — thin aroma inference service (only if needed) api/ ASP.NET Core — app, auth, endpoints, ONNX serving frontend/ React — the workbench UI infra/ Dockerfiles, docker-compose.yml, deploy diff --git a/ROADMAP.md b/ROADMAP.md index 34b7a19..ed1671e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -10,7 +10,7 @@ PRs reference their milestone (e.g. *"Part of M2"*), and progress is tracked in | **M1 — Training pipeline (Python)** | Dataset build, taste training, prediction core, ONNX export, demo workbench | ✅ Done | | **M2 — .NET API + ONNX serving** | ASP.NET Core skeleton, load taste ONNX, `/predict`, in-process ONNX Runtime | 🔜 Next | | **M3 — React workbench** | React UI against the fixed JSON contract; taste meters with confidence tags | Planned | -| **M4 — Aroma model** | Train OpenPOM on Leffingwell; ONNX-export or sidecar; wire `predict_aroma()` | Planned | +| **M4 — Aroma model** | Train on licensed (PMP 2001) / customer odor data; wire `predict_aroma()` — **deferred, see [docs/AROMA.md](docs/AROMA.md)** | Deferred | | **M5 — Packaging** | Dockerfiles, `docker-compose`, Postgres + pgvector, single-box deploy | Planned | | **M6 — Pilot-ready** | Auth + per-seat, pgvector substitution search, polish, demo script | Planned | diff --git a/aroma-sidecar/README.md b/aroma-sidecar/README.md deleted file mode 100644 index 8f0b18e..0000000 --- a/aroma-sidecar/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# aroma-sidecar/ - -Python, runtime — **only if needed**. A thin localhost service that runs aroma -(odor-descriptor) inference when the GNN model can't be exported to ONNX cleanly -for in-process serving in .NET. - -Best case this directory stays empty: the aroma model exports to ONNX and the -.NET app serves it directly, leaving zero Python at runtime. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 471b197..345153b 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -99,7 +99,7 @@ artifact, not bolted onto this.) ## Status: built vs to-build **Built (Track A / training, reusable by Track B):** -- `build_taste_dataset.py`, `train_taste.py`, `train_odor.py` (scaffold), `predict.py` +- `build_taste_dataset.py`, `train_taste.py`, `predict.py` (aroma training added when clean data exists — see [`AROMA.md`](AROMA.md)) - `app.py` + `workbench.html` — the Python/HTML *demo* serving prototype **Safety (defensive, caution-only — built into predict.py):** diff --git a/docs/AROMA.md b/docs/AROMA.md index 62bd672..7f1765a 100644 --- a/docs/AROMA.md +++ b/docs/AROMA.md @@ -83,8 +83,11 @@ We do **not** ship a negative-R² model — it would output confident, wrong sme a flavor chemist would catch it instantly (worse than nothing). `predict_aroma()` returns an honest "not available" marker, and **we lead with the taste engine**. -The **OpenPOM scaffold (`training/train_odor.py`) is kept** as the aroma engine for -when clean fuel exists: +We carry **no dead scaffold** — the architecture decision lives here, not in an +unrunnable stub. **OpenPOM is the chosen architecture** (MIT, re-addable in an +afternoon) for large sets; RandomForest suffices for small ones, exactly as the +taste heads and the `train_aroma.py` evaluation already demonstrate. We build the +training script when clean fuel exists: 1. **License PMP 2001** (~$2,775, Leffingwell & Associates) → re-curate → train. 2. **A customer's own odor data** (the paid pilot) → train on-prem. 3. A future open *expert-labeled* dataset, if one emerges. diff --git a/docs/SOURCES.md b/docs/SOURCES.md index c697d4c..8cdc72f 100644 --- a/docs/SOURCES.md +++ b/docs/SOURCES.md @@ -11,7 +11,7 @@ record of provenance, not legal advice. Get an IP/OSS-license review before ship **Cheminformatics / ML (training side — Python)** - **RDKit** — molecular parsing, fingerprints, descriptors, SMARTS, InChIKeys. (BSD-3-Clause.) -- **DeepChem** — GNN training framework under OpenPOM. (MIT.) Python-only; the one runtime-Python piece. +- **DeepChem** — GNN training framework under OpenPOM. (MIT.) For the **deferred** aroma model only; not currently installed or used. - **OpenPOM** (BioMachineLearning/openpom) — the message-passing GNN for odor; open reimplementation of the principal-odor-map work. (MIT — confirm.) The aroma model is theirs in spirit. - **scikit-learn** — RandomForest taste heads + sweetness-intensity regressor. (BSD-3-Clause.) - **PyTorch** — tensor/Autograd backend under DeepChem. (BSD-style.) @@ -25,7 +25,7 @@ record of provenance, not legal advice. Get an IP/OSS-license review before ship - **ASP.NET Core / .NET** — the app/API backbone. (MIT.) - **React** — frontend. (MIT.) - **PostgreSQL** + **pgvector** — DB + embedding/substitution search. (PostgreSQL License.) -- **FastAPI** — the Track-A demo serving layer (+ aroma sidecar if needed). (MIT.) +- **FastAPI** — the Track-A demo serving layer. (MIT.) - **Docker** — single-box deployment. (Apache-2.0.) --- diff --git a/infra/README.md b/infra/README.md index d17dd74..8a6d7c9 100644 --- a/infra/README.md +++ b/infra/README.md @@ -2,7 +2,7 @@ Dockerfiles, `docker-compose.yml`, and deployment for a single on-prem box. -- One Dockerfile per service (api, frontend, aroma-sidecar if used) +- One Dockerfile per service (api, frontend) - Compose stack wiring the app, Postgres + pgvector, and the models - Single-box deploy — Docker Compose, not Kubernetes (deliberate: small, known user count, on-prem simplicity is part of the product) diff --git a/training/SETUP.md b/training/SETUP.md index 7ffbe2a..86471ec 100644 --- a/training/SETUP.md +++ b/training/SETUP.md @@ -6,118 +6,88 @@ > lives in `/api` + `/frontend`; it consumes the ONNX models this training > pipeline produces. Train here, ship there. -Target box: Dell R620, Linux, **no GPU** → CPU training. Install the CPU -PyTorch build. Everything below assumes a fresh user/env so it stays -isolated and reproducible. +Target box: Dell R620, Linux, **no GPU** → CPU only. The taste + demo stack is +light (RDKit, scikit-learn, skl2onnx) and installs cleanly — no version-fussy +deep-learning dependencies. Everything below assumes a fresh, isolated env. ## 0. Prereqs -- Miniforge/Mambaforge (mamba resolves the finicky deepchem stack far faster - than conda). https://github.com/conda-forge/miniforge +- Python 3.12 + venv, **or** Miniforge/Mambaforge — either works for this stack. + https://github.com/conda-forge/miniforge - git -## 1. Create an isolated user + workspace +## 1. Create an isolated workspace ```bash -sudo adduser flavordemo # optional but clean -sudo su - flavordemo -mkdir -p ~/odor-demo && cd ~/odor-demo +mkdir -p ~/flavormancer-train && cd ~/flavormancer-train +python3 -m venv .venv && source .venv/bin/activate +# (or: mamba create -n flavor python=3.12 -y && mamba activate flavor) ``` -## 2. Conda env -OpenPOM is built on DeepChem and is **version-fussy** — this is the part most -likely to eat an afternoon. Pin against OpenPOM's own requirements, don't let -it resolve against "latest". Start here, then reconcile with the repo: - +## 2. Install the stack (light — no GPU, no DeepChem) ```bash -mamba create -n odor python=3.10 -y -mamba activate odor - -# CPU PyTorch (NO cuda wheels — this box has no GPU) -pip install torch --index-url https://download.pytorch.org/whl/cpu - -mamba install -c conda-forge rdkit pandas scikit-learn numpy openpyxl pyarrow -y +pip install rdkit pandas scikit-learn numpy openpyxl pyarrow # openpyxl: read the ChemTastesDB .xlsx | pyarrow: read/write the .parquet files -pip install deepchem -pip install dgl dgllife # OpenPOM's GNN backend -pip install pubchempy # name -> SMILES, for the demo UX -pip install umap-learn # for the odor-space map later +pip install skl2onnx onnxruntime # export + self-validate the taste ONNX models +pip install pubchempy # name -> SMILES, for the demo UX +pip install umap-learn # for the flavor-space map ``` -## 3. Get OpenPOM + data -```bash -git clone https://github.com/BioMachineLearning/openpom.git -pip install -e ./openpom -# Reconcile any version conflicts NOW using openpom/requirements — this is the -# expected friction point. If torch/dgl/deepchem fight, match openpom's pins. - -# Leffingwell odor dataset (SMILES + multilabel descriptors) — AROMA head -git clone --depth 1 https://github.com/pyrfume/pyrfume-data.git -# dataset lives under pyrfume-data/leffingwell/ +> **Aroma is deferred** (see [`docs/AROMA.md`](../docs/AROMA.md)) — no commercially +> usable public odor data is good enough to train it. The heavy OpenPOM/DeepChem GNN +> stack is therefore **not installed here**. When licensed (PMP 2001) or customer +> data exists, add it then; the aroma pipeline (`build_aroma_dataset.py` + +> `train_aroma.py`) is committed and ready to run. -# Taste data — TASTE heads (sweet, bitter, umami) + sweetness intensity. -# 1) ChemTastesDB v2.0 — PRIMARY source, CC-BY-4.0, 4075 molecules, 10 classes: +## 3. Get the taste data +```bash +# ChemTastesDB v2.0 — PRIMARY source, CC-BY-4.0, ~4075 molecules, multi-class taste: curl -L -o ChemTastesDB_database.xlsx \ "https://zenodo.org/records/14963136/files/ChemTastesDB_database.xlsx?download=1" -# 2) cosylab/bittersweet — extra sweet/bitter (AGPL; optional, ignore their py2.7 code): -git clone --depth 1 https://github.com/cosylabiiit/bittersweet.git -# 3) SweetenersDB (Cheron 2017) — sweetness INTENSITY regressor. [OBTAIN] -# Pull the ~316-compound table (SMILES + relative-to-sucrose sweetness) from the -# paper's supplementary; save as sweeteners_db.csv. Optional; intensity head -# is skipped cleanly if absent. -# 4) (optional) more sources the build script will auto-merge IF present: -# flavordb_taste.csv — FlavorDB export: columns SMILES + taste [OBTAIN] -# umami_list.csv — UMP442 / BIOPEP-UWM umami SMILES [OBTAIN] -# Each is optional and skipped cleanly if the file isn't there. + +# SweetenersDB v2.0 — sweetness INTENSITY regressor. MIT, from the authors' own lab: +git clone --depth 1 https://github.com/chemosim-lab/SweetenersDB.git +# relative-to-sucrose sweetness (SMILES + logS); the build script reads it if present. ``` +> Other taste sources are auto-merged **only if present**, but note the licensing +> decisions in [`docs/SOURCES.md`](../docs/SOURCES.md): cosylab/bittersweet is AGPL +> and FlavorDB is NonCommercial — both are **off by default and not used**. -## 4. Sanity checks (do these before training anything) +## 4. Sanity check ```bash python - <<'PY' -import torch, deepchem, rdkit -print("torch", torch.__version__, "cuda?", torch.cuda.is_available()) # expect False on R620 +import rdkit, sklearn, skl2onnx from rdkit import Chem print("rdkit ok:", Chem.MolToSmiles(Chem.MolFromSmiles("c1ccccc1"))) PY ``` -`cuda? False` is correct and expected here — it'll train on CPU. -## 5. Build the merged taste dataset, then train +## 5. Build the dataset, then train ```bash -python build_taste_dataset.py # merges all sources -> taste_master.parquet -python train_odor.py # AROMA head (OpenPOM) — multi-hour / overnight CPU -python train_taste.py # TASTE heads (sklearn) — minutes, even merged +python build_taste_dataset.py # merges sources -> taste_master.parquet (+ sweet_intensity.parquet) +python train_taste.py # taste heads (sklearn) — minutes, even merged +python export_onnx.py # export taste models to ONNX (+ roundtrip self-validation) ``` -`build_taste_dataset.py` writes `taste_master.parquet` (multi-label: sweet/ -bitter/umami/sour/salty) and, if SweetenersDB is present, `sweet_intensity.parquet`. - `train_taste.py` trains one head per taste that clears the data threshold -(sweet/bitter/umami train; sour/salty auto-skip -> rule), plus a sweetness- -intensity regressor. Prints AUROC / R2 you can quote. Saves to `taste_models/`. +(sweet/bitter/umami train; sour also gets a small-data *indicative* head; salty +stays a validated rule), plus a sweetness-intensity regressor. Prints AUROC / R² +you can quote. Saves to `taste_models/`. -`train_odor.py` writes `odor_model/`, `embeddings.parquet`, `metrics.json`. - -## 6. One molecule's full flavor read (taste runs today; aroma once wired) +## 6. One molecule's full flavor read (taste runs today) ```bash python predict.py "OC(=O)CC(O)(CC(=O)O)C(=O)O" # citric acid → sour=True, low sweet/bitter ``` -Returns every taste head present + sweetness intensity + the sour flag. This is -exactly what the workbench screen calls per molecule. - -## Notes -- The hard part is the install pins, not the training. Budget for it. -- Keep raw data + checkpoints small; whole project should sit well under 40GB. -- When you move to serving: the model file goes into a light CPU FastAPI - container; CUDA never has to be containerized because inference is CPU work. - +Returns every taste head present + sweetness intensity + sour/salty flags + the +physicochemical / stability / chemesthesis / safety packs. `substitute()` provides +the nearest-neighbor substitution search. This is what the workbench screen calls. -## Aroma model (OpenPOM) — the smell half -The aroma GNN is the one version-fussy install. On the R620 (CPU): +## 7. Run the demo workbench +```bash +pip install fastapi "uvicorn[standard]" +uvicorn app:app --host 0.0.0.0 --port 8000 # then open http://:8000/ +``` -1. `pip install deepchem openpom` (pin compatible torch/rdkit/deepchem; see OpenPOM's README). -2. Get the training data: the OpenPOM repo's curated `curated_GS_LF_merged_4983.csv` - (easiest), or build from `pyrfume-data/leffingwell`. -3. `python train_odor.py` -> writes `./odor_model/` (model + tasks.json + metrics.json) - and `odor_embeddings.parquet` (for the odor map + substitution search). Overnight on CPU. -4. `predict.py` auto-loads `./odor_model/` when present; until then `predict_aroma()` - returns an honest 'not trained yet' instead of fabricating smells. -5. Product (.NET): try ONNX-exporting the GNN; if it won't export cleanly, run it behind - the thin Python aroma sidecar (the architecture's documented fallback). +## Notes +- The taste/demo stack is light and CPU-only — no CUDA, no DeepChem. +- Keep raw data + model artifacts small; the whole project sits well under a few GB. +- Aroma, when it comes, trains on licensed/customer data and exports to ONNX + (RandomForest) or runs behind a thin Python sidecar (OpenPOM GNN) — built then, + not now. See `docs/AROMA.md`. diff --git a/training/app.py b/training/app.py index 8ebae39..7986816 100644 --- a/training/app.py +++ b/training/app.py @@ -8,55 +8,27 @@ Endpoints: GET / -> the workbench UI (workbench.html) - POST /api/predict -> {smiles|name} -> full flavor read (predict.predict) - POST /api/neighbors -> {smiles|name, k} -> substitution candidates - -Substitution runs today on Morgan-fingerprint Tanimoto similarity over the -labeled molecules — runnable now, no aroma model required. It upgrades to the -learned aroma-embedding space (embeddings.parquet from train_odor.py) once that -exists: swap the fingerprint index below for those vectors + cosine distance. - -Auth / per-seat is stubbed (single open instance) for the demo. For deployment, -put this behind login + per-user history as in the plan; the prediction core -doesn't change. + POST /api/predict -> {smiles|name} -> full flavor read (predict.predict) + POST /api/neighbors -> {smiles|name, k} -> substitution search (predict.substitute) + +Both endpoints delegate to predict.py — one source of truth for the flavor read AND +the substitution search (Tanimoto/Morgan nearest-neighbor over the labeled molecules; +runnable today, no aroma model required). Auth / per-seat is stubbed (single open +instance) for the demo; deployment puts this behind login + per-user history, and the +prediction core doesn't change. """ from pathlib import Path -import pandas as pd from fastapi import FastAPI from fastapi.responses import HTMLResponse from pydantic import BaseModel from rdkit import Chem -from rdkit.Chem import DataStructs, rdFingerprintGenerator -import predict as P # reuse the unified flavor read +import predict as P # the unified flavor read + substitution search app = FastAPI(title="Flavor Workbench (demo)") -_FPS, _SMI, _KNOWN = [], [], [] -_MORGAN = rdFingerprintGenerator.GetMorganGenerator(radius=2, fpSize=2048) - - -def _build_index(): - path = Path("taste_master.parquet") - if not path.exists(): - print("note: taste_master.parquet not found — substitution search disabled") - return - m = pd.read_parquet(path) - basic = [t for t in ("sweet", "bitter", "umami", "sour", "salty") if t in m.columns] - for _, r in m.iterrows(): - mol = Chem.MolFromSmiles(r["smiles"]) - if mol is None: - continue - _FPS.append(_MORGAN.GetFingerprint(mol)) - _SMI.append(r["smiles"]) - _KNOWN.append([t for t in basic if r[t] == 1]) - print(f"substitution index built: {len(_FPS)} molecules") - - -_build_index() - def _resolve(text: str): """Accept a SMILES or a compound name; return canonical SMILES or None.""" @@ -90,22 +62,9 @@ def api_predict(q: Query): @app.post("/api/neighbors") def api_neighbors(q: Query): smi = _resolve(q.smiles) - if not smi or not _FPS: + if not smi: return {"neighbors": []} - mol = Chem.MolFromSmiles(smi) - fp = _MORGAN.GetFingerprint(mol) - sims = DataStructs.BulkTanimotoSimilarity(fp, _FPS) - self_smi = Chem.MolToSmiles(mol) - ranked = sorted(range(len(sims)), key=lambda i: sims[i], reverse=True) - out = [] - for i in ranked: - if _SMI[i] == self_smi: - continue - out.append({"smiles": _SMI[i], "similarity": round(sims[i], 3), - "known_tastes": _KNOWN[i]}) - if len(out) >= q.k: - break - return {"neighbors": out} + return P.substitute(smi, k=q.k) @app.get("/", response_class=HTMLResponse) diff --git a/training/predict.py b/training/predict.py index 4892b24..3b33d0b 100644 --- a/training/predict.py +++ b/training/predict.py @@ -2,7 +2,7 @@ predict.py — the unified flavor read the workbench screen renders. One molecule in, one dict out, combining whatever heads exist in taste_models/: - aroma : odor descriptors (OpenPOM model from train_odor.py) [VERIFY hook] + aroma : DEFERRED — honest 'not available' (no clean public data; see docs/AROMA.md) sweet/bitter/umami : probabilities 0-1 (trained heads, if present) sweet_intensity : ~relative-to-sucrose estimate (if regressor present) sour : bool + which acid group (RULE — acidic groups) @@ -585,58 +585,17 @@ def analyze_balance(ingredients): } -_AROMA_DIR = Path("odor_model") -_AROMA = None # lazy: (model, featurizer, tasks) | ("unavailable", reason, None) - - -def _load_aroma(): - global _AROMA - if _AROMA is not None: - return _AROMA - try: - if not (_AROMA_DIR.exists() and (_AROMA_DIR / "tasks.json").exists()): - raise FileNotFoundError("no trained ./odor_model (run train_odor.py)") - import json - from openpom.feat.graph_featurizer import GraphFeaturizer - from openpom.models.mpnn_pom import MPNNPOMModel - tasks = json.load(open(_AROMA_DIR / "tasks.json")) - model = MPNNPOMModel(n_tasks=len(tasks), mode="classification", - n_classes=1, model_dir=str(_AROMA_DIR), device="cpu") - model.restore() - _AROMA = (model, GraphFeaturizer(), tasks) - except Exception as e: # noqa: BLE001 - _AROMA = ("unavailable", str(e), None) - return _AROMA - - def predict_aroma(smiles, top_k=8): - """Odor-descriptor profile from the trained OpenPOM GNN. - - Loads ./odor_model if present; otherwise degrades honestly (does NOT fabricate - smells, and does NOT crash predict()). Train it with train_odor.py. - """ + """Aroma is deferred. No commercially-clean *public* odor data yields a working + model (see docs/AROMA.md), so rather than fabricate smells we return an honest + 'not available'. A real head gets trained on licensed (PMP 2001) or customer + odor data — OpenPOM's MIT architecture for large sets, RandomForest for small — + and wired in here then. predict() never calls this unless include_aroma=True.""" m = Chem.MolFromSmiles(smiles) if m is None: return {"error": f"unparseable SMILES: {smiles}"} - loaded = _load_aroma() - if loaded[0] == "unavailable": - return {"available": False, - "note": "aroma model not trained/loadable yet — run train_odor.py to build " - "./odor_model (needs DeepChem + OpenPOM on the R620)", - "detail": loaded[1]} - model, feat, tasks = loaded - try: - import deepchem as dc - X = feat.featurize([Chem.MolToSmiles(m)]) - scores = np.array(model.predict(dc.data.NumpyDataset(X))) - scores = scores[:, :, -1] if scores.ndim == 3 else scores - ranked = sorted(zip(tasks, [float(v) for v in scores.ravel()[:len(tasks)]]), - key=lambda t: t[1], reverse=True)[:top_k] - return {"available": True, - "descriptors": [{"odor": d, "score": round(s, 3)} for d, s in ranked], - "note": "OpenPOM GNN (principal-odor-map reimplementation), loaded from ./odor_model"} - except Exception as e: # noqa: BLE001 - return {"available": False, "note": "aroma model load ok but prediction failed", "detail": str(e)} + return {"available": False, + "note": "aroma deferred — needs licensed/customer odor data; see docs/AROMA.md"} def _taste_profile(out): diff --git a/training/train_odor.py b/training/train_odor.py deleted file mode 100644 index f52c8d2..0000000 --- a/training/train_odor.py +++ /dev/null @@ -1,129 +0,0 @@ -""" -train_odor.py — structure -> odor-descriptor model (OpenPOM / principal odor map) - -This is the AROMA half of Flavormancer: a graph neural network that predicts odor -descriptors ("floral", "green", "citrus", ...) from molecular structure. Unlike -taste, odor does NOT track simple fingerprints (near-identical molecules can smell -unrelated), so this uses OpenPOM's message-passing GNN — the open reimplementation -of the Lee et al. 2023 "principal odor map" (Science). See SOURCES.md for credit. - -HONESTY NOTES -- Runs on the R620, CPU only (no CUDA). Training is an overnight-ish job. -- DeepChem + OpenPOM are Python-only and version-fussy; this is the one piece that - stays Python at runtime (ONNX export of GNNs is unreliable -> aroma sidecar). -- The OpenPOM constructor exposes many architecture hyperparameters. The block - below mirrors the OpenPOM README example; if your installed version renames an - argument, reconcile against that version's example notebook. The overall flow — - load -> featurize -> multitask GNN -> score -> save -> embeddings — is stable. -- This script was authored against OpenPOM's documented API and is syntax-checked, - but the actual training run happens on the R620 where DeepChem is installed; it - has not been executed in the planning sandbox. -""" - -import json -from pathlib import Path - -import numpy as np -import pandas as pd -from sklearn.metrics import roc_auc_score - -import deepchem as dc -from openpom.feat.graph_featurizer import GraphFeaturizer, GraphConvConstants -from openpom.models.mpnn_pom import MPNNPOMModel - -OUT = Path("odor_model") # trained model + tasks land here; predict.py reads it -OUT.mkdir(exist_ok=True) -EMB_OUT = Path("odor_embeddings.parquet") # powers the odor-space map + substitution search - -# --------------------------------------------------------------------------- -# 1. Data. Easiest path: OpenPOM ships a curated, deduped dataset -# (curated_GS_LF_merged_4983.csv, ~138 odor descriptors). Point DATA_CSV at it. -# Alternative: build your own from pyrfume-data/leffingwell (see SOURCES.md). -# --------------------------------------------------------------------------- -DATA_CSV = "curated_GS_LF_merged_4983.csv" # from the OpenPOM repo's datasets/ -SMILES_FIELD = "nonStereoSMILES" # OpenPOM's curated-dataset SMILES column - -_df = pd.read_csv(DATA_CSV) -TASKS = [c for c in _df.columns if c not in (SMILES_FIELD, "descriptors")] -print(f"{len(_df)} molecules, {len(TASKS)} odor descriptors") - -# --------------------------------------------------------------------------- -# 2. Featurize with the OpenPOM graph featurizer + DeepChem CSV loader -# --------------------------------------------------------------------------- -featurizer = GraphFeaturizer() -loader = dc.data.CSVLoader(tasks=TASKS, feature_field=SMILES_FIELD, featurizer=featurizer) -dataset = loader.create_dataset(DATA_CSV) - -splitter = dc.splits.RandomStratifiedSplitter() -train_ds, test_ds = splitter.train_test_split(dataset, frac_train=0.85, seed=42) - -# class imbalance ratio per task (odor labels are sparse) — OpenPOM uses this -train_ratios = [] -for j in range(len(TASKS)): - col = train_ds.y[:, j] - pos = max(int(col.sum()), 1) - train_ratios.append(float((len(col) - pos) / pos)) - -# --------------------------------------------------------------------------- -# 3. Train the message-passing GNN (CPU). Hyperparameters mirror the OpenPOM -# example; reconcile names with your installed version if needed. -# --------------------------------------------------------------------------- -model = MPNNPOMModel( - n_tasks=len(TASKS), - batch_size=128, - learning_rate=1e-3, - class_imbalance_ratio=train_ratios, - loss_aggr_type="sum", - node_out_feats=100, - edge_hidden_feats=75, - edge_out_feats=100, - num_step_message_passing=5, - mpnn_residual=True, - message_aggregator_type="sum", - mode="classification", - number_atom_features=GraphConvConstants.ATOM_FDIM, - number_bond_features=GraphConvConstants.BOND_FDIM, - n_classes=1, - nb_layers=2, - nb_timesteps=2, - self_loop=False, - model_dir=str(OUT), - device="cpu", # R620 has no GPU -) - -NB_EPOCH = 50 -model.fit(train_ds, nb_epoch=NB_EPOCH) -model.save_checkpoint(model_dir=str(OUT)) -json.dump(TASKS, open(OUT / "tasks.json", "w")) -print(f"saved model + {len(TASKS)} tasks to {OUT}/") - -# --------------------------------------------------------------------------- -# 4. Score — per-descriptor AUROC (the numbers you quote in the pitch) -# --------------------------------------------------------------------------- -y_pred = np.array(model.predict(test_ds)) -# normalize to [n_samples, n_tasks] positive-class scores -if y_pred.ndim == 3: - y_pred = y_pred[:, :, -1] -aurocs = {} -for j, label in enumerate(TASKS): - yt = test_ds.y[:, j] - if yt.sum() == 0 or yt.sum() == len(yt): - continue - try: - aurocs[label] = float(roc_auc_score(yt, y_pred[:, j])) - except ValueError: - continue -json.dump(aurocs, open(OUT / "metrics.json", "w"), indent=2) -if aurocs: - print(f"mean AUROC over {len(aurocs)} scorable descriptors: {np.mean(list(aurocs.values())):.3f}") - -# --------------------------------------------------------------------------- -# 5. Embeddings -> pgvector substitution search + UMAP odor map -# OpenPOM exposes a learned-embedding method; name may vary by version. -# --------------------------------------------------------------------------- -try: - emb = model.predict_embedding(dataset) # [VERIFY] method name in your version - pd.DataFrame(np.array(emb)).assign(smiles=_df[SMILES_FIELD].values).to_parquet(EMB_OUT) - print(f"saved odor embeddings -> {EMB_OUT}") -except Exception as e: # noqa: BLE001 - print(f"(embedding export skipped — confirm the embedding method name: {e})") diff --git a/training/train_taste.py b/training/train_taste.py index 0d3c810..72e0a50 100644 --- a/training/train_taste.py +++ b/training/train_taste.py @@ -8,7 +8,7 @@ deterministic cross-check. Add more data and the heads sharpen on the next run. Even with everything merged this trains in minutes on the R620 CPU. The -multi-day budget is the aroma model (train_odor.py), not this. +multi-day budget is the aroma model (deferred — see docs/AROMA.md), not this. Run order: python build_taste_dataset.py # writes taste_master.parquet (+ sweet_intensity.parquet)