Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,12 @@ reference table), and **qualitative** (a class/flag, not a number).

Python trains the models offline; a .NET application serves them at runtime. The
training language is a build-time detail — nothing at runtime depends on Python
(aside from an optional aroma sidecar if the GNN won't export cleanly).
(aside from a possible aroma sidecar later — aroma is deferred; see docs/AROMA.md).

```
data sources ─► Python training (build-time) ─► ONNX (taste) ──┐
RDKit · scikit-learn · OpenPOM aroma model ─┐
RDKit · scikit-learn
React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pgvector
Expand All @@ -55,7 +55,7 @@ React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pg

| Layer | Technology |
|-------|-----------|
| Model training (build-time) | Python · RDKit · scikit-learn · OpenPOM/DeepChem |
| Model training (build-time) | Python · RDKit · scikit-learn · skl2onnx |
| Model handoff | ONNX |
| App / API | ASP.NET Core (C#) |
| ML serving | ONNX Runtime, in-process in .NET |
Expand All @@ -67,7 +67,6 @@ React workbench ◄─ JSON API ◄─ ASP.NET Core + ONNX Runtime + Postgres/pg

```
training/ Python — dataset build + model training (build-time)
aroma-sidecar/ Python — thin aroma inference service (only if needed)
api/ ASP.NET Core — app, auth, endpoints, ONNX serving
frontend/ React — the workbench UI
infra/ Dockerfiles, docker-compose.yml, deploy
Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ PRs reference their milestone (e.g. *"Part of M2"*), and progress is tracked in
| **M1 — Training pipeline (Python)** | Dataset build, taste training, prediction core, ONNX export, demo workbench | ✅ Done |
| **M2 — .NET API + ONNX serving** | ASP.NET Core skeleton, load taste ONNX, `/predict`, in-process ONNX Runtime | 🔜 Next |
| **M3 — React workbench** | React UI against the fixed JSON contract; taste meters with confidence tags | Planned |
| **M4 — Aroma model** | Train OpenPOM on Leffingwell; ONNX-export or sidecar; wire `predict_aroma()` | Planned |
| **M4 — Aroma model** | Train on licensed (PMP 2001) / customer odor data; wire `predict_aroma()` — **deferred, see [docs/AROMA.md](docs/AROMA.md)** | Deferred |
| **M5 — Packaging** | Dockerfiles, `docker-compose`, Postgres + pgvector, single-box deploy | Planned |
| **M6 — Pilot-ready** | Auth + per-seat, pgvector substitution search, polish, demo script | Planned |

Expand Down
8 changes: 0 additions & 8 deletions aroma-sidecar/README.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ artifact, not bolted onto this.)
## Status: built vs to-build

**Built (Track A / training, reusable by Track B):**
- `build_taste_dataset.py`, `train_taste.py`, `train_odor.py` (scaffold), `predict.py`
- `build_taste_dataset.py`, `train_taste.py`, `predict.py` (aroma training added when clean data exists — see [`AROMA.md`](AROMA.md))
- `app.py` + `workbench.html` — the Python/HTML *demo* serving prototype

**Safety (defensive, caution-only — built into predict.py):**
Expand Down
7 changes: 5 additions & 2 deletions docs/AROMA.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,11 @@ We do **not** ship a negative-R² model — it would output confident, wrong sme
a flavor chemist would catch it instantly (worse than nothing). `predict_aroma()`
returns an honest "not available" marker, and **we lead with the taste engine**.

The **OpenPOM scaffold (`training/train_odor.py`) is kept** as the aroma engine for
when clean fuel exists:
We carry **no dead scaffold** — the architecture decision lives here, not in an
unrunnable stub. **OpenPOM is the chosen architecture** (MIT, re-addable in an
afternoon) for large sets; RandomForest suffices for small ones, exactly as the
taste heads and the `train_aroma.py` evaluation already demonstrate. We build the
training script when clean fuel exists:
1. **License PMP 2001** (~$2,775, Leffingwell & Associates) → re-curate → train.
2. **A customer's own odor data** (the paid pilot) → train on-prem.
3. A future open *expert-labeled* dataset, if one emerges.
Expand Down
4 changes: 2 additions & 2 deletions docs/SOURCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ record of provenance, not legal advice. Get an IP/OSS-license review before ship

**Cheminformatics / ML (training side — Python)**
- **RDKit** — molecular parsing, fingerprints, descriptors, SMARTS, InChIKeys. (BSD-3-Clause.)
- **DeepChem** — GNN training framework under OpenPOM. (MIT.) Python-only; the one runtime-Python piece.
- **DeepChem** — GNN training framework under OpenPOM. (MIT.) For the **deferred** aroma model only; not currently installed or used.
- **OpenPOM** (BioMachineLearning/openpom) — the message-passing GNN for odor; open reimplementation of the principal-odor-map work. (MIT — confirm.) The aroma model is theirs in spirit.
- **scikit-learn** — RandomForest taste heads + sweetness-intensity regressor. (BSD-3-Clause.)
- **PyTorch** — tensor/Autograd backend under DeepChem. (BSD-style.)
Expand All @@ -25,7 +25,7 @@ record of provenance, not legal advice. Get an IP/OSS-license review before ship
- **ASP.NET Core / .NET** — the app/API backbone. (MIT.)
- **React** — frontend. (MIT.)
- **PostgreSQL** + **pgvector** — DB + embedding/substitution search. (PostgreSQL License.)
- **FastAPI** — the Track-A demo serving layer (+ aroma sidecar if needed). (MIT.)
- **FastAPI** — the Track-A demo serving layer. (MIT.)
- **Docker** — single-box deployment. (Apache-2.0.)

---
Expand Down
2 changes: 1 addition & 1 deletion infra/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Dockerfiles, `docker-compose.yml`, and deployment for a single on-prem box.

- One Dockerfile per service (api, frontend, aroma-sidecar if used)
- One Dockerfile per service (api, frontend)
- Compose stack wiring the app, Postgres + pgvector, and the models
- Single-box deploy — Docker Compose, not Kubernetes (deliberate: small, known
user count, on-prem simplicity is part of the product)
136 changes: 53 additions & 83 deletions training/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,118 +6,88 @@
> lives in `/api` + `/frontend`; it consumes the ONNX models this training
> pipeline produces. Train here, ship there.

Target box: Dell R620, Linux, **no GPU** → CPU training. Install the CPU
PyTorch build. Everything below assumes a fresh user/env so it stays
isolated and reproducible.
Target box: Dell R620, Linux, **no GPU** → CPU only. The taste + demo stack is
light (RDKit, scikit-learn, skl2onnx) and installs cleanly — no version-fussy
deep-learning dependencies. Everything below assumes a fresh, isolated env.

## 0. Prereqs
- Miniforge/Mambaforge (mamba resolves the finicky deepchem stack far faster
than conda). https://github.com/conda-forge/miniforge
- Python 3.12 + venv, **or** Miniforge/Mambaforge — either works for this stack.
https://github.com/conda-forge/miniforge
- git

## 1. Create an isolated user + workspace
## 1. Create an isolated workspace
```bash
sudo adduser flavordemo # optional but clean
sudo su - flavordemo
mkdir -p ~/odor-demo && cd ~/odor-demo
mkdir -p ~/flavormancer-train && cd ~/flavormancer-train
python3 -m venv .venv && source .venv/bin/activate
# (or: mamba create -n flavor python=3.12 -y && mamba activate flavor)
```

## 2. Conda env
OpenPOM is built on DeepChem and is **version-fussy** — this is the part most
likely to eat an afternoon. Pin against OpenPOM's own requirements, don't let
it resolve against "latest". Start here, then reconcile with the repo:

## 2. Install the stack (light — no GPU, no DeepChem)
```bash
mamba create -n odor python=3.10 -y
mamba activate odor

# CPU PyTorch (NO cuda wheels — this box has no GPU)
pip install torch --index-url https://download.pytorch.org/whl/cpu

mamba install -c conda-forge rdkit pandas scikit-learn numpy openpyxl pyarrow -y
pip install rdkit pandas scikit-learn numpy openpyxl pyarrow
# openpyxl: read the ChemTastesDB .xlsx | pyarrow: read/write the .parquet files
pip install deepchem
pip install dgl dgllife # OpenPOM's GNN backend
pip install pubchempy # name -> SMILES, for the demo UX
pip install umap-learn # for the odor-space map later
pip install skl2onnx onnxruntime # export + self-validate the taste ONNX models
pip install pubchempy # name -> SMILES, for the demo UX
pip install umap-learn # for the flavor-space map
```

## 3. Get OpenPOM + data
```bash
git clone https://github.com/BioMachineLearning/openpom.git
pip install -e ./openpom
# Reconcile any version conflicts NOW using openpom/requirements — this is the
# expected friction point. If torch/dgl/deepchem fight, match openpom's pins.

# Leffingwell odor dataset (SMILES + multilabel descriptors) — AROMA head
git clone --depth 1 https://github.com/pyrfume/pyrfume-data.git
# dataset lives under pyrfume-data/leffingwell/
> **Aroma is deferred** (see [`docs/AROMA.md`](../docs/AROMA.md)) — no commercially
> usable public odor data is good enough to train it. The heavy OpenPOM/DeepChem GNN
> stack is therefore **not installed here**. When licensed (PMP 2001) or customer
> data exists, add it then; the aroma pipeline (`build_aroma_dataset.py` +
> `train_aroma.py`) is committed and ready to run.

# Taste data — TASTE heads (sweet, bitter, umami) + sweetness intensity.
# 1) ChemTastesDB v2.0 — PRIMARY source, CC-BY-4.0, 4075 molecules, 10 classes:
## 3. Get the taste data
```bash
# ChemTastesDB v2.0 — PRIMARY source, CC-BY-4.0, ~4075 molecules, multi-class taste:
curl -L -o ChemTastesDB_database.xlsx \
"https://zenodo.org/records/14963136/files/ChemTastesDB_database.xlsx?download=1"
# 2) cosylab/bittersweet — extra sweet/bitter (AGPL; optional, ignore their py2.7 code):
git clone --depth 1 https://github.com/cosylabiiit/bittersweet.git
# 3) SweetenersDB (Cheron 2017) — sweetness INTENSITY regressor. [OBTAIN]
# Pull the ~316-compound table (SMILES + relative-to-sucrose sweetness) from the
# paper's supplementary; save as sweeteners_db.csv. Optional; intensity head
# is skipped cleanly if absent.
# 4) (optional) more sources the build script will auto-merge IF present:
# flavordb_taste.csv — FlavorDB export: columns SMILES + taste [OBTAIN]
# umami_list.csv — UMP442 / BIOPEP-UWM umami SMILES [OBTAIN]
# Each is optional and skipped cleanly if the file isn't there.

# SweetenersDB v2.0 — sweetness INTENSITY regressor. MIT, from the authors' own lab:
git clone --depth 1 https://github.com/chemosim-lab/SweetenersDB.git
# relative-to-sucrose sweetness (SMILES + logS); the build script reads it if present.
```
> Other taste sources are auto-merged **only if present**, but note the licensing
> decisions in [`docs/SOURCES.md`](../docs/SOURCES.md): cosylab/bittersweet is AGPL
> and FlavorDB is NonCommercial — both are **off by default and not used**.

## 4. Sanity checks (do these before training anything)
## 4. Sanity check
```bash
python - <<'PY'
import torch, deepchem, rdkit
print("torch", torch.__version__, "cuda?", torch.cuda.is_available()) # expect False on R620
import rdkit, sklearn, skl2onnx
from rdkit import Chem
print("rdkit ok:", Chem.MolToSmiles(Chem.MolFromSmiles("c1ccccc1")))
PY
```
`cuda? False` is correct and expected here — it'll train on CPU.

## 5. Build the merged taste dataset, then train
## 5. Build the dataset, then train
```bash
python build_taste_dataset.py # merges all sources -> taste_master.parquet
python train_odor.py # AROMA head (OpenPOM) — multi-hour / overnight CPU
python train_taste.py # TASTE heads (sklearn) — minutes, even merged
python build_taste_dataset.py # merges sources -> taste_master.parquet (+ sweet_intensity.parquet)
python train_taste.py # taste heads (sklearn) — minutes, even merged
python export_onnx.py # export taste models to ONNX (+ roundtrip self-validation)
```
`build_taste_dataset.py` writes `taste_master.parquet` (multi-label: sweet/
bitter/umami/sour/salty) and, if SweetenersDB is present, `sweet_intensity.parquet`.

`train_taste.py` trains one head per taste that clears the data threshold
(sweet/bitter/umami train; sour/salty auto-skip -> rule), plus a sweetness-
intensity regressor. Prints AUROC / R2 you can quote. Saves to `taste_models/`.
(sweet/bitter/umami train; sour also gets a small-data *indicative* head; salty
stays a validated rule), plus a sweetness-intensity regressor. Prints AUROC / R²
you can quote. Saves to `taste_models/`.

`train_odor.py` writes `odor_model/`, `embeddings.parquet`, `metrics.json`.

## 6. One molecule's full flavor read (taste runs today; aroma once wired)
## 6. One molecule's full flavor read (taste runs today)
```bash
python predict.py "OC(=O)CC(O)(CC(=O)O)C(=O)O" # citric acid → sour=True, low sweet/bitter
```
Returns every taste head present + sweetness intensity + the sour flag. This is
exactly what the workbench screen calls per molecule.

## Notes
- The hard part is the install pins, not the training. Budget for it.
- Keep raw data + checkpoints small; whole project should sit well under 40GB.
- When you move to serving: the model file goes into a light CPU FastAPI
container; CUDA never has to be containerized because inference is CPU work.

Returns every taste head present + sweetness intensity + sour/salty flags + the
physicochemical / stability / chemesthesis / safety packs. `substitute()` provides
the nearest-neighbor substitution search. This is what the workbench screen calls.

## Aroma model (OpenPOM) — the smell half
The aroma GNN is the one version-fussy install. On the R620 (CPU):
## 7. Run the demo workbench
```bash
pip install fastapi "uvicorn[standard]"
uvicorn app:app --host 0.0.0.0 --port 8000 # then open http://<r620-ip>:8000/
```

1. `pip install deepchem openpom` (pin compatible torch/rdkit/deepchem; see OpenPOM's README).
2. Get the training data: the OpenPOM repo's curated `curated_GS_LF_merged_4983.csv`
(easiest), or build from `pyrfume-data/leffingwell`.
3. `python train_odor.py` -> writes `./odor_model/` (model + tasks.json + metrics.json)
and `odor_embeddings.parquet` (for the odor map + substitution search). Overnight on CPU.
4. `predict.py` auto-loads `./odor_model/` when present; until then `predict_aroma()`
returns an honest 'not trained yet' instead of fabricating smells.
5. Product (.NET): try ONNX-exporting the GNN; if it won't export cleanly, run it behind
the thin Python aroma sidecar (the architecture's documented fallback).
## Notes
- The taste/demo stack is light and CPU-only — no CUDA, no DeepChem.
- Keep raw data + model artifacts small; the whole project sits well under a few GB.
- Aroma, when it comes, trains on licensed/customer data and exports to ONNX
(RandomForest) or runs behind a thin Python sidecar (OpenPOM GNN) — built then,
not now. See `docs/AROMA.md`.
63 changes: 11 additions & 52 deletions training/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,55 +8,27 @@

Endpoints:
GET / -> the workbench UI (workbench.html)
POST /api/predict -> {smiles|name} -> full flavor read (predict.predict)
POST /api/neighbors -> {smiles|name, k} -> substitution candidates

Substitution runs today on Morgan-fingerprint Tanimoto similarity over the
labeled molecules — runnable now, no aroma model required. It upgrades to the
learned aroma-embedding space (embeddings.parquet from train_odor.py) once that
exists: swap the fingerprint index below for those vectors + cosine distance.

Auth / per-seat is stubbed (single open instance) for the demo. For deployment,
put this behind login + per-user history as in the plan; the prediction core
doesn't change.
POST /api/predict -> {smiles|name} -> full flavor read (predict.predict)
POST /api/neighbors -> {smiles|name, k} -> substitution search (predict.substitute)

Both endpoints delegate to predict.py — one source of truth for the flavor read AND
the substitution search (Tanimoto/Morgan nearest-neighbor over the labeled molecules;
runnable today, no aroma model required). Auth / per-seat is stubbed (single open
instance) for the demo; deployment puts this behind login + per-user history, and the
prediction core doesn't change.
"""

from pathlib import Path

import pandas as pd
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from pydantic import BaseModel
from rdkit import Chem
from rdkit.Chem import DataStructs, rdFingerprintGenerator

import predict as P # reuse the unified flavor read
import predict as P # the unified flavor read + substitution search

app = FastAPI(title="Flavor Workbench (demo)")

_FPS, _SMI, _KNOWN = [], [], []
_MORGAN = rdFingerprintGenerator.GetMorganGenerator(radius=2, fpSize=2048)


def _build_index():
path = Path("taste_master.parquet")
if not path.exists():
print("note: taste_master.parquet not found — substitution search disabled")
return
m = pd.read_parquet(path)
basic = [t for t in ("sweet", "bitter", "umami", "sour", "salty") if t in m.columns]
for _, r in m.iterrows():
mol = Chem.MolFromSmiles(r["smiles"])
if mol is None:
continue
_FPS.append(_MORGAN.GetFingerprint(mol))
_SMI.append(r["smiles"])
_KNOWN.append([t for t in basic if r[t] == 1])
print(f"substitution index built: {len(_FPS)} molecules")


_build_index()


def _resolve(text: str):
"""Accept a SMILES or a compound name; return canonical SMILES or None."""
Expand Down Expand Up @@ -90,22 +62,9 @@ def api_predict(q: Query):
@app.post("/api/neighbors")
def api_neighbors(q: Query):
smi = _resolve(q.smiles)
if not smi or not _FPS:
if not smi:
return {"neighbors": []}
mol = Chem.MolFromSmiles(smi)
fp = _MORGAN.GetFingerprint(mol)
sims = DataStructs.BulkTanimotoSimilarity(fp, _FPS)
self_smi = Chem.MolToSmiles(mol)
ranked = sorted(range(len(sims)), key=lambda i: sims[i], reverse=True)
out = []
for i in ranked:
if _SMI[i] == self_smi:
continue
out.append({"smiles": _SMI[i], "similarity": round(sims[i], 3),
"known_tastes": _KNOWN[i]})
if len(out) >= q.k:
break
return {"neighbors": out}
return P.substitute(smi, k=q.k)


@app.get("/", response_class=HTMLResponse)
Expand Down
Loading
Loading