Skip to content

feat(data): add SweetenersDB → sweetness-intensity regressor #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rvnminers-A-and-N merged 1 commit into from
Jun 26, 2026
Merged

feat(data): add SweetenersDB → sweetness-intensity regressor #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions docs/DATA-SOURCES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ What feeds the models, what each source unlocks, its license, and how to get it.
| **cosylab/bittersweet** | more sweet/bitter volume | AGPL-3.0 | ⬜ to get | `git clone github.com/cosylabiiit/bittersweet` → `bittersweet/data/*.tsv`. AGPL: keep a CC-BY-clean build with `INCLUDE_COSYLAB=False`, or opt in deliberately. |
| **FlavorDB** | ~25k molecules taste+odor + natural-source mapping | confirm terms | ⬜ to get | REST/JSON API at `cosylab.iiitd.edu.in/flavordb` → `flavordb_taste.csv`. Map taste fields; **[VERIFY] columns** before trusting. |
| **UMP442 / BIOPEP-UWM** | more umami examples (only 283 now) | confirm terms | ⬜ to get | BIOPEP-UWM umami DB (form-driven) → `umami_list.csv`. |
| **SweetenersDB (Chéron 2017)** | the sweetness-**intensity** regressor (currently skipped) | publisher supp. | ⬜ to get | paper supplementary (~316 compounds, relative-to-sucrose) → `sweeteners_db.csv`. Likely **paywalled → manual grab**. |
| **SweetenersDB v2.0** | the sweetness-**intensity** regressor | **MIT** | ✅ in use | direct CSV from `github.com/chemosim-lab/SweetenersDB` (`SweetenersDB_v2.0.csv`, 316 cmpds, `logSw` column). R²≈0.82. |
| **Pyrfume / Leffingwell** | the **aroma model** (OpenPOM) — issues #17 / #18 | per-set; confirm | ⬜ to get | `git clone github.com/pyrfume/pyrfume-data`; Leffingwell odor set. Separate, version-fragile effort. |
| **FEMA GRAS / FDA SAF** | GRAS cross-reference + dosing/OAV lookups | gov public / FEMA | ⬜ to get | FDA "Substances Added to Food" (public domain) → `gras_reference.parquet`; FEMA use-level PDFs → `properties.parquet`. **Verify every scraped dosing number.** |

Expand All @@ -27,6 +27,7 @@ What feeds the models, what each source unlocks, its license, and how to get it.
- **Column verification.** Each new source's column names must be checked against the
loaders (the `[VERIFY]` markers) — see issue **#25**. ChemTastesDB's mapping is
resolved (both the coarse `Class taste` and the granular `Taste` columns are parsed).
- **Suggested priority.** (1) `Taste`-column mining — **done** ✅ → (2) cosylab /
FlavorDB for taste volume → (3) SweetenersDB for the intensity head → (4)
Pyrfume / Leffingwell for the aroma model.
- **Suggested priority.** (1) `Taste`-column mining — **done** ✅ → (2) SweetenersDB
intensity head — **done** ✅ → (3) Pyrfume / Leffingwell for the aroma model →
(4) cosylab for taste volume (AGPL — pending decision). **FlavorDB is
CC BY-NC-SA (NonCommercial) — incompatible with a commercial product; skip.**
4 changes: 3 additions & 1 deletion training/build_taste_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,9 @@ def build_intensity(path):
return None
df = pd.read_csv(path)
sc = _find(df, ["smiles", "canonical smiles"])
vc = _find(df, ["relative_sweetness", "log_sweetness", "sweetness", "rs"])
# SweetenersDB v2.0 (MIT) names it 'logSw' (log sweetness vs sucrose, ~316 cmpds);
# also accept the generic names. [VERIFIED against SweetenersDB_v2.0.csv, 2026-06-26.]
vc = _find(df, ["log_sweetness", "logsw", "relative_sweetness", "sweetness", "rs"])
rows = []
for _, r in df.iterrows():
_, cs = canon(r[sc])
Expand Down
Loading