Skip to content

Commit 479f28a

Browse files
feat: calibrate every head, unify the chip language, fix the modal layout
Closes #265, #266, #267, #268. CALIBRATION EVERYWHERE (#265). Taste and Tox21 still ran on a flat 0.5 with only AUROC shown — the exact reporting gap #261 closed for aroma and mouthfeel. Both now use the same out-of-fold calibration with a 50% precision floor, and /api/heads publishes threshold, precision, recall and confident_capable for all four modalities. The taste heads were mildly shy, as expected with hundreds of positives each (sweet 0.33, umami 0.33, bitter 0.51), and all six clear the floor at 0.50-0.87. The Tox21 result is the one that matters: every one of the twelve calibrated UPWARD — NR-AR to 0.69, NR-AR-LBD 0.65, NR-ER 0.63 — which means a flat 0.5 had them OVER-flagging. That is the opposite direction from the thin aroma heads and the more dangerous one, because a caution flag that cries wolf teaches people to ignore the flags that matter. CHIP LANGUAGE (#266). Flavor, note, taste and mouthfeel chips had each been given a different border STYLE at a different time — dashed, dotted, thicker, plus per-taste text recolouring — so they read as four accidents rather than one system. They now share a pill, a size and a border weight, and carry identity in a leading dot in the family's own hue. Each studio section also gained a plain-language paragraph on what that dimension IS and why it matters: a flavor is the whole percept, a note is what the nose reads, taste is the basics the tongue reads, mouthfeel is touch reported by the trigeminal nerve. Menthol is cold without a temperature change; that distinction is load-bearing and nothing in the UI had ever said it. ALL MODALITIES ON CARDS (#267). Substitute and neighbor cards showed taste and aroma but not mouthfeel. Read straight off the profile matrix, whose mouthfeel columns were already there, so it costs no extra inference — and each sensation must clear its own calibrated threshold, so a card never shows something the modal would call indicative. MODAL LAYOUT (#268). Two real bugs, both measured rather than guessed: - Mobile: the swap grid used `1fr`, which is minmax(auto,1fr), and `auto` will not shrink below min-content — so the substitutes and neighbors cards rendered 468px wide inside a 358px modal. minmax(0,1fr) fixes it; .modal-top already had this and .swap-grid did not. - Desktop: #results is a flex column whose 14px gap separates the cards, but the render path set an inline display:block on it, and block boxes ignore `gap`. The Heads card had been sitting flush against the Substitutes card below it. Clearing the inline style lets the stylesheet win. Verified with a Playwright probe at 390px and 1440px: no card overflows its container at either size, and no two cards are within 8px of each other on desktop. Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>
1 parent 2bc9540 commit 479f28a

4 files changed

Lines changed: 121 additions & 30 deletions

File tree

training/predict.py

Lines changed: 42 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1218,19 +1218,29 @@ def predict_mouthfeel(mol):
12181218
}
12191219

12201220

1221-
def predict_tox(mol, threshold=0.5):
1221+
def predict_tox(mol, threshold=None):
12221222
"""Caution-only in-vitro tox-assay activity (Tox21 models). INDICATIVE flags for
1223-
review — NEVER a toxicity/safety determination. Honest/empty if heads untrained."""
1223+
review — NEVER a toxicity/safety determination. Honest/empty if heads untrained.
1224+
1225+
Each assay fires at its OWN calibrated threshold, and calibration matters more here than
1226+
anywhere else in the app: assay actives are rare, so a flat 0.5 made several heads
1227+
over-flag. Every one of the twelve calibrated UPWARD (NR-AR to 0.69, NR-ER to 0.63) — the
1228+
opposite direction from the thin aroma heads. A caution flag that cries wolf is worse than
1229+
no flag, because it teaches people to ignore the ones that matter.
1230+
"""
12241231
if not _TOX_MODELS:
12251232
return {"available": False,
12261233
"note": "tox heads not trained — run train_tox.py (Tox21, public domain)"}
12271234
x = _fp(mol)
12281235
assays = []
12291236
for name, clf in sorted(_TOX_MODELS.items()):
12301237
p = round(float(clf.predict_proba(x)[0, 1]), 3)
1238+
thr = _head_threshold(_TOX_META, name, threshold)
12311239
assays.append({"assay": name, "meaning": _TOX_MEANING.get(name, name), "probability": p,
1240+
"threshold": thr, "flagged": p >= thr,
1241+
"precision": _TOX_META.get(name, {}).get("cv_precision"),
12321242
"auroc": _TOX_META.get(name, {}).get("auroc")})
1233-
flags = [a["assay"] for a in assays if a["probability"] >= threshold]
1243+
flags = [a["assay"] for a in assays if a["flagged"]]
12341244
return {"available": True, "assays": assays, "flags": flags,
12351245
"note": "INDICATIVE in-vitro tox-assay activity (Tox21 RandomForest heads) — "
12361246
"caution-only, NOT a toxicity/safety determination; confirm with a toxicologist."}
@@ -1310,13 +1320,13 @@ def _aroma(a):
13101320
"desc": AROMA_DESC.get(h), **_cal(_MOUTHFEEL_META, h)}
13111321
for h in mouthfeel_heads]
13121322
return {
1313-
# taste heads keep a flat 0.5: hundreds of positives each, so they were never shy
1314-
"taste": [{"head": t, "auroc": _taste_auroc(t), "threshold": 0.5,
1315-
"confident_capable": True} for t in taste_heads],
1323+
"taste": [{"head": t, "auroc": _taste_auroc(t), **_cal(_TASTE_META, t)}
1324+
for t in taste_heads],
13161325
"aroma": [_aroma(a) for a in aroma_heads],
13171326
"mouthfeel": mouthfeel,
13181327
"safety": [{"head": t, "auroc": _TOX_META.get(t, {}).get("auroc"),
1319-
"meaning": _TOX_MEANING.get(t, t)} for t in sorted(_TOX_MODELS)],
1328+
"meaning": _TOX_MEANING.get(t, t), **_cal(_TOX_META, t)}
1329+
for t in sorted(_TOX_MODELS)],
13201330
}
13211331

13221332

@@ -1473,6 +1483,26 @@ def _predicted_tastes_at(profiles, i):
14731483
return [t for s, t in scored[:3] if s >= 0.2]
14741484

14751485

1486+
def _predicted_mouthfeel_at(profiles, i):
1487+
"""The MOUTHFEEL read for reference-set row i, straight off the profile matrix — no extra
1488+
inference, the columns are already there. Each sensation must clear its OWN calibrated
1489+
threshold (and be confident-capable), so a card never shows a sensation the modal would
1490+
call indicative. Returns the firing sensations, strongest first."""
1491+
if profiles is None:
1492+
return []
1493+
taste_heads, aroma_heads, mouth_heads = _profile_heads()
1494+
base = len(taste_heads) + len(aroma_heads) # mouthfeel columns follow taste then aroma
1495+
row, out = profiles[i], []
1496+
for j, name in enumerate(mouth_heads):
1497+
col = base + j
1498+
if col >= len(row):
1499+
break
1500+
score = float(row[col])
1501+
if score >= _head_threshold(_MOUTHFEEL_META, name) and _head_capable(_MOUTHFEEL_META, name):
1502+
out.append((score, name))
1503+
return [n for _, n in sorted(out, reverse=True)]
1504+
1505+
14761506
def structural_neighbors(smiles: str, k: int = 8, min_similarity: float = 0.0) -> dict:
14771507
"""STRUCTURAL neighbors: the k labeled molecules most structurally similar to the query
14781508
(Tanimoto over Morgan fingerprints), each with its known tastes. Structural look-alikes —
@@ -1504,6 +1534,7 @@ def structural_neighbors(smiles: str, k: int = 8, min_similarity: float = 0.0) -
15041534
neighbors.append({"smiles": smis[i], "similarity": round(float(sims[i]), 3),
15051535
"known_tastes": tastes[i],
15061536
"predicted_tastes": _predicted_tastes_at(profiles, i),
1537+
"mouthfeel": _predicted_mouthfeel_at(profiles, i),
15071538
# confident aromas precomputed once in the index — reused so the
15081539
# endpoint never re-runs the 24 aroma heads per neighbor (8x ~1.3s saved)
15091540
"aromas": _aromas[i] if i < len(_aromas) else []})
@@ -1545,6 +1576,7 @@ def substitutes(smiles: str, k: int = 8, min_match: float = 0.0) -> dict:
15451576
continue
15461577
subs.append({"smiles": smis[i], "profile_match": round(float(sims[i]), 3),
15471578
"known_tastes": tastes[i], "predicted_tastes": _predicted_tastes_at(profiles, i),
1579+
"mouthfeel": _predicted_mouthfeel_at(profiles, i),
15481580
"aromas": aromas[i] if i < len(aromas) else []})
15491581
if len(subs) >= k:
15501582
break
@@ -1584,6 +1616,7 @@ def mixture_to_molecule(smiles_list: list, weights: list | None = None, k: int =
15841616
continue
15851617
out.append({"smiles": smis[i], "profile_match": round(float(sims[i]), 3),
15861618
"known_tastes": tastes[i], "predicted_tastes": _predicted_tastes_at(profiles, i),
1619+
"mouthfeel": _predicted_mouthfeel_at(profiles, i),
15871620
"aromas": aromas[i] if i < len(aromas) else []})
15881621
if len(out) >= k:
15891622
break
@@ -1733,7 +1766,8 @@ def predict(smiles: str, include_aroma: bool = False) -> dict:
17331766
# If two+ taste heads both fire high, surface that as a complex-taste note —
17341767
# the model-side echo of ChemTastesDB's 'multitaste' class.
17351768
strong = [t for t in ("sweet", "bitter", "umami")
1736-
if isinstance(out.get(t), float) and out[t] >= 0.5]
1769+
if isinstance(out.get(t), float)
1770+
and out[t] >= _head_threshold(_TASTE_META, t)]
17371771
out["multitaste"] = len(strong) >= 2
17381772
out["taste_profile"] = _taste_profile(out)
17391773
out["physchem"] = physchem(mol)

training/train_taste.py

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,10 @@
2828
from rdkit.Chem import DataStructs, rdFingerprintGenerator
2929
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
3030
from sklearn.metrics import r2_score, roc_auc_score
31-
from sklearn.model_selection import train_test_split
31+
from sklearn.model_selection import cross_val_predict, train_test_split
32+
from train_aroma import (
33+
_calibrate, # same out-of-fold threshold + precision floor as the aroma heads
34+
)
3235

3336
BASIC = ["sweet", "bitter", "umami", "sour", "salty", "tasteless"]
3437
# Salty now ALSO trains as an INDICATIVE head (CV-AUROC ~0.96 once the PubChem documented-
@@ -102,9 +105,19 @@ def train_classifiers(master):
102105
clf = RandomForestClassifier(n_estimators=500, n_jobs=-1, random_state=42)
103106
clf.fit(Xtr, ytr)
104107
auc = roc_auc_score(yte, clf.predict_proba(Xte)[:, 1])
105-
print(f" {taste:7s} AUROC={auc:.3f} (pos={pos}, neg={neg})")
108+
# Per-head decision threshold + measured precision, same instrument as the aroma heads
109+
# (train_aroma._calibrate). These heads have hundreds of positives each and were never
110+
# expected to be shy — but "we measured and 0.5 was right" is a result, not a reason to
111+
# skip measuring. See docs/ACCURACY.md.
112+
oof = cross_val_predict(RandomForestClassifier(n_estimators=500, n_jobs=-1, random_state=42),
113+
Xv, yv, cv=5, method="predict_proba")[:, 1]
114+
thr, prec, rec, f1, capable = _calibrate(yv, oof)
115+
tag = "" if capable else " INDICATIVE (never reaches 50% precision)"
116+
print(f" {taste:7s} AUROC={auc:.3f} thr={thr:.2f} prec={prec:.2f}{tag} (pos={pos}, neg={neg})")
106117
joblib.dump(clf, OUT / f"{taste}_rf.joblib")
107-
manifest[taste] = {"auroc": round(float(auc), 3), "n_pos": pos, "n_neg": neg}
118+
manifest[taste] = {"auroc": round(float(auc), 3), "n_pos": pos, "n_neg": neg,
119+
"threshold": thr, "cv_precision": prec, "cv_recall": rec,
120+
"cv_f1": f1, "confident_capable": capable}
108121
return manifest
109122

110123

training/train_tox.py

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,10 @@
1818
from rdkit import Chem
1919
from rdkit.Chem import DataStructs, rdFingerprintGenerator
2020
from sklearn.ensemble import RandomForestClassifier
21-
from sklearn.model_selection import cross_val_score
21+
from sklearn.model_selection import cross_val_predict, cross_val_score
22+
from train_aroma import (
23+
_calibrate, # same out-of-fold threshold + precision floor as the aroma heads
24+
)
2225

2326
FP_BITS, FP_RADIUS = 2048, 2
2427
_MORGAN = rdFingerprintGenerator.GetMorganGenerator(radius=FP_RADIUS, fpSize=FP_BITS)
@@ -62,9 +65,19 @@ def fp(smiles):
6265
continue
6366
clf_args = {"n_estimators": 200, "n_jobs": -1, "random_state": 42, "class_weight": "balanced"}
6467
auc = cross_val_score(RandomForestClassifier(**clf_args), Xd, yd, cv=5, scoring="roc_auc").mean()
68+
# Calibration matters MORE here than anywhere else in the app. Assay actives are rare, so a
69+
# high AUROC can sit on top of terrible precision — and a safety flag that is wrong most of
70+
# the time is worse than no flag, because it teaches people to ignore the ones that matter.
71+
oof = cross_val_predict(RandomForestClassifier(**clf_args), Xd, yd, cv=5,
72+
method="predict_proba")[:, 1]
73+
thr, prec, rec, f1, capable = _calibrate(yd, oof)
6574
joblib.dump(RandomForestClassifier(**clf_args).fit(Xd, yd), OUT / f"{t}_rf.joblib")
66-
manifest[t] = {"auroc": round(float(auc), 3), "n_pos": int(yd.sum()), "n": int(mask.sum())}
75+
manifest[t] = {"auroc": round(float(auc), 3), "n_pos": int(yd.sum()), "n": int(mask.sum()),
76+
"threshold": thr, "cv_precision": prec, "cv_recall": rec, "cv_f1": f1,
77+
"confident_capable": capable}
6778
kept += 1
68-
print(f" {t:14s} n={int(mask.sum()):5d} pos={int(yd.sum()):4d} CV-AUROC={auc:.3f}")
79+
tag = "" if capable else " INDICATIVE (never reaches 50% precision)"
80+
print(f" {t:14s} n={int(mask.sum()):5d} pos={int(yd.sum()):4d} CV-AUROC={auc:.3f} "
81+
f"thr={thr:.2f} prec={prec:.2f}{tag}")
6982
(OUT / "manifest.json").write_text(json.dumps({"assays": manifest}, indent=2))
7083
print(f"\nkept {kept}/{len(TASKS)} tox heads -> tox_models/ (caution-only, Tox21 public domain)")

0 commit comments

Comments
 (0)