Resolve names from master_enrichment, the one base table

CHANGELOG.md

@@ -31,7 +31,7 @@ exists to install it somewhere other than the machine it was built on.
 - Numbers glossary in `HOW-IT-WORKS.md` and in the app's own How-it-works panel.
 
 ### Changed
-- Aroma roster **164 → 166 heads**; confident-capable heads **94 → 108**.
+- Aroma roster **164 → 172 heads**; confident-capable heads **94 → 108**.
 - Chip families (flavor / note / taste / mouthfeel) share one visual language instead of four
   accidental ones, and each studio section explains what its dimension *is*.
 - Every molecule has a display name: names fall back to molecular formula, with multi-component

docs/ACCURACY.md

@@ -128,7 +128,7 @@ because of it, and the reasoning is recorded in `train_aroma._calibrate` so nobo
 half the time when they fire.
 
 They are **not deleted, disabled, or hidden.** They keep their score, their column in the
-178-dimension flavor profile, their chips in search, their colour on the map, and every molecule
+183-dimension flavor profile, their chips in search, their colour on the map, and every molecule
 they find. What changes is one word: they are reported as **indicative** rather than *confident*,
 and the UI hatches their bar so you can see it at a glance.
 

docs/AROMA-AUDIT.md

@@ -2,7 +2,7 @@
 
 > **Update — supplement applied, then made food-only.** `build_aroma_supplement.py` adds a curated
 > **public-domain** character-impact set (`aroma_supplement.csv`, resolved via PubChem) for the sparse
-> descriptors below. Result: **42 → 167 heads** (food-safe core + non-food aroma-only tier; see the
+> descriptors below. Result: **42 → 172 heads** (food-safe core + non-food aroma-only tier; see the
 > multi-industry note below), after open-government sourcing (see below) took the
 > curated supplement to **~1,994 labelled molecules**. New heads include `coconut`, `nutty`, `caramel`,
 > `winey`, `onion`, `honey`, `herbal`, `vanilla`, `buttery`, `balsamic`, `smoky`, `cinnamon`, `spicy`,
@@ -28,10 +28,10 @@
 > 2-phenylethanol) instead of the E/Z terpene twins (nerol/geraniol) that dedupe to one row under the
 > connectivity-skeleton key. That was the **food-safe** ceiling at ~86.
 >
-> **Then we walked the road past it — the non-food aroma-only descriptor space — to 167 heads.** The
+> **Then we walked the road past it — the non-food aroma-only descriptor space — to 172 heads.** The
 > corpus is now deliberately **multi-industry**, not food-only: odor is physical, so to read
 > structure→smell well the corpus should include the fragrance space, and a per-molecule **safety flag**
-> (open-gov food registers) — not corpus membership — gates edibility per mode. The same 167 heads serve
+> (open-gov food registers) — not corpus membership — gates edibility per mode. The same 172 heads serve
 > every "Mancer" mode: **Flavormancer** (food chemistry), **Beveragemancer** (beverage), **Aromamancer**
 > (fragrance), **Vapemancer** (inhalation), **Oilmancer** (essential oils). Molecule names are resolved
 > to structures through PubChem and folded in flagged non-food. New heads cleared across several

docs/AROMA.md

@@ -18,7 +18,7 @@ enumerable via the annotations API. Pipeline:
   physicochemical descriptor block** (`chemfeatures.py`: MW, logP, TPSA, H-bond donors/
   acceptors, rotatable bonds, ring counts, fraction sp3, heteroatoms — the global properties
   bare bits miss). After broadening the vocabulary, folding the curated `flavors.csv` molecules
-  in (825 → **2403** molecules, incl. a curated public-domain supplement), and adding those features, **167 heads clear CV-AUROC ≥ 0.70**
+  in (825 → **2403** molecules, incl. a curated public-domain supplement), and adding those features, **172 heads clear CV-AUROC ≥ 0.70**
   (each score is that descriptor's own held-out AUROC): medicinal 0.97, ammoniacal 0.96,
   petroleum 0.96, citrus 0.94, camphor 0.93, almond 0.92, fatty 0.89, minty 0.88, ethereal 0.87,
   fruity 0.87, fishy 0.86, sulfurous 0.85, garlic 0.84, floral 0.83, pungent 0.81, earthy 0.73.

docs/CAPABILITIES.md

@@ -6,7 +6,7 @@ shape of the tool. The organizing discipline throughout: **every output is tagge
 by how it's derived, and nothing claims more certainty than its source supports.**
 
 > **Edition note.** This catalogues the **commercial** edition (Apache-2.0,
-> commercial-clean data). **Aroma ships here** as 167 presence/absence odor-descriptor
+> commercial-clean data). **Aroma ships here** as 172 presence/absence odor-descriptor
 > heads trained on public-domain HSDB text (plus 5 mouthfeel/chemesthesis heads and 12
 > Tox21 safety heads) — what's still gated is scored **intensity** (*how strong* a note
 > is), which needs research/customer panel data and lives in the **academic edition**
@@ -33,7 +33,7 @@ by how it's derived, and nothing claims more certainty than its source supports.
 - Multitaste — fires when 2+ taste heads are high (**trained**-derived).
 - Known-taste ground truth — verified labels override predictions (**lookup**).
 
-**Aroma** — **167 odor-descriptor heads ship** (**trained**)
+**Aroma** — **172 odor-descriptor heads ship** (**trained**)
 - Presence/absence per descriptor (citrus, floral, minty, almond, fatty, petroleum, earthy,
   medicinal, sulfurous, camphor, fruity, fishy, garlic, ethereal, ammoniacal, pungent) —
   RandomForests on fingerprint + physicochemical features, over public-domain HSDB odor text +

docs/DATA-PIPELINE.md

@@ -45,8 +45,8 @@ local compute.
 **Verify it worked** (this is the honest end-to-end check, not just "the server started"):
 
 ```bash
-curl -s localhost:8000/api/status                      # {"ready":true,"total":190,...}  (190 artifacts = 189 heads + the intensity regressor)
-curl -s localhost:8000/api/heads | jq '.aroma | length' # 166
+curl -s localhost:8000/api/status                      # {"ready":true,"total":190,...}  (190 artifacts = 195 heads + the intensity regressor)
+curl -s localhost:8000/api/heads | jq '.aroma | length' # 172
 # vanillin — the vanilla head should fire at 1.0, confident, with its calibrated threshold
 curl -s -X POST localhost:8000/api/predict \
   -H 'Content-Type: application/json' \

docs/HOW-IT-WORKS.md

@@ -98,7 +98,7 @@ features (trained on SweetenersDB).
 
 ---
 
-## 3. Aroma — 167 descriptor heads from public odor text
+## 3. Aroma — 172 descriptor heads from public odor text
 
 Odor is the hard, licensed part of this field — most rich odor datasets are NonCommercial. Our
 clean route:
@@ -111,7 +111,7 @@ clean route:
    `minty`), producing a presence/absence label per descriptor. We also fold in the curated
    character-impact facts from `flavors.csv`.
 3. **`train_aroma.py`** trains one RandomForest per descriptor (on fingerprint + descriptors) and
-   **keeps only the heads that clear CV-AUROC ≥ 0.70** — an honest bar. **167 heads** survive
+   **keeps only the heads that clear CV-AUROC ≥ 0.70** — an honest bar. **172 heads** survive
    (citrus, floral, minty, almond, fatty, petroleum, earthy, medicinal, sulfurous, camphor,
    fruity, fishy, garlic, ethereal, ammoniacal, pungent, pine, rose, rancid, alcoholic,
    woody, green, grassy, putrid), 0.71–0.98. The lower-population heads (≈10–20 documented

docs/METHODS.md

@@ -136,8 +136,8 @@ the thin heads — a genuine pine match could land at 0.42 and never be shown.
   recall climbs faster than precision falls. **A high AUROC does not protect you** — AUROC is
   computed on ranking and is insensitive to class imbalance; precision is not. `coffee` has
   AUROC 0.960 and out-of-fold precision **0.46**; both are true of the same head.
-- **Heads that cannot clear the floor are `indicative`, not deleted.** 73 of 167 aroma heads never
-  reach 50% precision at any threshold. They keep their score, their place in the 178-dim profile,
+- **Heads that cannot clear the floor are `indicative`, not deleted.** 65 of 172 aroma heads never
+  reach 50% precision at any threshold. They keep their score, their place in the 183-dim profile,
   their chips and their map colour — firing well above base rate is real evidence. They are simply
   never presented as a *confident* call: the UI marks them, and the read returns `indicative: true`
   plus the head's measured precision. Hiding them would delete reach to paper over a labelling

docs/MOUTHFEEL.md

@@ -10,7 +10,7 @@ without *feeling* cool, and WS-23 cools the mouth with almost no odour at all.
 
 **Five trained heads**, one RandomForest per sensation on the Morgan fingerprint + physicochemical
 block (the same stack as taste/aroma), in their own `mouthfeel_models/` directory, loaded and tagged
-`mouthfeel` by `predict.py` and folded into the 175-dim flavor-profile vector used for substitutes.
+`mouthfeel` by `predict.py` and folded into the 183-dim flavor-profile vector used for substitutes.
 
 | head | what it is | held-out CV-AUROC |
 |---|---|---|

docs/TOX.md

@@ -16,7 +16,7 @@ assay** heads that act as a *defensive screen*.
 |---|---|---|
 | label source | **documented + curated** sensory facts (a molecule is *known to be perceived* as X) | **experimental wet-lab screening** (a molecule *measurably was* active in an assay) |
 | what a positive means | people report this percept | this well lit up in a dish |
-| in the substitute-match vector? | **yes** (175 dims) | **no** — deliberately |
+| in the substitute-match vector? | **yes** (183 dims) | **no** — deliberately |
 | framing | prediction | **caution-only review flag** |
 
 Tox is excluded from the flavor-profile vector on purpose: you do not want reformulation

infra/initdb/01-schema.sql

@@ -1,8 +1,8 @@
 -- Flavormancer schema (#20).
 --
 -- The substitution index is a nearest-neighbour search over the flavour-profile vector, so the
--- vector lives in the database rather than being recomputed per query. 177 dimensions:
--- 6 taste + 166 aroma + 5 mouthfeel. Tox is deliberately NOT in the vector — safety is not a
+-- vector lives in the database rather than being recomputed per query. 183 dimensions:
+-- 6 taste + 172 aroma + 5 mouthfeel. Tox is deliberately NOT in the vector — safety is not a
 -- flavour-match dimension, and letting it steer "what tastes similar" would be wrong.
 CREATE EXTENSION IF NOT EXISTS vector;
 
@@ -19,7 +19,7 @@ CREATE TABLE IF NOT EXISTS molecule (
 
 CREATE TABLE IF NOT EXISTS molecule_profile (
     inchikey_skel   text PRIMARY KEY REFERENCES molecule(inchikey_skel) ON DELETE CASCADE,
-    profile         vector(177) NOT NULL,
+    profile         vector(183) NOT NULL,
     aromas          text[]                   -- heads clearing their own calibrated threshold
 );
 

mcp-server/server.py

@@ -256,7 +256,7 @@ def _flavor_map(label: str, limit: int, full: bool) -> dict:
 def read_flavor(molecule: str) -> dict:
     """Predict the flavor of a single molecule (name or SMILES).
 
-    Returns the six taste-head probabilities, confident aromas plus all 167 aroma scores, the
+    Returns the six taste-head probabilities, confident aromas plus all 172 aroma scores, the
     trained mouthfeel/chemesthesis scores (cooling/pungent/warming/astringent/tingling), GRAS
     status, structural alerts, caution-only Tox21 flags, and the applicability-domain flag.
     Prediction only — tox flags are indicative in-vitro activity, never a determination.

training/app.py

@@ -215,40 +215,35 @@ def _svg(smi, w=320, h=220):
 
 
 def _load_name_table():
-    """inchikey-skeleton -> (common, IUPAC) from the precomputed enrichment table, so the
-    whole labeled set resolves instantly and offline. Empty until build_properties.py has
-    written name columns; live PubChem stays the fallback for anything not in the table."""
+    """skeleton -> (common, IUPAC), sourced from master_enrichment — the ONE base table (#224).
+
+    This used to read properties.parquet and then merge iupac_backfill.parquet itself, which
+    duplicated work build_enrichment.py had already done and silently diverged from it in three
+    ways. The enrichment build resolves a name by curated list first, then PubChem common name,
+    then IUPAC name, then a molecular-formula fallback — and it un-inverts CAS-style ordering
+    along the way. None of that reached this table, so 754 molecules the grid displayed by name
+    (cedrol, fenchyl alcohol, hydroxycitronellal, musk ketone...) resolved to nothing here. The
+    old merge also read only `iupac_name` from the backfill and ignored the `common_name` column,
+    so every PubChem Title the crawler recovered was thrown away.
+
+    Reading the resolved name straight off the base table makes the UI's lookup and the grid
+    agree by construction rather than by coincidence.
+    """
     try:
         import pandas as pd
-        df = pd.read_parquet(P.artifact("properties.parquet"))
-        if "common_name" not in df.columns:
-            return {}
-        out = {}
-        for ik, c, u in zip(df["inchikey"], df["common_name"], df["iupac_name"]):
-            if isinstance(ik, str) and (isinstance(c, str) or isinstance(u, str)):
-                out[ik.split("-")[0]] = (c if isinstance(c, str) else None,
-                                         u if isinstance(u, str) else None)
-        return out
-    except Exception:  # noqa: BLE001 — no table / no pandas; just fall back to live lookups
+        df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
+    except Exception:  # noqa: BLE001 — no table / no pandas; live lookups still cover it
         return {}
+    out = {}
+    iupac = dict(zip(df.get("inchikey_skel", []), df.get("iupac_name", [])))  # optional column
+    for skel, name in zip(df["inchikey_skel"], df["name"]):
+        if isinstance(skel, str) and isinstance(name, str) and name.strip():
+            u = iupac.get(skel)
+            out[skel] = (name.strip(), u if isinstance(u, str) else None)
+    return out
 
 
-def _merge_iupac_backfill(table):
-    """Fold in IUPAC names that build_iupac_backfill.py recovered from PubChem for molecules
-    the main properties crawl missed (skeleton -> keep any common name, add the IUPAC)."""
-    try:
-        import pandas as pd
-        bf = pd.read_parquet(P.artifact("iupac_backfill.parquet"))
-    except Exception:  # noqa: BLE001 — backfill not built; nothing to merge
-        return table
-    for skel, u in zip(bf["inchikey_skel"], bf["iupac_name"]):
-        if isinstance(skel, str) and isinstance(u, str) and u:
-            common = table.get(skel, (None, None))[0]
-            table[skel] = (common, u)
-    return table
-
-
-_NAME_TABLE = _merge_iupac_backfill(_load_name_table())
+_NAME_TABLE = _load_name_table()
 
 
 @lru_cache(maxsize=8192)
@@ -445,7 +440,7 @@ def api_substitutes(q: Query):
 @app.post("/api/precomputed")
 def api_precomputed(q: Query):
     """Fast check: is this molecule's profile already in the index (instant read) or does it need a
-    fresh 178-head compute? Lets the UI show a 'conjuring a fresh reading' note for novel molecules."""
+    fresh 183-head compute? Lets the UI show a 'conjuring a fresh reading' note for novel molecules."""
     smi = _resolve(q.smiles)
     return {"precomputed": bool(smi and P.is_precomputed(smi))}
 
@@ -1326,7 +1321,7 @@ def _prewarm_formulation():
             m = Chem.MolFromSmiles(smi) if smi else None
             if m is not None:
                 canon = Chem.MolToSmiles(m)
-                P.predict_aroma(canon)   # fills the shared 167-head aroma cache
+                P.predict_aroma(canon)   # fills the shared 172-head aroma cache
                 P.substitutes(canon)     # taste heads + profile cosine, so the demo chips are instant
 
 

training/predict.py

@@ -214,8 +214,11 @@ def _load_rf(path):
 
 # Model heads are loaded on a BACKGROUND THREAD at import (see _load_all_models below) so that
 # `import predict` returns immediately and the web server can bind its port right away, showing a
-# friendly "warming up" page while the ~180 forests (628 MB) load — instead of a 34 s startup 502.
-# The load is fanned out across cores (joblib.load releases the GIL), which also cuts the wall time.
+# friendly "warming up" page while the ~195 forests (~700 MB) load — instead of a 50 s startup 502.
+# The load is SERIAL, and this comment used to claim the opposite ("fanned out across cores,
+# joblib.load releases the GIL"). Both halves of that were wrong: unpickling is GIL-bound, so
+# threads made it ~2.5x SLOWER, and a process pool deadlocks because this runs during module
+# import. See _load_all_models for the measurements and #225 for the real fix.
 _CLASSIFIERS = {}          # sweet/bitter/umami/... taste heads
 _INTENSITY = None          # sweet-intensity regressor
 _TASTE_META = {}           # taste -> {auroc, ...} from taste_models/manifest.json (held-out score)
@@ -1098,12 +1101,12 @@ def analyze_balance(ingredients):
 
 @lru_cache(maxsize=8192)
 def _aroma_scores_canon(canon):
-    """Run all 167 descriptor forests for a CANONICAL SMILES and return {head: score}."""
+    """Run all 172 descriptor forests for a CANONICAL SMILES and return {head: score}."""
     m = Chem.MolFromSmiles(canon)
     if m is None or not _AROMA_MODELS:
         return None
     fp = _feat(m)
-    # Fan the 167 forests across cores — each predict_proba releases the GIL, so this turns the
+    # Fan the 172 forests across cores — each predict_proba releases the GIL, so this turns the
     # ~40 s serial read (the only remaining cost, for genuinely novel/out-of-corpus molecules) into
     # a couple of seconds. In-corpus molecules never reach here (they read the precomputed index row).
     def _score(it):
@@ -1114,13 +1117,13 @@ def _score(it):
 
 
 def _aroma_scores(smiles):
-    """The expensive part of the aroma read: all 167 descriptor forests → {head: score}. Keyed on
+    """The expensive part of the aroma read: all 172 descriptor forests → {head: score}. Keyed on
     the CANONICAL SMILES (not threshold/top_k, not the raw string) so every caller shares one
     computation per molecule regardless of how they spelled it — predict_aroma, _query_profile
     (substitutes) and the /api/aroma endpoint all collapse to the same cache entry instead of each
-    re-running 167 forests (that double/mismatched inference was the ~6 s /api/substitutes).
+    re-running 172 forests (that double/mismatched inference was the ~6 s /api/substitutes).
 
-    In-corpus molecules skip the forests entirely: their 167 scores are read straight off the
+    In-corpus molecules skip the forests entirely: their 172 scores are read straight off the
     precomputed profile index (built at startup) — the same numbers, ~40 s cheaper on a cold hit."""
     m = Chem.MolFromSmiles(smiles)
     if m is None or not _AROMA_MODELS:
@@ -1365,7 +1368,7 @@ def _aroma(a):
 def _build_sub_index():
     global _SUB_INDEX
     import numpy as np
-    # Fast path: load the precomputed profile index (build_profile_index.py). The 178-dim
+    # Fast path: load the precomputed profile index (build_profile_index.py). The 183-dim
     # inference over ~8.8k molecules is slow (~3 min); the cache makes startup instant. We only
     # rebuild the cheap Morgan fingerprints from SMILES on load.
     cache = artifact("profile_index.npz")
@@ -1443,7 +1446,7 @@ def _ensure_sub_index():
 def _index_row(mol):
     """Row of `mol` in the profile index (matched by connectivity skeleton), or None if the
     molecule isn't in the reference corpus. In-corpus molecules can reuse their PRECOMPUTED
-    178-dim profile (built once at index build / startup) instead of re-running 167 forests at
+    183-dim profile (built once at index build / startup) instead of re-running 172 forests at
     query time — that inference is ~40 s cold on a novel molecule and was the real /api/substitutes
     and include_aroma cost. The precomputed row is the SAME model output, just paid up front."""
     _ensure_sub_index()
@@ -1465,7 +1468,7 @@ def _index_row(mol):
 
 def is_precomputed(smiles):
     """True if this molecule's full taste+aroma profile is already in the index (an instant read),
-    False if it's out-of-corpus and the 178 profile heads have to run fresh (the slower path). Used by the
+    False if it's out-of-corpus and the 183 profile heads have to run fresh (the slower path). Used by the
     UI to decide whether to show the 'conjuring a fresh reading' note while a read brews."""
     mol = Chem.MolFromSmiles(smiles or "")
     return mol is not None and _index_row(mol) is not None