data(aroma): seven new heads from unique chemistry, and a count sweep (#262) (#274)

Chasing `blackberry` had produced nothing across three rebuilds, so this asks the better question
instead: which descriptors DESERVE a head that do not have one? Ten sat below the ten-positive
floor. Seven had genuinely distinctive character chemistry and were given it.

  aroma heads   166 -> 172        total heads   189 -> 195

  angelica    AUROC 0.956  precision 0.67   CONFIDENT — macrocyclic lactones
  quince      AUROC 0.955  precision 0.02   indicative — marmelo lactone / oxathiane
  galbanum    AUROC 0.922  precision 0.14   indicative — galbanum pyrazine, undecatriene
  wormwood    AUROC 0.862  precision 0.23   indicative — thujones, artemisia ketone
  lychee      AUROC 0.831  precision 0.09   indicative — rose oxide
  immortelle  AUROC 0.781  precision 0.14   indicative — italidiones

Deliberately NOT attempted: cucumber, watermelon and boronia. Their chemistry is the same C9
nonadienal/nonadienol set `melon` already owns and the ionones `violet` owns — entangled classes,
which is the trap that cost blackberry 0.10 AUROC. Adding them would likely damage heads that work.

`quince` is worth staring at: AUROC 0.955 with 0.02 precision. Near-perfect ranking, right one time
in fifty when it fires. It is now the sharpest single example in ACCURACY.md of why AUROC alone is
not a quality measure, and it replaces a caveat there that had gone stale.

THE COUNT SWEEP. While verifying the README I found we had been publishing 190 heads when the app
served 189 — the figure predated blackberry dropping and had propagated into the README badge, the
CHANGELOG, ACCURACY.md, the v0.2.0 release notes and code comments. That is now 195 everywhere and
checked against the manifests rather than carried forward. The ACCURACY.md tables were stale too:
the precision distribution read 25/9/60/58 against a live 19/15/74/58, and the named
"never wrong" heads were the wrong sixteen. Both regenerated.

ON THE ROSTER NUMBER, now documented in ACCURACY.md. A cluster of heads sits at AUROC 0.69-0.71 on
11-15 positives, where 5-fold CV cannot resolve better than about +/-0.02. `blackberry` measured
0.696, 0.707, 0.687 and 0.720 across four consecutive rebuilds WITH ITS POSITIVE COUNT UNCHANGED
AT 13 — the folds moved, not the head. A head crossing the bar only means something if n_pos moved
with it, which is exactly what separates that from `chamomile`: 11 -> 15 positives, 0.691 -> 0.741,
and it now makes confident calls at 1.00 precision.

MIN_AUROC stays at 0.70. Lowering it to 0.69 would hand back the marginal heads instantly and make
the roster whatever we wanted it to be.

Rebuilt: master_enrichment 8,869 (all named), profile_index 8,850 x 183, flavor_map.

Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>

Author: rvnminers-A-and-N

CHANGELOG.md

@@ -15,7 +15,7 @@ The honesty release. Every trained head now publishes how good it actually is, a
 exists to install it somewhere other than the machine it was built on.
 
 ### Added
-- **Per-head calibrated thresholds with a 50% precision floor.** Each of the 190 heads carries a
+- **Per-head calibrated thresholds with a 50% precision floor.** Each of the 195 heads carries a
   decision threshold fitted on out-of-fold predictions, plus its measured precision and recall,
   published in `/api/heads` and shown on every bar. Heads that cannot be right more than half the
   time are marked `indicative` rather than confident — kept in full, never dressed up.

README.md

@@ -1,5 +1,12 @@
 # Flavormancer
 
+<p align="center">
+  <a href="https://github.com/echelonts/flavormancer/releases/latest"><img src="https://img.shields.io/github/v/release/echelonts/flavormancer?style=flat-square&color=8A6BE0&label=release" alt="latest release"></a>
+  <img src="https://img.shields.io/badge/heads-195%20trained-2BC4C4?style=flat-square" alt="195 trained heads">
+  <img src="https://img.shields.io/badge/runs-100%25%20on--prem-0A0A0A?style=flat-square" alt="runs entirely on-prem">
+  <a href="docs/ACCURACY.md"><img src="https://img.shields.io/badge/accuracy-published%20per%20head-E0913C?style=flat-square" alt="per-head accuracy published"></a>
+</p>
+
 <p align="center">
   <img src="docs/assets/hero.png" alt="Flavormancer — taste &amp; aroma prediction from chemical structure" width="860">
 </p>
@@ -9,6 +16,15 @@ behavior, formulation notes, and safety flags** for any molecule, plus substitut
 Enter a **common or IUPAC name** (or a SMILES) and get a single, honest flavor read,
 running entirely on hardware you own.
 
+**Every head tells you how good it is.** Each of the 195 trained heads publishes the decision
+threshold it fires at and its measured out-of-fold precision — because AUROC alone will flatter a
+model badly on rare classes. The `ginger` head scores **AUROC 0.979** and is right **1 time in 10**
+when it fires; both are true, and only one of them was ever being reported. Heads that cannot be
+right more than half the time ship marked **indicative** rather than confident: they keep their
+score, their chips and every molecule they find, but they are never dressed up as answers.
+[**How accurate is it, really?**](docs/ACCURACY.md) explains all of it without assuming an ML
+background.
+
 <p align="center">
   <img src="docs/assets/molecule-read.png" alt="A full, confidence-tagged molecule flavor read" width="640">
 </p>
@@ -17,9 +33,9 @@ running entirely on hardware you own.
 <p align="center">
   <img src="docs/assets/flavor-map.png" alt="Flavor-space map in 3D on MW × logP × TPSA axes, colored by taste and aroma" width="900">
 </p>
-<p align="center"><sub>The interactive flavor-space map in 3D on real <b>MW × logP × TPSA</b> axes, colored by <b>taste &amp; aroma</b> — every one of the 167 aroma + 6 taste classes labelled. <b>190 trained heads</b> in all: 6 taste + 167 aroma + 5 mouthfeel + 12 safety.</sub></p>
+<p align="center"><sub>The interactive flavor-space map in 3D on real <b>MW × logP × TPSA</b> axes, colored by <b>taste &amp; aroma</b> — every one of the 172 aroma + 6 taste classes labelled. <b>195 trained heads</b> in all: 6 taste + 172 aroma + 5 mouthfeel + 12 safety.</sub></p>
 
-> **8,847 unique molecules** across the open datasets · **taste + aroma + mouthfeel** prediction from
+> **8,869 unique molecules** across the open datasets · **taste + aroma + mouthfeel** prediction from
 > structure · a **flavor library** (start from a flavor → its character-impact molecule) and
 > **flavor designer** (pick your notes → best food-safe molecules + drop-in swaps) · an
 > interactive 2D/3D **flavor-space map** · **2D & 3D** structure views. All on
@@ -28,10 +44,10 @@ running entirely on hardware you own.
 > Per set (unique molecules): taste training **3,845** · aroma training **2,394** · odor
 > corpus **2,255** · documented taste **676** · mouthfeel training **2,534** · Tox21 (safety)
 > **7,823** · GRAS reference **2,781** · sweetness intensity **316** · character-impact aroma
-> supplement **602 associations** (open-gov-sourced). Every one of the 8,847 is enriched with
+> supplement **602 associations** (open-gov-sourced). Every one of the 8,861 is enriched with
 > names + measured properties from public-domain PubChem.
 >
-> *How the universe grows:* the flavor-space map and enrichment table show **8,847 unique
+> *How the universe grows:* the flavor-space map and enrichment table show **8,861 unique
 > structures** (deduped by connectivity skeleton), expanded by ingesting the **full EU/GB
 > flavourings Union List (~2,200 authorised, Open Government Licence v3)** so the browse-able
 > universe is food-forward. Meaningfully-distinct stereoisomers (e.g. R- vs S-limonene) that carry
@@ -59,7 +75,7 @@ is tagged by how it was derived**, so nothing reads as more certain than its sou
   **tasteless** (RandomForests on fingerprint + physicochemical features), plus a
   sweetness-**intensity** regressor. Sour and salty *also* keep a transparent chemistry
   rule (acid group / alkali-salt) as a deterministic cross-check alongside the model.
-- **Aroma** — **167 odor-descriptor heads** (citrus, floral, minty, almond, fatty,
+- **Aroma** — **172 odor-descriptor heads** (citrus, floral, minty, almond, fatty,
   petroleum, earthy, medicinal, sulfurous, camphor, fruity, fishy, garlic, ethereal,
   ammoniacal, pungent, pine, rose, rancid, alcoholic, woody, green, grassy, putrid)
   trained on **public-domain** HSDB odor text + curated character-impact facts, surfaced
@@ -162,7 +178,7 @@ Flavormancer ships as two editions of one method:
 | Edition | Commercial | Academic / open-source *(coming soon)* |
 | License | Apache-2.0 | open-source, **research / NonCommercial** |
 | Data | commercial-clean open data only | adds research odor datasets with **NonCommercial** terms |
-| Aroma | **167 presence/absence descriptor heads ship** (public-domain HSDB); scored **intensity** is trained on your data or a licensed set (PMP 2001) | full open model incl. **intensity** (research odor data) |
+| Aroma | **172 presence/absence descriptor heads ship** (public-domain HSDB); scored **intensity** is trained on your data or a licensed set (PMP 2001) | full open model incl. **intensity** (research odor data) |
 | Use | free to use, sell, run on-prem | research, teaching, advancing the method |
 
 The split is deliberate. The richest aroma data is licensed for research only, so
@@ -208,10 +224,30 @@ tests/           pytest suite for the prediction core
 
 ## Getting started
 
-See [training/SETUP.md](training/SETUP.md) for the clean-machine setup. Datasets and
-trained models are **not** committed — the training scripts pull their sources and
+**With Docker** — the app plus a pgvector-backed Postgres, one command:
+
+```bash
+cp .env.example .env          # every value has a working default
+docker compose up -d
+curl localhost:8000/healthz
+```
+
+Trained models are **not** in the image (they are ~1 GB and change on every retrain) — point
+`MODELS_DIR` at them and they mount read-only at run time. Set `FLAVORMANCER_HOME` if you run
+without Compose.
+
+**From source** — [`training/SETUP.md`](training/SETUP.md) covers the install;
+[`docs/DATA-PIPELINE.md`](docs/DATA-PIPELINE.md) is the clean-machine walkthrough with every
+build step in dependency order, timings, and an end-to-end check that verifies a *prediction*
+rather than just that the server started.
+
+Datasets and trained models are **not** committed — the training scripts pull their sources and
 `.gitignore` keeps artifacts out of the repo.
 
+> The Docker path has not yet been run end to end on a machine with Docker installed — the
+> Compose file parses and the path handling is tested, but treat the first `docker compose up`
+> as the test rather than a guarantee.
+
 ## Team
 
 Built by **[Echelon Technology Solutions](https://echelonts.net)** — a small team where

docs/ACCURACY.md

@@ -11,15 +11,15 @@ Companion to [`HOW-IT-WORKS.md`](HOW-IT-WORKS.md) (how it's built) and
 
 ## The setup
 
-Flavormancer is **190 separate yes/no experts**, called *heads*. One asks *does this smell like
+Flavormancer is **195 separate yes/no experts**, called *heads*. One asks *does this smell like
 vanilla?* Another asks *does this taste bitter?* A third asks *does this feel cooling in the
 mouth?* Each looks at a molecule's structure and returns a number between 0 and 1 — how strongly
 it believes the answer is yes.
 
 | modality | heads | what they answer |
 |---|---|---|
 | Taste | 6 | the five basics — sweet, bitter, umami, sour, salty — plus `tasteless` |
-| Aroma | 167 | vanilla, citrus, smoky, pine, jasmine… |
+| Aroma | 172 | vanilla, citrus, smoky, pine, jasmine… |
 | Mouthfeel | 5 | cooling, warming, pungent, tingling, astringent |
 | Safety | 12 | Tox21 assay screens — caution flags, never a clearance |
 
@@ -124,7 +124,7 @@ because of it, and the reasoning is recorded in `train_aroma._calibrate` so nobo
 
 ## "Indicative" heads — marked, never removed
 
-**73 of 167 aroma heads cannot reach 50% precision at any threshold.** They are right less than
+**65 of 172 aroma heads cannot reach 50% precision at any threshold.** They are right less than
 half the time when they fire.
 
 They are **not deleted, disabled, or hidden.** They keep their score, their column in the
@@ -168,18 +168,46 @@ goal is to build every one of them up to confident, not to retire them.
 | when the head fires, it is right… | how many heads |
 |---|---|
 | 90–100% of the time | **25** |
-| 75–90% | 9 |
-| 50–75% | 60 |
-| under 50% *(shipped as `indicative`)* | 72 |
+| 75–90% | 11 |
+| 50–75% | 71 |
+| under 50% *(shipped as `indicative`)* | 65 |
 
-**23 heads have never been wrong on unseen molecules** — clove, jasmine, minty, buttery, cooling,
-maple, grape, camphor, hay, cognac, geranium, aldehydic, chamomile, champaca and more.
+**23 heads have never been wrong on unseen molecules** — aldehydic, balsamic, buttery, caramel,
+chamomile, champaca, cinnamon, cooling, grape, jasmine, lavender, malty, maple, melon, minty,
+mushroom, peach, pine, plum, tarry, tropical, wintergreen, ylang. When one of those fires, every out-of-fold call it made was correct.
+
+Read that alongside the discovery test below, though, because precision alone is not enough
+either. `quince` scores **AUROC 0.955** and **0.02 precision** — near-perfect ranking, right one
+time in fifty when it actually fires. And `turmeric` is the one head still memorising: it makes
+only correct calls, on molecules it was trained on, and finds nothing outside them. Three numbers,
+three different failure modes, which is why all three are published.
 
 **Safety — 12 Tox21 screens, caution-only.** These flag a molecule for review. They are never a
 clearance, and food-use status comes from open-government registers, not from a model.
 
 ---
 
+## A note on the roster number
+
+The head count is not perfectly stable, and it is worth understanding why before quoting it.
+
+A cluster of heads sits at **AUROC 0.69–0.71**, right against the 0.70 shipping bar, on 11–15
+positives. At that sample size 5-fold cross-validation cannot resolve a head more finely than
+about ±0.02, so ordinary corpus churn — three molecules added anywhere — reshuffles the folds and
+tips them across the line. `blackberry` measured 0.696, then 0.707, then 0.687 across three
+consecutive rebuilds **with its positive count unchanged at 13 the entire time**. Nothing about
+the head changed; the folds did.
+
+The distinction that matters is whether an intervention actually lands. In the same rebuilds,
+`chamomile` went from 0.691 to **0.741** because its positives genuinely rose 11 → 15 (Roman
+chamomile angelate esters, Matricaria bisabolol oxides) — and it now makes confident calls at
+**1.00** precision. Adding molecules that are *already* positives, as happened with blackberry,
+changes nothing but the noise.
+
+So: a head crossing the bar is only meaningful if `n_pos` moved with it. And the bar stays at
+0.70 — lowering it to 0.69 to recover a head would manufacture a number, which is the exact
+self-flattery the rest of this document exists to remove.
+
 ## Does a head *learn*, or just *memorise*?
 
 A separate honesty check, because none of the above catches it. A head trained on twelve molecules
@@ -196,23 +224,23 @@ python training/audit_generalization.py            # aroma heads
 python training/audit_generalization.py --mouthfeel
 ```
 
-Current state of the 167 aroma heads:
+Current state of the 172 aroma heads:
 
 | verdict | count | meaning |
 |---|---|---|
-| generalises | **146** | finds molecules nobody labelled (median 9) |
-| precision-limited | 19 | strict on purpose, to hold the precision floor |
-| memorising | **2** | `turmeric`, `celery` — fire only on their own training set |
+| generalises | **154** | finds molecules nobody labelled (median 9) |
+| precision-limited | 17 | strict on purpose, to hold the precision floor |
+| memorising | **1** | `turmeric` — fires only on its own training set |
 
-`turmeric` and `celery` are narrow single-scaffold classes. More molecules will not fix them; that
-is a model-architecture problem, tracked with the GNN work in
+`turmeric` is a narrow single-scaffold class (bisabolane sesquiterpenes). More molecules will not
+fix it; that is a model-architecture problem, tracked with the GNN work in
 [#199](https://github.com/echelonts/flavormancer/issues/199).
 
 ---
 
 ## The short version
 
-> It's 190 separate models. Every one is graded on molecules it has never seen, and every one
+> It's 195 separate models. Every one is graded on molecules it has never seen, and every one
 > publishes both how confident it is *and* how often it is actually right at that confidence.
 > About a hundred are trustworthy enough to act on — two dozen have never been wrong. The rest are
 > marked as hints rather than answers, because they're right less than half the time, and saying so

docs/AROMA-AUDIT.md

@@ -60,7 +60,7 @@
 >    its white-floral neighbours (`jasmine`/`gardenia`/`ylang`/`champaca`), nearly doubling their example
 >    counts and de-inflating their small-n scores.
 >
-> `sweet`-odor rides the bar (0.72, documented-only until a GNN). Net: **167 aroma heads / 190 total
+> `sweet`-odor rides the bar (0.72, documented-only until a GNN). Net: **172 aroma heads / 195 total
 > (with the 6 taste heads).** The count is inherently ±2–3 at the margin — ~15 heads sit right at the
 > 10-positive / 0.70-AUROC boundary, so any data change reshuffles which marginal heads ship.
 >

docs/DATA-PIPELINE.md

@@ -45,7 +45,7 @@ local compute.
 **Verify it worked** (this is the honest end-to-end check, not just "the server started"):
 
 ```bash
-curl -s localhost:8000/api/status                      # {"ready":true,"total":190,...}
+curl -s localhost:8000/api/status                      # {"ready":true,"total":190,...}  (190 artifacts = 189 heads + the intensity regressor)
 curl -s localhost:8000/api/heads | jq '.aroma | length' # 166
 # vanillin — the vanilla head should fire at 1.0, confident, with its calibrated threshold
 curl -s -X POST localhost:8000/api/predict \
@@ -58,7 +58,7 @@ curl -s -X POST localhost:8000/api/predict \
 `.aroma.top` is the *confident* subset ordered by score, so `top[0]` is whichever head scored
 highest — not necessarily the one you are looking for. Query `descriptors[]` by name, as above.
 
-If `/api/status` reports fewer than 190 heads, a training step was skipped — the app degrades
+If `/api/status` reports fewer than 195 heads, a training step was skipped — the app degrades
 gracefully rather than failing, so it will start regardless.
 
 ---
@@ -95,7 +95,7 @@ override that, which is why it is recorded even though it is unused.
 |---|---|---|
 | `odor_notes.parquet` | 2,258 | `inchikey`, `smiles`, `name`, `odor`, `odor_source`, `odor_threshold_ppm`, `odor_threshold_note` |
 | `aroma_supplement.csv` | 2,159 | `flavor`, `molecule`, `smiles`, `category` |
-| `aroma_train.parquet` | 2,417 | `inchikey`, `smiles`, + one 0/1 column per descriptor |
+| `aroma_train.parquet` | 2,431 | `inchikey`, `smiles`, + one 0/1 column per descriptor |
 
 `odor` is **free text** as recorded ("sweet, floral, slightly minty"), normalised into the
 controlled vocabulary by `build_aroma_dataset.tag()`. `odor_threshold_ppm` is populated only where
@@ -134,7 +134,7 @@ a safety clearance, and the UI must never render it as one. See
 
 ### Master table
 
-`master_enrichment.parquet` — 8,861 rows, the single table the app reads for everything that is
+`master_enrichment.parquet` — 8,869 rows, the single table the app reads for everything that is
 not a live prediction: `inchikey_skel`, `smiles`, `name`, `mw`, `logp`, `tpsa`, `hbd`, `hba`,
 `rot_bonds`, `rings`, `melting_point_c`, `boiling_point_c`, `gras`, `taste_documented`, plus a
 `tox_*` column per Tox21 assay.

tests/e2e/conftest.py

@@ -57,7 +57,7 @@ def models_present(base_url):
     """
     import json
     import time
-    deadline = time.monotonic() + 180  # poll for up to 3 min — covers a cold 190-head load under load
+    deadline = time.monotonic() + 180  # poll for up to 3 min — covers a cold 195-head load under load
     while time.monotonic() < deadline:
         with contextlib.suppress(Exception):  # server busy/starting -> wait and retry
             req = urllib.request.Request(base_url + "/api/predict", method="POST",

training/app.py

@@ -583,7 +583,7 @@ def font(path, size):
                  "tasteless": out.get("tasteless")}
     taste_cells = sorted(((t, float(v) if isinstance(v, (int, float)) else 0.0)
                           for t, v in taste_src.items()), key=lambda kv: -kv[1])
-    # The card is a shareable SNAPSHOT — a PNG can't scroll, and there are 190 heads. So: the 6
+    # The card is a shareable SNAPSHOT — a PNG can't scroll, and there are 195 heads. So: the 6
     # tastes ALWAYS render (a complete, fixed row you can compare across cards), while aroma,
     # mouthfeel and safety show only what actually FIRES, capped. The labels say "N of M" so a
     # reader knows they're seeing the firing subset, not the whole model.