Skip to content

Commit 631ae43

Browse files
docs: define AUROC, threshold and precision where people actually land
HOW-IT-WORKS.md used CV-AUROC eleven times and never said what it is beyond a parenthetical, and never mentioned threshold, precision or recall at all β€” the real explanation lived only in ACCURACY.md, one click further than most readers go. A six-term glossary now sits at section 0.5, before any number is used: head, AUROC (0.5 is a coin flip, 1.0 is never wrong), threshold (not a flat 0.5 β€” 0.16 to 0.85, fitted per head), precision (when it says yes, how often is it right), recall, out-of-fold, and confident vs indicative. Two sentences each. The same glossary, compressed to a definition list, is now a card in the site's own How-it-works grid, so it reaches someone who will never open the docs folder. Also tightens the taste wording, which was loose in both places: there are FIVE basic tastes and SIX taste heads. `tasteless` is a head that predicts the documented absence of taste β€” genuinely useful, since it is how you find a clean carrier β€” but it is not a sixth basic taste, and calling it one is the kind of small inaccuracy a flavorist would notice immediately. Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>
1 parent 479f28a commit 631ae43

3 files changed

Lines changed: 54 additions & 4 deletions

File tree

β€Ždocs/ACCURACY.mdβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ it believes the answer is yes.
1818

1919
| modality | heads | what they answer |
2020
|---|---|---|
21-
| Taste | 6 | sweet, bitter, umami, sour, salty, tasteless |
21+
| Taste | 6 | the five basics β€” sweet, bitter, umami, sour, salty β€” plus `tasteless` |
2222
| Aroma | 167 | vanilla, citrus, smoky, pine, jasmine… |
2323
| Mouthfeel | 5 | cooling, warming, pungent, tingling, astringent |
2424
| Safety | 12 | Tox21 assay screens β€” caution flags, never a clearance |

β€Ždocs/HOW-IT-WORKS.mdβ€Ž

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,42 @@ the difference between **prediction** and **lookup**, and it's the whole point.
1717

1818
---
1919

20+
## 0.5. Reading the numbers β€” a two-sentence glossary
21+
22+
Every head reports the same handful of numbers. They are used throughout this document, and three
23+
of them are routinely misread, so here is what each one actually means. The long version, with
24+
worked examples, is in [`ACCURACY.md`](ACCURACY.md).
25+
26+
**Head** β€” one yes/no expert for one property. *Does this smell like vanilla?* is a head; so is
27+
*does this taste bitter?* There are 190 of them, and each returns a number from 0 to 1 for how
28+
strongly it believes the answer is yes.
29+
30+
**AUROC** β€” a **ranking** score. Hand a head one true vanilla molecule and one non-vanilla: how
31+
often does it score the vanilla one higher? **0.5 is a coin flip, 1.0 is never wrong.** It says
32+
nothing about how often the head is right when it actually fires β€” see precision.
33+
34+
**Threshold** β€” the score at or above which a head counts as *firing*. It is **not** a flat 0.5;
35+
each head has its own, fitted on data it never trained on, and they range 0.16–0.85. A head with
36+
13 examples hedges, so a real `pine` match can land at 0.42 β€” a shared cut-off would have silently
37+
withheld it.
38+
39+
**Precision** β€” **when it says yes, how often is it right?** This is trustworthiness, and it is the
40+
number AUROC cannot see. A head with 11 positives among 2,403 molecules can score AUROC 0.979 and
41+
have precision 0.10 β€” right one time in ten. Both are true of `ginger`.
42+
43+
**Recall** β€” **of all the real ones, how many did it catch?** This is thoroughness. It trades
44+
against precision: raise a head's threshold and you get fewer false alarms but more misses.
45+
46+
**Out-of-fold** β€” scored by a model that never saw that molecule. Every accuracy number here is
47+
out-of-fold; none of them are a model grading its own homework. See 5-fold cross-validation below.
48+
49+
**Confident vs indicative** β€” a head may only be called *confident* if it clears **50% precision**;
50+
it has to be right more often than not. The 73 aroma heads that cannot are shipped as
51+
**indicative** β€” they keep their score, their chips and every molecule they find, but they are
52+
never dressed up as a confident call.
53+
54+
---
55+
2056
## 1. How a molecule becomes numbers
2157

2258
Everything starts by turning a structure into something math can chew on:

β€Žtraining/workbench.htmlβ€Ž

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -529,6 +529,11 @@
529529
particular stuck out against the flavor and note chips they sit beside. */
530530
.atag.mf{border-style:solid;position:relative;padding-left:14px}
531531
.atag.mf::before{content:'';position:absolute;left:6px;top:50%;width:4px;height:4px;margin-top:-2px;border-radius:50%;background:currentColor}
532+
.how-card.gloss .gl{margin:0;display:grid;grid-template-columns:auto 1fr;gap:5px 10px;align-items:baseline}
533+
.how-card.gloss dt{font-family:var(--mono);font-size:10.5px;letter-spacing:.04em;text-transform:uppercase;color:var(--accent);white-space:nowrap}
534+
.how-card.gloss dd{margin:0;font-size:11.5px;line-height:1.55;color:var(--muted)}
535+
.how-card.gloss dd b{color:var(--ink);font-weight:600}
536+
@media(max-width:560px){.how-card.gloss .gl{grid-template-columns:1fr;gap:2px}.how-card.gloss dt{margin-top:7px}}
532537
.ss-why{font-size:11.5px;line-height:1.62;color:var(--muted);margin:2px 0 11px;max-width:74ch}
533538
.ss-why b{color:var(--ink);font-weight:600}
534539
.ss-why code{font-family:var(--mono);font-size:10.5px;background:var(--panel);border:1px solid var(--line);border-radius:4px;padding:0 4px}
@@ -826,8 +831,17 @@ <h1>Flavormancer</h1>
826831
<div class="how-card"><h4>3 Β· Aroma β€” 167 odor heads</h4><p>One random forest per descriptor (citrus, floral, woody…), trained on <b>public-domain odor text</b>. A head ships only if it clears <b>CV-AUROC β‰₯ 0.70</b> β€” 167 survive, each shown with its own score. It reads <b>presence, not intensity</b> (free text carries none) β€” an honest ceiling, stated in the UI.</p></div>
827832
<div class="how-card"><h4>4 Β· Honest by design</h4><p>Every value is tagged <b>measured / predicted / estimate</b>, so nothing reads as more precise than it is. Where a quantitative feature needs data we can't ship free-commercially (odor thresholds, panel intensities), the UI <b>says so</b> β€” and it lights up with <b>your</b> data.</p></div>
828833
<div class="how-card"><h4>5 Β· How good is a head, really?</h4><p><b>AUROC alone will lie to you.</b> It measures <i>ranking</i> and is blind to how rare a note is β€” a head with 11 examples among 2,403 molecules can score <b>0.979</b> and still be right just <b>1 time in 10</b> when it fires. So every head also publishes its <b>precision</b>: when it says yes, how often it's actually right, measured only on molecules it never trained on. Each head gets its <b>own</b> firing threshold (0.16&nbsp;–&nbsp;0.85, not a flat 0.5), fitted so a <b>confident</b> call is right more than half the time. <b>23 heads have never been wrong</b> on unseen molecules; the 73 that can't clear 50% are shown as <b>indicative</b> β€” hatched bars, kept in full, never dressed up as confident.</p></div>
829-
<div class="how-card"><h4>6 Β· Formulation, not just molecules</h4><p>The <b>Formulation Studio</b> reads a whole recipe before you pour β€” weighting each ingredient by odor impact, aggregating the blend's note-profile, flagging the overpowering component, and closing the gap to your target. That's single-molecule ML turned into a bench tool.</p></div>
830-
<div class="how-card"><h4>7 Β· On-prem &amp; commercial-clean</h4><p>Nothing leaves the box β€” a read makes <b>no cloud calls</b>. The shipped models train only on <b>public-domain or permissively-licensed</b> data, so the commercial edition stays clean (provenance tracked in the repo).</p></div>
834+
<div class="how-card gloss"><h4>6 Β· Reading the numbers</h4>
835+
<dl class="gl">
836+
<dt>AUROC</dt><dd>A <b>ranking</b> score: given one true example and one false one, how often is the true one scored higher? <b>0.5 = coin flip, 1.0 = never wrong.</b> Says nothing about how often a head is right when it fires.</dd>
837+
<dt>Threshold</dt><dd>The score at which a head counts as <b>firing</b>. Not a flat 0.5 β€” each head has its own, fitted on data it never trained on, ranging <b>0.16&nbsp;–&nbsp;0.85</b>.</dd>
838+
<dt>Precision</dt><dd><b>When it says yes, how often is it right?</b> Trustworthiness β€” and the number AUROC can't see.</dd>
839+
<dt>Recall</dt><dd><b>Of all the real ones, how many did it catch?</b> Thoroughness. Trades against precision.</dd>
840+
<dt>Out&#8209;of&#8209;fold</dt><dd>Scored by a model that <b>never saw that molecule</b>. Every number here is out-of-fold β€” no model grades its own homework.</dd>
841+
<dt>Indicative</dt><dd>A head that can't reach <b>50% precision</b> at any threshold. Kept in full, shown with a hatched bar, never called confident.</dd>
842+
</dl></div>
843+
<div class="how-card"><h4>7 Β· Formulation, not just molecules</h4><p>The <b>Formulation Studio</b> reads a whole recipe before you pour β€” weighting each ingredient by odor impact, aggregating the blend's note-profile, flagging the overpowering component, and closing the gap to your target. That's single-molecule ML turned into a bench tool.</p></div>
844+
<div class="how-card"><h4>8 Β· On-prem &amp; commercial-clean</h4><p>Nothing leaves the box β€” a read makes <b>no cloud calls</b>. The shipped models train only on <b>public-domain or permissively-licensed</b> data, so the commercial edition stays clean (provenance tracked in the repo).</p></div>
831845
</div>
832846
<p class="how-foot">Deeper dives in the repo: <a href="https://github.com/echelonts/flavormancer/blob/main/docs/HOW-IT-WORKS.md" target="_blank" rel="noopener">docs/HOW-IT-WORKS.md</a> (the full method) Β· <a href="https://github.com/echelonts/flavormancer/blob/main/docs/DATA-SOURCES.md" target="_blank" rel="noopener">docs/DATA-SOURCES.md</a> (every source + license) Β· <a href="https://github.com/echelonts/flavormancer/blob/main/docs/AROMA.md" target="_blank" rel="noopener">docs/AROMA.md</a> (the odor heads).</p>
833847
</div>
@@ -2468,7 +2482,7 @@ <h4>Software &amp; type</h4>
24682482
}
24692483
// Taste β€” the basic tastes, namespaced so they don't collide with the like-named aroma notes
24702484
if(tastes.length){
2471-
html += `<div class="studio-section"><div class="ss-head"><svg class="ic"><use href="#ic-flavors"/></svg> Taste <span class="ss-sub">what the tongue reads β€” distinct from a sweet <i>smell</i></span></div><p class="ss-why"><b>Taste is only the five basics</b> the tongue detects β€” sweet, bitter, umami, sour, salty (plus tasteless). It is a genuinely different channel from smell, which is why <code>taste:sweet</code> and the aroma note <code>sweet</code> are separate heads: vanillin <i>smells</i> sweet without being a sweetener. Use taste to set the backbone, notes to give it character.</p><div class="ss-body">`+
2485+
html += `<div class="studio-section"><div class="ss-head"><svg class="ic"><use href="#ic-flavors"/></svg> Taste <span class="ss-sub">what the tongue reads β€” distinct from a sweet <i>smell</i></span></div><p class="ss-why"><b>Taste is the five basics</b> the tongue detects β€” sweet, bitter, umami, sour, salty. We train <b>six</b> heads: those five plus <code>tasteless</code>, which predicts the documented <i>absence</i> of taste and is how you find a clean carrier. It is a genuinely different channel from smell, which is why <code>taste:sweet</code> and the aroma note <code>sweet</code> are separate heads: vanillin <i>smells</i> sweet without being a sweetener. Use taste to set the backbone, notes to give it character.</p><div class="ss-body">`+
24722486
`<div class="studio-group studio-group-notes">`+
24732487
tastes.map(t=>{const lbl=t.replace(/^taste:/,'');
24742488
// set --chip, don't override border/text colour: the per-taste hue now flows through the

0 commit comments

Comments
Β (0)