research(model): GNN for structure→property vs the RandomForest heads

[rvnminers-A-and-N]

Prototype a Graph Neural Network (message-passing over the molecular graph) as a stronger structure→property model; benchmark honestly vs the RF-on-fingerprints heads (same CV-AUROC bar). Framework TBD (chemprop / PyG / DGL). GPU-gated (#4).

Tracks internal task #28.

[rvnminers-A-and-N]

Concrete motivation from the 164-head expansion (#208): several heads sit at CV-AUROC ≈ 1.00 — wintergreen, oakmoss, myrrh, cinnamon, musky. I tested whether that's fixable overfitting by adding structurally-diverse examples to each; they stayed at 1.00. That confirms the percept is a single chemical family (salicylates = wintergreen, orcinol esters = oakmoss, furanosesquiterpenes = myrrh) — the Morgan-FP RandomForest separates them trivially. sweet-odor and musty sit at the opposite failure mode (diffuse, ~0.66–0.72). Both extremes are the case for a learned representation (GNN / principal-odor-map) rather than more fingerprint data. See docs/AROMA-AUDIT.md 'honesty rails'.

[rvnminers-A-and-N]

Folding in what #256 / #258 established, because it sharpens what this issue has to prove and gives it a ready-made scoreboard.

Scope: all 190 heads, not a subset. Re-running the whole roster is the only comparison that means anything — a GNN evaluated on a hand-picked handful would just be us choosing the cases it wins. 167 aroma + 6 taste + 5 mouthfeel + 12 safety, same corpus, same folds.

Two heads are already known to be unreachable by curation, and they are the sharpest test.

head	AUROC	n_pos	novel @0.5	novel @0.35
turmeric	0.983	11	0	0
celery	0.956	14	0	0

Both memorize at any threshold — they fire on their own training molecules and nothing else. #258 added structurally diverse positives to both and neither moved. They are single-scaffold classes (bisabolane sesquiterpenes; phthalides), and a 2048-bit Morgan fingerprint gives a forest no way to represent "a phthalide with a different side chain" as near-neighbours of the ones it saw. That is a representation limit, not a data limit, which is exactly the claim a GNN is supposed to answer.

The other end of the same argument: sweet odour (0.726 / 203 positives) and pungent (0.794 / 287) are broad, fuzzy, multi-scaffold classes that are not thin at all — they are simply hard. Curation will not lift them either.

So the roster hands this issue a clean two-sided benchmark. A GNN that is genuinely a better representation should move both ends:

• turmeric / celery — novel above zero. Learning the scaffold family rather than the members.
• sweet / pungent — AUROC up without novel collapsing. Learning a fuzzy class rather than memorising more of it.

Use the same instrument. training/audit_generalization.py runs against any head set; a GNN with a higher AUROC and the same zero discovery count would be no improvement at all, and only this test will show that. AUROC alone cannot — see the "Does a head LEARN, or just MEMORIZE?" section in docs/METHODS.md.

Baseline to beat, as of #258: 167 aroma heads — 153 generalize, 12 under-confident, 2 memorizing; median 6 novel discoveries at 0.5.

Everything stays additive: the GNN is evaluated against the forests, not as a replacement that quietly drops the heads it can't fit.

[rvnminers-A-and-N]

One more benchmark for the GNN, from #261/#262 — and it's a sharper one than AUROC.

Calibrating per-head thresholds revealed that 73 of 167 aroma heads never reach 50% out-of-fold precision at any threshold. AUROC completely hid this, because AUROC is computed on ranking and is insensitive to class imbalance while precision is not. ginger scores AUROC 0.979 / precision 0.10. coffee scores 0.960 / 0.50. The forests already look excellent on the metric we were reporting.

So a GNN that only beats the forests on AUROC has proved nothing we didn't already have. The claim worth testing is whether a learned representation raises precision where a 2048-bit Morgan fingerprint can't separate the class.

Baseline to beat, all 190 heads:

confident-capable (≥50% precision)	94 / 167
indicative	73
generalizing at calibrated threshold	146
precision-limited	19
memorizing	2 (turmeric, celery)

Run training/audit_generalization.py against the GNN heads for a like-for-like table. Watch for the degenerate win: a head can reach the precision floor by simply refusing to fire, so novel discoveries must hold up alongside precision.

[rvnminers-A-and-N]

Pre-registering a prediction from #257, so the GNN has something falsifiable to be measured against rather than a vague "should be better".

Curation was run on four heads that all looked like broad umbrella categories. The results split cleanly:

head	n_pos	AUROC	precision	after curation
fatty	27 → 41	0.854 → 0.917	0.21 → 0.51	fixed
sweet	208 → 210	0.718 → 0.727	0.50 → 0.50	unmoved
ethereal	98 → 106	0.924 → 0.920	0.50 → 0.50	unmoved
pungent	289 → 295	0.792 → 0.802	0.50 → 0.50	unmoved

"Broad" was not the discriminator — chemical coherence was. fatty is broad but structurally crisp (mid-chain aldehydes, free fatty acids, lactones), so a fingerprint can find it and sixteen molecules more than doubled its precision. sweet odour is broad and incoherent: a sweet smell can be a lactone, a furanone, a vanillin, an ester, or carbon tetrachloride. There is no shared substructure for a 2048-bit Morgan fingerprint to key on, so no amount of curation helps.

So the prediction to test: a learned representation should lift precision on the incoherent heads — sweet, ethereal, pungent, floral (0.19 precision on 39 positives) — while roughly matching the forests on the coherent ones. If a GNN improves fatty and leaves sweet where it is, it has not solved the problem this issue exists for; it has just been a slightly better fingerprint.

Baseline after #270 and the #257 round: 166 aroma heads, 108 confident-capable, 58 indicative. Measure with training/audit_generalization.py.