Skip to content

Commit 111b822

Browse files
refactor(data): resolve names from master_enrichment, the one base table (#224)
app.py built its own name lookup from properties.parquet and then merged iupac_backfill.parquet itself. build_enrichment.py had already done that work — and better — so the two diverged in three ways, all silent: 1. The enrichment build resolves a name curated-list-first, then PubChem common name, then IUPAC name, then a molecular-formula fallback, un-inverting CAS-style ordering along the way. None of that reached app.py's table. 2. The merge read only `iupac_name` from the backfill and ignored the `common_name` column, so every PubChem Title the crawler recovered was thrown away on arrival. 3. Nothing carried the formula fallback, so the molecules that only have one resolved to nothing. Measured before the change: 754 molecules had a name in master_enrichment that app._NAME_TABLE did not know — and not obscure ones. cedrol, fenchyl alcohol, hydroxycitronellal and musk ketone all rendered by name in the grid and came back empty from the UI's own lookup. After: 0. Sourcing the resolved name straight off the base table makes the grid and the lookup agree by construction instead of by coincidence, and deletes a whole merge function rather than adding one. That is what #224 is actually asking for, applied where the duplication genuinely was. Note what is deliberately NOT folded in. profile_index.npz is an 8,850 x 183 matrix — that 183 is the FLAVOUR-PROFILE vector (6 taste + 172 aroma + 5 mouthfeel), not the head count, which is 195. The 12 Tox21 heads are excluded on purpose: safety is not a flavour-match dimension and must never steer "what tastes similar". flavor_map.parquet is a UMAP embedding. Different shapes serving different questions; forcing either into a row-oriented molecule table would be worse, not tidier. odor_notes/taste_notes are documented-text SOURCES the enrichment build consumes — upstream of the base table, not competitors to it. Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>
1 parent 7e58b84 commit 111b822

1 file changed

Lines changed: 24 additions & 29 deletions

File tree

training/app.py

Lines changed: 24 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -215,40 +215,35 @@ def _svg(smi, w=320, h=220):
215215

216216

217217
def _load_name_table():
218-
"""inchikey-skeleton -> (common, IUPAC) from the precomputed enrichment table, so the
219-
whole labeled set resolves instantly and offline. Empty until build_properties.py has
220-
written name columns; live PubChem stays the fallback for anything not in the table."""
218+
"""skeleton -> (common, IUPAC), sourced from master_enrichment — the ONE base table (#224).
219+
220+
This used to read properties.parquet and then merge iupac_backfill.parquet itself, which
221+
duplicated work build_enrichment.py had already done and silently diverged from it in three
222+
ways. The enrichment build resolves a name by curated list first, then PubChem common name,
223+
then IUPAC name, then a molecular-formula fallback — and it un-inverts CAS-style ordering
224+
along the way. None of that reached this table, so 754 molecules the grid displayed by name
225+
(cedrol, fenchyl alcohol, hydroxycitronellal, musk ketone...) resolved to nothing here. The
226+
old merge also read only `iupac_name` from the backfill and ignored the `common_name` column,
227+
so every PubChem Title the crawler recovered was thrown away.
228+
229+
Reading the resolved name straight off the base table makes the UI's lookup and the grid
230+
agree by construction rather than by coincidence.
231+
"""
221232
try:
222233
import pandas as pd
223-
df = pd.read_parquet(P.artifact("properties.parquet"))
224-
if "common_name" not in df.columns:
225-
return {}
226-
out = {}
227-
for ik, c, u in zip(df["inchikey"], df["common_name"], df["iupac_name"]):
228-
if isinstance(ik, str) and (isinstance(c, str) or isinstance(u, str)):
229-
out[ik.split("-")[0]] = (c if isinstance(c, str) else None,
230-
u if isinstance(u, str) else None)
231-
return out
232-
except Exception: # noqa: BLE001 — no table / no pandas; just fall back to live lookups
234+
df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
235+
except Exception: # noqa: BLE001 — no table / no pandas; live lookups still cover it
233236
return {}
237+
out = {}
238+
iupac = dict(zip(df.get("inchikey_skel", []), df.get("iupac_name", []))) # optional column
239+
for skel, name in zip(df["inchikey_skel"], df["name"]):
240+
if isinstance(skel, str) and isinstance(name, str) and name.strip():
241+
u = iupac.get(skel)
242+
out[skel] = (name.strip(), u if isinstance(u, str) else None)
243+
return out
234244

235245

236-
def _merge_iupac_backfill(table):
237-
"""Fold in IUPAC names that build_iupac_backfill.py recovered from PubChem for molecules
238-
the main properties crawl missed (skeleton -> keep any common name, add the IUPAC)."""
239-
try:
240-
import pandas as pd
241-
bf = pd.read_parquet(P.artifact("iupac_backfill.parquet"))
242-
except Exception: # noqa: BLE001 — backfill not built; nothing to merge
243-
return table
244-
for skel, u in zip(bf["inchikey_skel"], bf["iupac_name"]):
245-
if isinstance(skel, str) and isinstance(u, str) and u:
246-
common = table.get(skel, (None, None))[0]
247-
table[skel] = (common, u)
248-
return table
249-
250-
251-
_NAME_TABLE = _merge_iupac_backfill(_load_name_table())
246+
_NAME_TABLE = _load_name_table()
252247

253248

254249
@lru_cache(maxsize=8192)

0 commit comments

Comments
 (0)