Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Keep the build context small and the image free of anything that isn't runtime code.
.git
.github
.claude
**/__pycache__
**/*.pyc
.venv
venv
# Trained artifacts are MOUNTED, never baked in — see the Dockerfile header.
**/*.joblib
**/*.parquet
**/*.npz
**/*.onnx
models/
aroma_models/
taste_models/
mouthfeel_models/
tox_models/
onnx_models/
# Dev-only
tests/
docs/
node_modules/
*.md
!README.md
7 changes: 7 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Copy to .env and adjust. Every value has a working default, so `docker compose up` runs as-is.
APP_PORT=8000
# Where the trained models + parquet tables live on the host (~1 GB, mounted read-only).
MODELS_DIR=./models
# 0 = auto (three quarters of available cores)
INFER_WORKERS=0
POSTGRES_PASSWORD=flavormancer
63 changes: 63 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Changelog

All notable changes to Flavormancer. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

Flavormancer is pre-1.0. **v1.0.0 is the MVP** — the point at which both tracks (the Python
service and the .NET/React Track B) are running, packaged and documented. Until then, minor
versions ship working increments.

## [Unreleased]

## [0.2.0] — 2026-07-30

The honesty release. Every trained head now publishes how good it actually is, and the packaging
exists to install it somewhere other than the machine it was built on.

### Added
- **Per-head calibrated thresholds with a 50% precision floor.** Each of the 190 heads carries a
decision threshold fitted on out-of-fold predictions, plus its measured precision and recall,
published in `/api/heads` and shown on every bar. Heads that cannot be right more than half the
time are marked `indicative` rather than confident — kept in full, never dressed up.
- **`docs/ACCURACY.md`** — a plain-language explanation of AUROC, precision, thresholds and
cross-validation, written to be read without a machine-learning background.
- **`training/audit_generalization.py`** — fires every head across the whole corpus and counts
discoveries outside its training set, to catch heads that memorise rather than learn.
- **Mouthfeel modality** — 5 trigeminal/chemesthesis heads (cooling, warming, pungent, tingling,
astringent), surfaced across reads, cards, chips and the map.
- **Docker packaging** — `Dockerfile`, `docker-compose.yml` with a pgvector-backed Postgres, and a
schema for the substitution index. Artifacts mount at runtime rather than baking into the image.
- `FLAVORMANCER_HOME` so the code and the ~1 GB of trained artifacts can live in different places.
- Numbers glossary in `HOW-IT-WORKS.md` and in the app's own How-it-works panel.

### Changed
- Aroma roster **164 → 166 heads**; confident-capable heads **94 → 108**.
- Chip families (flavor / note / taste / mouthfeel) share one visual language instead of four
accidental ones, and each studio section explains what its dimension *is*.
- Every molecule has a display name: names fall back to molecular formula, with multi-component
structures labelled as mixtures. **8,861 of 8,861 named**, down from 770 blank.
- Milestones relabelled to say what kind of work they hold (Foundations / Track B / Ship).

### Fixed
- `predict_aroma` had silently lost its `@lru_cache` to an orphaned decorator — the fix behind the
40s → 0.01s modal read.
- Substitute/neighbor cards overflowed the modal on mobile (`1fr` will not shrink below
min-content; needed `minmax(0,1fr)`).
- The modal's 14px card gap had never applied, because an inline `display:block` overrode the flex
column and block boxes ignore `gap`.
- A duplicate `pungent` key in the curated supplement that would have deleted six molecules.

### Known limits
- `sweet`, `ethereal` and `pungent` odour heads sit exactly at the precision floor. They are broad
*and* chemically incoherent, so curation cannot lift them — tracked for the GNN work (#199).
- 58 aroma heads remain `indicative` (#262).
- Track B (.NET API, React workbench) is scaffolded but not running (M2/M3).

## [0.1.0] — 2026-07-16

First public demo: taste heads, the aroma descriptor model, substitution search, the flavor-space
map, formulation studio, and the on-prem workbench UI.

[Unreleased]: https://github.com/echelonts/flavormancer/compare/v0.2.0...HEAD
[0.2.0]: https://github.com/echelonts/flavormancer/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/echelonts/flavormancer/releases/tag/v0.1.0
52 changes: 52 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Flavormancer — the on-prem prediction service.
#
# Deliberately a RUNTIME image, not a training one. Model training needs a GPU box, 32 cores and
# several hours; serving needs none of that. Trained artifacts (aroma_models/, taste_models/,
# mouthfeel_models/, tox_models/ and the parquet tables) are mounted at run time rather than baked
# in — they are ~1 GB, they change on every retrain, and burning them into a layer would make the
# image both enormous and stale the moment a head is retrained.
#
# docker build -t flavormancer:latest .
# docker compose up # see docker-compose.yml for the volume wiring
#
# RDKit is the reason for the slim-bookworm base rather than alpine: it ships manylinux wheels
# that need glibc, and building it from source on musl is hours of pain for no benefit.

FROM python:3.12-slim-bookworm AS base

# libxrender/libxext are RDKit's molecule-drawing dependencies (the structure SVGs); libgomp is
# OpenMP, which scikit-learn's forests use for parallel predict_proba.
RUN apt-get update && apt-get install -y --no-install-recommends \
libxrender1 libxext6 libgomp1 curl \
&& rm -rf /var/lib/apt/lists/*

ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PIP_NO_CACHE_DIR=1 \
FLAVORMANCER_HOME=/app/data

WORKDIR /app

# Dependencies first, in their own layer: they change far less often than the source, so an app
# edit rebuilds in seconds instead of reinstalling RDKit.
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt \
&& pip install --no-cache-dir "fastapi" "uvicorn[standard]" pydantic

COPY training/ ./training/

# Serving runs as an unprivileged user. The mounted model directory only needs to be readable.
RUN useradd --create-home --shell /usr/sbin/nologin flavormancer \
&& mkdir -p /app/data && chown -R flavormancer:flavormancer /app
USER flavormancer

WORKDIR /app/training
EXPOSE 8000

# /healthz answers before the models finish loading (the app serves a warming page meanwhile), so
# a long start-period is what keeps the container from being killed during a legitimate ~50s
# cold start. See docs/METHODS.md on why loading is serial.
HEALTHCHECK --interval=15s --timeout=5s --start-period=180s --retries=4 \
CMD curl -fsS http://localhost:8000/healthz || exit 1

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
62 changes: 62 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Flavormancer — single-box deployment.
#
# One machine, a handful of users, no orchestration. That is the actual shape of an on-prem
# install for a flavour house, so this is Compose rather than Kubernetes — see
# docs/ARCHITECTURE.md.
#
# docker compose up -d
# curl localhost:8000/healthz
#
# The trained models are NOT in the image. Point MODELS_DIR at wherever they live on the host
# (default ./models) — they are ~1 GB of joblib forests plus the parquet tables, rebuilt by the
# training scripts on a box with cores to spare.

name: flavormancer

services:
app:
build: .
image: flavormancer:latest
restart: unless-stopped
ports:
- "${APP_PORT:-8000}:8000"
environment:
# Serving is CPU-bound on forest inference; leave the box some headroom for Postgres.
FLAVORMANCER_INFER_WORKERS: "${INFER_WORKERS:-0}" # 0 = auto (3/4 of cores)
FLAVORMANCER_HOME: /app/data # where the mounted artifacts live
DATABASE_URL: "postgresql://flavormancer:${POSTGRES_PASSWORD:-flavormancer}@db:5432/flavormancer"
volumes:
# One clean mount, because the app now resolves artifacts under FLAVORMANCER_HOME rather
# than the working directory. Before that, ANY bind that reached the models also shadowed
# app.py — the container would have started with no application code.
- "${MODELS_DIR:-./models}:/app/data:ro"
depends_on:
db:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-fsS", "http://localhost:8000/healthz"]
interval: 15s
timeout: 5s
start_period: 180s # a cold start loads 190 heads; see docs/METHODS.md
retries: 4

db:
# pgvector, not plain Postgres: the substitution index is a nearest-neighbour search over the
# 177-dimension flavour-profile vector, which is exactly what pgvector exists for (#20).
image: pgvector/pgvector:pg16
restart: unless-stopped
environment:
POSTGRES_DB: flavormancer
POSTGRES_USER: flavormancer
POSTGRES_PASSWORD: "${POSTGRES_PASSWORD:-flavormancer}"
volumes:
- pgdata:/var/lib/postgresql/data
- ./infra/initdb:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U flavormancer -d flavormancer"]
interval: 10s
timeout: 5s
retries: 5

volumes:
pgdata:
31 changes: 31 additions & 0 deletions infra/initdb/01-schema.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
-- Flavormancer schema (#20).
--
-- The substitution index is a nearest-neighbour search over the flavour-profile vector, so the
-- vector lives in the database rather than being recomputed per query. 177 dimensions:
-- 6 taste + 166 aroma + 5 mouthfeel. Tox is deliberately NOT in the vector — safety is not a
-- flavour-match dimension, and letting it steer "what tastes similar" would be wrong.
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS molecule (
inchikey_skel text PRIMARY KEY, -- stereo-agnostic connectivity skeleton
smiles text NOT NULL,
name text, -- never null in practice: falls back to formula
mw real,
logp real,
tpsa real,
food_listed boolean DEFAULT false, -- open-gov register listing, NOT a safety clearance
taste_documented text
);

CREATE TABLE IF NOT EXISTS molecule_profile (
inchikey_skel text PRIMARY KEY REFERENCES molecule(inchikey_skel) ON DELETE CASCADE,
profile vector(177) NOT NULL,
aromas text[] -- heads clearing their own calibrated threshold
);

-- Cosine distance: the profile is a direction in flavour space, and two molecules with the same
-- balance of notes at different intensities should still read as neighbours.
CREATE INDEX IF NOT EXISTS molecule_profile_cos
ON molecule_profile USING hnsw (profile vector_cosine_ops);

CREATE INDEX IF NOT EXISTS molecule_food_listed ON molecule (food_listed);
26 changes: 13 additions & 13 deletions training/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,13 @@ def _load_name2smiles():
idx = {}
with contextlib.suppress(Exception): # table absent / no pandas; live lookup still covers it
import pandas as pd
df = pd.read_parquet("master_enrichment.parquet")
df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
for nm, smi in zip(df["name"], df["smiles"]):
if isinstance(nm, str) and isinstance(smi, str) and nm.strip() and smi.strip():
idx.setdefault(nm.strip().lower(), smi)
with contextlib.suppress(Exception): # no suggest file; fine
import csv
with open("flavor_volatiles.csv", encoding="utf-8") as fh:
with open(P.artifact("flavor_volatiles.csv"), encoding="utf-8") as fh:
for r in csv.DictReader(fh):
if r.get("name") and r.get("smiles"):
idx.setdefault(r["name"].strip().lower(), r["smiles"])
Expand Down Expand Up @@ -220,7 +220,7 @@ def _load_name_table():
written name columns; live PubChem stays the fallback for anything not in the table."""
try:
import pandas as pd
df = pd.read_parquet("properties.parquet")
df = pd.read_parquet(P.artifact("properties.parquet"))
if "common_name" not in df.columns:
return {}
out = {}
Expand All @@ -238,7 +238,7 @@ def _merge_iupac_backfill(table):
the main properties crawl missed (skeleton -> keep any common name, add the IUPAC)."""
try:
import pandas as pd
bf = pd.read_parquet("iupac_backfill.parquet")
bf = pd.read_parquet(P.artifact("iupac_backfill.parquet"))
except Exception: # noqa: BLE001 — backfill not built; nothing to merge
return table
for skel, u in zip(bf["inchikey_skel"], bf["iupac_name"]):
Expand Down Expand Up @@ -306,7 +306,7 @@ def _load_spectra():
public-domain PubChem availability metadata. Empty until the crawl has run."""
try:
import pandas as pd
df = pd.read_parquet("spectra.parquet")
df = pd.read_parquet(P.artifact("spectra.parquet"))
labels = [("has_ms", "MS"), ("has_ir", "IR"), ("has_nmr", "NMR"),
("has_uv", "UV"), ("has_raman", "Raman")]
out = {}
Expand Down Expand Up @@ -1403,7 +1403,7 @@ def _precompute_top_lists():
aroma heads over the odor corpus. Model-derived: honest 'what the tool predicts'."""
import pandas as pd
with contextlib.suppress(Exception): # no taste data; skip taste lists
tm = pd.read_parquet("taste_master.parquet")
tm = pd.read_parquet(P.artifact("taste_master.parquet"))
for taste, clf in P._CLASSIFIERS.items():
ranked = _rank(tm["smiles"], lambda X, c=clf: c.predict_proba(X)[:, 1])
_TOP_LISTS[f"taste:{taste}"] = {"label": f"Top {taste}", "items": _named_top(ranked)}
Expand All @@ -1418,7 +1418,7 @@ def _precompute_top_lists():
# skews industrial, so ranking by a head surfaces confident-but-odd picks (cyanide under
# "almond"). Documented examples are real, recognizable, and honest ("documented citrus").
from build_aroma_dataset import tag as _odor_tag
od = pd.read_parquet("odor_notes.parquet")
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
by_desc = {}
for _, r in od.iterrows():
nm, odor = r.get("name"), r.get("odor")
Expand Down Expand Up @@ -1456,7 +1456,7 @@ def _load_flavor_map():
+ 3D (x3,y3,z3) coordinates normalized to 0..1 with names — an interactive scatter / cloud."""
try:
import pandas as pd
df = pd.read_parquet("flavor_map.parquet")
df = pd.read_parquet(P.artifact("flavor_map.parquet"))
# UMAP occasionally emits NaN coords for a few near-duplicate rows — drop them so the
# JSON stays valid (NaN isn't JSON-compliant) and the scatter has no phantom points.
df = df.dropna(subset=[c for c in ("x", "y", "x3", "y3", "z3") if c in df.columns]).reset_index(drop=True)
Expand Down Expand Up @@ -1570,7 +1570,7 @@ def _precompute_design():
import numpy as np
import pandas as pd
from build_aroma_dataset import tag as _odor_tag
od = pd.read_parquet("odor_notes.parquet")
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
rows = [] # (smiles, name, mol_skeleton, {documented tags})
for smi, nm, odor in zip(od["smiles"], od.get("name", [None] * len(od)), od["odor"]):
mol = Chem.MolFromSmiles(str(smi)) if isinstance(smi, str) else None
Expand Down Expand Up @@ -1880,7 +1880,7 @@ def _load_enrichment():
"""Rows from master_enrichment.parquet with taste collapsed to a display string."""
try:
import pandas as pd
df = pd.read_parquet("master_enrichment.parquet")
df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
except Exception: # noqa: BLE001 — not built yet
return []
def _s(v): # NaN (a truthy float) -> "" ; keep real strings
Expand Down Expand Up @@ -2069,7 +2069,7 @@ def _load_odor_table():
build_odor_notes.py has run; tolerant of older tables without the threshold columns."""
try:
import pandas as pd
df = pd.read_parquet("odor_notes.parquet")
df = pd.read_parquet(P.artifact("odor_notes.parquet"))

def col(name):
return df[name] if name in df.columns else [None] * len(df)
Expand Down Expand Up @@ -2106,13 +2106,13 @@ def _load_documented_full():
out = {}
with contextlib.suppress(Exception):
import pandas as pd
od = pd.read_parquet("odor_notes.parquet")
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
for ik, odor in zip(od["inchikey"], od["odor"]):
if isinstance(ik, str) and isinstance(odor, str) and odor.strip():
out.setdefault(ik, {})["odor"] = odor.strip().split("\n")[0][:160]
with contextlib.suppress(Exception):
import pandas as pd
tn = pd.read_parquet("taste_notes.parquet")
tn = pd.read_parquet(P.artifact("taste_notes.parquet"))
for ik, taste in zip(tn["inchikey"], tn["taste"]):
if isinstance(ik, str) and isinstance(taste, str) and taste.strip():
out.setdefault(ik, {})["taste"] = taste.strip().split("\n")[0][:160]
Expand Down
Loading
Loading