Skip to content

Commit d8949bb

Browse files
rvnminers-A-and-Nrvnminers-A-and-N
authored
feat(infra): Docker + compose packaging, and a home for the artifacts (#19, #20) (#272)
Flavormancer could only run on the machine it was built on. This makes it installable. Dockerfile is deliberately a RUNTIME image. Training needs 32 cores and hours; serving needs neither, so the ~1 GB of trained forests and parquet tables mount at run time instead of baking into a layer that would go stale the moment a head is retrained. Base is slim-bookworm rather than alpine because RDKit ships manylinux wheels that want glibc, and building it on musl is hours of pain for nothing. Runs unprivileged; the health check allows a 180s start period because a cold start genuinely loads 190 heads. docker-compose.yml is Compose and not Kubernetes on purpose — one box, a handful of users, which is the real shape of an on-prem install for a flavour house. Postgres is the pgvector image because the substitution index is a nearest-neighbour search over the 177-dimension profile vector, which is exactly what pgvector exists for (#20). The schema in infra/initdb keeps tox OUT of that vector: safety is not a flavour-match dimension and must not steer "what tastes similar". The interesting part is FLAVORMANCER_HOME. Every artifact path in predict.py and app.py was relative to the working directory, because the systemd deployment has always had code and models sharing one folder. That makes the obvious container mount impossible: any bind reaching the models would also shadow app.py, and the container would start with no application code. I wrote that exact bug into the first draft of the compose file and caught it reviewing the mount. So artifacts now resolve through predict.artifact() under FLAVORMANCER_HOME, which defaults to "." — the existing systemd deployment is byte-for-byte unaffected, and a container can bake the code in while mounting the data. Verified both directions: unset resolves to the working directory, set redirects, full suite passes either way. Docker is not installed on the build box, so the image has NOT been built and run end to end. The compose file parses and the schema is plain SQL, but the first real `docker compose up` should be treated as the test. Called out in the PR rather than implied to be verified. Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>
1 parent 48e6929 commit d8949bb

8 files changed

Lines changed: 275 additions & 23 deletions

File tree

‎.dockerignore‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Keep the build context small and the image free of anything that isn't runtime code.
2+
.git
3+
.github
4+
.claude
5+
**/__pycache__
6+
**/*.pyc
7+
.venv
8+
venv
9+
# Trained artifacts are MOUNTED, never baked in — see the Dockerfile header.
10+
**/*.joblib
11+
**/*.parquet
12+
**/*.npz
13+
**/*.onnx
14+
models/
15+
aroma_models/
16+
taste_models/
17+
mouthfeel_models/
18+
tox_models/
19+
onnx_models/
20+
# Dev-only
21+
tests/
22+
docs/
23+
node_modules/
24+
*.md
25+
!README.md

‎.env.example‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Copy to .env and adjust. Every value has a working default, so `docker compose up` runs as-is.
2+
APP_PORT=8000
3+
# Where the trained models + parquet tables live on the host (~1 GB, mounted read-only).
4+
MODELS_DIR=./models
5+
# 0 = auto (three quarters of available cores)
6+
INFER_WORKERS=0
7+
POSTGRES_PASSWORD=flavormancer

‎CHANGELOG.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Changelog
2+
3+
All notable changes to Flavormancer. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
4+
versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
5+
6+
Flavormancer is pre-1.0. **v1.0.0 is the MVP** — the point at which both tracks (the Python
7+
service and the .NET/React Track B) are running, packaged and documented. Until then, minor
8+
versions ship working increments.
9+
10+
## [Unreleased]
11+
12+
## [0.2.0] — 2026-07-30
13+
14+
The honesty release. Every trained head now publishes how good it actually is, and the packaging
15+
exists to install it somewhere other than the machine it was built on.
16+
17+
### Added
18+
- **Per-head calibrated thresholds with a 50% precision floor.** Each of the 190 heads carries a
19+
decision threshold fitted on out-of-fold predictions, plus its measured precision and recall,
20+
published in `/api/heads` and shown on every bar. Heads that cannot be right more than half the
21+
time are marked `indicative` rather than confident — kept in full, never dressed up.
22+
- **`docs/ACCURACY.md`** — a plain-language explanation of AUROC, precision, thresholds and
23+
cross-validation, written to be read without a machine-learning background.
24+
- **`training/audit_generalization.py`** — fires every head across the whole corpus and counts
25+
discoveries outside its training set, to catch heads that memorise rather than learn.
26+
- **Mouthfeel modality** — 5 trigeminal/chemesthesis heads (cooling, warming, pungent, tingling,
27+
astringent), surfaced across reads, cards, chips and the map.
28+
- **Docker packaging** — `Dockerfile`, `docker-compose.yml` with a pgvector-backed Postgres, and a
29+
schema for the substitution index. Artifacts mount at runtime rather than baking into the image.
30+
- `FLAVORMANCER_HOME` so the code and the ~1 GB of trained artifacts can live in different places.
31+
- Numbers glossary in `HOW-IT-WORKS.md` and in the app's own How-it-works panel.
32+
33+
### Changed
34+
- Aroma roster **164 → 166 heads**; confident-capable heads **94 → 108**.
35+
- Chip families (flavor / note / taste / mouthfeel) share one visual language instead of four
36+
accidental ones, and each studio section explains what its dimension *is*.
37+
- Every molecule has a display name: names fall back to molecular formula, with multi-component
38+
structures labelled as mixtures. **8,861 of 8,861 named**, down from 770 blank.
39+
- Milestones relabelled to say what kind of work they hold (Foundations / Track B / Ship).
40+
41+
### Fixed
42+
- `predict_aroma` had silently lost its `@lru_cache` to an orphaned decorator — the fix behind the
43+
40s → 0.01s modal read.
44+
- Substitute/neighbor cards overflowed the modal on mobile (`1fr` will not shrink below
45+
min-content; needed `minmax(0,1fr)`).
46+
- The modal's 14px card gap had never applied, because an inline `display:block` overrode the flex
47+
column and block boxes ignore `gap`.
48+
- A duplicate `pungent` key in the curated supplement that would have deleted six molecules.
49+
50+
### Known limits
51+
- `sweet`, `ethereal` and `pungent` odour heads sit exactly at the precision floor. They are broad
52+
*and* chemically incoherent, so curation cannot lift them — tracked for the GNN work (#199).
53+
- 58 aroma heads remain `indicative` (#262).
54+
- Track B (.NET API, React workbench) is scaffolded but not running (M2/M3).
55+
56+
## [0.1.0] — 2026-07-16
57+
58+
First public demo: taste heads, the aroma descriptor model, substitution search, the flavor-space
59+
map, formulation studio, and the on-prem workbench UI.
60+
61+
[Unreleased]: https://github.com/echelonts/flavormancer/compare/v0.2.0...HEAD
62+
[0.2.0]: https://github.com/echelonts/flavormancer/compare/v0.1.0...v0.2.0
63+
[0.1.0]: https://github.com/echelonts/flavormancer/releases/tag/v0.1.0

‎Dockerfile‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Flavormancer — the on-prem prediction service.
2+
#
3+
# Deliberately a RUNTIME image, not a training one. Model training needs a GPU box, 32 cores and
4+
# several hours; serving needs none of that. Trained artifacts (aroma_models/, taste_models/,
5+
# mouthfeel_models/, tox_models/ and the parquet tables) are mounted at run time rather than baked
6+
# in — they are ~1 GB, they change on every retrain, and burning them into a layer would make the
7+
# image both enormous and stale the moment a head is retrained.
8+
#
9+
# docker build -t flavormancer:latest .
10+
# docker compose up # see docker-compose.yml for the volume wiring
11+
#
12+
# RDKit is the reason for the slim-bookworm base rather than alpine: it ships manylinux wheels
13+
# that need glibc, and building it from source on musl is hours of pain for no benefit.
14+
15+
FROM python:3.12-slim-bookworm AS base
16+
17+
# libxrender/libxext are RDKit's molecule-drawing dependencies (the structure SVGs); libgomp is
18+
# OpenMP, which scikit-learn's forests use for parallel predict_proba.
19+
RUN apt-get update && apt-get install -y --no-install-recommends \
20+
libxrender1 libxext6 libgomp1 curl \
21+
&& rm -rf /var/lib/apt/lists/*
22+
23+
ENV PYTHONUNBUFFERED=1 \
24+
PYTHONDONTWRITEBYTECODE=1 \
25+
PIP_NO_CACHE_DIR=1 \
26+
FLAVORMANCER_HOME=/app/data
27+
28+
WORKDIR /app
29+
30+
# Dependencies first, in their own layer: they change far less often than the source, so an app
31+
# edit rebuilds in seconds instead of reinstalling RDKit.
32+
COPY requirements.txt ./
33+
RUN pip install --no-cache-dir -r requirements.txt \
34+
&& pip install --no-cache-dir "fastapi" "uvicorn[standard]" pydantic
35+
36+
COPY training/ ./training/
37+
38+
# Serving runs as an unprivileged user. The mounted model directory only needs to be readable.
39+
RUN useradd --create-home --shell /usr/sbin/nologin flavormancer \
40+
&& mkdir -p /app/data && chown -R flavormancer:flavormancer /app
41+
USER flavormancer
42+
43+
WORKDIR /app/training
44+
EXPOSE 8000
45+
46+
# /healthz answers before the models finish loading (the app serves a warming page meanwhile), so
47+
# a long start-period is what keeps the container from being killed during a legitimate ~50s
48+
# cold start. See docs/METHODS.md on why loading is serial.
49+
HEALTHCHECK --interval=15s --timeout=5s --start-period=180s --retries=4 \
50+
CMD curl -fsS http://localhost:8000/healthz || exit 1
51+
52+
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

‎docker-compose.yml‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Flavormancer — single-box deployment.
2+
#
3+
# One machine, a handful of users, no orchestration. That is the actual shape of an on-prem
4+
# install for a flavour house, so this is Compose rather than Kubernetes — see
5+
# docs/ARCHITECTURE.md.
6+
#
7+
# docker compose up -d
8+
# curl localhost:8000/healthz
9+
#
10+
# The trained models are NOT in the image. Point MODELS_DIR at wherever they live on the host
11+
# (default ./models) — they are ~1 GB of joblib forests plus the parquet tables, rebuilt by the
12+
# training scripts on a box with cores to spare.
13+
14+
name: flavormancer
15+
16+
services:
17+
app:
18+
build: .
19+
image: flavormancer:latest
20+
restart: unless-stopped
21+
ports:
22+
- "${APP_PORT:-8000}:8000"
23+
environment:
24+
# Serving is CPU-bound on forest inference; leave the box some headroom for Postgres.
25+
FLAVORMANCER_INFER_WORKERS: "${INFER_WORKERS:-0}" # 0 = auto (3/4 of cores)
26+
FLAVORMANCER_HOME: /app/data # where the mounted artifacts live
27+
DATABASE_URL: "postgresql://flavormancer:${POSTGRES_PASSWORD:-flavormancer}@db:5432/flavormancer"
28+
volumes:
29+
# One clean mount, because the app now resolves artifacts under FLAVORMANCER_HOME rather
30+
# than the working directory. Before that, ANY bind that reached the models also shadowed
31+
# app.py — the container would have started with no application code.
32+
- "${MODELS_DIR:-./models}:/app/data:ro"
33+
depends_on:
34+
db:
35+
condition: service_healthy
36+
healthcheck:
37+
test: ["CMD", "curl", "-fsS", "http://localhost:8000/healthz"]
38+
interval: 15s
39+
timeout: 5s
40+
start_period: 180s # a cold start loads 190 heads; see docs/METHODS.md
41+
retries: 4
42+
43+
db:
44+
# pgvector, not plain Postgres: the substitution index is a nearest-neighbour search over the
45+
# 177-dimension flavour-profile vector, which is exactly what pgvector exists for (#20).
46+
image: pgvector/pgvector:pg16
47+
restart: unless-stopped
48+
environment:
49+
POSTGRES_DB: flavormancer
50+
POSTGRES_USER: flavormancer
51+
POSTGRES_PASSWORD: "${POSTGRES_PASSWORD:-flavormancer}"
52+
volumes:
53+
- pgdata:/var/lib/postgresql/data
54+
- ./infra/initdb:/docker-entrypoint-initdb.d:ro
55+
healthcheck:
56+
test: ["CMD-SHELL", "pg_isready -U flavormancer -d flavormancer"]
57+
interval: 10s
58+
timeout: 5s
59+
retries: 5
60+
61+
volumes:
62+
pgdata:

‎infra/initdb/01-schema.sql‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
-- Flavormancer schema (#20).
2+
--
3+
-- The substitution index is a nearest-neighbour search over the flavour-profile vector, so the
4+
-- vector lives in the database rather than being recomputed per query. 177 dimensions:
5+
-- 6 taste + 166 aroma + 5 mouthfeel. Tox is deliberately NOT in the vector — safety is not a
6+
-- flavour-match dimension, and letting it steer "what tastes similar" would be wrong.
7+
CREATE EXTENSION IF NOT EXISTS vector;
8+
9+
CREATE TABLE IF NOT EXISTS molecule (
10+
inchikey_skel text PRIMARY KEY, -- stereo-agnostic connectivity skeleton
11+
smiles text NOT NULL,
12+
name text, -- never null in practice: falls back to formula
13+
mw real,
14+
logp real,
15+
tpsa real,
16+
food_listed boolean DEFAULT false, -- open-gov register listing, NOT a safety clearance
17+
taste_documented text
18+
);
19+
20+
CREATE TABLE IF NOT EXISTS molecule_profile (
21+
inchikey_skel text PRIMARY KEY REFERENCES molecule(inchikey_skel) ON DELETE CASCADE,
22+
profile vector(177) NOT NULL,
23+
aromas text[] -- heads clearing their own calibrated threshold
24+
);
25+
26+
-- Cosine distance: the profile is a direction in flavour space, and two molecules with the same
27+
-- balance of notes at different intensities should still read as neighbours.
28+
CREATE INDEX IF NOT EXISTS molecule_profile_cos
29+
ON molecule_profile USING hnsw (profile vector_cosine_ops);
30+
31+
CREATE INDEX IF NOT EXISTS molecule_food_listed ON molecule (food_listed);

‎training/app.py‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -164,13 +164,13 @@ def _load_name2smiles():
164164
idx = {}
165165
with contextlib.suppress(Exception): # table absent / no pandas; live lookup still covers it
166166
import pandas as pd
167-
df = pd.read_parquet("master_enrichment.parquet")
167+
df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
168168
for nm, smi in zip(df["name"], df["smiles"]):
169169
if isinstance(nm, str) and isinstance(smi, str) and nm.strip() and smi.strip():
170170
idx.setdefault(nm.strip().lower(), smi)
171171
with contextlib.suppress(Exception): # no suggest file; fine
172172
import csv
173-
with open("flavor_volatiles.csv", encoding="utf-8") as fh:
173+
with open(P.artifact("flavor_volatiles.csv"), encoding="utf-8") as fh:
174174
for r in csv.DictReader(fh):
175175
if r.get("name") and r.get("smiles"):
176176
idx.setdefault(r["name"].strip().lower(), r["smiles"])
@@ -220,7 +220,7 @@ def _load_name_table():
220220
written name columns; live PubChem stays the fallback for anything not in the table."""
221221
try:
222222
import pandas as pd
223-
df = pd.read_parquet("properties.parquet")
223+
df = pd.read_parquet(P.artifact("properties.parquet"))
224224
if "common_name" not in df.columns:
225225
return {}
226226
out = {}
@@ -238,7 +238,7 @@ def _merge_iupac_backfill(table):
238238
the main properties crawl missed (skeleton -> keep any common name, add the IUPAC)."""
239239
try:
240240
import pandas as pd
241-
bf = pd.read_parquet("iupac_backfill.parquet")
241+
bf = pd.read_parquet(P.artifact("iupac_backfill.parquet"))
242242
except Exception: # noqa: BLE001 — backfill not built; nothing to merge
243243
return table
244244
for skel, u in zip(bf["inchikey_skel"], bf["iupac_name"]):
@@ -306,7 +306,7 @@ def _load_spectra():
306306
public-domain PubChem availability metadata. Empty until the crawl has run."""
307307
try:
308308
import pandas as pd
309-
df = pd.read_parquet("spectra.parquet")
309+
df = pd.read_parquet(P.artifact("spectra.parquet"))
310310
labels = [("has_ms", "MS"), ("has_ir", "IR"), ("has_nmr", "NMR"),
311311
("has_uv", "UV"), ("has_raman", "Raman")]
312312
out = {}
@@ -1403,7 +1403,7 @@ def _precompute_top_lists():
14031403
aroma heads over the odor corpus. Model-derived: honest 'what the tool predicts'."""
14041404
import pandas as pd
14051405
with contextlib.suppress(Exception): # no taste data; skip taste lists
1406-
tm = pd.read_parquet("taste_master.parquet")
1406+
tm = pd.read_parquet(P.artifact("taste_master.parquet"))
14071407
for taste, clf in P._CLASSIFIERS.items():
14081408
ranked = _rank(tm["smiles"], lambda X, c=clf: c.predict_proba(X)[:, 1])
14091409
_TOP_LISTS[f"taste:{taste}"] = {"label": f"Top {taste}", "items": _named_top(ranked)}
@@ -1418,7 +1418,7 @@ def _precompute_top_lists():
14181418
# skews industrial, so ranking by a head surfaces confident-but-odd picks (cyanide under
14191419
# "almond"). Documented examples are real, recognizable, and honest ("documented citrus").
14201420
from build_aroma_dataset import tag as _odor_tag
1421-
od = pd.read_parquet("odor_notes.parquet")
1421+
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
14221422
by_desc = {}
14231423
for _, r in od.iterrows():
14241424
nm, odor = r.get("name"), r.get("odor")
@@ -1456,7 +1456,7 @@ def _load_flavor_map():
14561456
+ 3D (x3,y3,z3) coordinates normalized to 0..1 with names — an interactive scatter / cloud."""
14571457
try:
14581458
import pandas as pd
1459-
df = pd.read_parquet("flavor_map.parquet")
1459+
df = pd.read_parquet(P.artifact("flavor_map.parquet"))
14601460
# UMAP occasionally emits NaN coords for a few near-duplicate rows — drop them so the
14611461
# JSON stays valid (NaN isn't JSON-compliant) and the scatter has no phantom points.
14621462
df = df.dropna(subset=[c for c in ("x", "y", "x3", "y3", "z3") if c in df.columns]).reset_index(drop=True)
@@ -1570,7 +1570,7 @@ def _precompute_design():
15701570
import numpy as np
15711571
import pandas as pd
15721572
from build_aroma_dataset import tag as _odor_tag
1573-
od = pd.read_parquet("odor_notes.parquet")
1573+
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
15741574
rows = [] # (smiles, name, mol_skeleton, {documented tags})
15751575
for smi, nm, odor in zip(od["smiles"], od.get("name", [None] * len(od)), od["odor"]):
15761576
mol = Chem.MolFromSmiles(str(smi)) if isinstance(smi, str) else None
@@ -1880,7 +1880,7 @@ def _load_enrichment():
18801880
"""Rows from master_enrichment.parquet with taste collapsed to a display string."""
18811881
try:
18821882
import pandas as pd
1883-
df = pd.read_parquet("master_enrichment.parquet")
1883+
df = pd.read_parquet(P.artifact("master_enrichment.parquet"))
18841884
except Exception: # noqa: BLE001 — not built yet
18851885
return []
18861886
def _s(v): # NaN (a truthy float) -> "" ; keep real strings
@@ -2069,7 +2069,7 @@ def _load_odor_table():
20692069
build_odor_notes.py has run; tolerant of older tables without the threshold columns."""
20702070
try:
20712071
import pandas as pd
2072-
df = pd.read_parquet("odor_notes.parquet")
2072+
df = pd.read_parquet(P.artifact("odor_notes.parquet"))
20732073

20742074
def col(name):
20752075
return df[name] if name in df.columns else [None] * len(df)
@@ -2106,13 +2106,13 @@ def _load_documented_full():
21062106
out = {}
21072107
with contextlib.suppress(Exception):
21082108
import pandas as pd
2109-
od = pd.read_parquet("odor_notes.parquet")
2109+
od = pd.read_parquet(P.artifact("odor_notes.parquet"))
21102110
for ik, odor in zip(od["inchikey"], od["odor"]):
21112111
if isinstance(ik, str) and isinstance(odor, str) and odor.strip():
21122112
out.setdefault(ik, {})["odor"] = odor.strip().split("\n")[0][:160]
21132113
with contextlib.suppress(Exception):
21142114
import pandas as pd
2115-
tn = pd.read_parquet("taste_notes.parquet")
2115+
tn = pd.read_parquet(P.artifact("taste_notes.parquet"))
21162116
for ik, taste in zip(tn["inchikey"], tn["taste"]):
21172117
if isinstance(ik, str) and isinstance(taste, str) and taste.strip():
21182118
out.setdefault(ik, {})["taste"] = taste.strip().split("\n")[0][:160]

0 commit comments

Comments
 (0)