feat(training): GRAS cross-check from FDA SAF (public domain) - #57
Merged
Merged
Conversation
…AF (public domain) The second clean data win. The FDA "Substances Added to Food" (SAF) inventory is a US-government work in the public domain; build_gras_reference.py downloads it, parses the cp1252 CSV, extracts CAS numbers, resolves them to InChIKey via PubChem, and writes gras_reference.parquet — the table predict.py cross-checks for "is this a recognized food ingredient?" (defensive signal, never a clearance). Tested end-to-end on the live SAF: 3,971 rows -> 3,969 valid CAS, resolving cleanly (styrene, anisole, benzyl alcohol, ...). Full build ~20 min, run locally; data is gitignored, only the fetcher is committed. Signed-off-by: Austin L. <86896075+rvnminers-A-and-N@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The second clean data win — built and tested.
The FDA "Substances Added to Food" (SAF) inventory is a US-government work in the public domain.
training/build_gras_reference.pydownloads it, parses the cp1252 CSV (4-line preamble, comma-delimited), extracts CAS numbers, resolves them to InChIKey via PubChem, and writesgras_reference.parquet— the tablepredict.pycross-checks for "is this a recognized food ingredient at all?" (a defensive signal, never a clearance).Tested live on the real SAF: 3,971 rows → 3,969 valid CAS, resolving correctly (styrene → PPBRXRYQALVLMV…, etc.; the few misses are obscure extracts with no single structure).
ruffclean. Full build ~20 min; data is gitignored, only the fetcher is committed.With this, both data-gated features that can be filled with commercial-clean data are now buildable: measured BP/VP (PubChem, #56) and GRAS (FDA SAF). Quantitative dosing + RI remain customer/licensed (documented).