Skip to content

feat: package the crawler as a multi-source aroma data skill - #2

Merged
rastagan-git merged 1 commit into
mainfrom
agent/multi-source-aroma-skill
Jul 19, 2026
Merged

rastagan-git merged 1 commit into
mainfrom
agent/multi-source-aroma-skill

Conversation

@rastagan-git

Copy link
Copy Markdown
Owner

Summary

  • Replace the four standalone workbook scripts with an installable flavor-data-crawler package and a unified flavor-data CLI.
  • Preserve the original entry points as compatibility wrappers while adding typed statuses, provenance, caching, retries, atomic writes, and recovery checkpoints.
  • Add source adapters for PubChem, selected pinned Pyrfume archives, and pinned M2OR receptor-assay evidence alongside the existing NIST and MFFI workflows.
  • Add an English-first README with a reciprocal Chinese README, a reusable Codex skill, packaging metadata, offline tests, and cross-platform CI.

Compatibility

  • The original .py and .bat entry points remain available with their established workbook defaults.
  • Commands create sibling outputs and refuse accidental overwrites unless --force is explicit.
  • ChemicalBook compatibility remains permission-gated because the current robots policy excludes the legacy routes; the implementation does not bypass CAPTCHA or access controls.

Data sources

  • PubChem: identifiers, properties, synonyms, and attributed odor annotations.
  • Pyrfume: allowlisted, commit-pinned archives with archive-level source and rights notes.
  • M2OR: commit-pinned molecule–olfactory-receptor assay evidence with species, responsiveness, receptor, and DOI context.
  • NIST and MFFI: retained and wrapped with conservative access behavior and provenance.

Validation

  • ruff check .
  • ruff format --check .
  • pytest -q — 46 passed
  • Official skill validator — passed
  • Package dependency check and wheel build — passed
  • Live smoke checks — PubChem, NIST, Pyrfume, and M2OR returned expected records

Licensing and access

No project license is added by this PR, and no downloaded upstream dataset is bundled. Provider-specific citation, access, and redistribution notes remain explicit in the documentation and output provenance.

Closes #1

@rastagan-git
rastagan-git marked this pull request as ready for review July 19, 2026 12:21
@rastagan-git
rastagan-git merged commit 1a97647 into main Jul 19, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Modernize the crawler into a provenance-aware, multi-source odorant data skill

1 participant