Add m2v-multilingual test gate to end2end suite - #86
Conversation
Boots the skill through the candidate default intent engine (the model2vec multilingual classifier) via ovoscope's get_m2v_minicroft and exercises a representative slice of the golden utterances, asserting both routing and the spoken fact content (stubbing numbersapi.com the same way the existing padacioso golden suite does). All five golden cases route to the correct intent and speak the stubbed fact content under the candidate model.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Beep boop! The automated check sequence is complete. 🦾I've aggregated the results of the automated checks for this PR below. 🏷️ Release PreviewWhat's in the box? A preview of the next release! 📦 Current:
🚀 Release Channel Compatibility Predicted next version:
🔍 LintEnsuring the codebase remains stable and healthy. 🛡️ ❌ ruff: issues found — see job log 🎙️ SkillI've peered into the heart of this skill. 💖 ℹ️ Not an OVOS skill repo — check skipped. 🔒 Security (pip-audit)Checking if we're following security best practices. 📏 ✅ No known vulnerabilities found (78 packages scanned). 📋 Repo HealthI've checked the repo's flexibility (aka refactorability). 🧘♂️ ✅ All required files present. Latest Version: ✅ 🌍 Locale BuildThe automated gnomes have finished their shift. 🍄 ✅ Locale properly configured (114 files, 10 languages) Locale directories found:
Localization coverage:
pyproject.toml: ✅
Build manifest: ✅ 228 locale files included in package 🧪 Spec LintEvaluating the overall quality of your PR. ✨ 11 error(s), 10 warning(s) 📊 CoverageA bird's eye view of your test coverage landscape. 🦅 ❌ 40.2% total coverage Per-file coverage (2 files)
Full report: download the ⚖️ License CheckAre we all good on the legal front? Let's find out! 📑 ✅ No license violations found. Policy: Apache 2.0 (universal donor). StrongCopyleft / NetworkCopyleft / WeakCopyleft / Other / Error categories fail. MPL allowed. 🔨 Build TestsChecking the plumbing of your data flows. 🚰 ✅ All versions pass
🔌 Skill Tests (ovoscope)I've checked the skill's 'privacy' (aka data handling). 🔒 ✅ 29/29 passed ✅ TestDateTrivia — 1/1 🚌 Bus CoverageA forensic look at what's being emitted. 🔎 Stay curious and keep coding! 🚀 |
This adds an m2v-multilingual test gate to the end2end suite, part of the ecosystem-wide campaign to validate every default skill against the candidate default intent engine before it ships.
The new test (
test/end2end/test_m2v_gate.py) boots the skill throughovoscope'sget_m2v_minicrofthelper, which loads the model2vec multilingual classifier (OpenVoiceOS/ovos-m2v-intents-multi-128M-v5), and sends five utterances lifted from the existing golden corpus, one per intent plus an extra number-fact phrasing. It stubs thenumbersapi.comfetchers the same waytest_golden_utterances.pyalready does, so the suite stays network-free, and checks that the spoken text is exactly the stubbed fact sentinel for the intent that should have handled it.All five cases route correctly and speak the right stubbed fact under the candidate model — no mispredicts found. The existing padacioso-based golden suite is untouched — this is an addition, not a replacement.
Test plan
pytest test/end2end/test_m2v_gate.py -v— 6 passed (booted live against the real model in a throwaway venv)