fix(stt/llm): multilingual recognition fixes — auto-detect, code-switching, mangled tech terms, and answer-repair stubs

[Treamz]

Summary

Six stacked fixes for multilingual (ru/uk/en) meetings, found while debugging why Auto-Detect Language and mixed-language speech worked poorly end-to-end:

1. Auto Detect silently pinned Google STT to English. AppState.setRecognitionLanguage rewrote 'auto' → 'english-us' for every non-Natively provider, and GoogleSTT's auto branch hardcoded fr/es/de alternates (Google v1 caps them at 3), so Russian/Ukrainian could never be detected. The substitution is removed (every provider handles 'auto' natively — Deepgram multi, Soniox/ElevenLabs omit the hint), and Google's alternates are now derived from the OS preferred languages via googleAutoDetectAlternates().

2. Every Google STT restart lost 1s+ of speech. The abandoned gRPC stream's async close/end events fired after a restart had created the new stream and nulled this.stream, so audio buffered until the 1/s lazy reconnect. Handlers now capture their own stream and ignore stale events. Restarts (language change, 4:30 proactive) also switched from stream.destroy() to a graceful end() swap, so the tail final of the last phrase is flushed instead of killed mid-flight.

3. Auto mode never adapted its primary language. Google reports result.languageCode per phrase but the client ignored it — a meeting held in Russian ran every phrase through the slower alternate-language path forever. After two consecutive finals in a non-primary language, the stream now re-pins (detected language becomes primary, old primary joins the alternates; switching back re-pins again). Explicit language choice never re-pins.

4. English tech terms inside Russian speech got phonetically mangled ("в чём разница между стейтлес виджет…"). Two-layer fix: provider-side vocabulary biasing (Google speechContexts, ElevenLabs Scribe v2 keyterms — note: ElevenLabs charges ~20% extra for keyterms) from a new curated glossary, plus a deterministic local post-processor (transcriptTermFix.ts) that restores glossary terms in final Cyrillic segments via consonant-skeleton matching — exact-match only, so ordinary Russian/Ukrainian speech is never rewritten. Zero network, works with every provider.

5. Streamed answers got replaced by "I don't have the original question or answer to rewrite." The post-stream scaffold-contamination repair asked a stateless LLM call to rewrite "your previous response" without embedding it, and the canned inability stub passed the ≥5-char acceptance check, overwriting the real answer. The draft now travels in the repair prompt as a <draft_response> block, and a new isRepairInabilityStub() guard rejects such stubs at both repair acceptance sites.

6. Russian coding questions got all-English answers. The coding contract mandates exact English headings (## Approach, ## Dry Run), which models read as "the whole answer is English", overriding the soft auto-language instruction. The instruction now explicitly decouples mandated scaffolding from prose language: headings/code stay as contracted, all explanatory prose follows the user's language.

Type of Change

• 🐛 Bug Fix (items 1, 2, 5, 6)
• ✨ New Feature (items 3, 4)

Testing & Environment

• Manual test performed on: macOS 26 (Apple Silicon), live meetings with ElevenLabs and Google STT, ru/uk/en speech
• npm run typecheck:electron clean; npm run build:electron clean.
• 24 new tests across 6 files, all passing on top of current main:
  • GoogleAutoDetectAlternates.test.mjs — locale-driven alternates, fallback, caps; guards against the auto → english-us substitution returning.
  • GoogleSTTLanguageRepin.test.mjs — re-pin threshold/reset/round-trip, stale close/end/error immunity, tail-final forwarding (fake gRPC client).
  • SttPhraseHints.test.mjs — glossary limits, Google speechContexts request shape, ElevenLabs keyterm cap enforcement.
  • TranscriptTermFix.test.mjs — skeleton convergence (стейтлес→stateless etc.), punctuation preservation, ordinary-speech no-op guarantees.
  • RepairInabilityStub.test.mjs — stub detection + false-positive guards + structural asserts on both repair sites.
  • LanguageInstructionCarveOut.test.mjs — language-instruction carve-out stays in place.

To verify manually: set STT language to Auto Detect with the Google provider on a system whose preferred languages include Russian/Ukrainian, speak Russian mid-meeting (expect re-pin log Auto mode: re-pinning primary language en-US → ru-RU), say "стейтлес виджет" (expect Stateless Widget in the final transcript), and ask a DSA question in Russian (expect Russian prose under the English contract headings).

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR improves multilingual speech recognition and answer generation:

• derives Google auto-detection alternatives from system language preferences and dynamically re-pins the primary language;
• adds generation-aware Google stream swapping, stale-event guards, and graceful tail draining;
• adds provider vocabulary hints and local repair for mangled technical terms;
• hardens answer-repair prompts and preserves the user’s prose language in structured coding responses.

Confidence Score: 3/5

The PR is not yet safe to merge because an in-flight old-stream final can still bypass generation ordering after the drain deadline.

Destroying the expired stream prevents future data production, but the deadline immediately removes its generation and releases younger transcripts; an already queued data callback can then use the absent-generation fallback and append older speech after newer context.

Files Needing Attention: electron/audio/GoogleSTT.ts

Important Files Changed

Filename	Overview
electron/audio/GoogleSTT.ts	Adds locale-driven auto detection, language re-pinning, phrase hints, stale-stream protection, and generation-aware stream draining.
electron/audio/ElevenLabsStreamingSTT.ts	Adds bounded, URL-encoded technical keyterms to ElevenLabs realtime transcription.
electron/config/languages.ts	Adds preferred-locale selection for Google’s three alternative-language slots.
electron/llm/transcriptTermFix.ts	Adds deterministic glossary-based restoration of mangled technical terms in final Cyrillic transcripts.
electron/IntelligenceEngine.ts	Embeds the original draft in stateless repair prompts and rejects canned inability responses.
electron/LLMHelper.ts	Clarifies that structured English headings do not override the language of explanatory prose.
electron/main.ts	Preserves auto-detect language selection and applies final-transcript term repair in the shared STT pipeline.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Audio[Audio chunks] --> Current[Current Google stream]
  Current -->|language re-pin or restart| Swap[Generation-aware stream swap]
  Swap --> Old[Old stream drains]
  Swap --> New[New stream receives audio]
  Old --> Queue[Ordered transcript generation queue]
  New --> Queue
  Queue --> Transcript[Transcript pipeline]
  Transcript --> Repair[Term and answer repair]
  Repair --> UI[Assistant output]

Loading

_{Reviews (5): Last reviewed commit: "fix(stt): destroy the drain straggler on..." | Re-trigger Greptile}