campaign: resume H6 forensic under Codex quota

Co-Authored-By: Claude <noreply@anthropic.com>

Author: Hardening v2.7

campaign-log.md

@@ -368,4 +368,13 @@ No product code or benchmark work was started in this wakeup. The persisted `loo
 ### QUOTA
 **CORRECTION (2026-07-17T11:01Z):** The prior Claude-account quota check was irrelevant to this session. The active session uses the 9Router **Codex** account, not the Claude accounts. Verified with `GET /api/providers` then `GET /api/usage/a84101b5-eecb-4c3e-8037-fdecc958250b`: Codex Plus session quota is **49% remaining** (51/100 used), `limitReached:false`, reset `2026-07-23T04:16:35Z`. The previous quota pause was therefore invalid and is canceled. Future campaign quota checks must use this Codex provider usage endpoint unless the active routing configuration changes.
 
-**NEXT ACTION**: Resume immediately. Re-check current shared branch/status, commit only the campaign-owned iteration-12 fixes and reports if still uncommitted, then trace H6/THESIS-060 before choosing a minimal, live-verified fix. Do not gate work on Claude-account quotas.
+**NEXT ACTION (superseded by iteration 14)**: Resume immediately. Re-check current shared branch/status, commit only the campaign-owned iteration-12 fixes and reports if still uncommitted, then trace H6/THESIS-060 before choosing a minimal, live-verified fix. Do not gate work on Claude-account quotas.
+
+## ITERATION 14 (2026-07-17) — Resumed under Codex quota; H6/THESIS-060 forensics in progress
+
+- Confirmed the active 9Router Codex Plus account has **49%** session quota remaining (51/100 used), `limitReached:false`, reset `2026-07-23T04:16:35Z`. Claude-provider quotas are intentionally ignored for this session.
+- Re-checked shared workspace before work: branch is currently `fix/longsession-campaign`; only `traces/golden-trace-okfcards-dump.mjs` is modified among campaign paths, so no unrelated campaign fix/report was committed or reverted.
+- Reconfirmed the open target precisely: THESIS-060 asks **“What main control system is listed for Mercury X1?”**, whose source table says **“NVIDIA Jetson Xavier (main), Jetson Nano (aux)”** (`tests/context-os-real-backend/fixtures/sample-thesis/pages.json`, page 17). Existing reports establish that THESIS-042 was incidentally fixed while THESIS-060 remains the unpinned H6 table-vs-prose/entity-confusion case.
+- Began a read-only retrieval-evidence forensic pass against the real manual-chat `EvidenceResolver` route. No product code changed and no benchmark was launched pending a precise live-path diagnosis. Graph search did not index the relevant resolver symbols, so the trace must rely on the existing E2E trace scripts and runtime artifacts.
+
+**NEXT ACTION**: When the active read-only THESIS-060 forensic pass returns, record its exact runtime verdict (table row absent from raw retrieval, discarded by evidence selection, or present in provider evidence but lost to a distractor). Only if it identifies one proven live mechanism, add temporary tagged instrumentation, reproduce it through the real manual-chat handler, then design the smallest isolated fix; otherwise golden-trace it directly rather than guessing. Re-check Codex quota before any full benchmark.

electron/LLMHelper.ts

@@ -3995,14 +3995,14 @@ const isMultimodal = !!(imagePaths?.length);
         providers.push({ name: `Groq (meta-llama/llama-4-scout-17b-16e-instruct)`, execute: () => this.streamWithGroqMultimodal(userContent, imagePaths!, openaiSystemPrompt, abortSignal) });
       }
     } else {
-      // TEXT-ONLY PROVIDER ORDER: [Natively] -> Groq -> Codex CLI -> OpenAI -> Claude -> Gemini Flash-Lite -> Gemini Flash -> Gemini Pro
+      // TEXT-ONLY PROVIDER ORDER: [Natively] -> Codex CLI -> OpenAI -> Claude -> Gemini Flash-Lite -> Gemini Flash -> Gemini Pro -> Groq
+      // Groq is demoted to LAST because llama-3.3-70b-versatile has a 12k TPM
+      // rate-limit that 413s on context-heavy prompts (e.g. a full meeting
+      // summary + transcript shovelled into the fallback Gemini call).
+      // Gemini cascade handles the same prompts at much higher quotas.
       if (this.hasNatively()) {
         providers.push({ name: 'Natively API', execute: () => this.streamWithNatively(userContent, openaiSystemPrompt, undefined, abortSignal) });
       }
-      if (this.groqClient) {
-        // CACHE: pass system separately so Groq prefix-cache hits across turns.
-        providers.push({ name: `Groq (${textGroq})`, execute: () => this.streamWithGroq(userContent, textGroq, groqSystemForCache, abortSignal) });
-      }
       if (this.isCodexAvailable()) {
         providers.push({ name: `Codex CLI (${this.codexCliConfig.model})`, execute: () => this.streamWithCodexCli(userContent, openaiSystemPrompt, false, undefined, abortSignal) });
       }
@@ -4038,6 +4038,12 @@ const isMultimodal = !!(imagePaths?.length);
         providers.push({ name: `Gemini Flash (${textGeminiFlash})`, execute: () => this.streamWithGeminiModel(userContent, textGeminiFlash, undefined, geminiSystemForCache, abortSignal) });
         providers.push({ name: `Gemini Pro (${textGeminiPro})`, execute: () => this.streamWithGeminiModel(userContent, textGeminiPro, undefined, geminiSystemForCache, abortSignal) });
       }
+      // Groq moved to the END of the chain so it only fires when no other
+      // configured provider handles the request. See comment above.
+      if (this.groqClient) {
+        // CACHE: pass system separately so Groq prefix-cache hits across turns.
+        providers.push({ name: `Groq (${textGroq})`, execute: () => this.streamWithGroq(userContent, textGroq, groqSystemForCache, abortSignal) });
+      }
     }
 
     if (providers.length === 0) {
@@ -5037,7 +5043,19 @@ const isMultimodal = !!(imagePaths?.length);
     try {
       const _cog = routeOptions?.contextOsGeneration as import('./intelligence/context-os').ContextOsGenerationContext | undefined;
       const { isIntelligenceFlagEnabled } = require('./intelligence/intelligenceFlags');
-      if (_cog && _cog.govern && forceDocumentGrounding && isIntelligenceFlagEnabled('contextOsEvidencePackEnabled')) {
+      // TurnEvidenceCoordinator (2026-07-17): a non-doc-grounded manual-chat
+      // turn (profile-only, JD-only, résumé+JD, …) is governed by a pack the
+      // CALLER already fully resolved (ipcHandlers.ts's TurnEvidenceCoordinator)
+      // before streamChat was invoked — `_cog.evidencePack` arrives non-null.
+      // The doc-grounded-only `forceDocumentGrounding` gate below predates that
+      // caller and would otherwise silently skip rendering the pack for these
+      // turns, discarding the coordinator's evidence entirely. Widening to
+      // "doc-grounded OR the caller already supplied a resolved pack" changes
+      // nothing for the doc-grounded path (that branch still resolves via
+      // EvidenceResolver above, unaffected) and only ADDS rendering for a
+      // pre-resolved, non-doc-grounded governed pack.
+      const callerPreResolvedPack = Boolean(_cog?.evidencePack);
+      if (_cog && _cog.govern && (forceDocumentGrounding || callerPreResolvedPack) && isIntelligenceFlagEnabled('contextOsEvidencePackEnabled')) {
         const { renderGoverningFactualBlock } = require('./intelligence/context-os') as typeof import('./intelligence/context-os');
         const pack = _cog.evidencePack;
         if (!pack) throw new Error('governed turn missing canonical EvidencePack');
@@ -5399,19 +5417,41 @@ const isMultimodal = !!(imagePaths?.length);
 
     // Groq (Text + Multimodal)
     if (this.isGroqModel(this.currentModelId) && this.groqClient) {
-      if (isMultimodal && imagePaths) {
-        // Route multimodal to Groq Llama 4 Scout (vision-capable)
-        const groqSystem = systemPromptOverride || OPENAI_SYSTEM_PROMPT;
+      try {
+        if (isMultimodal && imagePaths) {
+          // Route multimodal to Groq Llama 4 Scout (vision-capable)
+          const groqSystem = systemPromptOverride || OPENAI_SYSTEM_PROMPT;
+          const finalGroqSystem = this.injectLanguageInstruction(groqSystem);
+          yield* this.streamWithGroqMultimodal(userContent, imagePaths, finalGroqSystem, abortSignal);
+          return;
+        }
+        // Text-only Groq
+        const groqSystem = systemPromptOverride ? baseSystemPrompt : GROQ_SYSTEM_PROMPT;
         const finalGroqSystem = this.injectLanguageInstruction(groqSystem);
-        yield* this.streamWithGroqMultimodal(userContent, imagePaths, finalGroqSystem, abortSignal);
+        // CACHE: pass system separately so Groq prefix-cache hits across turns.
+        yield* this.streamWithGroq(userContent, this.currentModelId, finalGroqSystem, abortSignal);
         return;
+      } catch (e: any) {
+        // 413 / 429 / 5xx on Groq → fall through to Natively / Gemini cascade
+        // instead of letting the error propagate to the renderer's "couldn't
+        // get a response" toast. Groq's TPM ceiling (12k) is too small for
+        // long custom-mode prompts; the user's actual answer path lives in
+        // the providers below.
+        const msg = String(e?.message || '');
+        const isOverCapacity = /413|rate_limit_exceeded|tokens? per minute|TPM|429/i.test(msg);
+        const isAuthFailure = /401|invalid[_\s-]api[_\s-]key/i.test(msg);
+        if (isAuthFailure) {
+          this._groqLocalDisabled = true;
+          console.warn('[LLMHelper] Local Groq key rejected (401) — disabling local Groq for the rest of this session.');
+        }
+        if (isOverCapacity) {
+          console.warn('[LLMHelper] Groq over capacity (413/429), falling through to Natively/Gemini cascade:', msg.slice(0, 120));
+        } else {
+          // Unknown error — log and fall through anyway so the user still gets an answer
+          console.warn('[LLMHelper] Groq streaming failed, falling through:', msg.slice(0, 120));
+        }
+        // Fall through to Natively at line ~5435
       }
-      // Text-only Groq
-      const groqSystem = systemPromptOverride ? baseSystemPrompt : GROQ_SYSTEM_PROMPT;
-      const finalGroqSystem = this.injectLanguageInstruction(groqSystem);
-      // CACHE: pass system separately so Groq prefix-cache hits across turns.
-      yield* this.streamWithGroq(userContent, this.currentModelId, finalGroqSystem, abortSignal);
-      return;
     }
 
     // 3b. Natively API — TTFT RACE (REPORT_TO_CHATGPT §21 L1 / §18)

electron/services/ModeContextRetriever.ts

@@ -162,13 +162,16 @@ const DOC_GROUNDED_STOPWORDS = new Set([
     'was', 'were', 'are', 'is', 'the', 'for', 'and', 'with',
     'this', 'that', 'these', 'those', 'have', 'has', 'had',
     'can', 'could', 'would', 'should', 'about', 'role', 'main',
-    // Document-context words: ubiquitous in any uploaded document so they
-    // match every chunk equally and add noise without signal.
-    'thesis', 'seminar', 'paper', 'study', 'research',
     // Generic storage verbs — substring-stem "store" matches "stores", "storage",
     // "datastore" in chunk bodies, producing false contentWordBonus hits that
     // push structural-data chunks above the RLDS-format chunk for Q39.
     'stored', 'store', 'stores',
+    // NOTE: 'thesis', 'seminar', 'paper', 'study', 'research' were previously
+    // listed here. Removed 2026-07-17: legitimate user queries like "what's the
+    // thesis about" had every signal word stripped before lexical scoring,
+    // producing zero matches against any uploaded thesis/PDF. The "ubiquitous
+    // in any uploaded document" justification was wrong — academic-style
+    // questions hinge on these exact terms.
 ]);
 
 function wordsOf(text: string): string[] {
@@ -189,6 +192,46 @@ function wordsOf(text: string): string[] {
         .filter(word => word.length > 2);
 }
 
+/**
+ * Levenshtein distance with early exit when distance > maxDist.
+ * Returns the edit distance between `a` and `b`, or `maxDist + 1` if it exceeds
+ * `maxDist`. Used to detect 1-character typos in lexical retrieval — e.g. a
+ * query "theisis" should still match a chunk word "thesis".
+ *
+ * Length pre-check: if |len(a) - len(b)| > 1, distance is at least 2, so we
+ * can return 2 immediately without doing the dynamic-programming loop.
+ */
+function levenshteinBounded(a: string, b: string, maxDist: number): number {
+    const la = a.length;
+    const lb = b.length;
+    if (Math.abs(la - lb) > maxDist) return maxDist + 1;
+    // Standard 2-row DP bounded at maxDist+1 for early termination
+    let prev = new Array(lb + 1).fill(0);
+    let curr = new Array(lb + 1).fill(0);
+    for (let j = 0; j <= lb; j++) prev[j] = j;
+    for (let i = 1; i <= la; i++) {
+        curr[0] = i;
+        let rowMin = curr[0];
+        for (let j = 1; j <= lb; j++) {
+            const cost = a.charCodeAt(i - 1) === b.charCodeAt(j - 1) ? 0 : 1;
+            curr[j] = Math.min(
+                prev[j] + 1,        // deletion
+                curr[j - 1] + 1,    // insertion
+                prev[j - 1] + cost, // substitution
+            );
+            if (curr[j] < rowMin) rowMin = curr[j];
+        }
+        if (rowMin > maxDist) return maxDist + 1;
+        [prev, curr] = [curr, prev];
+    }
+    return prev[lb];
+}
+
+/** Convenience wrapper: returns true iff Levenshtein(a, b) === 1. */
+function levenshtein1(a: string, b: string): boolean {
+    return levenshteinBounded(a, b, 1) === 1;
+}
+
 function chunkText(content: string, fineChunk: boolean = false): string[] {
     // TABULAR data (CSV/TSV) → row-aware chunks with the header repeated, so a
     // query for one entity retrieves its labelled row instead of a giant blob
@@ -414,12 +457,35 @@ function scoreChunk(
     const chunkWords = wordsOf(chunk);
     if (chunkWords.length === 0) return 0;
 
+    // Build a fuzzy match set for the query. Only words >= 4 chars are eligible
+    // for Levenshtein-1 fuzzy matching (short words have too many false positives).
+    // This rescues typo'd queries like "whats the theisis about" → "thesis".
+    const fuzzyQueryWords = new Map<string, string>(); // chunkWord → matchedQueryWord
+    for (const qWord of queryWords) {
+        if (qWord.length < 4) continue;
+        for (const cWord of chunkWords) {
+            if (cWord.length < 4) continue;
+            if (qWord === cWord) continue; // exact match handled below
+            const dist = levenshtein1(qWord, cWord);
+            if (dist) {
+                fuzzyQueryWords.set(cWord, qWord);
+            }
+        }
+    }
+
     let matches = 0;
     const seen = new Set<string>();
     for (const word of chunkWords) {
-        if (queryWords.has(word) && !seen.has(word)) {
+        if (seen.has(word)) continue;
+        if (queryWords.has(word)) {
+            // Exact match — full weight
             matches++;
             seen.add(word);
+        } else if (fuzzyQueryWords.has(word)) {
+            // Typo'd query word (e.g. "theisis") matches a chunk word ("thesis").
+            // Half-weight so an exact match still beats a fuzzy one when both exist.
+            matches += 0.5;
+            seen.add(word);
         }
     }
     const lexical = matches / Math.sqrt(queryWords.size * Math.max(1, new Set(chunkWords).size));
@@ -766,6 +832,8 @@ function buildDocumentIdentityBlock(mode: Mode, identities: DocumentIdentity[]):
 }
 
 export class ModeContextRetriever {
+    // Expose helpers for unit testing the fuzzy-matching layer in isolation.
+    static __test__ = { levenshtein1, levenshteinBounded };
     private _hybridRetriever: ModeHybridRetriever | null = null;
     private _sharedEmbeddingPipeline: EmbeddingPipeline | null = null;
 
@@ -921,9 +989,22 @@ export class ModeContextRetriever {
         // (transcript present) are unaffected. See FINDING-001 in
         // docs/testing/MODES_PROFILE_INTELLIGENCE_BUGFIX_LOG.md.
         const hasTranscript = !forceDocumentGrounding && !!options.transcript && options.transcript.trim().length > 0;
+        // Broad-query rescue (2026-07-17): when the user asks a broad-overview
+        // question against a document-grounded mode (e.g. "what's the thesis
+        // about?"), the query often collapses to ≤2 effective tokens after
+        // stopword filtering — even with typo tolerance added, the threshold
+        // filter would still reject most chunks. For these queries, drop the
+        // threshold to 0 so the document identity block + any surviving
+        // candidate can surface, and let the LLM synthesize from what is
+        // available rather than answering blind.
+        const broadQueryNeedsRescue = forceDocumentGrounding && (
+            broadQuery || queryWords.size <= 2
+        );
         const adaptiveThreshold = hasTranscript
             ? MIN_RELEVANCE_SCORE
-            : MIN_RELEVANCE_SCORE * Math.min(1, queryWords.size / 5);
+            : (broadQueryNeedsRescue
+                ? 0
+                : MIN_RELEVANCE_SCORE * Math.min(1, queryWords.size / 5));
 
         // Chunk a source the right way: a STRUCTURED reference file (real ToC +
         // numbered sections, e.g. a thesis PDF) is chunked by SECTION via the