Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/bugs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Answering Pipeline Bug Report — August 2026

## Overview

This PR documents **three user-reported bugs** in the answering pipeline (v2.8.5).
All bugs were discovered from Telegram user reports and validated against the source code
with static analysis + jcodemunch blast-radius assessment.

**No source-code changes are in this PR** — these are documentation-only fixes for developer review.

---

## Bug Index

| ID | Title | Severity | Files to Fix |
|---|---|---|---|
| [Bug 001](./bug-001-document-grounded-refusal.md) | Document-Grounded Refusal on General Questions | 🔴 High | `AnswerPlanner.ts`, `contextRoute.ts` |
| [Bug 002](./bug-002-screenshot-code-generation.md) | Screenshot Attached but Code Not Generated | 🟠 Medium | `ipcHandlers.ts`, `IntelligenceEngine.ts` |
| [Bug 003](./bug-003-skill-injection-failure.md) | Skill Injection Ignored in V3 Engine | 🟠 Medium | `WhatToAnswerLLM.ts` |

---

## Blast Radius Summary (jcodemunch analysis)

All three bugs touch the core answering loop. Risk is **concentrated** in a small set of files
but the flag `documentGroundedCustomModeActive` fans out to 26 dependent files.

```
documentGroundedCustomModeActive (AnswerPlanner.ts)
├── WhatToAnswerLLM.ts [6 refs — also affected by Bug 003]
├── contextRoute.ts [5 refs — secondary fix site for Bug 001]
├── ModesManager.ts [8 refs]
├── conversationHistoryPolicy.ts [1 ref]
└── modeProfiles.ts [1 ref]

buildCustomModeExecutionContract
├── IntelligenceEngine.ts [5 refs — fix site for Bug 002]
└── ipcHandlers.ts [6 refs — fix site for Bug 002]

WhatToAnswerLLM (class)
└── llm/index.ts [2 refs]
```

> **⚠️ All confirmed dependents have `has_test_reach: false`** — meaning none of the above files
> are currently covered by a test that verifies end-to-end answering behaviour.
> Each bug doc includes a checklist of tests to add or update.

---

## Key Additional Finding (Bug 001)

`documentGroundedFromContract()` in [`modeSourceContract.ts`](../../electron/services/modeSourceContract.ts)
**already implements the correct guard** (`if (!hasReferenceFiles) return false`).
It has **zero call-sites** in the entire codebase.

The recommended fix for Bug 001 is to wire this function in rather than adding a new inline check.

---

## How to Review

1. Read each bug file in [`docs/bugs/`](./) for full root-cause analysis, diffs, and safe-change checklists.
2. Fixes are **surgical** — each change is a single guard condition or a 3-line append.
3. Run `ModePolicyShadowDivergence2026_07_26.test.mjs` and `Issue303SkillInvocation.test.mjs`
after implementing fixes to catch regressions early.
142 changes: 142 additions & 0 deletions docs/bugs/bug-001-document-grounded-refusal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Bug 001 — Document-Grounded Refusal on General / YouTube Questions

## User Reports (Telegram)

> *"I just asked a random question from a YouTube, and it answers: 'This is not directly uploaded document.'"*
> *"I noticed that instead of generating an answer based on the conversation context, it keeps saying 'no document has been uploaded.'"*

---

## Root Cause

### 1. `documentGroundedCustomModeActive` ignores `hasReferenceFiles`

**File:** [`electron/llm/AnswerPlanner.ts`](../../electron/llm/AnswerPlanner.ts) — line 2335

```typescript
// CURRENT (broken)
const documentGroundedCustomModeActive =
input.activeMode?.documentGroundedCustomModeActive === true;
```

This reads the pre-computed flag directly from the mode object. The flag is set by `ModesManager` for any mode whose `sourceAuthority` is `reference_files_primary` (or similar) — **even when no files have been uploaded**.

### 2. The safe guard function exists but is never called

**File:** [`electron/services/modeSourceContract.ts`](../../electron/services/modeSourceContract.ts) — line 730

```typescript
function documentGroundedFromContract(
contract: ModeSourceContract,
hasReferenceFiles: boolean, // ← correctly short-circuits
): boolean {
if (!hasReferenceFiles) return false; // ← guard we need
return contract.sourceAuthority === 'reference_files_only'
|| contract.sourceAuthority === 'reference_files_primary'
|| contract.sourceAuthority === 'reference_files_plus_transcript';
}
```

> **This function has 0 call-sites in the entire codebase** (confirmed via `find_references`).
> It was built to solve exactly this problem but was never wired in.

### 3. Forced document-grounded routing with empty context

**File:** [`electron/llm/AnswerPlanner.ts`](../../electron/llm/AnswerPlanner.ts) — line 2822

```typescript
if (documentGroundedCustomModeActive && !explicitDocumentModeCodingAsk && !explicitDocumentModeProfileAsk) {
const docShape = classifyDocumentQuestionShape(question, ...);
answerType = docShape === 'broad_overview' ? 'lecture_answer' : docShape;
}
```

Because the flag is `true` (incorrectly), the turn is classified as `lecture_answer` / `definitional_answer` / etc.

### 4. Validation fails against empty `docContextBlock`

**Files:** [`electron/ipcHandlers.ts`](../../electron/ipcHandlers.ts) (line 4029) and [`electron/IntelligenceEngine.ts`](../../electron/IntelligenceEngine.ts) (line 3156)

The WTA gate checks that a non-empty `docContextBlock` exists for document-grounded answer types. Since no files were uploaded, the block is empty → validation fails → refusal message returned.

---

## Also Affected: `contextRoute.ts`

**File:** [`electron/llm/contextRoute.ts`](../../electron/llm/contextRoute.ts) — line 76

```typescript
// Same pattern, same bug:
const documentGroundedCustomModeActive = plan.documentGroundedCustomModeActive === true;
```

This copy of the flag is used for conversation-history routing and WhatToAnswerLLM prompt shaping — it inherits the same incorrect `true` value.

---

## Proposed Fix

### Fix A — `AnswerPlanner.ts` (primary routing gate)

```diff
- const documentGroundedCustomModeActive =
- input.activeMode?.documentGroundedCustomModeActive === true;
+ const hasReferenceFiles = input.activeMode?.hasReferenceFiles === true;
+ const documentGroundedCustomModeActive =
+ input.activeMode?.documentGroundedCustomModeActive === true && hasReferenceFiles;
```

**Alternative (preferred):** wire in the already-existing safe utility:

```diff
+ import { documentGroundedFromContract } from '../services/modeSourceContract';
// …
const documentGroundedCustomModeActive =
- input.activeMode?.documentGroundedCustomModeActive === true;
+ documentGroundedFromContract(
+ input.activeMode?.sourceContract,
+ input.activeMode?.hasReferenceFiles === true,
+ );
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Outdated
```

### Fix B — `contextRoute.ts` (conversation-history / WTA routing)

Apply the same guard so that the `plan` object propagated to WhatToAnswerLLM is consistent:

```diff
- const documentGroundedCustomModeActive = plan.documentGroundedCustomModeActive === true;
+ const documentGroundedCustomModeActive =
+ plan.documentGroundedCustomModeActive === true && (plan.hasReferenceFiles === true);
```

---

## Blast Radius

| Symbol | Confirmed Dependents | Potential (namespace import) |
|---|---|---|
| `documentGroundedCustomModeActive` (AnswerPlanner) | 5 files | 21 files |
| `classifyDocumentQuestionShape` | 3 files | 6 files |
| `buildCustomModeExecutionContract` | 2 files | 4 files |

**High-blast files that must be regression-tested after this fix:**
- `electron/llm/WhatToAnswerLLM.ts` (6 references to the flag)
- `electron/llm/contextRoute.ts` (5 references)
- `electron/services/ModesManager.ts` (8 references)
- `electron/llm/conversationHistoryPolicy.ts`
- `electron/llm/modeProfiles.ts`

**Files with no test coverage flagged by jcodemunch:**
- All confirmed dependents listed above have `has_test_reach: false`
- Recommendation: add a dedicated Jest test for the `documentGroundedFromContract` path with `hasReferenceFiles: false`

---

## Safe-Change Checklist

- [ ] Fix A applied in `AnswerPlanner.ts`
- [ ] Fix B applied in `contextRoute.ts`
- [ ] `documentGroundedFromContract` verified to return `false` when `hasReferenceFiles` is falsy
- [ ] Manual smoke-test: open "General" mode, ask a question, verify no refusal
- [ ] Existing doc-grounded tests still pass when files are uploaded
- [ ] No regression in `ModePolicyShadowDivergence2026_07_26.test.mjs` (tests the `documentGroundedCustomModeActive` flag parity)
122 changes: 122 additions & 0 deletions docs/bugs/bug-002-screenshot-code-generation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Bug 002 — Screenshot Attached but Code Not Generated

## User Report (Telegram)

> *"when i take a screenshot and chat shows screenshot attached, and i ask to give me the code, it doesn't give me the code"*

---

## Root Cause Chain

### 1. `hasScreenContext` omitted in manual-chat IPC handler

**File:** [`electron/ipcHandlers.ts`](../../electron/ipcHandlers.ts) — inside `_geminiChatStreamHandler` (~line 1163)

When `buildV3Prompt` is called for the `gemini-chat-stream` IPC path, `hasScreenContext` is **completely absent** from the argument object:

```typescript
const composed = await buildV3Prompt({
surface: 'manual-chat',
pathTag: 'ipc',
question: String(message || ''),
modeTemplateType: rawMode,
modeUniqueId: modeInfo?.id ?? null,
attachedSourceCount: files.length,
// … other fields …
// hasScreenContext: ← MISSING
});
```

The field defaults to `undefined` / `false` inside `buildV3Prompt`.

### 2. `IntelligenceEngine` only checks `options.screenContext`, not `imagePaths`

**File:** [`electron/IntelligenceEngine.ts`](../../electron/IntelligenceEngine.ts) — inside `runWhatShouldISay` (~line 2494)

```typescript
// CURRENT (broken):
hasScreenContext: Boolean(options?.screenContext),
```

`options.screenContext` is the **OCR object** built from the periodic screen-capture service — it is not set for manually-attached screenshots sent through `imagePaths`. When a user attaches a screenshot through the chat UI, `imagePaths` is populated but `screenContext` is `null`, so `hasScreenContext` is `false`.

### 3. Turn classifier misses `SCREEN_SPECIFIC` / `SCREEN_FACT`

**File:** [`electron/context-intelligence/question/turn-classifier.ts`](../../electron/context-intelligence/question/turn-classifier.ts)

The classifier uses `hasScreenContext` to mark turns as screen-anchored. With `hasScreenContext: false`, a query like *"give me the code"* receives no screen-related label.

### 4. Coding persona never activated

**File:** [`electron/llm/AnswerPlanner.ts`](../../electron/llm/AnswerPlanner.ts)

Without a `SCREEN_SPECIFIC` or `CODING_TASK_RE` match, the orchestrator falls through to a generic `general_meeting_answer` type and never activates the coding persona or code-extraction prompt.

> **Note:** the phrase *"give me the code"* does not match `CODING_TASK_RE` (which requires verbs like write / implement / solve). The primary fix must therefore be at the `hasScreenContext` propagation level, not keyword-tuning.

---

## Proposed Fixes

### Fix A — `electron/ipcHandlers.ts`

Inside `_geminiChatStreamHandler`, pass `hasScreenContext` from `imagePaths`:

```diff
const composed = await buildV3Prompt({
surface: 'manual-chat',
pathTag: 'ipc',
question: String(message || ''),
modeTemplateType: rawMode,
modeUniqueId: modeInfo?.id ?? null,
attachedSourceCount: files.length,
attachedFileNames: (files as Array<{ fileName?: string }>)
.map((f) => f.fileName ?? '').filter(Boolean),
profileSourceCount: v3ProfileCounts.profileResume + ...,
resolvedProfileSources: v3ProfileResolved,
extraAllowedSourceTypes: extraSourceTypes,
debugSources: v3DebugSources as never,
deferDebugCompletion: true,
requestId: `v3-${myStreamId}`,
requestSequence: myStreamId,
+ hasScreenContext: Boolean(imagePaths && imagePaths.length > 0),
});
```

### Fix B — `electron/IntelligenceEngine.ts`

Expand the `hasScreenContext` guard to cover the `imagePaths` channel:

```diff
- hasScreenContext: Boolean(options?.screenContext),
+ hasScreenContext: Boolean(
+ options?.screenContext || (imagePaths && imagePaths.length > 0)
+ ),
```

---

## Blast Radius

| Symbol / File | Confirmed Dependents | Risk |
|---|---|---|
| `ScreenContextService` | `IntelligenceEngine`, `IntelligenceManager`, `WhatToAnswerLLM` | Low — existing callers only read `captureScreen()`, not `imagePaths` |
| `buildCustomModeExecutionContract` | `IntelligenceEngine` (×5), `ipcHandlers` (×6) | Medium — touches both primary entry points |
| `ipcHandlers.ts` change | Only `gemini-chat-stream` handler | Low — scoped to manual-chat surface |
| `IntelligenceEngine.ts` change | All `runWhatShouldISay` callers | Medium — also affects overlay auto-answer flow |

**Regression risk for overlay auto-answer:**
The overlay path sends both `options.screenContext` (OCR object) and `imagePaths`. After Fix B, both still resolve to `true` — no behaviour change for the existing overlay path.

**Test file to update:**
- [`electron/services/__tests__/IntelligenceEngineScreenContext.test.mjs`](../../electron/services/__tests__/IntelligenceEngineScreenContext.test.mjs) — add a case where `screenContext` is `null` but `imagePaths` is non-empty, assert `hasScreenContext === true`.

---

## Safe-Change Checklist

- [ ] Fix A applied in `ipcHandlers.ts`
- [ ] Fix B applied in `IntelligenceEngine.ts`
- [ ] Manual smoke-test: attach a screenshot in manual chat, type "give me the code", verify coding response
- [ ] Overlay auto-answer with screen capture still works (no regression)
- [ ] `IntelligenceEngineScreenContext.test.mjs` updated with new `imagePaths`-only case
Loading