Skip to content

Commit 3c8d318

Browse files
ndemiancclaude
andcommitted
Atom++ AI: M5 auto-verify loop, context-window UI, 400-recovery, step-limit Continue
Four features on the agentic core (all in extensions/atom-ai/, plain JS): - M5 diagnostics-driven auto-verify loop: after edits land and the model goes to finish, pull getDiagnostics for the files touched THIS run (new errors only — diffed vs a pre-run baseline keyed by severity+message so pre-existing/unrelated problems and line-shifts don't trigger a chase) and, if a verify command is set, run it through the live-terminal plumbing. Failures are injected back as a synthetic turn so the agent self-corrects, bounded by verify.maxRounds AND maxSteps, abort-safe, shown as a live verify card. A command that can't run (missing npm script / not found / exit 127) is classified config-not-code: warn the user, never loop the agent on it. New pure, unit-tested verify.js; settings atompp.ai.verify.{enabled,command, maxRounds,includeWarnings} (diagnostics-only by default, zero-config). - Context-window UI (Claude-Code style): footer donut ring that fills + recolors by pressure; click opens a breakdown popover (segmented bar + Messages / System prompt / Tools / Free-space rows with tokens + %), live-updating during a run. agent.js now reports system/tools token estimates so the split is real. - Malformed-tool-use 400 recovery: a tool_use truncated by max_tokens used to leak an internal `_malformed` marker into the re-sent message, which the API rejected ("Extra inputs are not permitted"). providers.js now builds API-clean blocks by construction via finalizeAgentBlocks() and carries the malformed signal in a separate Set; the agent recovers ("retry smaller") instead of erroring. - Step-limit Continue affordance: when the agent stops at maxSteps it now shows an explanation + a one-click Continue (resumes the persistent transcript, preserves the original goal for Retry) instead of a dead-end "Stopped". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 823306c commit 3c8d318

6 files changed

Lines changed: 383 additions & 34 deletions

File tree

‎extensions/atom-ai/agent.js‎

Lines changed: 80 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ const fs = require('fs');
1414
const path = require('path');
1515
const cp = require('child_process');
1616
const { streamClaudeAgentTurn } = require('./providers');
17+
const { formatVerifyFeedback, verifyOutcome, looksUnrunnable } = require('./verify');
1718

1819
const SYSTEM = [
1920
"You are Atom++'s built-in autonomous coding agent. You accomplish the user's goal in their",
@@ -27,7 +28,8 @@ const SYSTEM = [
2728
'- Paths are relative to the workspace root.',
2829
'- For a multi-step goal, call update_plan FIRST with a short checklist (3-8 short items, all "pending"), then call it again to set an item "in_progress" when you start it and "done" when finished. Skip the plan for trivial single-step goals.',
2930
'- If the goal truly depends on a decision only the user can make (tech stack, scope, where to create files, must-have features), call ask_user ONCE with concise multiple-choice questions (a short header + 2-4 concrete options each) INSTEAD of writing the questions as prose. Then act on their answers and do not ask again. Do NOT ask about things you can reasonably decide yourself — prefer a sensible default and proceed.',
30-
'- When the goal is finished, end with a one-line summary starting with "Done:" and STOP (no more tools).'
31+
'- When the goal is finished, end with a one-line summary starting with "Done:" and STOP (no more tools).',
32+
'- After you finish, your work is verified automatically: editor diagnostics for the files you changed (plus a verify command, if the user configured one) are checked. If problems are found you will get them back as a follow-up message — fix the ones you introduced, then finish again with "Done:". If a reported problem is clearly pre-existing and unrelated to the goal, do not chase it: note it in one line and finish.'
3133
].join('\n');
3234

3335
const TOOLS = [
@@ -41,6 +43,11 @@ const TOOLS = [
4143
{ name: 'ask_user', description: 'Ask the user one or more multiple-choice questions when the goal genuinely depends on a decision only they can make (tech stack, scope, where to put files, must-have features). The user picks by CLICKING — do NOT write questions as prose. Ask ONCE up front with all your questions, then proceed with the answers and never re-ask. Prefer sensible defaults over asking; only ask when a wrong guess would waste real work.', input_schema: { type: 'object', properties: { questions: { type: 'array', items: { type: 'object', properties: { header: { type: 'string', description: 'a 1-3 word tag for the question' }, question: { type: 'string' }, multiSelect: { type: 'boolean', description: 'true if several options can be picked at once' }, options: { type: 'array', items: { type: 'object', properties: { label: { type: 'string' }, description: { type: 'string' } }, required: ['label'] } } }, required: ['question', 'options'] } } }, required: ['questions'] } }
4244
];
4345

46+
// Rough token estimates (chars/4) for the static prompt segments, so the context popover can break
47+
// down "what's filling the window" — system + tools are sent on every request, the rest is messages.
48+
const SYSTEM_TOKENS_EST = Math.round(SYSTEM.length / 4);
49+
const TOOLS_TOKENS_EST = Math.round(JSON.stringify(TOOLS).length / 4);
50+
4451
const FILE_EXCLUDES = '{**/node_modules/**,**/.git/**,**/out/**,**/dist/**,**/.vscode-test/**,**/*.map}';
4552

4653
// ---- ripgrep search (self-contained) ---------------------------------------
@@ -248,6 +255,7 @@ async function runTool(tu, ctx) {
248255
const ok = await ctx.applyEdit({ path: input.path, exists: true, proposed: res.proposed });
249256
if (!ok) { return 'ERROR: could not apply the edit to ' + input.path + ' (file may be read-only or in conflict).'; }
250257
ctx.editCount = (ctx.editCount || 0) + 1;
258+
if (ctx.touched) { ctx.touched.add(abs); } // this run's edited files → verified afterward
251259
return 'Applied edit to ' + input.path + ' (the user is reviewing it with Keep/Undo; do not re-edit it).';
252260
}
253261
if (tu.name === 'write_file') {
@@ -263,6 +271,7 @@ async function runTool(tu, ctx) {
263271
const ok = await ctx.applyEdit({ path: input.path, exists: existed, proposed: newStr });
264272
if (!ok) { return 'ERROR: could not write ' + input.path + ' (path may be read-only or in conflict).'; }
265273
ctx.editCount = (ctx.editCount || 0) + 1;
274+
if (ctx.touched) { ctx.touched.add(abs); } // this run's edited files → verified afterward
266275
return 'Applied edit to ' + input.path + ' (pending the user\'s Keep/Undo review).';
267276
}
268277
if (tu.name === 'run_command') {
@@ -316,6 +325,67 @@ async function runAgent(ctx) {
316325
let step = 0;
317326
let reason = 'done';
318327
let nudges = 0;
328+
329+
// --- M5 auto-verify loop ------------------------------------------------
330+
// When the model wants to finish AND it edited files this run, check its work before handing back:
331+
// run the configured verify command (one-shot — typecheck/lint/test) + pull editor diagnostics for
332+
// the touched files, then if anything failed feed it back so the agent self-corrects. Bounded by
333+
// verify.maxRounds (re-entries) AND maxSteps; respects abort. A failing verify never blocks the user —
334+
// after the budget it hands the remaining problems over honestly.
335+
let feedbackUsed = 0, verifySeq = 0;
336+
async function runVerifyOnce() {
337+
const id = 'verify-' + (++verifySeq);
338+
const cmd = ((ctx.verify && ctx.verify.command) || '').trim();
339+
ctx.post({ type: 'verifyRun', id, command: cmd });
340+
let ran = false, exitCode = 0, cmdTail = '';
341+
if (cmd) {
342+
ran = true;
343+
cmdTail = await runCommand(root, cmd,
344+
(chunk, stream) => ctx.post({ type: 'verifyOutput', id, chunk, stream }),
345+
(code) => { if (ctx.commandStops) { ctx.commandStops.delete(id); } exitCode = (code == null ? -1 : code); },
346+
(child, stop) => { if (ctx.commandStops) { ctx.commandStops.set(id, stop); } },
347+
ctx.commandTimeout);
348+
}
349+
// A command that couldn't even start (missing npm script / command-not-found) is a CONFIG problem,
350+
// not the agent's code — flag it as unrunnable so we warn the user instead of looping the agent on it.
351+
const unrunnable = ran && looksUnrunnable(exitCode, cmdTail);
352+
const cmdProblem = ran && exitCode !== 0 && !unrunnable;
353+
let diag = { text: '', count: 0 };
354+
if (ctx.getNewDiagnostics) { try { diag = await ctx.getNewDiagnostics(); } catch (e) { dbg('verify.diagErr', { msg: String((e && e.message) || e) }); } }
355+
return { id, ok: !cmdProblem && diag.count === 0, cmdProblem, unrunnable, cmdTail, diag };
356+
}
357+
/** Returns true if the loop should CONTINUE (verification failed, fix-feedback pushed). */
358+
async function attemptVerify() {
359+
const v = ctx.verify;
360+
if (!v || !v.enabled) { return false; }
361+
if (ctx.signal.aborted) { return false; }
362+
if (!(ctx.editCount > 0)) { return false; } // no edits this run → nothing to verify
363+
const cmd = (v.command || '').trim();
364+
const touchedCount = ctx.getTouchedUris ? ctx.getTouchedUris().length : 0;
365+
if (!cmd && !touchedCount) { return false; } // nothing to check
366+
const max = Math.max(0, v.maxRounds || 0);
367+
ctx.post({ type: 'agentStatus', text: 'verifying edits…' });
368+
const r = await runVerifyOnce();
369+
if (ctx.signal.aborted) { return false; }
370+
const outcome = verifyOutcome(r.ok, feedbackUsed, max);
371+
ctx.post({ type: 'verifyDone', id: r.id, ok: r.ok, exhausted: outcome === 'exhausted', errorCount: r.diag.count, cmdFail: r.cmdProblem, unrunnable: r.unrunnable, hadCommand: !!cmd });
372+
if (r.unrunnable) {
373+
// The verify command itself is misconfigured — tell the user, never make the agent "fix" it.
374+
dbg('verify.unrunnable', { tail: String(r.cmdTail || '').slice(0, 160) });
375+
ctx.post({ type: 'agentTool', icon: 'warning', text: 'verify command couldn’t run — check the atompp.ai.verify.command setting' });
376+
}
377+
if (outcome === 'pass') { dbg('verify.pass', { cmd: !!cmd, touched: touchedCount, unrunnable: r.unrunnable }); return false; }
378+
if (outcome === 'exhausted') {
379+
dbg('verify.exhausted', { errors: r.diag.count, cmdFail: r.cmdProblem });
380+
ctx.post({ type: 'agentTool', icon: 'warning', text: 'verification still failing — left for your review' });
381+
return false;
382+
}
383+
feedbackUsed++;
384+
dbg('verify.fail', { round: feedbackUsed, errors: r.diag.count, cmdFail: r.cmdProblem });
385+
messages.push({ role: 'user', content: formatVerifyFeedback({ cmdFail: r.cmdProblem, cmdTail: r.cmdTail, diag: r.diag }, feedbackUsed, max, cmd) });
386+
return true;
387+
}
388+
319389
try {
320390
while (step++ < ctx.maxSteps) {
321391
if (ctx.signal.aborted) { reason = 'stopped'; break; }
@@ -335,11 +405,11 @@ async function runAgent(ctx) {
335405
}
336406
});
337407
if (streamed) { ctx.post({ type: 'agentTurnEnd' }); }
338-
dbg('turn.response', { step, stop: turn.stop_reason, blocks: turn.content.length, textChars, tools: turn.content.filter((c) => c.type === 'tool_use').map((c) => c.name + (c._malformed ? '!malformed' : '')) });
408+
dbg('turn.response', { step, stop: turn.stop_reason, blocks: turn.content.length, textChars, tools: turn.content.filter((c) => c.type === 'tool_use').map((c) => c.name + (turn.malformed && turn.malformed.has(c.id) ? '!malformed' : '')) });
339409
// Report real context usage (input_tokens = how full the transcript is) so the UI can meter + warn.
340410
if (turn.usage) {
341411
dbg('usage', { input: turn.usage.input_tokens, output: turn.usage.output_tokens, cacheRead: turn.usage.cache_read_input_tokens });
342-
ctx.post({ type: 'contextUsage', input: (turn.usage.input_tokens || 0) + (turn.usage.cache_read_input_tokens || 0) + (turn.usage.cache_creation_input_tokens || 0), output: turn.usage.output_tokens || 0, limit: ctx.contextLimit || 200000, model: ctx.model });
412+
ctx.post({ type: 'contextUsage', input: (turn.usage.input_tokens || 0) + (turn.usage.cache_read_input_tokens || 0) + (turn.usage.cache_creation_input_tokens || 0), output: turn.usage.output_tokens || 0, limit: ctx.contextLimit || 200000, model: ctx.model, system: SYSTEM_TOKENS_EST, tools: TOOLS_TOKENS_EST });
343413
}
344414

345415
// Guard: never push an empty assistant message — Anthropic 400s on content:[] (e.g. a
@@ -365,7 +435,7 @@ async function runAgent(ctx) {
365435
let cancelled = false;
366436
for (const tu of toolUses) {
367437
if (cancelled || ctx.signal.aborted) { cancelled = true; dbg('tool.cancelled', { name: tu.name }); results.push({ type: 'tool_result', tool_use_id: tu.id, content: 'Cancelled by the user.' }); continue; }
368-
if (tu._malformed) { dbg('tool.malformed', { name: tu.name }); results.push({ type: 'tool_result', tool_use_id: tu.id, content: 'ERROR: your tool arguments were cut off (truncated JSON). Retry with smaller input — for edits use edit_file with a short snippet.' }); continue; }
438+
if (turn.malformed && turn.malformed.has(tu.id)) { dbg('tool.malformed', { name: tu.name }); results.push({ type: 'tool_result', tool_use_id: tu.id, content: 'ERROR: your tool arguments were cut off (truncated JSON). Retry with smaller input — for edits use edit_file with a short snippet.' }); continue; }
369439
dbg('tool.call', { name: tu.name, input: inputPreview(tu.input) });
370440
const out = await runTool(tu, ctx);
371441
dbg('tool.result', { name: tu.name, chars: String(out).length, error: String(out).startsWith('ERROR') });
@@ -391,7 +461,10 @@ async function runAgent(ctx) {
391461
if (nudges++ < 3) { continue; }
392462
reason = 'limit'; break;
393463
}
394-
if (/(^|\n)\s*done\s*:/i.test(text) || /\bdone\.?\s*$/i.test(text.trim())) { reason = 'done'; break; }
464+
if (/(^|\n)\s*done\s*:/i.test(text) || /\bdone\.?\s*$/i.test(text.trim())) {
465+
if (await attemptVerify()) { continue; } // verification failed → fix feedback pushed, keep going
466+
reason = 'done'; break;
467+
}
395468
// Only nudge when the model PROMISED an action but didn't call a tool (a real stall) —
396469
// not when it gave a complete conversational answer (e.g. a joke, an explanation, a question).
397470
const promisesAction = /[:…]\s*$/.test(text.trim())
@@ -403,6 +476,7 @@ async function runAgent(ctx) {
403476
continue;
404477
}
405478
dbg(promisesAction ? 'stall.giveup' : 'turn.complete', { nudges, promised: promisesAction });
479+
if (await attemptVerify()) { continue; } // verification failed → fix feedback pushed, keep going
406480
reason = 'done';
407481
break;
408482
}
@@ -420,7 +494,7 @@ async function runAgent(ctx) {
420494
} finally {
421495
dbg('agent.done', { reason, steps: step - 1, edits: ctx.editCount || 0, credits: ctx.credits != null ? ctx.credits : null });
422496
// credits: null until the M10 gateway returns real usage; the response bar shows it when present.
423-
ctx.post({ type: 'agentDone', reason, edits: ctx.editCount || 0, credits: ctx.credits != null ? ctx.credits : null });
497+
ctx.post({ type: 'agentDone', reason, edits: ctx.editCount || 0, credits: ctx.credits != null ? ctx.credits : null, maxSteps: ctx.maxSteps });
424498
}
425499
}
426500

‎extensions/atom-ai/extension.js‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ const { registerLmProvider } = require('./lmProvider');
1818
const { registerInlineComplete } = require('./inlineComplete');
1919
const { runAgent } = require('./agent');
2020
const { registerReview } = require('./reviewSession');
21+
const { formatDiagnosticLines, diagKey } = require('./verify');
2122

2223
const CLAUDE_MODELS = [
2324
{ label: 'Claude Opus 4.8', id: 'claude-opus-4-8', detail: 'Most capable' },
@@ -437,6 +438,46 @@ function repairAgentMemory() {
437438

438439
/** Keep agent memory bounded. Trim oldest messages, but only at a goal boundary (a
439440
* string-content user message) so we never orphan a tool_use/tool_result pair. */
441+
// --- M5 auto-verify: editor-diagnostics gathering (vscode glue; pure formatting lives in verify.js) ---
442+
/** Pre-run snapshot of every diagnostic so we only ever report problems the agent NEWLY introduced
443+
* (keyed by severity+message, not position, so they survive the line shifts an edit causes). */
444+
function snapshotDiagnostics() {
445+
const base = new Map();
446+
for (const [uri, diags] of vscode.languages.getDiagnostics()) {
447+
const set = new Set();
448+
for (const d of diags) { set.add(diagKey(d.severity, d.message)); }
449+
base.set(uri.toString(), set);
450+
}
451+
return base;
452+
}
453+
/** Language servers publish diagnostics asynchronously after a save — wait until they settle (a quiet
454+
* period after the last relevant change) or a hard cap, so we don't read stale/empty results. */
455+
function waitForDiagnostics(uris, maxMs) {
456+
return new Promise((resolve) => {
457+
if (!uris.length) { resolve(); return; }
458+
const want = new Set(uris.map((u) => u.toString()));
459+
let done = false, quiet = null;
460+
const finish = () => { if (done) { return; } done = true; if (quiet) { clearTimeout(quiet); } clearTimeout(hard); sub.dispose(); resolve(); };
461+
const sub = vscode.languages.onDidChangeDiagnostics((e) => {
462+
if (e.uris.some((u) => want.has(u.toString()))) { if (quiet) { clearTimeout(quiet); } quiet = setTimeout(finish, 250); }
463+
});
464+
const hard = setTimeout(finish, maxMs);
465+
});
466+
}
467+
async function collectNewDiagnostics(uris, baseline, includeWarnings) {
468+
if (!uris.length) { return { text: '', count: 0 }; }
469+
await waitForDiagnostics(uris, 1500);
470+
const perFile = uris.map((uri) => ({
471+
rel: vscode.workspace.asRelativePath(uri),
472+
uriKey: uri.toString(),
473+
diags: (vscode.languages.getDiagnostics(uri) || []).map((d) => ({
474+
severity: d.severity, message: d.message, source: d.source,
475+
line: d.range.start.line, character: d.range.start.character
476+
}))
477+
}));
478+
return formatDiagnosticLines(perFile, baseline, includeWarnings, 40);
479+
}
480+
440481
function trimAgentMemory() {
441482
const MAX = 80, KEEP = 60;
442483
if (agentMessages.length <= MAX) { return; }
@@ -464,6 +505,17 @@ async function agentFlow(text) {
464505
agentMessages.push({ role: 'user', content: text });
465506
trimAgentMemory();
466507
dbg('agent.start', { model: cfg.get('claude.model', 'claude-sonnet-4-6'), maxSteps: cfg.get('agent.maxSteps', 25), transcriptMsgs: agentMessages.length, goalChars: text.length });
508+
// Auto-verify: capture a pre-run diagnostics baseline (so we only flag NEW problems) and a fresh
509+
// per-run set of edited files to verify afterward.
510+
const verifyCfg = {
511+
enabled: cfg.get('verify.enabled', true),
512+
command: cfg.get('verify.command', ''),
513+
maxRounds: Math.max(0, cfg.get('verify.maxRounds', 2)),
514+
includeWarnings: cfg.get('verify.includeWarnings', false)
515+
};
516+
const runTouched = new Set(); // abs paths edited this run (agent.js fills it via ctx.touched)
517+
const diagBaseline = verifyCfg.enabled ? snapshotDiagnostics() : new Map();
518+
dbg('verify.config', { enabled: verifyCfg.enabled, hasCommand: !!verifyCfg.command, maxRounds: verifyCfg.maxRounds, includeWarnings: verifyCfg.includeWarnings });
467519
try {
468520
await runAgent({
469521
messages: agentMessages, // persists across runs → the agent remembers the session
@@ -477,6 +529,10 @@ async function agentFlow(text) {
477529
commandStops: commandStops, // runId → stop() (process-group kill); used by Stop button / ■
478530
commandTimeout: cfg.get('commandTimeout', 120000), // backstop before a command is force-killed (0 = off)
479531
applyEdit: (req) => review.applyEdit(req), // file edits: apply-then-review
532+
verify: verifyCfg, // M5 auto-verify settings
533+
touched: runTouched, // agent.js adds each edited abs path here
534+
getTouchedUris: () => [...runTouched].map((p) => vscode.Uri.file(p)),
535+
getNewDiagnostics: () => collectNewDiagnostics([...runTouched].map((p) => vscode.Uri.file(p)), diagBaseline, verifyCfg.includeWarnings),
480536
signal: abort.signal
481537
});
482538
} finally {
@@ -623,6 +679,7 @@ class ChatViewProvider {
623679
case 'accountManage': await accountManage(); break;
624680
case 'copy': try { await vscode.env.clipboard.writeText(String(msg.text || '')); } catch (e) { /* clipboard unavailable */ } break;
625681
case 'retry': if (lastAgentGoal && !abort) { dbg('retry', { goalChars: lastAgentGoal.length }); await agentFlow(lastAgentGoal); } break;
682+
case 'continueAgent': if (!abort) { const g = lastAgentGoal; dbg('continue', { transcriptMsgs: agentMessages.length }); await agentFlow('Continue from where you left off and finish the task. Pick up exactly where you stopped — do not restart or repeat work that is already done.'); lastAgentGoal = g; } break;
626683
case 'feedback': dbg('feedback', { value: msg.value }); break;
627684
case 'openFile': await openWorkspaceFile(msg.path); break;
628685
case 'reviewKeepFile': dbg('review.keep', { id: msg.id }); review.keepFile(msg.id, 'kept'); break;

0 commit comments

Comments
 (0)