Skip to content

Commit 0c2add0

Browse files
runner: persist session records and recover after a restart
1 parent 5445e87 commit 0c2add0

6 files changed

Lines changed: 344 additions & 14 deletions

File tree

‎README.md‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,8 @@ Week 1 (Oct 19 to 23, 2026), plus the first part of week 2.
6969
| Piece | File | State |
7070
| --- | --- | --- |
7171
| Fake runner, the stand-in the daemon builds against | [runner/src/fake-runner.ts](runner/src/fake-runner.ts) | Done |
72-
| Session records and legal state moves | [runner/src/session-store.ts](runner/src/session-store.ts) | Done |
72+
| Session records and legal state moves | [runner/src/session-store.ts](runner/src/session-store.ts) | Done. The real runner saves them under `~/.codehands/sessions`, so they outlive a restart |
73+
| `recover()`: clean up after a restart | [runner/src/runner.ts](runner/src/runner.ts) | Done. Not in the contract's `Runner`: the daemon calls it once at startup, on the type `createRunner` returns |
7374
| `seq` 1, 2, 3 … per session | [runner/src/events.ts](runner/src/events.ts) | Done |
7475
| Claude Code arguments and stream-json parser | [runner/src/agents/claude-code.ts](runner/src/agents/claude-code.ts) | Done, tested against a recorded run |
7576
| Worktrees: branch names, add, remove, prune | [runner/src/git/worktree.ts](runner/src/git/worktree.ts) | Done |
@@ -113,17 +114,20 @@ const { branch, commit } = await runner.approve(sessionId, "Fix the date bug");
113114

114115
| Export | What |
115116
| --- | --- |
116-
| `createRunner(options?)` | The real runner. `worktreeRoot` overrides `~/.codehands/worktrees`. `memory` swaps where repo memory lives (default: files under `~/.codehands/memory`). `agentCommand` swaps the program it runs, which is how the tests replay a recording instead of calling Claude |
117+
| `createRunner(options?)` | The real runner. `worktreeRoot` overrides `~/.codehands/worktrees`. `sessionRoot` overrides `~/.codehands/sessions`. `memory` swaps where repo memory lives (default: files under `~/.codehands/memory`). `agentCommand` swaps the program it runs, which is how the tests replay a recording instead of calling Claude |
117118
| `createFakeRunner({ delayMs? })` | Emits 12 scripted events, 300 ms apart, then `approval.requested`, and supports every method including stop and resume. Costs nothing, needs no repo. Build against it, and keep it as the demo backup |
118119
| `EventSequence` | Hands out events with the next `seq` |
119-
| `SessionStore`, `SessionRecord` | In-memory session records with the legal-move check from the contracts |
120+
| `SessionStore`, `SessionRecord` | Session records with the legal-move check from the contracts. Saved as files when given a root, otherwise in memory (the fake runner) |
121+
| `RecoverableRunner`, `RecoveryReport` | What `createRunner` returns: the contract's `Runner` plus `recover()`, and what `recover()` reports |
120122
| `branchName(prompt, sessionId)`, `slugify(text)` | `codehands/fix-the-failing-test-3f2a9c1e` |
121123
| `MemoryStore`, `createMemoryStub()`, `createFileMemory(root?)` | Repo memory: `notes`, `addNote`, `profile`, `saveProfile`. The stub keeps it in a `Map` for tests |
122124
| `formatNote`, `runNote`, `trimNotes`, `notesForPrompt` | How a note is written, built from a run, trimmed to about 200 lines, and capped at 6,000 characters for the prompt |
123125
| `detectRepoProfile(dir)`, `RepoProfile` | What a repo is, read from its files. Never throws: an unknown repo gives `language: "unknown"` |
124126
| `repoKey(repoPath)` | The repo's id, which names its worktree and memory folders |
125127

126-
The runner keeps session records in memory only. The daemon owns the real database. A runner that restarts forgets its sessions; stop, resume and crash recovery are where that changes.
128+
The daemon owns the real database. The runner keeps only what it needs to finish a session, in `~/.codehands/sessions/<sessionId>.json`, so `approve`, `reject`, `resume` and `diff` still work after a restart. The agent itself does not survive one.
129+
130+
When the daemon starts it should call `recover()` once, before anything else. Sessions still marked `queued` or `running` become `failed`, and their ids come back in `report.failed`. Failed sessions lose their worktree, along with anything uncommitted in it, because `failed` is final and nothing else would remove it. Their branch goes too, unless the agent committed to it. `awaiting_approval` and `stopped` sessions are left alone. A worktree that can't be removed comes back in `report.cleanupErrors` and is tried again next time.
127131

128132
### How Claude Code's output becomes events
129133

@@ -222,7 +226,8 @@ The scripts in `runner/scripts/`:
222226
| File | Covers |
223227
| --- | --- |
224228
| [claude-parser.test.ts](runner/test/claude-parser.test.ts) | The whole recorded run replays in order; each mapping above; blocked commands; failed tests hidden behind exit code 0; error results; broken lines; `raw` on every event |
225-
| [runner.test.ts](runner/test/runner.test.ts) | A worktree on a new branch with your repo untouched; clean failures for a repo with no commits, an agent with no result line, a missing binary and an unsupported agent; diff with new files; approve commits on the branch and leaves `main` alone; reject deletes the branch; nothing to approve; stop kills the agent and its children, keeps the worktree and refuses a session that is not running; resume continues the numbering, passes the prompt, can be stopped again and refuses a session that is not stopped; `approval.requested` right after `task.finished`; memory: the second run gets the first run's note, the profile is saved, and failed or stopped runs add no note |
229+
| [runner.test.ts](runner/test/runner.test.ts) | A worktree on a new branch with your repo untouched; clean failures for a repo with no commits, an agent with no result line, a missing binary and an unsupported agent; diff with new files; approve commits on the branch and leaves `main` alone; reject deletes the branch; nothing to approve; stop kills the agent and its children, keeps the worktree and refuses a session that is not running; resume continues the numbering, passes the prompt, can be stopped again and refuses a session that is not stopped; `approval.requested` right after `task.finished`; memory: the second run gets the first run's note, the profile is saved, and failed or stopped runs add no note; after a restart: approve, reject, diff and resume still work, and `recover()` fails interrupted sessions, removes their worktrees, keeps branches with commits, leaves live sessions alone, and reports what it can't clean or read |
230+
| [session-store.test.ts](runner/test/session-store.test.ts) | A new store finds what the old one saved; the pid is never saved; moves are still checked after loading; ids that aren't plain names never reach the disk; no root means memory only |
226231
| [fake-runner.test.ts](runner/test/fake-runner.test.ts) | Event order and `seq`; approve only after finishing; stop then resume with `seq` continuing; no `approval.requested` after a stop |
227232
| [memory.test.ts](runner/test/memory.test.ts) | The stub keeps notes in order and repos apart; trimming drops whole notes and keeps the newest; the prompt cap keeps the newest end; a note's summary stays on one line; the file store reads back from disk and treats missing files as empty |
228233
| [profile.test.ts](runner/test/profile.test.ts) | npm, pnpm, npm's placeholder test script, `.nvmrc`, `engines.node`, needs install, broken `package.json`, Python, empty folder |

‎runner/src/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
export { createRunner, type RunnerOptions } from "./runner.js";
1+
export { createRunner, type RecoverableRunner, type RecoveryReport, type RunnerOptions } from "./runner.js";
22
export { createFakeRunner, type FakeRunnerOptions } from "./fake-runner.js";
33
export { EventSequence } from "./events.js";
44
export { SessionStore, type SessionRecord } from "./session-store.js";

‎runner/src/runner.ts‎

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,14 @@
11
import { randomUUID } from "node:crypto";
2+
import { existsSync } from "node:fs";
3+
import { homedir } from "node:os";
4+
import { join } from "node:path";
25
import { createInterface } from "node:readline";
36
import type { Runner, RunnerEvent, StartOptions } from "@codehands/contracts";
47
import { execa } from "execa";
58
import { claudeArgs, claudeResumeArgs, newClaudeParseState, parseClaudeLine } from "./agents/claude-code.js";
69
import { EventSequence } from "./events.js";
710
import { commitAll, diffSince } from "./git/diff.js";
8-
import { addWorktree, baseCommit, branchName, pruneWorktrees, removeWorktree, worktreePath } from "./git/worktree.js";
11+
import { addWorktree, baseCommit, branchName, git, pruneWorktrees, removeWorktree, worktreePath } from "./git/worktree.js";
912
import { createFileMemory, repoKey } from "./memory/notes.js";
1013
import { detectRepoProfile } from "./memory/profile.js";
1114
import { notesForPrompt, runNote, type MemoryStore } from "./memory/store.js";
@@ -16,32 +19,46 @@ import { SessionStore, type SessionRecord } from "./session-store.js";
1619

1720
export interface RunnerOptions {
1821
worktreeRoot?: string; // default ~/.codehands/worktrees
22+
sessionRoot?: string; // default ~/.codehands/sessions. Session records, so they outlive a restart
1923
memory?: MemoryStore; // default: files under ~/.codehands/memory. Tests pass createMemoryStub()
2024
// Which program to run for a task. Tests swap in a script that replays a recording.
2125
// notes: the repo's memory for the prompt, undefined when there is none.
2226
agentCommand?: (o: StartOptions, notes?: string) => { file: string; args: string[] };
2327
resumeCommand?: (o: StartOptions, prompt: string) => { file: string; args: string[] };
2428
}
2529

30+
// What recover() did. failed: sessions a restart interrupted, for the daemon to mark failed too.
31+
// cleanupErrors: sessions it couldn't clean up. They stay on disk and the next recover() tries again.
32+
export interface RecoveryReport {
33+
failed: string[];
34+
cleanupErrors: { sessionId: string; reason: string }[];
35+
}
36+
37+
export interface RecoverableRunner extends Runner {
38+
recover(): Promise<RecoveryReport>;
39+
}
40+
2641
const claudeCommand = (o: StartOptions, notes?: string) => ({ file: "claude", args: claudeArgs(o, notes) });
2742
// start() passes our sessionId as Claude's --session-id, so the same id resumes it.
2843
const claudeResumeCommand = (o: StartOptions, prompt: string) =>
2944
({ file: "claude", args: claudeResumeArgs(o.sessionId, prompt, o.budgetUsd) });
3045

3146
export function createRunner({
3247
worktreeRoot,
48+
sessionRoot = join(homedir(), ".codehands", "sessions"),
3349
memory = createFileMemory(),
3450
agentCommand = claudeCommand,
3551
resumeCommand = claudeResumeCommand,
36-
}: RunnerOptions = {}): Runner {
37-
const store = new SessionStore();
52+
}: RunnerOptions = {}): RecoverableRunner {
53+
const store = new SessionStore(sessionRoot);
3854
// The running agent of each session, so stop() can wait for it to exit.
3955
const exits = new Map<string, Promise<unknown>>();
4056

4157
// Every event goes through emit, so lastSeq is always right for resume().
4258
function emitters(record: SessionRecord, seq: EventSequence) {
4359
const emit = (event: RunnerEvent) => {
4460
record.lastSeq = seq.last;
61+
store.save(record);
4562
return event;
4663
};
4764
const fail = (reason: string, extra: Record<string, unknown> = {}) => {
@@ -67,6 +84,7 @@ export function createRunner({
6784
record.branch = branchName(o.prompt, o.sessionId);
6885
record.repoKey = await repoKey(o.repoPath);
6986
record.worktreePath = worktreePath(record.repoKey, o.sessionId, worktreeRoot);
87+
store.save(record); // before the worktree exists, so a crash here can't leave one nobody knows about
7088
await pruneWorktrees(o.repoPath);
7189
await addWorktree(o.repoPath, record.branch, record.worktreePath, record.baseCommit);
7290
} catch (err) {
@@ -169,8 +187,40 @@ export function createRunner({
169187
return notesForPrompt(await memory.notes(key));
170188
}
171189

190+
// A failed session can't be approved or rejected, so nothing else would remove its worktree.
191+
async function discardWorktree(record: SessionRecord): Promise<void> {
192+
const { repoPath } = record.options;
193+
await removeWorktree(repoPath, record.worktreePath!);
194+
if (existsSync(record.worktreePath!)) throw new Error(`could not remove ${record.worktreePath}`);
195+
// The branch goes only if it never moved: commits the agent made are kept.
196+
const tip = await git(repoPath, ["rev-parse", "--verify", "--quiet", `refs/heads/${record.branch}`]).catch(() => "");
197+
if (tip && tip === record.baseCommit) await git(repoPath, ["branch", "-D", record.branch!]);
198+
record.worktreePath = undefined;
199+
store.save(record);
200+
}
201+
172202
return {
173203
start,
204+
// Call once when the daemon starts, before it starts or commands anything.
205+
// A session still marked queued or running lost its agent, so it becomes failed.
206+
// awaiting_approval and stopped sessions are left alone: approve, reject and resume still work.
207+
async recover() {
208+
const report: RecoveryReport = { failed: [], cleanupErrors: [] };
209+
for (const sessionId of store.ids()) {
210+
if (exits.has(sessionId)) continue; // its agent is alive in this process
211+
try {
212+
const record = store.get(sessionId);
213+
if (record.status === "queued" || record.status === "running") {
214+
store.transition(sessionId, "failed");
215+
report.failed.push(sessionId);
216+
}
217+
if (record.status === "failed" && record.worktreePath) await discardWorktree(record);
218+
} catch (err) {
219+
report.cleanupErrors.push({ sessionId, reason: (err as Error).message });
220+
}
221+
}
222+
return report;
223+
},
174224
// Kills the agent and everything it started. task.stopped comes out of the start() stream,
175225
// and the session is stopped by the time this resolves. The worktree stays.
176226
async stop(sessionId) {

‎runner/src/session-store.ts‎

Lines changed: 55 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import { mkdirSync, readdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
2+
import { join } from "node:path";
13
import { canTransition, type SessionStatus, type StartOptions } from "@codehands/contracts";
24

35
// Only what the runner needs to stop, resume, diff, approve or reject.
@@ -9,32 +11,61 @@ export interface SessionRecord {
911
repoKey?: string; // set by start(). Where this repo's memory lives
1012
branch?: string;
1113
baseCommit?: string;
12-
pid?: number;
13-
stopRequested?: boolean; // set by stop(), so the agent's exit reads as task.stopped, not task.failed
14+
pid?: number; // never saved: after a restart the number may belong to another program
15+
stopRequested?: boolean; // set by stop(), so the agent's exit reads as task.stopped, not task.failed. Never saved
1416
lastSeq: number;
1517
diff?: { patch: string; stat: string };
1618
}
1719

20+
// Session ids name files, so one like ../../x must never reach the disk.
21+
const SAFE_ID = /^[\w-]+$/;
22+
1823
export class SessionStore {
1924
private sessions = new Map<string, SessionRecord>();
25+
private root?: string;
26+
27+
// With a root, every record is also a file (<root>/<sessionId>.json), so it outlives a restart.
28+
// Without one it stays in memory, which is all the fake runner needs.
29+
constructor(root?: string) {
30+
this.root = root;
31+
if (root) mkdirSync(root, { recursive: true });
32+
}
2033

2134
create(options: StartOptions): SessionRecord {
22-
if (this.sessions.has(options.sessionId)) throw new Error(`session ${options.sessionId} already exists`);
35+
if (this.root && !SAFE_ID.test(options.sessionId)) throw new Error(`bad session id ${options.sessionId}`);
36+
if (this.find(options.sessionId)) throw new Error(`session ${options.sessionId} already exists`);
2337
const record: SessionRecord = { options, status: "queued", lastSeq: 0 };
2438
this.sessions.set(options.sessionId, record);
39+
this.save(record);
2540
return record;
2641
}
2742

2843
get(sessionId: string): SessionRecord {
29-
const record = this.sessions.get(sessionId);
44+
const record = this.find(sessionId);
3045
if (!record) throw new Error(`unknown session ${sessionId}`);
3146
return record;
3247
}
3348

49+
// Every saved session, including ones from before a restart.
50+
ids(): string[] {
51+
if (!this.root) return [...this.sessions.keys()];
52+
return readdirSync(this.root).filter((name) => name.endsWith(".json")).map((name) => name.slice(0, -".json".length));
53+
}
54+
55+
// Writes to a temp file first, so a crash mid-write can't leave half a record.
56+
save(record: SessionRecord): void {
57+
if (!this.root) return;
58+
const { pid: _pid, stopRequested: _stopRequested, ...saved } = record;
59+
const file = this.file(record.options.sessionId);
60+
writeFileSync(`${file}.tmp`, JSON.stringify(saved));
61+
renameSync(`${file}.tmp`, file);
62+
}
63+
3464
// Refuses any move not in SESSION_TRANSITIONS, e.g. approving a running session.
3565
transition(sessionId: string, to: SessionStatus): SessionRecord {
3666
const record = this.check(sessionId, to);
3767
record.status = to;
68+
this.save(record);
3869
return record;
3970
}
4071

@@ -46,4 +77,24 @@ export class SessionStore {
4677
}
4778
return record;
4879
}
80+
81+
// Memory first, then the disk. undefined when it's in neither.
82+
private find(sessionId: string): SessionRecord | undefined {
83+
const known = this.sessions.get(sessionId);
84+
if (known || !this.root || !SAFE_ID.test(sessionId)) return known;
85+
let text: string;
86+
try {
87+
text = readFileSync(this.file(sessionId), "utf8");
88+
} catch (err) {
89+
if ((err as NodeJS.ErrnoException).code === "ENOENT") return undefined;
90+
throw err;
91+
}
92+
const record = JSON.parse(text) as SessionRecord;
93+
this.sessions.set(sessionId, record);
94+
return record;
95+
}
96+
97+
private file(sessionId: string): string {
98+
return join(this.root!, `${sessionId}.json`);
99+
}
49100
}

0 commit comments

Comments
 (0)