Skip to content

Commit fa037c9

Browse files
mchwangclaude
andcommitted
Merge main into F2d: keep F1c/F1d beside the v7 PR records
runner/store.ts: both sides appended a section at the end of Store. Kept F2d's schema v7 methods and F1d's startup recovery methods; main is still at schema v6, so v7 stays F2d's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 parents a0b5206 + b1c6b1d commit fa037c9

39 files changed

Lines changed: 3165 additions & 323 deletions

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,4 @@ dist/
66
.codeboost-local/
77
playwright-report/
88
test-results/
9+
.vitest/

‎AGENTS.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,10 @@ Every reproduced race requires a failing-before and passing-after regression. As
117117
- Neutralise issue references (`#N`, `GH-N`, `owner/repo#N`, issue URLs) in any text codeboost writes that can become a commit message, such as a PR title or description, and neutralise @-mentions in any of that text that is not fenced (a title). Code fences do not protect commit messages, and GitHub closes issues from closing keywords in default-branch commits.
118118
- Never spread a collection whose size follows unbounded input into function arguments (`Math.max(...runs)`); engines limit the argument count, so use a loop.
119119
- A hardened Git invocation must also keep Git out of nested repositories and populated submodules, whose own config and hooks are the agent's: pass `--ignore-submodules` on the command line (the config default does not bind plumbing or override `.gitmodules`), and never run Git with a nested repository as its working directory.
120+
- Decide a filesystem fact (whether a name exists, where a link resolves, what a path's entries are) on the file system that will serve it, not on a copy. A host that folds case or Unicode normalization, or decodes names as text, answers differently from the container's; check links and names inside the container, as its kernel resolves them, by raw bytes.
121+
- Never infer that agent-written content is unchanged from timestamps, sizes or an index's stat cache: not every write moves them (tmpfs, for one). Compare the content itself.
122+
- Hand agent-chosen names to tools as literal data: disable or sidestep pathspec magic (a leading `:`), quote them in the tool's own input format (a leading `"` in `--stdin-paths`), pass them as bytes, and keep empty fields when splitting NUL-separated output. Check a name's safety only where it is reported, never refusing unchanged input for a name the report never carries.
123+
- Check a trust root before anything reads it. When a tool reads config or rules from storage an agent could have reached (Git and the metadata volume), compare that storage with its trusted baseline first, and run nothing if it changed.
120124

121125
## Blinded experiments
122126

‎agents/container/changes.ts‎

Lines changed: 332 additions & 0 deletions
Large diffs are not rendered by default.

‎agents/container/run.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import { BASE_IMAGE, CLAUDE_VERSION, CODEX_VERSION } from './image.ts';
99
import { taskFilesystemOwner } from './storage.ts';
1010
import { hasOwnerLabels, ownerLabels } from '../labels.ts';
1111
import { assertPhasePolicy } from '../policy.ts';
12-
export { prepareTaskFilesystems, removeTaskFilesystems } from './storage.ts';
12+
export { prepareTaskFilesystems, removeTaskFilesystems, UnusableRepositoryError } from './storage.ts';
1313
export type { TaskFilesystems, TaskStorageLimits } from './storage.ts';
1414

1515
const validateSecrets = (profile: ContainerProfile, secrets: Readonly<Record<string, string>>) => {

‎agents/container/storage.ts‎

Lines changed: 159 additions & 124 deletions
Large diffs are not rendered by default.

‎agents/container/tree-script.ts‎

Lines changed: 619 additions & 0 deletions
Large diffs are not rendered by default.

‎agents/recovery.ts‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,10 @@ export interface UnownedResource extends RecoveredResource {
2424
readonly reason: 'no-runner-label' | 'unknown-kind' | 'inconsistent-storage';
2525
}
2626
export interface RecoveryReport {
27-
/** Agent containers, egress proxies, seeders, export containers and networks of this runner, now confirmed gone. */
27+
/**
28+
* Agent containers, egress proxies, seeders, export and inspection containers and networks of this runner, now
29+
* confirmed gone.
30+
*/
2831
readonly removed: readonly RecoveredResource[];
2932
/** One handle per task-storage allocation of this runner, kept whole for export and `removeTaskFilesystems`. */
3033
readonly storage: readonly RecoveredTaskStorage[];
@@ -119,8 +122,8 @@ const removeById = async (resource: RecoveredResource, remaining: () => number)
119122

120123
/**
121124
* Crash recovery for one database (#51 item 4). Acts only on objects whose `io.codeboost.runner` label is
122-
* `runnerOwner`: it removes agent containers, egress proxies, seeders, export containers and networks, and resolves
123-
* only once they are gone. It keeps task storage whole (volumes and keeper) and returns a recovery handle per
125+
* `runnerOwner`: it removes agent containers, egress proxies, seeders, export and inspection containers and networks,
126+
* and resolves only once they are gone. It keeps task storage whole (volumes and keeper) and returns a recovery handle per
124127
* allocation. Objects from older builds without a runner label, and anything it does not recognise, are reported and
125128
* never touched.
126129
*
@@ -181,8 +184,9 @@ export async function recoverLeftovers(runnerOwner: string, timeoutMs = 120_000)
181184
for (const resource of owned.container) {
182185
const kind = kindOf(resource);
183186
if (kind === 'storage:keeper') keep(resource, 'keeper');
184-
// Seeders and export containers are transient: a leftover one is removed, never kept with the storage.
185-
else if (kind === 'storage:seeder' || kind === 'storage:export' || kind === 'agent' || kind === 'egress')
187+
// Seeders, export and inspection containers are transient: a leftover one is removed, never kept with the storage.
188+
else if (kind === 'storage:seeder' || kind === 'storage:export' || kind === 'storage:inspect' || kind === 'agent'
189+
|| kind === 'egress')
186190
remove.push(resource);
187191
else unknown(resource);
188192
}

‎docs/implementation/agent-isolation.md‎

Lines changed: 41 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,10 @@ Use only these entry points to run an agent:
6868
if not, it removes what it made and refuses. Cleanup removes an object by the ID captured at its create, even if
6969
its labels are wrong; an object found only by name (including task storage) must carry all three owner labels. Call
7070
`removeTaskFilesystems` when the task ends. It refuses a repository that has a
71-
symbolic link with an absolute target or a target outside the checkout, before it
72-
creates any storage. Report this to the user as a repository the agent cannot run
73-
on; do not retry it.
71+
symbolic link with an absolute target or one that can lead outside the checkout (resolved in the container as its
72+
kernel would, even once the agent creates a missing directory on the way), or any link in its Git metadata; the
73+
seeder checks this and the allocation removes what it made, then throws an `UnusableRepositoryError`. Report
74+
this to the user as a repository the agent cannot run on; do not retry it.
7475
3. `captureInvocation` freezes the request. Capture each attempt ID once. A new
7576
attempt needs a new attempt ID.
7677
4. `startCodexInvocation` or `startClaudeInvocation` returns a handle at once and runs
@@ -80,17 +81,48 @@ Use only these entry points to run an agent:
8081
call. The request carries `networkAllocationId`, a UUID you choose for the vendor network. Pass the vendor
8182
credential only as the function argument.
8283
5. To keep a stopped writable attempt's partial output, call `exportTaskDiff(storage, { base, imageId })` before
83-
`removeTaskFilesystems`. `base` is the full ID of the last commit codeboost made in that storage. It returns at
84-
most 1 MiB of diff (`truncated` says whether it was cut), and takes `signal` and a deadline for the Docker work.
84+
`removeTaskFilesystems`. `base` is the full ID of the commit the storage was seeded from (the clone's head). For a
85+
recovery handle, also pass the `metadataBaseline` you recorded (step 6). It returns at most 1 MiB of diff
86+
(`truncated` says whether it was cut), and takes `signal` and a deadline for the Docker work.
8587
A changed file over 8 MiB, an untracked nested repository, a path Git will not add, an entry named `.git`, a fifo
8688
or socket, an ignored untracked path, a submodule directory with content, and a changed file whose `ident` or
8789
`working-tree-encoding` attribute changes what the diff shows each appear as a `codeboost:` notice line. When Git
8890
fails, the error carries Git's last two error lines.
89-
6. After a crash or restart, call `recoverLeftovers(runnerOwner)` (`agents/recovery.ts`) while holding the
91+
6. To audit what a writable attempt changed (#66, `agents/container/changes.ts`):
92+
- **Record the baseline.** The storage value carries `metadataBaseline`, a digest of the metadata volume that the
93+
seeder takes as its last step. Record it with the allocation. `snapshotDeclaredLinks`, `inspectTaskChanges` and
94+
`exportTaskDiff` each check the metadata against it before running any Git command, and need it back
95+
(`metadataBaseline`) for a recovery handle.
96+
- **Before launch**, call `snapshotDeclaredLinks(storage, paths, { imageId })` with the item's declared paths.
97+
For each declared symlink it records where it resolves, one part at a time as the kernel would, and the state of
98+
the target and everything beneath it. A link on the way, a target that is a link, or a link inside a directory
99+
target shows as `through-link`. Do not launch an item with a `through-link` declared link: a write through it
100+
lands somewhere its target does not cover. Keep the result.
101+
- **After the handle settles**, call `inspectTaskChanges(storage, { base, linkSnapshot, imageId })`. It returns the
102+
change manifest: every difference between the work tree and `base`, read without following links, with content
103+
IDs as a commit would store them. New ignored files, fifos and entries under a `.git` part are listed; a new
104+
directory that `base`'s own ignore rules ignore, with no tracked entry beneath it, is one entry (`ignored: true`). It also returns `agentCommits`,
105+
`metadataChanged`, `linkTargetChanges`, `nestedGitlinkContent` and `digest`. If the metadata changed, no Git
106+
command runs: the manifest has `metadataChanged: true` and every other list empty.
107+
- **Needs human.** The metadata is read-only to agents, so any agent commit or metadata change means a protection
108+
failed. Route it to needs human, as for link target changes and nested gitlink content.
109+
- **Refusals.** It refuses, and never returns part of the answer, when:
110+
- there are more than 10,000 changes (a populated submodule counts as one);
111+
- a name or link target it reports is longer than 1,024 bytes, is not strict UTF-8, or holds a Unicode control,
112+
format, line or paragraph separator, or unassigned character (unchanged names are never checked);
113+
- the recorded targets hold more than 20,000 entries;
114+
- it cannot read something;
115+
- `base` is not a commit in the storage, or Git fails;
116+
- for a snapshot, the metadata changed since seeding (an export refuses then too).
117+
118+
Treat a refusal as needs human.
119+
- **Both calls** run in a read-only container with no network, take `signal`, `onProcessGroup` and `timeoutMs`
120+
(default 120 s), and settle only after their container is gone.
121+
7. After a crash or restart, call `recoverLeftovers(runnerOwner)` (`agents/recovery.ts`) while holding the
90122
database's single-runner lock and before admitting work. It touches only objects labelled with that runner
91-
token. It removes agent containers, egress proxies, seeders, export containers and networks, and resolves once they are gone. It keeps
92-
task storage whole (both volumes and the keeper) and returns one recovery handle per allocation, carrying its
93-
attempt and allocation IDs. `removeTaskFilesystems` accepts a handle as it accepts the value
123+
token. It removes agent containers, egress proxies, seeders, export and inspection containers and networks, and
124+
resolves once they are gone. It keeps task storage whole (both volumes and the keeper) and returns one recovery
125+
handle per allocation, carrying its attempt and allocation IDs. `removeTaskFilesystems` accepts a handle as it accepts the value
94126
`prepareTaskFilesystems` returned. D issues a handle only after checking every part's owner labels. Objects without
95127
a runner label (from older builds), objects of this runner that D does not create, and storage whose parts
96128
disagree are listed in `unowned` and never touched. An object counts as D's only with exactly one kind label,

0 commit comments

Comments
 (0)