Skip to content

Commit f53227c

Browse files
mchwangclaude
andauthored
D: inspect what an agent changed in task storage (#66, part 1) (#85)
* D: inspect what an agent changed in task storage (#66, part 1) snapshotDeclaredLinks records, before launch, where each declared link resolves (one part at a time, as the kernel does) and the state of its target. inspectTaskChanges returns the change manifest F2a audits: every difference between the work tree and base, walked without following links, with content IDs as a commit would store them; agent commits and any metadata change against a baseline the seeder now records; link target changes; content in gitlink directories; and a digest. Ignored paths follow base's own rules, and an ignored directory is reported once. It fails rather than returns part of the answer on too many changes, a name that is not strict UTF-8, or anything unreadable. Both run a Perl program in a read-only container through runStorageScript, which the export now uses too. Recovery removes leftover inspection containers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 1 found in change inspection - Names reach check-ignore behind "./", so ":/x" or ":(glob)x" is only a name, not pathspec magic that hid a file or failed the run. - A declared link whose target is itself a link, or whose directory target holds a link, is through-link: a write through it lands elsewhere. - Inspection re-resolves links without walking their targets, so each recorded target is counted once against the entry limit; a target's status is compared from its full walk only. - The walk asks check-ignore about each new entry as it reaches it and never enters an ignored directory, so a large ignored tree cannot exhaust the container's memory, and it stops once new entries alone pass the change limit. - Link targets are resolved from the filesystem root, so "/work/x" and "../work/x" are inside the work tree. - A dangling last part is watched as a target; output is bounded explicitly; docs and comments describe what the code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 2 found in change inspection - Ignore checks mirror each checked path's type under /tmp/ignore and go in without a trailing slash, so "dir/*" with a negation, and directory-only patterns against a file, decide as Git does. - A directory target holding a link records and compares every entry; F must not launch an item with a through-link declared link. - Resolution continues by name past a missing directory, so removing a target's parent is a status change, not a retarget; an anchor walks up to the nearest existing entry. - The walk keeps only type and mode per entry, and inspection containers get 1 GB, so a 200,000-file checkout inspects. - Inspection takes a link count, not a separator, so a declared path named "--" works. - Names are capped at 1,024 bytes and declared links at 1,000, the output bound follows from those limits, C1 and bidi controls are refused, and any unexpected Perl failure exits 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 3 found in change inspection - The ignore mirror creates every parent of a checked path and moves aside whatever stands in the way, so a tracked file turned into a directory with new subdirectories inspects. - A ".." after a missing directory leaves the link absent with no target; anchors are found walking down without ever reading through a link; a target's parents are checked at inspection, so a parent turned into a link is reported and nothing outside the work tree is read. - Names refuse Unicode control, format and separator characters (Cc, Cf, Zl, Zp), in Perl and TypeScript alike. - The argument budget holds every declared link and target at the name limit, so any snapshot accepted can be inspected; declared links are capped at 200. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read every file at inspection instead of trusting the index's stat check Review round 4 got a clean inspection over changed content on the tmpfs work volume: a file's times do not always move with its content, so a stat that matches the index proves nothing. Inspection now hashes every tracked and new file as a commit would store it (19 s for 200,000 files at half a CPU), and the metadata digest covers each file's content, not only its stat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 5 found in change inspection - The ignore mirror never moves one of base's .gitignore files: a directory the agent puts where one was cannot switch its rules off. Below such a directory nothing is mirrored, which can only list more. - A file inspection cannot read is refused by name in the walk (exit 6) rather than failing inside Git. - A test checks that inspection runs with 1 GB and takes a result far past 16 MiB. - The docs and comments list every refusal, and the collapsed-directory rule in full; the digest comment says the commit step is still to come. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 6 found in change inspection - Files are hashed into a scratch index built from base, with update-index --info-only, so IDs are what git add would store (a text=auto file base keeps with CRLF stays CRLF), and anything Git reports on stderr fails the run with its line (an encoding it could not apply). - A .gitattributes that is not a regular file is refused by name instead of blocking Git until the deadline. - The seeder's digest failure carries Perl's error; a Git killed by a signal says so; a regular .gitignore of any mode is mirrored; the metadata digest hashes content outside objects/, which kept allocation fast. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 7 found in change inspection - Paths that are gone or changed type leave the scratch index before hashing, as git add -A handles them, so a deleted .gitattributes no longer applies its rules from the index. - Only an error or fatal line from Git fails the run, or an unprefixed line that does not continue a warning: Git's warnings about attribute files it ignores (a negative pattern, a misplaced macro, a symlinked .gitattributes) are what a commit does too, and an honest repository with them now inspects. - The metadata digest hashes every file except loose objects and packs, so pack indexes and info/alternates are covered; a Git failure while checking base reads as one, not as a missing commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 8 found in change inspection - Git's two unprefixed complaints about an attribute line it skips ("is not a valid attribute name", "not allowed") no longer fail the run on their own; a test covers them without a warning in front. - The comments say what the hashing guarantees: the work tree's attributes alone decide (git add -A can apply a deleted .gitattributes depending on path order), and the commit step must build from the same scratch index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 9 found in change inspection - The metadata digest is compared with the baseline before any Git command: config the agent could have changed (a filter driver) never runs. If it changed, a snapshot refuses and an inspection reports only metadataChanged, having read nothing else. Both operations take the baseline. - A submodule checked out as a directory stays in the scratch index, so a commit built from it keeps the submodule. - Git's attribute complaints are matched whatever the path holds (a colon, or the root file), and only the lines right after a warning continue it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 10 found in change inspection - The export checks the metadata digest against the baseline before running Git, as inspection does, and a recovery handle needs the recorded baseline; the resolution is shared in storage.ts. - A snapshot with no declared links still runs the script, so the metadata check before launch always happens. - The scratch index only computes file IDs: the comments now say the commit step must store files the same way and apply links and deletions from the manifest; the unfounded submodule exception is gone. - The export tests no longer write the metadata: base-side state (the tracked ssl directory, tracked and excluded directories, a tracked directory that becomes a repository, large tracked files, a recorded submodule, a macOS ignorecase config) is set up in the source repository or the clone before seeding, as production has it. - Fake-Docker tests cover the metadata-only result and its guard; the docs give every operation's baseline and refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 11 found in change inspection - A submodule checked out as a directory stays in the scratch index again: the commit step removes the same entries, and without the exception would drop every submodule. - The export docs and comments say what it now covers (edits and new files; agents cannot commit or stage), that `base` is the seeded commit, and that a recovery handle needs `metadataBaseline`; the baseline comment no longer promises a new baseline after a runner commit. - A fake-Docker test checks that F's baseline reaches the export script for a recovery handle. - The ignored-and-unreadable export case uses a base .gitignore, as production can have, instead of info/exclude. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 12 found in change inspection - A path below a .gitignore file standing where the agent made a directory is taken as not ignored, without asking Git (which could not tell what is a directory there): it is listed, never collapsed. - An entry under a .git part is never the new half of a rename, so a deletion stays a deletion. - Every .gitattributes leaves the scratch index before hashing, so where Git will not read the work tree's own (over 100 MB, a symlink) it cannot fall back to base's: only attribute files the work tree has, and Git reads, decide. - A snapshot walks each distinct target once, so two links to one target count its entries once. - The script's header gives both modes' arguments and every exit status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 13 found in change inspection - Only a regular .gitattributes leaves the scratch index: a tracked symlink holds a link target, never rules, and the commit step keeps it. - A snapshot reports each target's state once, in a `targets` map keyed by path, as inspection does; link records carry the target and its status. A hundred links to one large target no longer repeat its entries past the output bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gap review round 14 found in change inspection Every .gitattributes leaves the scratch index again, a symlink included: Git falls back to the index when it will not open a symlinked one, and parses the link's target text as rules. The scratch index only computes file IDs; the commit step builds its tree from base plus the manifest's changes, so an unchanged symlinked .gitattributes or submodule stays as base has it, and the submodule exception is no longer needed. A test gives a symlinked .gitattributes whose target text is a rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Refuse names with code points the image's Perl does not assign Review round 15: the format-character ban used Perl 5.36's Unicode 14, so a format character added later (U+13439) passed into the manifest while Node's check refused it. Unassigned code points (Cn) are now refused on both sides, so a character newer than the check knows fails closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 16 found in change inspection - A tracked file whose bytes are exactly what checking out base's blob writes is untouched, even when re-hashing it would give another ID (base committed it with CRLF before a text rule, or with an expanded $Id$ before an ident rule). Only files whose checkout converts are compared, by base's attributes and the repository's line-ending config, with cat-file --filters. - The lifecycle doc no longer says the export diffs against the last codeboost commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 17 found in change inspection - The checkout comparison reads base's attribute files, mirrored under /tmp/attributes, not the work tree's: a file equal to what the clone's checkout wrote is unchanged even if the agent deleted .gitattributes, and a rule the agent adds cannot make its own content count as base's checkout. - Past the change limit, the run refuses before starting one cat-file per converting file. - Both Git runners share one stderr filter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 18 found in change inspection - A symlinked .gitattributes in base is mirrored as its target text, as the clone's checkout read it, so a file untouched since that checkout is not reported. - Names are checked only as they go into the output (changes, nested content, links, target entries): an unchanged file in base may have any name, such as a Persian name with a zero-width non-joiner. - The refusal message, the script header, the TS doc and agent-isolation.md state the name rules in full. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 19 found in change inspection - A tracked file whose bytes are exactly base's blob is untouched whatever rules the work tree has now: an agent adding a text rule no longer makes an untouched CRLF file a change. This cheap check runs first; only what is left and converts is compared with base's checkout, under its own cap with its own message. - A populated gitlink counts against the change limit, so the output bound covers nestedGitlinkContent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 20 found in change inspection - The checkout-comparison cap is gone: untouched legacy files can need comparing as well as changed ones, so their number is not a change count, and the run's deadline bounds that work. - TaskChange states all three ways a file's content counts as unchanged, including bytes exactly base's blob. - A test with 10,001 deletions reaches the final change count (the walk's early limit only counts new entries). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep empty attribute values when reading check-attr's answer Review round 21: Perl's split drops trailing empty fields, so an empty value ("* filter=") as the last field crashed an honest inspection. Empty fields are kept, only the final terminator's is dropped, and an answer that is not whole triples fails as a Git failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gaps review round 22 found in change inspection - A declared path that is not a link is not-a-link whether or not it or its directory exists, so adding or deleting a declared plain file is not reported as a change to a link; a link on the way still makes it through-link. - The lifecycle doc and LinkTargetChange say that `retargeted` on a link the item itself edits is that edit, judged by the plan's link rules, while any change to what a link points at sends the task to needs human. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Close the gap review round 23 found in change inspection - A declared link's way is followed by name past any part that is missing or not a directory, ".." included, so its old target is always recorded and watched: a write through a dangling link the item then retargets is reported. A link on that way still makes it through-link. - The host-side link check treats a path under a file (ENOTDIR) as dangling, as it does a missing part, instead of failing to seed a repository with such a link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check every existing part of a link's way, even past a missing one Review round 24: once a part was missing, both resolvers stopped checking the rest, but a ".." can climb back into real directories. The snapshot then missed a link on the way (m/../d/esc/../x reported absent, not through-link), and the host-side link check let a repository seed with a link that leaves the checkout once the agent creates the missing directory or turns a file into one (f/x/../../d/esc/../etc/hostname). Every part that exists is now checked in both, and a link met there makes it through-link, or is followed and refused if it leaves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Match link parts by exact name on a case-insensitive host Review round 25: on macOS the host-side link check found the link "d" when a link's text said "D", followed it, and accepted a link that leaves the checkout once the agent creates D in the container, where names are exact. A part now counts only when its directory holds exactly that name. The new seed-refusal case fails without the fix on a case-insensitive host; on Linux the part is simply missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check repository links in the container, as its kernel resolves them Review rounds 24-26 kept finding ways the host-side link check disagreed with the container: a ".." past a missing part back to a real link, a host that folds case or Unicode normalization, names decoded as UTF-8, and a check that read whole directories synchronously. The check now runs in the seeder, over the storage it just filled, as a Perl `links` mode: raw bytes, exact names, every link on the way followed, missing parts taken as directories the agent may create until a ".." climbs back. Any link in the metadata, or one that can lead out of the work tree, fails the seeder, and the allocation removes what it made. The host-side walk is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Report an unusable repository as its own error, reason first Review round 27: the seeder's refusal reached the caller as a DockerError cut at 512 characters, so a long name could push the reason out, and nothing told it apart from a Docker failure. The seeder's exit 11 now becomes an UnusableRepositoryError with up to 2 KB of the seeder's message, which puts the reason before the name. The script's header lists its fourth mode, and a stale test comment about the removed host check is corrected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Take the refusal's reason from the script's own line; export the error from run.ts Review round 28: Docker can print warnings on stderr before the seeder's output, so the UnusableRepositoryError now carries the script's "Repository ..." line rather than all of stderr. run.ts, which production callers import from, re-exports the error, and the metadata-link test checks its type and message as the link test does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test that storage allocation walks nothing on the host The walk-pauses test for the host-side link walk is replaced: that walk is gone, and the link check runs in the seeder. The test now asserts that allocation reads none of the checkout's directories on the host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add review rules from the change-inspection review rounds Four patterns recurred across the #66 part 1 review: filesystem facts decided on a host copy instead of the container, unchanged content inferred from timestamps, agent-chosen names passed to tools as anything but literal data, and Git reading config before the metadata was checked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hash the content of every metadata file, objects and packs included Copilot: the metadata digest trusted loose objects and packs to their names and stat fields, so a same-size overwrite that did not move the times would leave the baseline matching, and Git would then run on changed objects. That is the timestamp rule this branch adds to AGENTS.md; every file's content is now hashed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent f2f1551 commit f53227c

12 files changed

Lines changed: 1971 additions & 197 deletions

‎AGENTS.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,10 @@ Every reproduced race requires a failing-before and passing-after regression. As
102102
- When a subprocess reports a problem only as a warning and carries on, decide pass or fail by what each message means for the result, not by whether anything was printed: fail on messages that mean it did less than it should (for example could not read a path), and let through messages about harmless input the agent controls. Test both a benign case and a failing case, and filter the output as it arrives so that no volume of benign messages can push a failure out of a bounded buffer.
103103
- Quote or escape agent-controlled text (file names, paths, branch names) wherever it lands in output that people or tools parse, such as diffs, notices, logs or reports, so it cannot forge that output's structure.
104104
- A hardened Git invocation must also keep Git out of nested repositories and populated submodules, whose own config and hooks are the agent's: pass `--ignore-submodules` on the command line (the config default does not bind plumbing or override `.gitmodules`), and never run Git with a nested repository as its working directory.
105+
- Decide a filesystem fact (whether a name exists, where a link resolves, what a path's entries are) on the file system that will serve it, not on a copy. A host that folds case or Unicode normalization, or decodes names as text, answers differently from the container's; check links and names inside the container, as its kernel resolves them, by raw bytes.
106+
- Never infer that agent-written content is unchanged from timestamps, sizes or an index's stat cache: not every write moves them (tmpfs, for one). Compare the content itself.
107+
- Hand agent-chosen names to tools as literal data: disable or sidestep pathspec magic (a leading `:`), quote them in the tool's own input format (a leading `"` in `--stdin-paths`), pass them as bytes, and keep empty fields when splitting NUL-separated output. Check a name's safety only where it is reported, never refusing unchanged input for a name the report never carries.
108+
- Check a trust root before anything reads it. When a tool reads config or rules from storage an agent could have reached (Git and the metadata volume), compare that storage with its trusted baseline first, and run nothing if it changed.
105109

106110
## Blinded experiments
107111

‎agents/container/changes.ts‎

Lines changed: 332 additions & 0 deletions
Large diffs are not rendered by default.

‎agents/container/run.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import { BASE_IMAGE, CLAUDE_VERSION, CODEX_VERSION } from './image.ts';
99
import { taskFilesystemOwner } from './storage.ts';
1010
import { hasOwnerLabels, ownerLabels } from '../labels.ts';
1111
import { assertPhasePolicy } from '../policy.ts';
12-
export { prepareTaskFilesystems, removeTaskFilesystems } from './storage.ts';
12+
export { prepareTaskFilesystems, removeTaskFilesystems, UnusableRepositoryError } from './storage.ts';
1313
export type { TaskFilesystems, TaskStorageLimits } from './storage.ts';
1414

1515
const validateSecrets = (profile: ContainerProfile, secrets: Readonly<Record<string, string>>) => {

‎agents/container/storage.ts‎

Lines changed: 159 additions & 124 deletions
Large diffs are not rendered by default.

‎agents/container/tree-script.ts‎

Lines changed: 619 additions & 0 deletions
Large diffs are not rendered by default.

‎agents/recovery.ts‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,10 @@ export interface UnownedResource extends RecoveredResource {
2424
readonly reason: 'no-runner-label' | 'unknown-kind' | 'inconsistent-storage';
2525
}
2626
export interface RecoveryReport {
27-
/** Agent containers, egress proxies, seeders, export containers and networks of this runner, now confirmed gone. */
27+
/**
28+
* Agent containers, egress proxies, seeders, export and inspection containers and networks of this runner, now
29+
* confirmed gone.
30+
*/
2831
readonly removed: readonly RecoveredResource[];
2932
/** One handle per task-storage allocation of this runner, kept whole for export and `removeTaskFilesystems`. */
3033
readonly storage: readonly RecoveredTaskStorage[];
@@ -119,8 +122,8 @@ const removeById = async (resource: RecoveredResource, remaining: () => number)
119122

120123
/**
121124
* Crash recovery for one database (#51 item 4). Acts only on objects whose `io.codeboost.runner` label is
122-
* `runnerOwner`: it removes agent containers, egress proxies, seeders, export containers and networks, and resolves
123-
* only once they are gone. It keeps task storage whole (volumes and keeper) and returns a recovery handle per
125+
* `runnerOwner`: it removes agent containers, egress proxies, seeders, export and inspection containers and networks,
126+
* and resolves only once they are gone. It keeps task storage whole (volumes and keeper) and returns a recovery handle per
124127
* allocation. Objects from older builds without a runner label, and anything it does not recognise, are reported and
125128
* never touched.
126129
*
@@ -181,8 +184,9 @@ export async function recoverLeftovers(runnerOwner: string, timeoutMs = 120_000)
181184
for (const resource of owned.container) {
182185
const kind = kindOf(resource);
183186
if (kind === 'storage:keeper') keep(resource, 'keeper');
184-
// Seeders and export containers are transient: a leftover one is removed, never kept with the storage.
185-
else if (kind === 'storage:seeder' || kind === 'storage:export' || kind === 'agent' || kind === 'egress')
187+
// Seeders, export and inspection containers are transient: a leftover one is removed, never kept with the storage.
188+
else if (kind === 'storage:seeder' || kind === 'storage:export' || kind === 'storage:inspect' || kind === 'agent'
189+
|| kind === 'egress')
186190
remove.push(resource);
187191
else unknown(resource);
188192
}

‎docs/implementation/agent-isolation.md‎

Lines changed: 41 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,10 @@ Use only these entry points to run an agent:
6868
if not, it removes what it made and refuses. Cleanup removes an object by the ID captured at its create, even if
6969
its labels are wrong; an object found only by name (including task storage) must carry all three owner labels. Call
7070
`removeTaskFilesystems` when the task ends. It refuses a repository that has a
71-
symbolic link with an absolute target or a target outside the checkout, before it
72-
creates any storage. Report this to the user as a repository the agent cannot run
73-
on; do not retry it.
71+
symbolic link with an absolute target or one that can lead outside the checkout (resolved in the container as its
72+
kernel would, even once the agent creates a missing directory on the way), or any link in its Git metadata; the
73+
seeder checks this and the allocation removes what it made, then throws an `UnusableRepositoryError`. Report
74+
this to the user as a repository the agent cannot run on; do not retry it.
7475
3. `captureInvocation` freezes the request. Capture each attempt ID once. A new
7576
attempt needs a new attempt ID.
7677
4. `startCodexInvocation` or `startClaudeInvocation` returns a handle at once and runs
@@ -80,17 +81,48 @@ Use only these entry points to run an agent:
8081
call. The request carries `networkAllocationId`, a UUID you choose for the vendor network. Pass the vendor
8182
credential only as the function argument.
8283
5. To keep a stopped writable attempt's partial output, call `exportTaskDiff(storage, { base, imageId })` before
83-
`removeTaskFilesystems`. `base` is the full ID of the last commit codeboost made in that storage. It returns at
84-
most 1 MiB of diff (`truncated` says whether it was cut), and takes `signal` and a deadline for the Docker work.
84+
`removeTaskFilesystems`. `base` is the full ID of the commit the storage was seeded from (the clone's head). For a
85+
recovery handle, also pass the `metadataBaseline` you recorded (step 6). It returns at most 1 MiB of diff
86+
(`truncated` says whether it was cut), and takes `signal` and a deadline for the Docker work.
8587
A changed file over 8 MiB, an untracked nested repository, a path Git will not add, an entry named `.git`, a fifo
8688
or socket, an ignored untracked path, a submodule directory with content, and a changed file whose `ident` or
8789
`working-tree-encoding` attribute changes what the diff shows each appear as a `codeboost:` notice line. When Git
8890
fails, the error carries Git's last two error lines.
89-
6. After a crash or restart, call `recoverLeftovers(runnerOwner)` (`agents/recovery.ts`) while holding the
91+
6. To audit what a writable attempt changed (#66, `agents/container/changes.ts`):
92+
- **Record the baseline.** The storage value carries `metadataBaseline`, a digest of the metadata volume that the
93+
seeder takes as its last step. Record it with the allocation. `snapshotDeclaredLinks`, `inspectTaskChanges` and
94+
`exportTaskDiff` each check the metadata against it before running any Git command, and need it back
95+
(`metadataBaseline`) for a recovery handle.
96+
- **Before launch**, call `snapshotDeclaredLinks(storage, paths, { imageId })` with the item's declared paths.
97+
For each declared symlink it records where it resolves, one part at a time as the kernel would, and the state of
98+
the target and everything beneath it. A link on the way, a target that is a link, or a link inside a directory
99+
target shows as `through-link`. Do not launch an item with a `through-link` declared link: a write through it
100+
lands somewhere its target does not cover. Keep the result.
101+
- **After the handle settles**, call `inspectTaskChanges(storage, { base, linkSnapshot, imageId })`. It returns the
102+
change manifest: every difference between the work tree and `base`, read without following links, with content
103+
IDs as a commit would store them. New ignored files, fifos and entries under a `.git` part are listed; a new
104+
directory that `base`'s own ignore rules ignore, with no tracked entry beneath it, is one entry (`ignored: true`). It also returns `agentCommits`,
105+
`metadataChanged`, `linkTargetChanges`, `nestedGitlinkContent` and `digest`. If the metadata changed, no Git
106+
command runs: the manifest has `metadataChanged: true` and every other list empty.
107+
- **Needs human.** The metadata is read-only to agents, so any agent commit or metadata change means a protection
108+
failed. Route it to needs human, as for link target changes and nested gitlink content.
109+
- **Refusals.** It refuses, and never returns part of the answer, when:
110+
- there are more than 10,000 changes (a populated submodule counts as one);
111+
- a name or link target it reports is longer than 1,024 bytes, is not strict UTF-8, or holds a Unicode control,
112+
format, line or paragraph separator, or unassigned character (unchanged names are never checked);
113+
- the recorded targets hold more than 20,000 entries;
114+
- it cannot read something;
115+
- `base` is not a commit in the storage, or Git fails;
116+
- for a snapshot, the metadata changed since seeding (an export refuses then too).
117+
118+
Treat a refusal as needs human.
119+
- **Both calls** run in a read-only container with no network, take `signal`, `onProcessGroup` and `timeoutMs`
120+
(default 120 s), and settle only after their container is gone.
121+
7. After a crash or restart, call `recoverLeftovers(runnerOwner)` (`agents/recovery.ts`) while holding the
90122
database's single-runner lock and before admitting work. It touches only objects labelled with that runner
91-
token. It removes agent containers, egress proxies, seeders, export containers and networks, and resolves once they are gone. It keeps
92-
task storage whole (both volumes and the keeper) and returns one recovery handle per allocation, carrying its
93-
attempt and allocation IDs. `removeTaskFilesystems` accepts a handle as it accepts the value
123+
token. It removes agent containers, egress proxies, seeders, export and inspection containers and networks, and
124+
resolves once they are gone. It keeps task storage whole (both volumes and the keeper) and returns one recovery
125+
handle per allocation, carrying its attempt and allocation IDs. `removeTaskFilesystems` accepts a handle as it accepts the value
94126
`prepareTaskFilesystems` returned. D issues a handle only after checking every part's owner labels. Objects without
95127
a runner label (from older builds), objects of this runner that D does not create, and storage whose parts
96128
disagree are listed in `unowned` and never touched. An object counts as D's only with exactly one kind label,

0 commit comments

Comments
 (0)