Skip to content

Commit 879e62a

Browse files
mchwangclaude
andcommitted
Create and start the egress proxy separately; document Ask's host recovery
Review fixes (Copilot, #69): - `docker run --detach` mixed two outcomes: a refused create (for example, a name held by another container) made nothing, while a failed start left our container. Treating both as "created" reported and inspected a foreign same-named container. The proxy is now created, then started: a refused create is absence, a failed start leaves a container owned by its ID. - Ask's handling of D's host staging directories was questioned. They are recovered through the Ask root (the worker's TMPDIR, recorded durably before setup and deleted at startup before Ask is enabled); say so at the point where Ask marks unreleased results. - Refresh the isolation doc's `unreleased` and retired-profile rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent b161e52 commit 879e62a

4 files changed

Lines changed: 50 additions & 18 deletions

File tree

‎agents/network/network.ts‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -200,17 +200,15 @@ export function createVendorNetwork(invocation: InvocationInput, imageId: string
200200
return value;
201201
};
202202
// Run one create step; a client killed by its deadline leaves the daemon outcome for `object` unknown.
203-
// `answeredFailureMayCreate`: a command such as `docker run --detach` that can create the object and then exit
204-
// nonzero (the start failed), so even a daemon-answered failure leaves the object possibly present.
205-
const create = (object: string, args: readonly string[], answeredFailureMayCreate = false) => {
203+
// Each create is a pure create, so a daemon-answered failure (such as a name held by someone else) made nothing.
204+
const create = (object: string, args: readonly string[]) => {
206205
const timeout = remaining();
207206
try {
208207
const output = docker(args, timeout);
209208
created.add(object);
210209
return output;
211210
} catch (error) {
212211
if (typeof (error as { status?: unknown }).status !== 'number') unsettled.add(object);
213-
else if (answeredFailureMayCreate) created.add(object);
214212
throw error;
215213
}
216214
};
@@ -251,11 +249,13 @@ export function createVendorNetwork(invocation: InvocationInput, imageId: string
251249
networkId = createdId(create(name, ['network', 'create', '--internal', '--driver', 'bridge', '--subnet', subnet,
252250
'--label', `io.codeboost.egress=${allocationId}`, name]), 'network');
253251
proxyPlanned = true;
254-
proxyId = createdId(create(proxyContainer, ['run', '--detach', '--name', proxyContainer, '--read-only', '--user', '10001:10001',
252+
// Create and start separately: a refused create made nothing, while a failed start leaves a container we own by ID.
253+
proxyId = createdId(create(proxyContainer, ['create', '--name', proxyContainer, '--read-only', '--user', '10001:10001',
255254
'--cap-drop=ALL', '--security-opt=no-new-privileges', '--security-opt=seccomp=builtin', '--runtime=runc', '--pids-limit=64', '--memory=64m', '--memory-swap=64m',
256255
'--cpus=.25', '--network', name, '--network-alias', 'codeboost-proxy',
257256
'--label', `io.codeboost.egress=${allocationId}`, '--env', `CODEBOOST_ALLOWED_HOSTS=${VENDOR_HOSTS[vendor].join(',')}`,
258-
'--entrypoint', 'node', imageId, '/usr/local/lib/codeboost-egress-proxy.mjs'], true), 'proxy');
257+
'--entrypoint', 'node', imageId, '/usr/local/lib/codeboost-egress-proxy.mjs']), 'proxy');
258+
docker(['start', proxyId], remaining());
259259
docker(['network', 'connect', 'bridge', proxyId], remaining());
260260
docker(['exec', proxyId, 'node', '-e', [
261261
"const net=require('node:net');let attempts=0;",

‎docs/implementation/agent-isolation.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -87,9 +87,12 @@ The caller must do the following:
8787
the resources it could not confirm removed in `unreleased`; the container may still be running.
8888
- When `unreleased` is present, record those resources durably and keep them owned until their removal is
8989
confirmed. Each Docker entry has its creation-time ID (when known) and its ownership label; remove one only
90-
if both still match. The agent container is listed only if this invocation's `docker create` ran, so a
91-
same-named container that belongs to another invocation is never reported.
92-
- A profile that settled with `unreleased` cannot be started again.
90+
if both still match. An object is listed only if this invocation created it or may have (its create succeeded,
91+
or its client was killed before the daemon answered). A create the daemon refused, for example because another
92+
invocation holds the name, made nothing, so that name is never reported or touched. The egress proxy is created
93+
and started as two steps for this reason. Directory entries are host paths under the caller's `TMPDIR`.
94+
- A profile that settled with `unreleased` is retired: every launch path (`startProfileInvocation`,
95+
`createValidatedContainer`, `startValidatedContainer`, `runContainer`) refuses it, while its cleanup still runs.
9396
- Call `cancel` to stop an invocation. The first stop reason is kept.
9497
- Treat `stopReason` as the result of the invocation. A missing `stopReason` means
9598
the agent finished normally.

‎runner/question-container.ts‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,10 @@ export async function askInContainer(question: ContainerQuestion, deps: Containe
222222
finally { signal.removeEventListener('abort', cancel); }
223223
// D gave up on cleanup: these resources are no longer owned by anything in this process. Fail closed so the
224224
// worker reports them at release and no new question starts until a restart finds none left.
225-
// Presence, not length, is the signal: an empty list still means D stopped before cleanup was confirmed.
225+
// Presence, not length, is the signal: an empty list still means D stopped before cleanup was confirmed. Docker
226+
// leftovers are found again by their labels. Host leftovers (D's input and auth staging directories) are covered by
227+
// the Ask root: the worker's TMPDIR is that root, D stages under tmpdir(), and the root is recorded durably before
228+
// setup and deleted at startup before Ask is enabled again.
226229
if (result.unreleased !== undefined) retained.markUntracked();
227230
return answerFromResult(question.provider, result, invocation);
228231
} finally {

‎test/agent-network.test.ts‎

Lines changed: 34 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -101,26 +101,52 @@ describe('vendor-only egress', () => {
101101
it('removes a proxy that was created even though its start failed', () => {
102102
const shim = mkdtempSync(join(tmpdir(), 'docker-shim-')), recorded = join(shim, 'proxy');
103103
const realDocker = execFileSync('sh', ['-c', 'command -v docker'], { encoding: 'utf8' }).trim();
104-
// `docker run --detach` creates the proxy, then the start fails: the client exits nonzero with a daemon answer.
104+
// The proxy container is created, then its start fails: a container this allocation owns by ID.
105105
writeFileSync(join(shim, 'docker'), ['#!/bin/sh',
106-
`if [ "$1" = run ] && [ "$2" = --detach ]; then shift 2; printf %s "$2" > '${recorded}'; `
107-
+ `'${realDocker}' create "$@" >/dev/null; echo 'Error response from daemon: failed to start' >&2; exit 125; fi`,
106+
`if [ "$1" = start ] && [ "$2" != --attach ]; then printf %s "$2" > '${recorded}'; `
107+
+ `echo 'Error response from daemon: failed to start' >&2; exit 1; fi`,
108108
`exec '${realDocker}' "$@"`].join('\n'), { mode: 0o755 });
109109
const failing = captureInvocation({ ...invocation, attemptId: `proxy-start-${randomUUID()}`,
110110
deadline: Date.now() + 60_000 });
111111
const path = process.env.PATH;
112112
process.env.PATH = `${shim}:${path}`;
113-
let proxy = '';
113+
let proxyId = '';
114114
try {
115115
expect(() => createVendorNetwork(failing, imageId)).toThrow('failed to start');
116-
proxy = readFileSync(recorded, 'utf8');
116+
proxyId = readFileSync(recorded, 'utf8');
117117
} finally { process.env.PATH = path; rmSync(shim, { recursive: true, force: true }); }
118-
const leaked = spawnSync('docker', ['container', 'inspect', proxy], { stdio: 'ignore' }).status === 0;
119-
if (leaked) spawnSync('docker', ['rm', '--force', proxy], { stdio: 'ignore' });
120-
expect(proxy).toMatch(/^codeboost-proxy-/);
118+
const leaked = spawnSync('docker', ['container', 'inspect', proxyId], { stdio: 'ignore' }).status === 0;
119+
if (leaked) spawnSync('docker', ['rm', '--force', proxyId], { stdio: 'ignore' });
120+
expect(proxyId).toMatch(/^[0-9a-f]{64}$/);
121121
expect(leaked).toBe(false);
122122
}, 60_000);
123123

124+
it('neither reports nor removes a same-named proxy when the daemon refuses the create', () => {
125+
const shim = mkdtempSync(join(tmpdir(), 'docker-shim-')), recorded = join(shim, 'foreign');
126+
const realDocker = execFileSync('sh', ['-c', 'command -v docker'], { encoding: 'utf8' }).trim();
127+
// Someone else's container takes the proxy name just before our create, so the daemon refuses it.
128+
writeFileSync(join(shim, 'docker'), ['#!/bin/sh',
129+
`if [ "$1" = create ] && [ "$2" = --name ] && case "$3" in codeboost-proxy-*) true;; *) false;; esac; then `
130+
+ `'${realDocker}' create --name "$3" --entrypoint true ${imageId} >/dev/null; printf %s "$3" > '${recorded}'; fi`,
131+
`exec '${realDocker}' "$@"`].join('\n'), { mode: 0o755 });
132+
const refused = captureInvocation({ ...invocation, attemptId: `refused-proxy-${randomUUID()}`,
133+
deadline: Date.now() + 60_000 });
134+
const path = process.env.PATH;
135+
process.env.PATH = `${shim}:${path}`;
136+
let foreign = '', error: unknown;
137+
try {
138+
try { createVendorNetwork(refused, imageId); } catch (caught) { error = caught; }
139+
foreign = readFileSync(recorded, 'utf8');
140+
} finally { process.env.PATH = path; rmSync(shim, { recursive: true, force: true }); }
141+
const survived = spawnSync('docker', ['container', 'inspect', foreign], { stdio: 'ignore' }).status === 0;
142+
spawnSync('docker', ['rm', '--force', foreign], { stdio: 'ignore' });
143+
// The refused create is reported as itself: our network was removed, nothing is left to retry or report.
144+
expect(error).toBeInstanceOf(Error);
145+
expect(error).not.toBeInstanceOf(VendorNetworkCreationCleanupError);
146+
expect(String(error)).toMatch(/already in use|Conflict/);
147+
expect(survived).toBe(true);
148+
}, 60_000);
149+
124150
it('does not delete a same-named stand-in when setup fails after the proxy exists', () => {
125151
const shim = mkdtempSync(join(tmpdir(), 'docker-shim-')), recorded = join(shim, 'impostor');
126152
const realDocker = execFileSync('sh', ['-c', 'command -v docker'], { encoding: 'utf8' }).trim();

0 commit comments

Comments
 (0)