Skip to content

Commit 82661d6

Browse files
mchwangclaude
andcommitted
Merge F6a 7ffd084 (shutdown push guard) into F6b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 parents fbde7d4 + 7ffd084 commit 82661d6

4 files changed

Lines changed: 33 additions & 8 deletions

File tree

‎docs/implementation/pre-merge-rebase.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,12 @@ This ordering is intentionally local-only. The push below extends the durable li
3939
| Step | What happens |
4040
|---|---|
4141
| Admission | The task state version is captured, and issue trust is refreshed. `Store.beginPrePush` then records `{ attemptId, branch, from, to, snapshotId, startedAt }` in `tasks.push_in_progress` (schema v18) and advances the task state version. It requires that captured state version (a reassignment during the trust read refuses), a mergeable status, no cancel, no active attempt, merge, rebase or push, `to` equal to the snapshot head, and `from` a durable rewrite predecessor of `to`. |
42-
| Last local check | Run by the pusher as its synchronous finalizer, as Git starts: trust is revalidated and the task state version, review version and snapshot must be unchanged since admission. After a push, the same three must still be exactly what the push left before any `cmd:` check runs, so a plan revision or review edit made while the push was in flight fails the preparation. The same check runs before and after the new head is fetched for review (after a refusal, a `moved` outcome, or a pull request that moved after the push): a change made during the push, the wait or the fetch fails the preparation as stale instead of being reviewed. |
42+
| Last local check | Run by the pusher as its synchronous finalizer, as Git starts: a shutdown already begun refuses the push (the runner's admission flag, set in shutdown's first step, before the HTTP drain), trust is revalidated and the task state version, review version and snapshot must be unchanged since admission. After a push, the same three must still be exactly what the push left before any `cmd:` check runs, so a plan revision or review edit made while the push was in flight fails the preparation. The same check runs before and after the new head is fetched for review (after a refusal, a `moved` outcome, or a pull request that moved after the push): a change made during the push, the wait or the fetch fails the preparation as stale instead of being reviewed. |
4343
| Push | `GitBranchPusher` pushes `to` to the task PR's `codeboost/` branch with a lease pinned to exactly `from`, the PR head read before the rebase. Any other value at the tip refuses with nothing pushed, even a commit codeboost made. The branch is read back after the push. |
4444
| Refused | Nothing was pushed. The marker clears. Once GitHub's PR API reports the branch's new head (the wait below), that head is reviewed and preparation returns `review-required`. |
4545
| Visible on the PR | After a push or a refusal, GitHub's pull-request API reports the branch's new head asynchronously. Preparation inspects the PR with backoff until it reports a head other than the pre-push head. It starts no inspection after 30 seconds, so the wait lasts at most 30 seconds plus one inspection (a 6-second deadline in production, plus up to 6 seconds for a stopped `gh` call to settle), within the preparation deadline. A base change alone does not end the wait. The pushed head continues to the `cmd:` checks; any other head or base is reviewed and returns `review-required`; a PR still at the pre-push head fails the preparation without touching the review. |
4646
| Unknown outcome | A failure, a timeout or a cancel during the push reads the branch once, unless the preparation's own deadline ended the push (see the next row). The read and Git's settlement after an abort together fit within 30 seconds and what is left of the preparation's deadline: the read is aborted the 16-second process-settlement reserve early, and it does not start when less than that reserve is left. A process that left Git's group and still holds its output does not hold the read either: the read changes nothing, so after the 1-second pipe drain it returns failed rather than waiting, and the marker stays. A process that stays in Git's group after Git exits is waited for, as for every Git call: it cannot be signalled safely once Git has exited, because its group ID may be reused, so that one case is not bounded. Its answer: `to` means pushed, `from` means not pushed yet, anything else means moved. The marker then clears. After `moved`, the preparation reviews that head as after a refusal: it waits for the pull request to show it, refreshes attribution and approvals, and returns `review-required`. After `pushed` or `not pushed`, the push's error fails the preparation. Only shutdown aborts that read; a cancel requested during the push lets it finish and closes the task when the marker clears. |
47-
| Unreadable, deadline or shutdown | The marker stays when the branch cannot be read, when the preparation's own deadline ended the push (less than the settlement reserve is left, so no read starts), or on shutdown. Merge admission, runner admission, status changes, reassignment, another rebase and another push all refuse while it exists, and merge status names it. Publishing cannot overlap it either: it runs only from `running` or `needs human`, and a push requires a mergeable status. After a restart, a tracked job started under the verified runner lock reads the branch and clears it; a cancel requested while no preparation runs starts the same job, and repeating that cancel starts it again if its read failed; preparation requests are refused as already running until it ends, and a task it closes as cancelled has its PRs closed. Otherwise the next preparation reads the branch, clears the marker, and asks for a reload. |
47+
| Unreadable, deadline or shutdown | The marker stays when the branch cannot be read, when the preparation's own deadline ended the push (less than the settlement reserve is left, so no read starts), or on shutdown (from shutdown's first step: the push's own failure path then starts no read). Merge admission, runner admission, status changes, reassignment, another rebase and another push all refuse while it exists, and merge status names it. Publishing cannot overlap it either: it runs only from `running` or `needs human`, and a push requires a mergeable status. After a restart, a tracked job started under the verified runner lock reads the branch and clears it; a cancel requested while no preparation runs starts the same job, and repeating that cancel starts it again if its read failed; preparation requests are refused as already running until it ends, and a task it closes as cancelled has its PRs closed. Otherwise the next preparation reads the branch, clears the marker, and asks for a reload. |
4848

4949
A push that was killed (by a cancel, a deadline or a crash) after GitHub received it may still land after the settlement read saw `from`. Its lease still names `from`, so it overwrites nothing newer, and a later preparation reviews the branch as it finds it.
5050

‎runner/pre-merge.ts‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -322,7 +322,10 @@ export class PreMergeCoordinator {
322322
const owned = (cleared: { before: number; after: number }) => cleared.before === binding ? cleared.after : binding;
323323
try {
324324
await this.remote.push({ branch: marker.branch, from: marker.from, to: marker.to, beforePush: () => {
325-
// The last local checks before the external write: trust, and no task or review change since the claim.
325+
// The last local checks before the external write: shutdown, trust, and no task or review change since the
326+
// claim. Shutdown sets the runner's admission flag first and closes this coordinator only after the HTTP drain,
327+
// so between the two only the runner's flag shows that shutdown began.
328+
if (this.runner.closing) throw new GuardRefusal('The server is shutting down.');
326329
authorization?.validate();
327330
const task = this.service.store.getTask(identity);
328331
if (task.stateVersion !== binding || task.cancelRequested !== null
@@ -333,14 +336,14 @@ export class PreMergeCoordinator {
333336
} }, signal);
334337
} catch (error) {
335338
if (error instanceof BranchPushRefused) return { result: 'refused', stateVersion: owned(this.#finishPush(marker)) };
336-
// Shutdown: no new GitHub read may start; startup settles the marker. Otherwise one read, within what is left of
337-
// the preparation's deadline with Git's settlement reserve kept inside it (except for a process that stays in
339+
// Shutdown (from its first step, when the runner stops admission): this failure path starts no settlement read;
340+
// startup settles the marker. Otherwise one read, within what is left of the preparation's deadline with Git's settlement reserve kept inside it (except for a process that stays in
338341
// Git's group after Git exits, which is waited for; see #settlePush), settles it; a cancel does not stop that
339342
// read. When the preparation's own deadline ended the push, less than that reserve is left, so no read starts and
340343
// the marker stays, as for an unreadable branch.
341344
let settled = binding, outcome: PushOutcome | null = null;
342345
const left = Math.floor(deadline - performance.now());
343-
if (!this.#closing && left >= 1) {
346+
if (!this.#closing && !this.runner.closing && left >= 1) {
344347
try { const read = await this.#settlePush(marker, left); settled = owned(read); outcome = read.outcome; }
345348
catch { /* The marker stays; the next preparation or startup reads the branch again. */ }
346349
}

‎test/pre-merge.test.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1476,6 +1476,24 @@ it('keeps waiting while GitHub still reports the pre-push head, even if the base
14761476
await fixture.coordinator.close(); await fixture.runner.close();
14771477
});
14781478

1479+
it('pushes nothing when shutdown begins while the push reads the remote, before this coordinator is closed', async () => {
1480+
let closingAtBoundary: boolean | null = null;
1481+
const fixture = await rebaseFixture('feature\n', undefined, { duringPushRead(_service, coordinator) {
1482+
// Server shutdown step 1 stops runner admission; the coordinator's own close waits for the HTTP drain (step 2).
1483+
coordinator.runner.rejectAdmission();
1484+
closingAtBoundary = coordinator.runner.closing;
1485+
// The coordinator itself is still open: a new preparation would be refused only as already running.
1486+
expect(() => coordinator.assertStartable()).toThrow(/already running/);
1487+
} });
1488+
expect(closingAtBoundary).toBe(true);
1489+
expect(fixture.result).toMatchObject({ state: 'failed', reason: 'The server is shutting down.' });
1490+
expect(fixture.remote().head).not.toBe(fixture.rebased);
1491+
// Shutdown starts no new GitHub read: the marker waits for startup.
1492+
expect(fixture.branchReads()).toBe(0);
1493+
expect(pushMarker(fixture)).toMatchObject({ to: fixture.rebased });
1494+
await fixture.coordinator.close(); await fixture.runner.close();
1495+
});
1496+
14791497
it('pushes nothing when a cancel is committed while the push reads the remote, and closes the task once the outcome is read', async () => {
14801498
const fixture = await rebaseFixture('feature\n', undefined, { duringPushRead(service, coordinator) {
14811499
// The production route: the coordinator cancels the task and aborts this preparation after the cancel commits.

‎test/runner-branch-push.test.ts‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -333,8 +333,10 @@ describe('GitBranchPusher', () => {
333333
mkdirSync(bin);
334334
// Git's place on PATH: other commands succeed at once with no output; `cat-file` leaves a process in a new session
335335
// holding its output.
336+
// Each cat-file call leaves its own process, so every PID is appended and every one is killed afterwards.
336337
const escape = `const c = require('node:child_process').spawn('sleep', ['60'], { detached: true, stdio: ['ignore', 'inherit', 'inherit'] });`
337-
+ `require('node:fs').writeFileSync(${JSON.stringify(pidFile)}, String(c.pid)); c.unref();`;
338+
+ `require('node:fs').appendFileSync(${JSON.stringify(pidFile)}, c.pid + '\\n'); c.unref();`;
339+
const escaped = () => { try { return readFileSync(pidFile, 'utf8').split('\n').filter(Boolean).map(Number); } catch { return []; } };
338340
writeFileSync(join(bin, 'git'), `#!/bin/sh\ncase " $* " in *" cat-file "*) exec ${JSON.stringify(process.execPath)} -e ${JSON.stringify(escape)} ;; esac\nexit 0\n`, { mode: 0o755 });
339341
try {
340342
const began = performance.now();
@@ -343,8 +345,10 @@ describe('GitBranchPusher', () => {
343345
// A push checks its head with cat-file before anything is written: that read is bounded the same way.
344346
await expect(instance.push(IDENTITY, { head: s.base, branch: BRANCH })).rejects.toThrow(/outside its group still holds its output/);
345347
expect(performance.now() - began).toBeLessThan(30_000);
348+
// One holder per cat-file call: both are recorded, so the cleanup below reaches both.
349+
expect(escaped()).toHaveLength(2);
346350
} finally {
347-
try { process.kill(Number(readFileSync(pidFile, 'utf8')), 'SIGKILL'); } catch { /* already gone */ }
351+
for (const pid of escaped()) try { process.kill(pid, 'SIGKILL'); } catch { /* already gone */ }
348352
}
349353
}, 40_000);
350354

0 commit comments

Comments
 (0)