merge-queue.md

Merge-queue lifecycle (#24)

Queue-backed merging is enabled only when the configured GitHub adapter implements the recorded K1 queue-observation contract. The reviewed head remains the authority throughout the lifecycle; browser requests never supply a head, attempt ID, or queue conclusion.

Durable lifecycle

SQLite owns the merge-attempt record and retains its plan revision, snapshot, review version, exact reviewed head, and the pull request it merges (#121). The queue watermark, enqueue and every later queue inspection use that pull request, never the one the gate would choose now (see "Which pull request" in guarded-merge.md). An attempt saved before #121 names none; it queued github.pullRequest, which is read instead, and without that setting a poll reports an observation error and leaves the attempt unchanged. Legal transitions are:

submitting -> queued -> merged
                    -> removed
                    -> failed

An observation may settle submitting after a process restart, because the enqueue command may have completed before the local queued update. Enqueue command success is never recorded as merged. Once submission begins, cancellation, timeout, transport failure, or another ambiguous command outcome leaves the persisted submitting record disabled, retains its diagnostic, and remains recoverable through queue inspection. Direct merges use the same durable ownership: a later pull-request read can reconcile an ambiguous direct command to merged, while any non-merged state remains disabled rather than being treated as proof that retry is safe. Only a confirmed GitHub refusal becomes a retryable failed attempt. Once the external command succeeds, a later local refresh failure likewise does not turn that committed action into a command failure.

Every update compares the current attempt ID and legal source state. A delayed poll for an older attempt therefore cannot overwrite a retry. Immediately before enqueue, the coordinator persists GitHub's opaque connection cursor for the latest queue-timeline event; later inspection paginates forward from that cursor, with a bounded fail-closed page limit. Correlated state requires exactly one post-cursor add sequence and its matching active or terminal shape; multiple add sequences fail closed. An older or later attempt for the same reviewed head therefore cannot settle this attempt, and long-running restart recovery does not depend on the boundary remaining in a recent-event window. Removed and failed attempts retain GitHub's terminal reason. Retry creates a new attempt only when the same revision, snapshot, review version, and reviewed head are still current. Any snapshot replacement or plan amendment after a stored attempt requires every plan item to have an approval bound to the current snapshot and revision. When GitHub explicitly requires fresh review without a locally observed replacement snapshot, those approvals must come from a review generation at or after the attempted generation. A distinct replacement snapshot can satisfy the gate after complete re-review even if its commit SHA was restored to the original value. After those approvals are recorded, the old attempt is historical rather than retryable.

Runtime ownership

The coordinator owns at most one enqueue and one shared queue inspection. Cancellation does not release either operation until its GitHub call settles. One fourteen-second deadline covers every sequential validation, queue-correlation, and submission stage rather than restarting for each remote call. Shutdown rejects every new API request at admission and rechecks irreversible work after partially received request bodies, then gives admitted requests a bounded fourteen-and-a-half-second drain. At the deadline it aborts request-scoped status inspections, destroys requests still blocked on partial bodies, and cancels coordinator work before waiting for server closure and closing SQLite, so an admitted request or queue poll cannot deadlock teardown.

GET /api/merge is the narrow polling path for queue-backed attempts. It reads only the persisted attempt and the K1 queue observation; it does not reload Git history or reconstruct the review. Ambiguous direct submissions remain disabled and reconcile through a full review refresh instead of repeatedly calling the queue endpoint. The browser patches only the merge control and status banner, so current selection, scroll, code attachment, and composer drafts remain unchanged. Queue polling starts at two seconds, doubles while the same active state persists, and caps at thirty seconds; a lifecycle transition or explicit user action resets the interval. A review action advances the merge-poll generation, preventing an older response from re-enabling retry against pre-action review state. One merge poll runs at a time (#48): a render while a poll is in flight starts no second chain, and only the latest poll's answer or error is applied, and only while its generation is current. An older generation's poll that returns late neither applies nor releases the newer generation's poll. The page sets no timeout of its own on a poll; the server bounds /api/merge with its own read deadlines (traced, not tested). The merge click also clears the poll timer, as every generation change does; that is defensive, because a click needs ready, which excludes a running job or an active attempt (traced, not tested). The action stays disabled while submitting or queued and after confirmed merge. Removal or failure exposes the reason but keeps retry disabled until a full review refresh revalidates current GitHub blockers. Retry is then offered only for an unchanged reviewed context, and the full gate is revalidated twice before another enqueue.

Transient or incomplete GitHub observations leave the attempt active, surface an observation error, and continue polling rather than clearing the browser's active attempt. Submitting attempts use unknown-outcome wording until GitHub proves that the attempt is queued. Only a validated queued, merged, removed, or failed observation changes durable state. Queue rules count as the server-side current-base guard; adapters without queue inspection continue to fail closed.

Both pre-action validation reads must agree on whether merge queues apply. Queue mode performs one final fresh validation after capturing its timeline cursor, immediately before durable ownership and gh pr merge. Any mode change aborts before the command; the coordinator never decides whether to create durable queue ownership from an earlier observation.

Stop reasons and answers (#48)

A GitHub read that stops reports the first reason it stopped for, and the merge path and the server's trust read keep a timeout or a gh answer (CommandFailed, gh's own exit) that came before a later abort, as AGENTS.md requires: the original reason is kept and an actionable error is not replaced with cancellation text. Only what the abort itself stopped is answered with the abort (503 before admission, a 409 unknown outcome after it, or the queue poll's 409 shutdown error). Each path is listed below, with its exceptions.

• Gateway reads. A GitHub read that runs out of its own time throws RemoteTimeout (github/timeout.ts): the merge-state inspection, the queue watermark, the queue inspection and the issue read. When the caller aborts later, while gh is still stopping, the read still reports its timeout. A failure gh reported before either abort (its own exit, CommandFailed) is reported as itself, not replaced by the abort, except in the rules read, which treats any failure as unknown rules (traced, not tested). When one inspection stage's gh exit stops the other stages, the inspection waits for every stage to settle and then reports that exit, even when its own deadline fired meanwhile (tested). The already-fixed check's result is handled from the moment it starts, so a check that a failed stage stopped and that settles first is not an unhandled rejection, which Node would raise process-wide (found by the #48 tests; it predated #48; tested); a successful answer that arrives after the abort is still discarded, and so is an error the gateway raised while checking a successful answer (for example a different issue returned; traced, not tested). An inspection stage's own validation error that comes before the abort, while the other stages are still settling, is also answered with the abort: resendable, never a wrong saved answer (traced, not tested). The issue gateway's rule also covers execution's issue text and the issue board's refresh; the board's prioritizer still answers a refresh stopped by shutdown with the shutdown reason (traced, not tested).
• Issue-trust reads. The server wraps a failed trust read as an upstream failure and keeps the timeout as its cause. On the merge path, a trust read that timed out, or that gh answered, before the caller's abort still reports that; the runner actions keep a trust read's own timeout too, even when shutdown started while gh stopped (tested), and answer any other failed trust read as shutting down once shutdown has started (the server is stopping, the runner is closing, or the request was aborted, which only shutdown does). Either way nothing is admitted.
  • The trust read that gates the runner actions (start, resume, approve-continuation, publish, prepare-merge) is not saved as the action's outcome when it times out (tested for start; the other actions share the same branch): the action answers 503, and the same action ID reads trust again; an outcome already saved under that ID is still the answer, also when shutdown started while gh stopped (tested).
  • The trust read inside merge preparation (runner/pre-merge.ts) is different: a timeout or gh answer there settles the preparation as failed, which is saved and replayed, and a shutdown that arrives after it is reported as that timeout or answer (tested: the preparation layer in test/pre-merge.test.ts, with the error shape the server's trust read builds, and that read itself through publishing in test/runner-publishing.test.ts); a cancel takes the same path (traced, not tested). The rebase conflict resolver's trust read (runner/rebase-conflict.ts) is not covered: Rebaser (runner/rebase.ts) reports a cancel, shutdown or expired rebase deadline that is pending when the resolver's error is handled in its place. This is unchanged by #48 and left as a follow-up (#174).
  • A publish whose trust read timed out, or that gh answered, before shutdown or cancellation is recorded as failed with that reason, not as stopped (tested for shutdown, test/runner-publishing.test.ts); cancellation takes the same path, both leave the publish owed, and a cancelled task's PRs are still closed (traced, not tested).
  • Execution's issue read reports its own timeout when a cancel arrives after it; the runner decides the attempt's outcome from its recorded cancel, not from that error (traced, not tested).
  • Out of this change: the issue board's trust toggle still saves a timed-out author read as that action's failure, and planning's issue reads (web/server.ts, web/planning.ts) still answer a timeout as an upstream failure without that cause.
• Before admission. A RemoteTimeout (even wrapped), the merge deadline or shutdown applied nothing. The click answers 503 and may be resent; it is never saved as the click's definite refusal. An outcome another coordinator saved under the same click meanwhile is the answer instead, also when this coordinator's own abort followed the timeout (tested). A replay after the click's abort loads no review, starts or joins no GitHub read, and asks for a refresh (tested). A replay before it reads GitHub fresh under the click's own signal; if that read fails, the saved outcome is answered with a refresh blocker instead of the page status's own blocker (traced, not tested). A gh answer that came before a later abort, and that the read reports as its error, is handled as it would be without the abort: it is saved as the click's refusal. A result built from a gh failure, such as the already-fixed check's unknown blocker, that arrives after the abort is discarded for the abort instead. That holds for the merge deadline (tested through the server's trust refresh just before admission; the first trust read settles within 12.75 s of its start: its own 12 s plus gh's stop wait; it starts after the task and review are loaded, so on a slow load it can still be running when the click's 14 s deadline fires, which takes the same path). At shutdown the server closes the Store's write gate before it aborts the coordinator. A refusal handled during the drain is still saved; one handled after the write gate closed cannot be: the click answers 503 with gh's text, and nothing is saved. Any refusal that cannot be saved now keeps its text in that 503. Clicks are refused once shutdown starts. The drain waits up to 14.5 s, while a click settles only after the step it was in has stopped: about 14.75 s when that is a gh call (its 14 s deadline plus the slowest gh stop wait, the trust read's 0.75 s; the already-fixed check runs on the merge gateway's runner, with 0.4 s), and about 15 s or a little more when it is a review load, whose history worker gets 1 s after the abort before it is terminated (#170; traced, not tested), and the drain ends early when the server has no open connections (a click whose browser went away). When the drain times out, the click's own deadline has already aborted it (clicks are refused once shutdown starts, so its 14 s ends before the 14.5 s drain does); gh's answer, which can surface up to 0.75 s after that abort, is saved if it is handled before the write gate closes and answers 503 otherwise. When the drain ends early, the coordinator's shutdown aborts the click after the write gate closed; the coordinator test that closes writes before shutdown models that path. gh's own 30-second timeout is a plain error, but every read before admission has a shorter RemoteTimeout (traced, not tested).
  • The already-fixed check reports its timeout as a blocker instead (alreadyFixed: 'unknown'), and a blocker is saved and replayed like a refusal: it says the merge is not ready on the state read for this click, which a new click reads again. Treating it as resendable is a possible follow-up.
  • A rules read cannot report a timeout that way: the inspection's own deadline fires first. A gh failure inside the inspection (for example the PR read or a published target's PR list) that came before its deadline is reported as itself and saved like a refusal with gh's text (traced, not tested). The already-fixed check turns its own gh failure into an alreadyFixed: 'unknown' blocker, also saved; its own timer stops it before the inspection's deadline. When a caller's abort comes after that failure, while the check is still settling, the blocker arrives after the abort and is discarded like any other answer: the inspection reports the abort, which may be resent (tested). After an abort, every other error passes through the check, including the abort's own reason. When that reason is a failed stage's exit, the inspection reports the failed stage's own rejection, the same exit (tested); for the caller's reason or an error that is not gh's, it answers with the abort (traced, not tested).
• gh's exit. Once gh's exit has been observed, its exit status is the answer (github/run-with-input.ts). A deadline or shutdown that fires while its output pipes drain does not turn GitHub's refusal, or its success, into an unknown outcome. The rule holds for every gh call: a publish whose gh command had already failed when it was cancelled or stopped is recorded as failed or refused with gh's message rather than as stopped, and still stays owed; one whose gh command had already succeeded goes on, and its next external step (a gh call or a git push) stops at once on the aborted signal (traced, not tested).
• After admission. GitHub's refusal is the click's answer even when the coordinator's deadline or shutdown came later, including a refusal whose failed state could not be saved (then answered as an unknown outcome with GitHub's text). Any other gh answer is reported with its text as an unknown outcome; a failure the abort itself caused reports the abort, as an unknown outcome.
• Queue reads. A queue read that timed out, or that gh answered, before shutdown answers with that as an observation error (200), not with the shutdown error (409). A queue read that fails answers from the attempt on record, which may have become queued or been replaced by a retry while GitHub was read; the observation error is attached only to the attempt that was read. Reading that attempt is a store read; if it fails, the poll fails (503) instead of answering with the observation error.
• A merge click's answer. It takes its readiness from the attempt on record, so after a submission it does not offer Merge again. Its GitHub readiness was read before the command, so it never offers an action from it: if a queue poll ended the attempt while the server reloaded the review, the answer is not ready and asks for a refresh (tested), and a retry is offered only after a fresh GitHub read (traced, not tested). A coordinator built without a store (only tests do this; the demo server has no merge coordinator) has no attempt on record, so its answer asks for a refresh the same way (traced, not tested).

The status cache in GhMergeGateway is cleared before and after every merge command, successful or refused, and a read that started before the merge cannot fill it afterwards. Admission always reads GitHub fresh; only the read-only page status uses the cache. Publishing (push, ready, close) does not clear it, so the page may show the status from before publishing for up to five seconds; no merge decision uses that status. The clearing and the refused stale fill are tested; that admission always reads fresh and the publishing staleness were traced in the #48 investigation, not tested.

Acceptance evidence

Controlled unit and browser regressions cover enqueue success, delayed merge, queue removal, unmergeable failure, head replacement, retry, stale observation publication, active-attempt retry refusal, shutdown settlement, durable restart recovery, and preservation of UI input while polling. Final evidence is recorded against the exact pushed PR head.

#48 regressions: test/merge-stop-reasons.test.ts (stop reasons and resendable timeouts; a saved success answered after the click's own abort with no further load or GitHub read; output past the limit while the pipes drain still fails the command; an outcome saved meanwhile answered after a timeout and this click's own abort; gh's answer kept when shutdown finds writes closed; one inspection stage's gh exit reported after its stopped siblings settle, including past the deadline, with no unhandled rejection; a caller abort answered in place of the already-fixed check's own earlier gh failure), test/merge-queue-poll.test.ts (a failed, timed-out or gh-answered queue read answers from the current attempt; GitHub's refusal survives a later shutdown, saved or not; the click's answer is inactive, also when a poll ends the attempt before the answer is built; an issue-trust timeout answers 503, alone or before the merge deadline, and a gh answer before the deadline is saved and replayed), test/merge-status-cache.test.ts (cache evidence; these passed before the fixes), test/pre-merge.test.ts and test/runner-publishing.test.ts (a preparation's or a publish's trust read that timed out or that gh answered before shutdown is saved or recorded as failed with that answer, not as the shutdown), and test/runner-start.test.ts (a runner action's timed-out trust read is not saved and answers 503, also when shutdown started while gh stopped; a saved outcome still wins). The #48 browser regressions (test/browser/issue-48-poll-ordering.spec.ts, test/browser/issue-48-stale-errors.spec.ts) cover one poll chain under a re-render, a held error, an older generation's late poll, and an unusable current answer; a re-render during a merge-job poll takes the same guard (traced, not tested).