Skip to content

Commit e577e43

Browse files
committed
fix(daemon): re-issue the interrupted request against a replacement daemon
When the shared daemon dies under a live MCP frontend (#2796), the worker's daemon exchange fails and the current failure path calls _Exit(EXIT_FAILURE) — "a failed daemon transport cannot wake a thread blocked in stdio portably". The agent then sees server EOF and every pooled connection to the same daemon breaks at once. On Windows this happens whenever a client reaps its spawned endpoint tree: DETACHED_PROCESS detaches the console, not the process tree, so the shared daemon dies with the endpoint that spawned it. Keep the reader untouched and recover on the worker instead: when cbm_daemon_application_client_mcp_tagged reports a transport failure and the caller supplied a replacement supplier (new daemon_lost callback on mcp_run), swap in the replacement client, re-mint the request token (tokens are scoped to the client that reserved them), and re-issue the very request whose exchange was interrupted. Only transport errors retry: BUSY, UNAVAILABLE, REJECTED, HANDLER_ERROR and CANCELLED are daemon answers that keep their existing meaning, so a cancellation is still answered as a cancellation. Bounded by FRONTEND_DAEMON_RECONNECT_MAX; without a callback or once the budget is spent the old immediate-exit path stays. A replayed request may already have executed in the dead daemon, so the interrupted call has at-least-once semantics; frontend.h states this explicitly. The replaced client stays owned by the frontend and is only close_begin'd at swap time: the stdin reader and the maintenance monitor can still hold a pointer read before the swap, so close_finish runs in the run tail after the worker is joined and the monitor is stopped. All non-worker client reads are snapshotted under the state mutex. A supplier that returns the current or an already-retired client is rejected instead of being double-freed. Cancellation survives the reconnect: the published token is invalidated while the callback runs, a matching cancel is remembered in cancel_pending, and the retry delivers it against the freshly minted token. The retry reserves and publishes that token under one lock, and CANCEL_ERROR no longer tears down the reader — the in-flight exchange observes the same failure and the worker owns recovery. A maintenance stop that lands in the reconnect window now takes the monitor's bounded grace instead of exiting immediately. The EOF drain derives its deadline each iteration from the last completed item, grants a bounded FRONTEND_EOF_RECONNECT_GRACE_MS grace per reconnect attempt (tracked by an atomic attempt counter, so a sub-poll failure still resets it), gives the retried exchange a fresh stuck window, and re-samples the reconnect flag before stopping. main() supplies the callback: it re-runs the connect-or-spawn leg plus session context and returns the replacement. It drops its global handle before the run so ownership follows the code, logs the reconnect through cbm_log_warn, and reports failures on stderr only — main_report_client_bootstrap_failure() would inject an unsolicited id:null JSON-RPC frame into the live session's stdout. The POSIX spawn child prefers close_range/closefrom and keeps only async-signal-safe calls, because the callback runs on a worker thread while the reader and maintenance monitor are live. UI flags are not re-applied; the daemon persists its UI configuration. Verified on WSL/gcc 13.3: scripts/test.sh --suites daemon_frontend reports 19 passed, including a transport-failure reconnect, a two-replacement retry (exercising the budget loop and repeated retirement), a CANCELLED-no-retry case that fails when the retry gate is widened, a NULL supplier that fail-stops with a marker proving the callback ran, and a supplier returning the dead client that is rejected without a double close. A real daemon kill -9 mid-session is served by a re-spawned daemon with one daemon_lost_reconnect warning; pre-fix the same scenario exits 1 and the request is lost. Fixes #2796 Signed-off-by: Minervaowl7 <127482476+Minervaowl7@users.noreply.github.com>
1 parent 10ab613 commit e577e43

5 files changed

Lines changed: 755 additions & 52 deletions

File tree

‎src/daemon/bootstrap.c‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,9 @@
3333
#include <sys/types.h>
3434
#include <sys/wait.h>
3535
#include <unistd.h>
36+
#ifdef __linux__
37+
#include <sys/syscall.h>
38+
#endif
3639
#ifdef __APPLE__
3740
#include <pthread.h>
3841
#include <spawn.h>
@@ -1302,11 +1305,21 @@ static bool bootstrap_production_spawn(void *context,
13021305
}
13031306
#else
13041307
static void bootstrap_child_close_fds(void) {
1305-
long open_max = sysconf(_SC_OPEN_MAX);
1306-
if (open_max < 0 || open_max > 1048576L) {
1307-
open_max = 65536L;
1308+
/* async-signal-safe: this runs in a forked child of a multi-threaded
1309+
* process (the MCP frontend reconnects from its worker thread), so no
1310+
* libc allocation or lock may be touched. Prefer the kernel's one-shot
1311+
* close when it exists (raw syscall); the bounded loop is only a
1312+
* fallback. */
1313+
#if defined(__linux__) && defined(SYS_close_range)
1314+
if (syscall(SYS_close_range, 3U, ~0U, 0U) == 0) {
1315+
return;
13081316
}
1309-
for (int fd = 3; fd < open_max; fd++) {
1317+
#elif defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__) || defined(__DragonFly__)
1318+
closefrom(3);
1319+
return;
1320+
#endif
1321+
enum { BOOTSTRAP_CHILD_FD_CAP = 65536 };
1322+
for (int fd = 3; fd < BOOTSTRAP_CHILD_FD_CAP; fd++) {
13101323
(void)close(fd);
13111324
}
13121325
}

0 commit comments

Comments
 (0)