Skip to content

Commit 92cb3b9

Browse files
DeusDataclaude
andcommitted
fix(test): bound the cli activation-guard scope fixture's host waits (DeusData#2115)
The four namespace-scope tests fork a real host daemon and, in the parent, waited on it with two UNBOUNDED calls: read() for the child's one-byte readiness signal in cli_scope_fixture_start, and waitpid(host, 0) in cli_scope_fixture_finish. If the forked child stalls before it can signal — a fork-time sanitizer allocator stall is the classic cause, the same hazard the earlier posix_spawn work addressed — the parent blocks forever and the WHOLE cli suite hangs to the CI wall-clock kill (the observed rc=124 at 900 s on the ubuntu-24.04-arm gcc shard, which passed on every other DeusData#2115 shard). Attribution: the production drain path is already bounded (DRAIN 15 s, CONTROL 2 s) and a foreign-namespace install takes the non-blocking scope probe and skips draining entirely, so this is not a production deadlock — it is a test-fixture determinism gap, amplified by ASan on a loaded arm runner. Fix, test-only: - cli_scope_wait_ready(): poll the ready pipe to a generous, bounded deadline (90 s, comfortably past the child's own 45 s cohort-admission deadline plus service start), so a wedged child yields a clean ASSERT_TRUE(ready) failure instead of an unbounded read(). - cli_scope_reap_host(): reap with WNOHANG to a bound (60 s), then SIGKILL and reap, so a wedged child is force-killed rather than hanging finish(); a killed child leaves host_exit == -1, which fails the caller's assertion cleanly. This also prevents a stuck child from holding cohort locks that would poison later tests. - cli_scope_host_serving(): raise the status probe deadline from 5 s to 15 s so a slow-but-alive daemon on a loaded runner is not misread as "drained" (the flaky ASSERT(host_serving) failure this fixture showed); it still fails cleanly when the daemon is genuinely gone or reports stopping. None of these decides a healthy test — the child signals in well under a second on a healthy runner; the bounds only convert an otherwise unbounded hang into a deterministic pass/fail. The scoping contract still binds: with cli_activation_resolve_scope forced to always return ACTIVE, the three "must-not-drain" tests go RED at ASSERT(host_serving) and the host-namespace "must-drain" test stays GREEN. cli suite: 313 passed locally (macOS, twice); 30-round pre-fix and 8-round post-fix stress of the four tests were clean; a fault-injected never-ready child fails all four cleanly in ~37 s with no suite hang. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 013e1f7 commit 92cb3b9

1 file changed

Lines changed: 82 additions & 7 deletions

File tree

‎tests/test_cli.c‎

Lines changed: 82 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
#include <sys/un.h>
4343
#include <sys/wait.h>
4444
#include <poll.h>
45+
#include <signal.h>
4546
#endif
4647
#ifdef __APPLE__
4748
#include <mach-o/dyld.h>
@@ -1510,6 +1511,16 @@ TEST(cli_install_recovers_markerless_stale_rendezvous) {
15101511
* install under test and asks the daemon itself afterwards. */
15111512
#define CLI_SCOPE_HOST_DRAINED 3
15121513
#define CLI_SCOPE_TIMEOUT_MS 5000U
1514+
/* Generous, BOUNDED waits so a wedged host child (a fork-time sanitizer
1515+
* allocator stall is the classic cause — see the posix_spawn fix history)
1516+
* fails the test cleanly instead of hanging the whole suite to the CI
1517+
* wall-clock kill. Each comfortably exceeds the child's own 45 s cohort
1518+
* admission deadline plus a multi-second service start/teardown, so none trips
1519+
* for a merely slow-but-healthy runner; they only convert an otherwise
1520+
* unbounded hang into a deterministic pass/fail. */
1521+
#define CLI_SCOPE_READY_TIMEOUT_MS 90000U
1522+
#define CLI_SCOPE_HOST_REAP_TIMEOUT_MS 60000U
1523+
#define CLI_SCOPE_HOST_SERVING_TIMEOUT_MS 15000U
15131524

15141525
typedef struct {
15151526
char tmpdir[256];
@@ -1598,6 +1609,68 @@ static _Noreturn void cli_scope_host_child(const cli_scope_fixture_t *fixture, i
15981609
_exit(!ready_ok ? 1 : drained ? CLI_SCOPE_HOST_DRAINED : 0);
15991610
}
16001611

1612+
/* Bounded read of the host child's one-byte readiness signal. A child that
1613+
* deadlocks before it can write (a fork-time allocator stall under a sanitizer
1614+
* is the classic cause) must never hang the whole suite on an unbounded read:
1615+
* poll to a generous deadline, then let the caller's ASSERT_TRUE(ready) fail
1616+
* cleanly. Returns the byte, or 0 when the child died, closed the pipe, or
1617+
* never answered in time. */
1618+
static char cli_scope_wait_ready(int fd, uint32_t timeout_ms) {
1619+
uint64_t deadline = cbm_now_ms() + timeout_ms;
1620+
for (;;) {
1621+
int64_t remaining = (int64_t)deadline - (int64_t)cbm_now_ms();
1622+
if (remaining <= 0) {
1623+
return 0;
1624+
}
1625+
struct pollfd pfd = {.fd = fd, .events = POLLIN, .revents = 0};
1626+
int r = poll(&pfd, 1, (int)remaining);
1627+
if (r < 0) {
1628+
if (errno == EINTR) {
1629+
continue;
1630+
}
1631+
return 0;
1632+
}
1633+
if (r == 0) {
1634+
return 0; /* deadline reached with no signal */
1635+
}
1636+
char ready = 0;
1637+
ssize_t got = read(fd, &ready, 1);
1638+
if (got == 1) {
1639+
return ready;
1640+
}
1641+
if (got < 0 && errno == EINTR) {
1642+
continue;
1643+
}
1644+
return 0; /* EOF (child gone) or error */
1645+
}
1646+
}
1647+
1648+
/* Reap the host child within a bound: the release signal makes a healthy child
1649+
* break within ~50 ms and finish teardown in a few seconds, so a child still
1650+
* alive past the deadline is wedged — SIGKILL it and reap so the suite always
1651+
* makes progress. Returns the child's exit code, or -1 when it had to be
1652+
* killed or did not exit cleanly; a -1 fails the caller's host_exit assertion
1653+
* cleanly rather than hanging. */
1654+
static int cli_scope_reap_host(pid_t host, uint32_t timeout_ms) {
1655+
uint64_t deadline = cbm_now_ms() + timeout_ms;
1656+
int status = 0;
1657+
for (;;) {
1658+
pid_t reaped = waitpid(host, &status, WNOHANG);
1659+
if (reaped == host) {
1660+
return WIFEXITED(status) ? WEXITSTATUS(status) : -1;
1661+
}
1662+
if (reaped < 0 && errno != EINTR) {
1663+
return -1;
1664+
}
1665+
if (cbm_now_ms() >= deadline) {
1666+
(void)kill(host, SIGKILL);
1667+
(void)waitpid(host, &status, 0);
1668+
return -1;
1669+
}
1670+
cbm_usleep(2000);
1671+
}
1672+
}
1673+
16011674
static bool cli_scope_fixture_start(cli_scope_fixture_t *fixture, const char *tag) {
16021675
memset(fixture, 0, sizeof(*fixture));
16031676
fixture->host = -1;
@@ -1657,8 +1730,8 @@ static bool cli_scope_fixture_start(cli_scope_fixture_t *fixture, const char *ta
16571730
}
16581731
close(ready_pipe[1]);
16591732
close(release_pipe[0]);
1660-
char ready = 0;
1661-
bool host_ready = child > 0 && read(ready_pipe[0], &ready, 1) == 1 && ready == 'R';
1733+
char ready = child > 0 ? cli_scope_wait_ready(ready_pipe[0], CLI_SCOPE_READY_TIMEOUT_MS) : 0;
1734+
bool host_ready = ready == 'R';
16621735
close(ready_pipe[0]);
16631736
fixture->host = child;
16641737
fixture->release_fd = release_pipe[1];
@@ -1682,9 +1755,14 @@ static bool cli_scope_fixture_start(cli_scope_fixture_t *fixture, const char *ta
16821755
* committed client still admitted. */
16831756
static bool cli_scope_host_serving(const cli_scope_fixture_t *fixture) {
16841757
cbm_daemon_runtime_status_t status = {0};
1758+
/* A generous, bounded status deadline: a foreign-namespace install leaves
1759+
* this daemon serving, so a slow response on a loaded runner must not be
1760+
* misread as "drained" (the flaky failure this fixture showed). The call
1761+
* still fails cleanly — a genuinely drained daemon is unreachable or
1762+
* reports stopping — it just no longer decides survival on a 5 s budget. */
16851763
return fixture->endpoint &&
16861764
cbm_daemon_runtime_request_status(fixture->endpoint, &fixture->identity,
1687-
CLI_SCOPE_TIMEOUT_MS, &status) &&
1765+
CLI_SCOPE_HOST_SERVING_TIMEOUT_MS, &status) &&
16881766
!status.stopping && status.committed_clients == 1;
16891767
}
16901768

@@ -1718,10 +1796,7 @@ static int cli_scope_fixture_finish(cli_scope_fixture_t *fixture) {
17181796
}
17191797
int host_exit = -1;
17201798
if (fixture->host > 0) {
1721-
int status = 0;
1722-
if (waitpid(fixture->host, &status, 0) == fixture->host && WIFEXITED(status)) {
1723-
host_exit = WEXITSTATUS(status);
1724-
}
1799+
host_exit = cli_scope_reap_host(fixture->host, CLI_SCOPE_HOST_REAP_TIMEOUT_MS);
17251800
fixture->host = -1;
17261801
}
17271802
cbm_daemon_ipc_endpoint_free(fixture->endpoint);

0 commit comments

Comments
 (0)