Description
On macOS, in session-user installation mode, the agent can permanently stop running after an upgrade cycle, with no application error log, no crash report, and no automatic restart. The service simply disappears from launchd and never comes back until the user manually reinstalls it.
Environment
- OS: macos
- Agent version at time of repro: 2.3.5
- Install dir:
~/.local/openaev-agent-session
- launchd label:
io.filigran.openaev-agent-session (LaunchAgent, gui/<uid> domain)
Reproducible steps
Steps to create the smallest reproducible scenario:
- install a brand new session agent on a mac
ps aux | grep openaev-agent should list it as running
- wait for 32 secs and
ps aux | grep openaev-agent again
- Agent is now down and will never come back
Expected output
Upgrade should continue and restart the agent
Actual output
Agent is now down and will never come back
Additional information
Root cause
The upgrade job is executed like any other agent job: the agent spawns the upgrade script as a child process via Command::new("bash").spawn() (src/process/agent_exec.rs), with stdout/stderr redirected to /dev/null. This means the upgrade script (and everything it spawns, e.g. curl) inherits the same process group as the agent itself, which is the process supervised by launchd under the LaunchAgent label.
The generated upgrade script (agent-upgrade-session-user.sh, materialized at runtime as runtimes/execution-<job-id>/execution.sh) ends with:
echo "03. Starting agent service"
launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/io.filigran.${session_name}.plist || (echo "Fail restarting io.filigran.${session_name}" >&2 && exit 1)
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/io.filigran.${session_name}.plist
Per launchd.plist(5):
AbandonProcessGroup: If false, the job's process group is sent a SIGTERM followed by a SIGKILL when the job is stopped/removed. If true, the process group is left alone. Default is false.
The generated .plist does not set AbandonProcessGroup, so it defaults to false. As a consequence, when the upgrade script calls launchctl bootout on its own job label, launchd tears down the entire process group — including the very script that issued the bootout command, before it can reach the launchctl bootstrap line right after it.
Result:
- The service is cleanly removed from launchd (
launchd: service inactive: ... / removing service: ... visible in the unified log).
- The
bootstrap call is never reached → the agent never comes back.
- The script's stdout/stderr were already redirected to
/dev/null by the parent agent process, and the kill is an external SIGTERM/SIGKILL (not an application error), so nothing is logged anywhere — no entry in openaev-agent.log, no crash report in DiagnosticReports, empty runner.log (StandardOutPath/StandardErrorPath).
Suggested fixes (any of these would address the root cause)
- Set
AbandonProcessGroup to true in the generated LaunchAgent/LaunchDaemon .plist templates, so that bootout/bootstrap (or any job-triggered stop) does not tear down descendant processes such as the upgrade script.
- Fully detach the upgrade script from the agent's process group before it reaches the
bootout step (e.g. setsid/double-fork/nohup at the very start of the generated script), so it survives the parent job being stopped.
- Avoid the stop/start round-trip entirely: replace
launchctl bootout + launchctl bootstrap with launchctl kickstart -k gui/$(id -u)/io.filigran.${session_name}, which restarts the job in place without ever removing it from launchd's registry, closing the window where the service can vanish and never come back.
Any of the above should also apply to the equivalent LaunchDaemon (system-wide, sudo) upgrade script if it uses the same stop/start pattern (agent-upgrade-daemon.sh or similar) — that variant wasn't repro'd here but shares the same structural risk if it also self-boots-out from within its own process group.
Diagnostic done the following way:
- Agent running normally,
openaev-agent.log shows a healthy startup and job polling:
{"timestamp":"2026-07-09T13:58:11.691077Z","level":"INFO","fields":{"message":"Starting OpenAEV agent 2.3.5 (production)", ...}}
{"timestamp":"2026-07-09T13:58:11.831818Z","level":"INFO","fields":{"message":"Starting ping thread", ...}}
{"timestamp":"2026-07-09T13:58:11.953229Z","level":"INFO","fields":{"message":"Starting listening jobs thread", ...}}
{"timestamp":"2026-07-09T13:58:12.040431Z","level":"INFO","fields":{"message":"Starting cleanup thread", ...}}
- ~32s later (next job poll cycle), an upgrade job is received and dispatched:
{"timestamp":"2026-07-09T13:58:43.438991Z","level":"INFO","fields":{"message":"Start handling inject: None", ...}}
{"timestamp":"2026-07-09T13:58:43.439029Z","level":"INFO","fields":{"message":"Cleaning job: \"30a742f7-2248-427a-9146-655a621c8891\"", ...}}
{"timestamp":"2026-07-09T13:58:43.572804Z","level":"INFO","fields":{"message":"Invoking execution", ...}}
{"timestamp":"2026-07-09T13:58:43.574544Z","level":"INFO","fields":{"message":"Revoking execution", ...}}
{"timestamp":"2026-07-09T13:58:43.574628Z","level":"INFO","fields":{"message":"Done handling inject: None", ...}}
No further log line ever appears — the log file ends here. "Invoking"→"Revoking" happens in ~1.7ms, consistent with a fire-and-forget spawn (the agent does not wait for the child script to finish before marking the job as done).
log show/log stream (unified log) at the corresponding timestamp shows launchd tearing down the job:
launchd: [gui/501 [100023]:] service inactive: io.filigran.openaev-agent-session
launchd: [gui/501 [100023]:] removing service: io.filigran.openaev-agent-session
No subsequent launchd line ever shows the service being re-registered/bootstrapped.
launchctl print gui/<uid>/io.filigran.openaev-agent-session afterwards returns:
Bad request.
Could not find service "io.filigran.openaev-agent-session" in domain for user gui: 501
confirming the job was fully removed from launchd and never came back.
- The actual generated
execution.sh used at repro time (paths/tenant/token redacted where relevant):
#!/bin/sh
set -e
base_url=http://localhost:3001
architecture=$(uname -m)
install_dir="/Users/yann/.local/openaev-agent-session"
session_name="openaev-agent-session"
tenant_id="b50f11c2-30f8-4055-b181-31df7740996a"
os=$(uname | tr '[:upper:]' '[:lower:]')
if [ "${os}" = "darwin" ]; then
os="macos"
fi
echo "Starting upgrade script for ${os} | ${architecture}"
openaev_dir=$(printf %s "${install_dir}" | sed 's/openbas/openaev/g')
if [ -d "$openaev_dir" ]; then
echo "01. Downloading OpenAEV Agent into ${install_dir}..."
curl -sSfL ${base_url}/api/tenants/${tenant_id}/agent/executable/openaev/${os}/${architecture} -o ${install_dir}/openaev-agent_upgrade
mv ${install_dir}/openaev-agent_upgrade ${install_dir}/openaev-agent
chmod +x ${install_dir}/openaev-agent
echo "02. Updating OpenAEV configuration file"
cat > ${install_dir}/openaev-agent-config.toml <<EOF
...
EOF
echo "03. Starting agent service"
launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/io.filigran.${session_name}.plist || (echo "Fail restarting io.filigran.${session_name}" >&2 && exit 1)
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/io.filigran.${session_name}.plist
else
...
fi
Description
On macOS, in
session-userinstallation mode, the agent can permanently stop running after an upgrade cycle, with no application error log, no crash report, and no automatic restart. The service simply disappears fromlaunchdand never comes back until the user manually reinstalls it.Environment
~/.local/openaev-agent-sessionio.filigran.openaev-agent-session(LaunchAgent,gui/<uid>domain)Reproducible steps
Steps to create the smallest reproducible scenario:
ps aux | grep openaev-agentshould list it as runningps aux | grep openaev-agentagainExpected output
Upgrade should continue and restart the agent
Actual output
Agent is now down and will never come back
Additional information
Root cause
The upgrade job is executed like any other agent job: the agent spawns the upgrade script as a child process via
Command::new("bash").spawn()(src/process/agent_exec.rs), with stdout/stderr redirected to/dev/null. This means the upgrade script (and everything it spawns, e.g.curl) inherits the same process group as the agent itself, which is the process supervised bylaunchdunder the LaunchAgent label.The generated upgrade script (
agent-upgrade-session-user.sh, materialized at runtime asruntimes/execution-<job-id>/execution.sh) ends with:Per
launchd.plist(5):The generated
.plistdoes not setAbandonProcessGroup, so it defaults tofalse. As a consequence, when the upgrade script callslaunchctl bootouton its own job label, launchd tears down the entire process group — including the very script that issued thebootoutcommand, before it can reach thelaunchctl bootstrapline right after it.Result:
launchd: service inactive: .../removing service: ...visible in the unified log).bootstrapcall is never reached → the agent never comes back./dev/nullby the parent agent process, and the kill is an external SIGTERM/SIGKILL (not an application error), so nothing is logged anywhere — no entry inopenaev-agent.log, no crash report inDiagnosticReports, emptyrunner.log(StandardOutPath/StandardErrorPath).Suggested fixes (any of these would address the root cause)
AbandonProcessGrouptotruein the generated LaunchAgent/LaunchDaemon.plisttemplates, so thatbootout/bootstrap(or any job-triggered stop) does not tear down descendant processes such as the upgrade script.bootoutstep (e.g.setsid/double-fork/nohupat the very start of the generated script), so it survives the parent job being stopped.launchctl bootout+launchctl bootstrapwithlaunchctl kickstart -k gui/$(id -u)/io.filigran.${session_name}, which restarts the job in place without ever removing it from launchd's registry, closing the window where the service can vanish and never come back.Any of the above should also apply to the equivalent LaunchDaemon (system-wide, sudo) upgrade script if it uses the same stop/start pattern (
agent-upgrade-daemon.shor similar) — that variant wasn't repro'd here but shares the same structural risk if it also self-boots-out from within its own process group.Diagnostic done the following way:
openaev-agent.logshows a healthy startup and job polling:{"timestamp":"2026-07-09T13:58:11.691077Z","level":"INFO","fields":{"message":"Starting OpenAEV agent 2.3.5 (production)", ...}} {"timestamp":"2026-07-09T13:58:11.831818Z","level":"INFO","fields":{"message":"Starting ping thread", ...}} {"timestamp":"2026-07-09T13:58:11.953229Z","level":"INFO","fields":{"message":"Starting listening jobs thread", ...}} {"timestamp":"2026-07-09T13:58:12.040431Z","level":"INFO","fields":{"message":"Starting cleanup thread", ...}}{"timestamp":"2026-07-09T13:58:43.438991Z","level":"INFO","fields":{"message":"Start handling inject: None", ...}} {"timestamp":"2026-07-09T13:58:43.439029Z","level":"INFO","fields":{"message":"Cleaning job: \"30a742f7-2248-427a-9146-655a621c8891\"", ...}} {"timestamp":"2026-07-09T13:58:43.572804Z","level":"INFO","fields":{"message":"Invoking execution", ...}} {"timestamp":"2026-07-09T13:58:43.574544Z","level":"INFO","fields":{"message":"Revoking execution", ...}} {"timestamp":"2026-07-09T13:58:43.574628Z","level":"INFO","fields":{"message":"Done handling inject: None", ...}}No further log line ever appears — the log file ends here. "Invoking"→"Revoking" happens in ~1.7ms, consistent with a fire-and-forget spawn (the agent does not wait for the child script to finish before marking the job as done).
log show/log stream(unified log) at the corresponding timestamp shows launchd tearing down the job:No subsequent
launchdline ever shows the service being re-registered/bootstrapped.launchctl print gui/<uid>/io.filigran.openaev-agent-sessionafterwards returns:confirming the job was fully removed from launchd and never came back.
execution.shused at repro time (paths/tenant/token redacted where relevant):