Skip to content

Commit d02b496

Browse files
committed
[act.deploy.sigma.D7+] three processes with PPID 1 and nobody watching them is not a deployment
Ivan asked whether any of this is deployed. It was not. The unit files had been sitting in deploy/systemd/ since they were written, with "NOT INSTALLED BY THIS COMMIT" in their own headers, while all three services ran as hand-started foreground processes reparented to PID 1 — no supervision, no restart, no start-at-boot. The only symptom of any of them dying is "backend offline" in the UI, which is exactly the sentence that cost a Mac-side debugging session. NOW ACTUALLY RUNNING, under systemctl --user with linger ON so they come up at boot without a login: dirac-fields.service :8901 enabled + active dirac-web.service :1338 enabled + active dirac-physics.service :8902 installed, deliberately NOT started SUPERVISION PROVEN, NOT ASSUMED — the whole value of a unit file is the restart, and a restart nobody has watched is a claim: kill -9 on the fields MainPID (948941) → new pid 949241 within 8 s, /health 200. Linger checked separately (loginctl ... -p Linger = yes), because enable+active still gives you nothing after a reboot without it, and that gap is invisible until the reboot. THE UNIT WAS SPLIT, and the port conflict is why the file's own argument finally won. dirac-backend.service supervised both python daemons under one `wait -n` wrapper and said in its header that two units were almost certainly right. Two concrete costs, not one: a physics crash would restart the fields daemon and throw away its in-memory SCF cache (minutes of CPU for an Fe-heme wavefunction); and TODAY the physics daemon on :8902 is hand-run by another session, so the merged unit would fail to bind, exit the wrapper, and restart-loop BOTH — taking down a working fields daemon in order to fail at starting a physics one. Merged file moved to _archive/ rather than deleted: it carries the measured memory ceiling derivation (pyscf plans around 4000 MB per SCF; idle RSS ~160 MB fields / ~1.06 GB physics) that both new units inherit. deploy/README.md corrected everywhere, not just at the top — the table now names files that exist, and the `journalctl -u dirac-backend` line is fixed with the trap written next to it: journalctl against a nonexistent unit prints nothing and exits 0, so silence there is "no such unit", never "no errors". A correction that reaches only the summary leaves the copy-pasteable commands lying.
1 parent f9dabd6 commit d02b496

4 files changed

Lines changed: 121 additions & 11 deletions

File tree

‎deploy/README.md‎

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,23 @@ three): PPID = 1 for all three processes.
1111
This directory does not fix that by itself — it documents the fix
1212
(`deploy/systemd/*.service`) and gives you a day-to-day tool for the same
1313
three services (`bin/dev`) plus a safe way to reclaim stale compute
14-
(`bin/dirac-sweep`). **The systemd units are not installed by this commit.**
14+
(`bin/dirac-sweep`).
15+
16+
> **STATUS 2026-08-11 — INSTALLED AND RUNNING.** `dirac-fields.service` (:8901)
17+
> and `dirac-web.service` (:1338) are enabled under `systemctl --user`, with
18+
> linger ON, so both start at boot without a login. Supervision was PROVEN, not
19+
> assumed: `kill -9` on the fields daemon's MainPID and it returned with a new
20+
> pid inside 8 s, `/health` 200. `dirac-physics.service` (:8902) is installed but
21+
> NOT started — that port is held by a hand-run daemon belonging to another
22+
> session, and a second one cannot bind it. Hand it over with
23+
> `systemctl --user enable --now dirac-physics.service` once the manual process
24+
> is stopped.
25+
>
26+
> `dirac-backend.service` was SPLIT into `dirac-fields` + `dirac-physics` (the
27+
> merged file is in `_archive/`). Its own header had already argued for the
28+
> split; the port conflict above is what made it concrete, because a merged unit
29+
> restart-loops BOTH daemons when one port is taken — taking down a working
30+
> fields daemon in order to fail at starting a physics one.
1531
Installing them is a deliberate step — see below.
1632

1733
## What's here
@@ -20,7 +36,9 @@ Installing them is a deliberate step — see below.
2036
|---|---|
2137
| `bin/dev` | up / down / status / build / logs for fields, physics, web. Never touches a daemon it didn't start itself — another session's daemon survives `bin/dev down`. |
2238
| `bin/dirac-sweep` | the admin DELETE boundary for stale `app.field_cube` rows + orphaned `app.blob` bytes. CLI, not HTTP — see "why a CLI" below. |
23-
| `deploy/systemd/dirac-backend.service` | supervises the fields (:8901) + physics (:8902) python daemons. |
39+
| `deploy/systemd/dirac-fields.service` | supervises the fields daemon (:8901). INSTALLED + RUNNING. |
40+
| `deploy/systemd/dirac-physics.service` | supervises the physics daemon (:8902). Installed, NOT started — port held by a hand-run process. |
41+
| `deploy/systemd/_archive/dirac-backend.service.superseded` | the merged both-daemons unit these two replaced. Kept, not deleted: its header holds the measured memory-ceiling derivation both inherit. |
2442
| `deploy/systemd/dirac-web.service` | supervises the static web server (:1338). |
2543

2644
## Verified-live facts (re-check before trusting an older doc)
@@ -135,10 +153,11 @@ existing `dirac-sync.timer` in this same directory), not system units — no
135153

136154
```bash
137155
# from the repo root
138-
systemctl --user link "$PWD/deploy/systemd/dirac-backend.service"
156+
systemctl --user link "$PWD/deploy/systemd/dirac-fields.service"
157+
systemctl --user link "$PWD/deploy/systemd/dirac-physics.service" # :8902 — see STATUS above before starting
139158
systemctl --user link "$PWD/deploy/systemd/dirac-web.service"
140159
systemctl --user daemon-reload
141-
systemctl --user enable --now dirac-backend.service
160+
systemctl --user enable --now dirac-fields.service
142161
systemctl --user enable --now dirac-web.service
143162
```
144163

@@ -150,25 +169,29 @@ unit starts isn't fighting an existing one for the port.
150169

151170
Check it worked:
152171
```bash
153-
systemctl --user status dirac-backend.service dirac-web.service
154-
journalctl --user -u dirac-backend.service -f
172+
systemctl --user status dirac-fields.service dirac-web.service
173+
# and the one nobody checks until after a reboot:
174+
loginctl show-user "$USER" -p Linger --value # must be "yes"
175+
journalctl --user -u dirac-fields.service -f
176+
# NB: a journalctl -u against a unit name that does not exist prints NOTHING
177+
# and exits 0 — silence there is not "no errors", it is "no such unit".
155178
```
156179

157180
### Uninstalling
158181

159182
```bash
160-
systemctl --user disable --now dirac-backend.service dirac-web.service
183+
systemctl --user disable --now dirac-fields.service dirac-physics.service dirac-web.service
161184
systemctl --user daemon-reload
162185
# the link target lives in this repo; nothing to delete outside it.
163186
# to fully remove the symlinks systemd creates in ~/.config/systemd/user/:
164-
systemctl --user link --now 2>/dev/null; rm -f ~/.config/systemd/user/dirac-backend.service ~/.config/systemd/user/dirac-web.service
187+
systemctl --user link --now 2>/dev/null; rm -f ~/.config/systemd/user/dirac-fields.service ~/.config/systemd/user/dirac-physics.service ~/.config/systemd/user/dirac-web.service
165188
systemctl --user daemon-reload
166189
```
167190

168191
### Why the memory numbers are what they are
169192

170193
Full derivation is in the comment block inside
171-
`deploy/systemd/dirac-backend.service` (it's the load-bearing copy — this is
194+
`deploy/systemd/dirac-fields.service` (it's the load-bearing copy — this is
172195
a summary, not a second source of truth). Headline: `MemoryHigh=6G`,
173196
`MemoryMax=8G`. Measured inputs:
174197

@@ -197,7 +220,9 @@ a summary, not a second source of truth). Headline: `MemoryHigh=6G`,
197220

198221
### Known limitation, stated rather than hidden
199222

200-
`dirac-backend.service` runs both python daemons under one unit via a
223+
RESOLVED 2026-08-11 — split into two units; kept because the reasoning is
224+
the reason the split happened. `dirac-backend.service` ran both python
225+
daemons under one unit via a
201226
`wait -n`-based bash wrapper (see the unit file's own top comment for the
202227
full reasoning) because this task granted exactly one backend unit path.
203228
Consequence: if physics crashes, fields restarts too (and vice versa) — a
@@ -206,7 +231,8 @@ worse blast radius than the two independent units
206231
implies would be correct. If that shared-fate restart becomes a real
207232
problem, the fix is a second unit file
208233
(`deploy/systemd/dirac-physics.service`, splitting the current
209-
`dirac-backend.service` into fields-only), which is a small change but is
234+
`dirac-backend.service` into fields-only) — DONE, see STATUS at the top —
235+
which was a small change but is
210236
explicitly out of scope for what was authorized here.
211237

212238
## `dirac-sync.timer` — read this before you rely on `main` being what you think it is
File renamed without changes.
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# dirac-fields.service — Dirac fields daemon — classical + quantum cube fields (:8901)
2+
#
3+
# SPLIT FROM dirac-backend.service, which supervised BOTH python daemons under
4+
# one wrapper. That file said so itself: "the correct systemd shape is almost
5+
# certainly two independent units, so a crash in one does not restart the
6+
# other", and it only existed in the merged form because the task that wrote it
7+
# was granted a single unit path. Two reasons the split is now worth the two
8+
# files:
9+
#
10+
# ① SHARED FATE. `wait -n` means a physics crash restarts the fields daemon,
11+
# throwing away its in-memory SCF cache (a converged Fe-heme wavefunction is
12+
# minutes of CPU) for a fault in a different process.
13+
# ② PORT OWNERSHIP, which is the concrete reason today: the physics daemon on
14+
# :8902 is run by hand from another session right now. A merged unit would
15+
# try to bind a port already in use, the wrapper would exit, and systemd
16+
# would restart-loop BOTH — taking down a working fields daemon to fail at
17+
# starting a physics one. Separate units let :8901 come under supervision
18+
# immediately and :8902 be handed over whenever its owner is ready.
19+
20+
[Unit]
21+
Description=Dirac fields daemon — classical + quantum cube fields (:8901)
22+
After=postgresql.service network.target
23+
StartLimitIntervalSec=300
24+
StartLimitBurst=5
25+
26+
[Service]
27+
Type=simple
28+
WorkingDirectory=/home/ivan/dirac
29+
ExecStart=/home/ivan/dirac/backend/env/bin/python backend/field_server.py
30+
Restart=on-failure
31+
RestartSec=5
32+
33+
# Ceilings carried over unchanged from dirac-backend.service, where they were
34+
# derived from measurement: pyscf plans around a 4000 MB max_memory per SCF
35+
# object, and idle RSS was measured at ~160 MB (fields) and ~1.06 GB (physics).
36+
# PER-UNIT now rather than shared, which is strictly safer: one runaway SCF can
37+
# no longer consume the other daemon's headroom before either gets throttled.
38+
MemoryHigh=6G
39+
MemoryMax=8G
40+
41+
[Install]
42+
WantedBy=default.target
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# dirac-physics.service — Dirac physics daemon — sigma-hole surface MEP + torsion (:8902)
2+
#
3+
# SPLIT FROM dirac-backend.service, which supervised BOTH python daemons under
4+
# one wrapper. That file said so itself: "the correct systemd shape is almost
5+
# certainly two independent units, so a crash in one does not restart the
6+
# other", and it only existed in the merged form because the task that wrote it
7+
# was granted a single unit path. Two reasons the split is now worth the two
8+
# files:
9+
#
10+
# ① SHARED FATE. `wait -n` means a physics crash restarts the fields daemon,
11+
# throwing away its in-memory SCF cache (a converged Fe-heme wavefunction is
12+
# minutes of CPU) for a fault in a different process.
13+
# ② PORT OWNERSHIP, which is the concrete reason today: the physics daemon on
14+
# :8902 is run by hand from another session right now. A merged unit would
15+
# try to bind a port already in use, the wrapper would exit, and systemd
16+
# would restart-loop BOTH — taking down a working fields daemon to fail at
17+
# starting a physics one. Separate units let :8901 come under supervision
18+
# immediately and :8902 be handed over whenever its owner is ready.
19+
20+
[Unit]
21+
Description=Dirac physics daemon — sigma-hole surface MEP + torsion (:8902)
22+
After=postgresql.service network.target
23+
StartLimitIntervalSec=300
24+
StartLimitBurst=5
25+
26+
[Service]
27+
Type=simple
28+
WorkingDirectory=/home/ivan/dirac
29+
ExecStart=/home/ivan/dirac/backend/env/bin/python backend/physics/server.py
30+
Restart=on-failure
31+
RestartSec=5
32+
33+
# Ceilings carried over unchanged from dirac-backend.service, where they were
34+
# derived from measurement: pyscf plans around a 4000 MB max_memory per SCF
35+
# object, and idle RSS was measured at ~160 MB (fields) and ~1.06 GB (physics).
36+
# PER-UNIT now rather than shared, which is strictly safer: one runaway SCF can
37+
# no longer consume the other daemon's headroom before either gets throttled.
38+
MemoryHigh=6G
39+
MemoryMax=8G
40+
41+
[Install]
42+
WantedBy=default.target

0 commit comments

Comments
 (0)