Skip to content

Commit 463f71d

Browse files
chore(plans): mark boot-oom-sequential-load done (PR #134)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent aad035b commit 463f71d

1 file changed

Lines changed: 28 additions & 6 deletions

File tree

‎plans/boot-oom-sequential-load.md‎

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
2-
status: in-progress
2+
status: done
33
depends: []
44
specs: []
55
issues:
66
- 132
7-
pr:
7+
pr: 134
88
---
99

1010
# Plan: fix boot OOM — load sheets sequentially, not concurrently
@@ -38,10 +38,12 @@ returned state shape/order.
3838

3939
## Validation
4040

41-
- [ ] Live pod boots and reaches `/api/health/ready` 200 on the standard node
42-
size (heap measured ~0.48 GB retained via temporary boot instrumentation).
43-
- [ ] `npm run -w apps/api type-check && npm run lint` clean; loader/store tests pass.
44-
- [ ] Deployed `:sandbox` image boots without OOM.
41+
- [x] Live pod boots and reaches `/api/health/ready` 200 on the standard node
42+
size, 0 restarts, ~0.48 GB retained (per-phase boot instrumentation).
43+
- [x] `npm run -w apps/api type-check && npm run lint` clean; loader/store/memory
44+
tests pass (12/12).
45+
- [x] Deployed `:sandbox` image boots without OOM on the normal config
46+
(heap 2048 / limit 2560Mi).
4547

4648
## Risks
4749

@@ -50,4 +52,24 @@ returned state shape/order.
5052

5153
## Notes
5254

55+
- **The real bug was concurrency, not memory size or data volume.** Per-phase
56+
heap logging in the live pod showed reconcile + push-daemon + private-store
57+
load all sat at ~60 MB; the entire balloon was the concurrent `Promise.all`
58+
read in `loadInMemoryState`. Retained state is ~0.48 GB.
59+
- The diagnosis ruled out several earlier hypotheses: public people count (the
60+
prune didn't change the crash), the private store (11 MB), and reconcile/
61+
push-daemon git-object caching (flat at boot). The gitsheets read path *is*
62+
memory-heavy per read, but the OOM was specifically the eleven reads peaking
63+
together.
64+
- **Sibling changes from the same investigation:** #131 (heap 1536→2048,
65+
limit 2Gi→2.5Gi) was a mitigation, now redundant but harmless headroom — left
66+
as-is. #133 (spam prune) shipped to production `published` (31,832→18,203
67+
people); independently worthwhile but not the OOM fix.
68+
5369
## Follow-ups
70+
71+
- **Tracked as #132** — broader heap-footprint work (the gitsheets per-read
72+
cost and ~2× retention of raw TOML + parsed records). Optional now that boot
73+
fits comfortably, but the lever if the dataset grows.
74+
- **Deferred (ops):** the throwaway `ghcr.io/codeforphilly/codeforphilly-ng:heapdiag`
75+
image tag was pushed during diagnosis; harmless but can be deleted from GHCR.

0 commit comments

Comments
 (0)