Skip to content

Commit 19fb503

Browse files
Merge pull request #131 from CodeForPhilly/fix/boot-oom-heap-bump
fix(deploy): raise heap + memory limit for full published import
2 parents 888dce6 + 03e0acd commit 19fb503

2 files changed

Lines changed: 20 additions & 4 deletions

File tree

‎deploy/kustomize/base/configmap.yaml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,16 @@ data:
2121
# containers) so the in-memory record set + secondary indices + FTS fit.
2222
# Keep ~25% headroom below the container's memory limit for ephemeral
2323
# work (request handling, gitsheets tree mutations, etc.).
24-
NODE_OPTIONS: "--max-old-space-size=1536"
24+
#
25+
# 2048 (was 1536): a fresh boot rebuilding in-memory state from the full
26+
# `published` import (~31.8k people + ~10.4k tag-assignments + secondary
27+
# indices) OOM'd at the 1536 ceiling. The native FTS5 store is off-heap;
28+
# the V8 heap holds the record maps + indices. 2048 boots cleanly and runs
29+
# stable; it is deliberately kept modest because these nodes are only
30+
# ~3.9Gi — an earlier 3072/3.5Gi trial let the pod grow until it starved
31+
# the node's kubelet (NodeNotReady). See the memory-optimization issue for
32+
# the (suspiciously large ~60x) on-disk-to-heap expansion worth reducing.
33+
NODE_OPTIONS: "--max-old-space-size=2048"
2534
PORT: "3001"
2635
STORAGE_BACKEND: "filesystem"
2736
# The runtime-served branch. `published` is the long-term sandbox + prod

‎deploy/kustomize/base/deployment.yaml‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,12 +64,19 @@ spec:
6464
resources:
6565
requests:
6666
cpu: 100m
67-
memory: 768Mi
67+
memory: 1Gi
6868
limits:
6969
cpu: 1000m
7070
# Holds the full public dataset + secondary indices + FTS in
71-
# memory (~31k people, 268 projects, 10k tag-assignments, …)
72-
memory: 2Gi
71+
# memory (~31.8k people, 268 projects, 10.4k tag-assignments, …).
72+
# 2.5Gi (was 2Gi): a cold boot rebuilding state from the full
73+
# `published` import exceeded the prior 1536Mi heap; raised to
74+
# NODE_OPTIONS=--max-old-space-size=2048 (see configmap.yaml),
75+
# with the container limit just above that for the off-heap FTS5
76+
# store + ephemeral request work. Kept at 2.5Gi (not higher) so
77+
# a single pod can't starve a node — these nodes are only ~3.9Gi
78+
# and a 3.5Gi trial drove one NodeNotReady. ~1.4Gi node headroom.
79+
memory: 2560Mi
7380
securityContext:
7481
runAsNonRoot: true
7582
runAsUser: 1000

0 commit comments

Comments
 (0)