Skip to content

Commit aad035b

Browse files
fix(api): load sheets sequentially at boot to avoid OOM spike
A cold boot OOM'd (FATAL: Reached heap limit) building in-memory state from the full `published` import. Per-phase heap instrumentation in the live pod traced it to `loadInMemoryState` reading all eleven sheets via Promise.all: every sheet's transient read/decompress/parse buffers peaked simultaneously, blowing past a 1.5 GB heap — even though the retained state is only ~0.5 GB. Read the sheets sequentially instead. Same records, same order, same retained footprint; the peak is bounded to the single largest sheet. Boot is not latency-sensitive, so the serialization costs nothing in practice. Verified: the live pod boots to /api/health/ready in ~0.48 GB on the standard node size. Tracked against #132 (heap footprint). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ef79e0c commit aad035b

1 file changed

Lines changed: 20 additions & 27 deletions

File tree

‎apps/api/src/store/memory/loader.ts‎

Lines changed: 20 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -24,33 +24,26 @@ import {
2424
export async function loadInMemoryState(publicStore: PublicStore): Promise<InMemoryState> {
2525
const state = createEmptyState();
2626

27-
const [
28-
projects,
29-
people,
30-
tags,
31-
tagAssignments,
32-
memberships,
33-
updates,
34-
buzzes,
35-
blogPosts,
36-
roles,
37-
interests,
38-
slugHistoryRecords,
39-
] = await Promise.all([
40-
publicStore.projects.queryAll(),
41-
publicStore.people.queryAll(),
42-
publicStore.tags.queryAll(),
43-
publicStore['tag-assignments'].queryAll(),
44-
publicStore['project-memberships'].queryAll(),
45-
publicStore['project-updates'].queryAll(),
46-
publicStore['project-buzz'].queryAll(),
47-
// Blog-posts may be absent on data repos that haven't merged the sheet
48-
// config PR yet — queryAll returns [] in that case, which is fine.
49-
publicStore['blog-posts'].queryAll(),
50-
publicStore['help-wanted-roles'].queryAll(),
51-
publicStore['help-wanted-interest'].queryAll(),
52-
publicStore['slug-history'].queryAll(),
53-
]);
27+
// Read each sheet sequentially, NOT via Promise.all. Running all eleven
28+
// queryAll() reads concurrently makes every sheet's transient
29+
// read/decompress/parse buffers peak at the same instant; with the full
30+
// import that combined spike exceeded a 1.5 GB heap on the runtime nodes and
31+
// OOM'd the boot — even though the *retained* state is only ~0.5 GB.
32+
// Sequential reads bound the peak to the single largest sheet. Boot speed is
33+
// not latency-sensitive, so the serialization is free in practice.
34+
const projects = await publicStore.projects.queryAll();
35+
const people = await publicStore.people.queryAll();
36+
const tags = await publicStore.tags.queryAll();
37+
const tagAssignments = await publicStore['tag-assignments'].queryAll();
38+
const memberships = await publicStore['project-memberships'].queryAll();
39+
const updates = await publicStore['project-updates'].queryAll();
40+
const buzzes = await publicStore['project-buzz'].queryAll();
41+
// Blog-posts may be absent on data repos that haven't merged the sheet
42+
// config PR yet — queryAll returns [] in that case, which is fine.
43+
const blogPosts = await publicStore['blog-posts'].queryAll();
44+
const roles = await publicStore['help-wanted-roles'].queryAll();
45+
const interests = await publicStore['help-wanted-interest'].queryAll();
46+
const slugHistoryRecords = await publicStore['slug-history'].queryAll();
5447

5548
for (const p of projects) indexProject(state, p);
5649
for (const p of people) indexPerson(state, p);

0 commit comments

Comments
 (0)