fix(responses): bound the durable spill directory with an aggregate byte cap

src/responses/state.ts

@@ -35,6 +35,30 @@ const SNAPSHOT_DEBOUNCE_MAX_MS = 30_000;
  * continuation chains) stores the full expanded input each turn — ~quadratic bytes per chain —
  * so a count cap alone cannot bound memory. Oldest-first eviction applies past this mark. */
 export const MAX_STORED_RESPONSE_BYTES = 64 * 1024 * 1024;
+/**
+ * Aggregate ceiling for the durable spill directory: the disk-side counterpart to
+ * the RAM ceiling above. Without it the spilled set is bounded only per-file
+ * (MAX_RESPONSE_SPILL_PAYLOAD_BYTES, 256 MiB) and per-entry (MAX_STORED_RESPONSES,
+ * 1000), whose product is 250 GiB — larger than the disk of any host this runs on.
+ * The only effective bound was therefore RESPONSE_TTL_MS, which makes disk use a
+ * function of client request rate rather than of anything this process controls.
+ *
+ * Measured on one macOS host, 2026-08-30: a client spilling ~150 MB payloads at
+ * ~1.4/min held 6.8 GB after 44 minutes, still climbing toward the ~12 GB an
+ * hour-long window implies, and filled the volume. Retention itself was correct
+ * throughout — the TTL evicted that whole cohort an hour later — so what was
+ * missing is a budget, not a sweep.
+ *
+ * 1 GiB comes from the same sample (n=31), whose spilled sizes are strongly
+ * bimodal: median 1.1 MiB against a p90 of 198.7 MiB, near the per-file ceiling.
+ * At that median the count cap and this ceiling bind within 8% of each other
+ * (1000 x 1.1 MiB = 1.07 GiB), so ordinary traffic sees no eviction it would not
+ * already have seen and only the large tail is cut. Erring small is the safe
+ * direction: too low costs a replay miss, an already-handled path surfaced as
+ * previous_response_not_found, while too high costs the host's disk and every
+ * unrelated process on it.
+ */
+export const MAX_SPILLED_RESPONSE_BYTES = 1024 * 1024 * 1024;
 /** Legacy snapshot selection only. Spill demotion is governed solely by the RAM cap above. */
 const SNAPSHOT_ENTRY_MAX_BYTES = 2 * 1024 * 1024;
 const SNAPSHOT_TOTAL_MAX_BYTES = 24 * 1024 * 1024;
@@ -603,6 +627,41 @@ export function getStoredResponseBytesForTests(): number {
   return storedResponseBytes;
 }
 
+let spillByteCapOverride: number | null = null;
+
+function spillByteCap(): number {
+  return spillByteCapOverride ?? MAX_SPILLED_RESPONSE_BYTES;
+}
+
+/**
+ * Live total of durable spill payloads. Recomputed per call rather than carried as
+ * a running counter: spilled entries reach `states` through several insertion paths
+ * (demotion swap, direct oversized admission, snapshot reload), and one missed
+ * increment there would silently disable the cap, where an O(MAX_STORED_RESPONSES)
+ * walk cannot drift.
+ */
+function spilledResponseBytes(): number {
+  let total = 0;
+  for (const entry of states.values()) {
+    if (entry.kind === "spill") total += entry.spill.payloadBytes;
+  }
+  // Superseded generations awaiting a durable snapshot are still files on disk.
+  // Counting only `states` would let PENDING_SPILL_UNLINKS_MAX of them sit outside
+  // the budget while it reports itself satisfied.
+  for (const ref of pendingSpillUnlinks) total += ref.payloadBytes;
+  return total;
+}
+
+/** Test-only: lower/restore the durable spill cap (null restores the default). */
+export function setSpilledResponseByteCapForTests(bytes: number | null): void {
+  spillByteCapOverride = bytes;
+}
+
+/** Test-only: current durable spill accounting (proves evictions unlink their files). */
+export function getSpilledResponseBytesForTests(): number {
+  return spilledResponseBytes();
+}
+
 function serializedBytes(value: unknown): number | null {
   try {
     const serialized = JSON.stringify(value);
@@ -1495,6 +1554,58 @@ export function replayOverlapSkipsForTests(): number {
   return replayOverlapSkips;
 }
 
+/**
+ * Bring the durable spill set inside MAX_SPILLED_RESPONSE_BYTES, and report the
+ * bytes released.
+ *
+ * One owner, three callers: mutation pruning, the lazy load that follows a
+ * restart, and the periodic sweep. The periodic caller is not redundant — the
+ * mutation path only runs when traffic arrives, and a process can come up over
+ * budget from a snapshot written under a larger ceiling and then sit idle. That
+ * was observed in production at 1.8 GiB against a 1 GiB cap, held until the first
+ * request.
+ *
+ * NOT covered here: spill files orphaned by a crash. They are absent from
+ * `states`, so this function can neither see nor price them, and they stay with
+ * recoverOrphanedResponseSpills and its RESPONSE_SPILL_ORPHAN_GRACE_MS window.
+ * This ceiling therefore bounds what the store owns, which is every file it can
+ * account for, and not the directory as a whole.
+ */
+function enforceSpilledResponseBudget(): number {
+  let spilledBytes = spilledResponseBytes();
+  if (spilledBytes <= spillByteCap()) return 0;
+  const before = spilledBytes;
+  // Deferred generations go first. They are already superseded, so releasing one
+  // costs only the crash window the queue exists to cover — the same trade
+  // PENDING_SPILL_UNLINKS_MAX already makes against unbounded disk. Evicting a
+  // live continuation to make room for a dead file would be the wrong order.
+  while (spilledBytes > spillByteCap() && pendingSpillUnlinks.length > 0) {
+    const ref = pendingSpillUnlinks.shift()!;
+    spilledBytes -= ref.payloadBytes;
+    deleteResponseSpill(ref);
+  }
+  // Ordered by createdAt, not by map order. `states` is not an age index:
+  // demotion and spill replacement delete and reinsert entries, and
+  // writeBoundedSnapshot serializes the map reversed, so map order can put a
+  // newer continuation first — and evicting that one spends a resume the older
+  // entry would not have cost. Sorting is O(k log k) over the spilled subset and
+  // runs only on a tick already over budget.
+  const spilled = [...states]
+    .filter((pair): pair is [string, SpilledResponseState] => pair[1].kind === "spill")
+    // createdAt is millisecond-resolution, so ties are ordinary under load. A
+    // stable sort would then fall back to insertion order — the very order this
+    // is avoiding — so break ties on the response id. Not localeCompare: the
+    // order must not depend on the host locale.
+    .sort((a, b) => a[1].createdAt - b[1].createdAt
+      || (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
+  for (const [id, entry] of spilled) {
+    if (spilledBytes <= spillByteCap()) break;
+    spilledBytes -= entry.spill.payloadBytes;
+    deleteEntry(id);
+  }
+  return before - spilledBytes;
+}
+
 function pruneResponses(at = now()): void {
   for (const [id, state] of states) {
     if (at - state.createdAt > RESPONSE_TTL_MS) deleteEntry(id);
@@ -1537,6 +1648,7 @@ function pruneResponses(at = now()): void {
       replaceWithSpillFailure(oldestId, entry);
     }
   }
+  enforceSpilledResponseBudget();
 }
 
 /** Periodic TTL-only sweep; count/byte eviction remains owned by mutation paths. */
@@ -1547,7 +1659,10 @@ export function sweepExpiredResponseStates(at = now()): number {
     deleteEntry(id);
     removed += 1;
   }
-  if (removed > 0) schedulePersist();
+  // The disk ceiling needs a caller that does not depend on traffic. The return
+  // value stays the TTL count so this function's existing contract is unchanged.
+  const reclaimed = enforceSpilledResponseBudget();
+  if (removed > 0 || reclaimed > 0) schedulePersist();
   return removed;
 }
 

structure/00_overview.md

@@ -84,7 +84,7 @@ opencodex state root does not undo those writes. Putting native Codex back is th
 | `~/.opencodex/ocx.pid`, `runtime-port.json`, `system-env-port` | opencodex runtime | Live process identity and the port a client should reach; rewritten on start. `runtime-port.json` also carries the protected per-process listener-attestation key used before CLI diagnostics attach a management bearer. |
 | `~/.opencodex/codex-runtime.json`, `codex-runtime-clamp.json` | opencodex Codex runtime | Selected Codex executable/version state and effort-clamp diagnostics. Not process identity: these persist a resolved choice and a diagnostic, so losing them changes behavior until re-resolved. |
 | `~/.opencodex/service-state.json`, `service.log`, `service-api-token`, `opencodex-service-launcher.vbs`, `opencodex-service-task.xml`, `opencodex-service.cmd`, `winsw`, `tray-state.json`, `tray-heartbeat.json`, `opencodex-tray.ps1`, `opencodex-tray-*.ico`, `update-job.json` | opencodex operators | Installed-service, Windows tray, and self-update artifacts and bookkeeping. The update record carries its worker PID so a dead worker recovers instead of blocking later runs. |
-| `~/.opencodex/responses-state.json`, `usage-debug.jsonl`, `crash.log`, `artifacts/` | opencodex diagnostics and artifacts | Bounded caches, diagnostics, and generated image/video artifacts served locally. |
+| `~/.opencodex/responses-state.json`, `responses-state-spill/`, `usage-debug.jsonl`, `crash.log`, `artifacts/` | opencodex diagnostics and artifacts | Bounded caches, diagnostics, and generated image/video artifacts served locally. The spill directory holds continuation state demoted out of the in-memory cap and is bounded in aggregate, not only per file. |
 | `~/.opencodex/codex-shim.json`, `*.lock`, `kimi-device-id`, `mimo-client-id`, `.star-prompted` | opencodex bookkeeping | Shim restore obligations, cross-process locks, per-install client identifiers, one-shot UI flags. |
 | `~/.opencodex/.opencodex-owner.json`, `.opencodex-uninstall.json` | opencodex | Ownership marker and the manifest that bounds what uninstall may remove. Both live in the OpenCodex state root, not in `$CODEX_HOME`. |
 | `$CODEX_HOME/config.toml` | Codex, edited by opencodex | Active provider and provider table. |

structure/02_config-and-codex-home.md

@@ -157,6 +157,34 @@ Hidden` inside an already-running PowerShell script, nor to .NET/VBS process-win
 - 다른 대안 대신 이 방식을 선택한 이유: Names and environment paths are caller-controlled, required secret writes must not silently skip ACLs, and elevation has a larger authority boundary that should remain FFI-only.
 - 장점, 단점 및 영향: Default Windows ARM64 installations can start and harden secrets; non-default Windows roots continue to fail closed until Bun exposes a trustworthy native system-directory API without FFI.
 
+The durable response-spill directory `~/.opencodex/responses-state-spill/` is bounded in
+aggregate, not only per file. Continuation state demoted out of the in-memory cap
+(`MAX_STORED_RESPONSE_BYTES`) is written there, and eviction past
+`MAX_SPILLED_RESPONSE_BYTES` removes oldest-first through the same deletion point that serves
+TTL and count eviction, so an evicted entry unlinks its file. One function owns that ceiling and
+three callers drive it: mutation pruning, the lazy load that follows a restart, and the periodic
+sweep. The periodic caller is not redundant — the mutation path runs only when traffic arrives, so a
+process that comes up over budget from a snapshot written under a larger ceiling would otherwise
+stay over it while idle.
+
+The ceiling bounds what the store can account for, which is every entry in the map plus the
+superseded generations queued for unlink, and deliberately not the directory as a whole. Spill files
+orphaned by a crash are absent from the map, so this accounting can neither see nor price them; they
+remain with the `recoverOrphanedResponseSpills` grace sweep described below, which is the only
+mechanism that reclaims them. A host that crashes repeatedly can therefore hold spill bytes above
+this ceiling for up to `RESPONSE_SPILL_ORPHAN_GRACE_MS` past each crash. Without that aggregate bound the
+directory was limited only per file (256 MiB) and per entry (1000) — a 250 GiB product — which
+left `RESPONSE_TTL_MS` as the only effective limit and made disk use a function of client
+request rate rather than of anything the process controls.
+
+[Decision Log]
+- 목적과 의도: Bound the durable spill directory in aggregate so demoted continuation state cannot consume the host disk.
+- 기존 구현 및 제약 조건: The resident map has an unconditional byte cap and demotes past it, but the disk it demotes onto had only a per-file ceiling and the shared 1000-entry count cap. Retention itself worked — the hour-long TTL did evict — so the gap was a missing budget, not a leak.
+- 검토한 주요 대안: Lower the per-file ceiling; shorten the TTL; sweep the directory on a timer; add a configurable budget key; carry a running byte counter.
+- 선택한 방식: A constant aggregate ceiling checked at the end of the existing prune, evicting oldest-first, with the total recomputed per prune rather than carried as a counter.
+- 다른 대안 대신 이 방식을 선택한 이유: Per-file or TTL changes alter retention semantics other bounds depend on; a timer adds a second owner for eviction; a config key would surface a knob the sibling bounds (count, TTL, per-file) do not have; and a running counter could silently disable the cap if any of the several insertion paths missed an increment, where a walk over at most 1000 entries cannot drift.
+- 장점, 단점 및 영향: Disk use stops tracking client request rate. Ordinary traffic is unaffected because the count cap binds at a comparable point for median-sized payloads; a workload of unusually large continuations loses its oldest spills earlier than the TTL would, surfacing as the existing `previous_response_not_found` continuation miss.
+
 Response-state loading performs a bounded recovery pass for interrupted snapshot writes. It only
 matches regular files named `responses-state.json.ocx.<pid>.<sequence>.tmp`, waits at least 15
 minutes, and skips the current or any live PID. Eligible files are truncated before unlinking so a