Skip to content

fix(memory): journal the index accounting, and stop three ways of losing it (#1447) - #1450

Merged
ozand merged 1 commit into
mainfrom
feat/1447-memory-index-journal
Sep 9, 2026
Merged

fix(memory): journal the index accounting, and stop three ways of losing it (#1447)#1450
ozand merged 1 commit into
mainfrom
feat/1447-memory-index-journal

Conversation

@ozand

@ozand ozand commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Closes #1447.

The issue asked for one thing: get memory_index onto the ledger. Probing the path before editing turned up two further defects in the same function, both more severe than the one filed.

1. The record went stale, which is worse than absent

When memory/index.md was deleted, get_memory_context returned "" without touching last_index_fit, so the record kept the previous build's byte counts. A cycle with no memory would have journalled another cycle's accounting as its own. Verified before fixing:

after deleting the index -> context: ''
last_index_fit: STALE, source_chars=3654

Wrong data presented as current is a worse failure than no data.

2. An unreadable index raised out of prompt construction

undecodable index RAISES: UnicodeDecodeError 'utf-8' codec can't decode byte 0xff

memory/index.md is instance-owned. The loop could fail its own cycle by writing one bad byte into its own file. ADR-002 / #1173 is explicit that an unreadable input is a value, not an exception.

3. An allowlist ate the signal added by #1443

ContextBuilder copied six named keys out of the record. #1443 added resident_matched / resident_expected / resident_missing after that list was written, so the report that a rule entry stopped matching never reached a caller at all — the guard's output was filtered out on the way to its reader. Copying the whole record removes the class rather than this instance of it.

What changed

last_index_fit now always carries a status: present / missing / empty / unavailable, with the exception type as reason for the last. Those four are exactly the states ADR-002 requires a caller be able to tell apart.

The bridge journals memory_index on the phase: "system_prompt" row alongside sections, never instead of it — existing readers of sections.memory are untouched, pinned by a test.

Grouper safety, checked not assumed

This adds a key to an existing row rather than a new phase, but #1374 is the precedent for new keys minting phantom entries. scripts/loop_metrics_report.py:160 allowlists phases (the #1374 ratchet) and every outcome reader filters phase == "outcome" explicitly, so nothing keys on the presence of a system_prompt field.

Tests

Six new. Five fail against the unfixed sources — verified in an isolated copy, not by stashing. The sixth is the sections behaviour-preservation test, which must pass in both states and does.

Full suite: 4 failed, 3595 passed, 18 skipped. The four are the known Windows baseline, failing identically on a clean origin/main.

…ing it

#1447 asked for one thing — get `memory_index` onto the ledger. Probing the
path first turned up two more defects in the same function, both worse:

1. STALE, not absent. When memory/index.md was deleted, `get_memory_context`
   returned "" without touching `last_index_fit`, so the record kept the
   PREVIOUS build's byte counts. A cycle with no memory would have journalled
   another cycle's accounting as its own. Verified: after deleting the index,
   `source_chars` still read 3654.

2. An unreadable index RAISED. `index.read_text()` propagated
   UnicodeDecodeError out through build_system_prompt. memory/index.md is
   instance-owned, so the loop could fail its own cycle by writing one bad
   byte into its own file. ADR-002/#1173 is explicit that an unreadable input
   is a value, not an exception.

3. The allowlist ate the signal. ContextBuilder copied six named keys out of
   the record, and #1443 added `resident_matched` / `resident_expected` /
   `resident_missing` after that list was written — so the report that a rule
   entry stopped matching never reached a caller at all. Copying the whole
   record removes the class, not just this instance of it.

`last_index_fit` now always carries a `status`: present / missing / empty /
unavailable, with the exception type as `reason` for the last. The bridge
journals `memory_index` on the `system_prompt` row ALONGSIDE `sections`,
never instead of it.

Grouper safety: this adds a key to an existing row rather than a new phase.
loop_metrics_report._group_cycles allowlists phases (#1374 ratchet) and every
outcome reader filters `phase == "outcome"`, so nothing keys on the presence
of a system_prompt field.

Five of the six new tests fail against the unfixed sources; the sixth is the
behaviour-preservation test for `sections`, which must pass in both states.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ozand
ozand merged commit 8092848 into main Sep 9, 2026
3 checks passed
@ozand
ozand deleted the feat/1447-memory-index-journal branch September 9, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Journal memory-index fit diagnostics on system_prompt rows

1 participant