feat(memory): rebuild agent memory on one Postgres fact store - #284
Open
eastriverlee wants to merge 11 commits into
Open
feat(memory): rebuild agent memory on one Postgres fact store#284eastriverlee wants to merge 11 commits into
eastriverlee wants to merge 11 commits into
Conversation
eastriverlee
force-pushed
the
feat/memory-store
branch
from
September 2, 2026 13:29
57abafa to
b6bc9cf
Compare
…b queue The memory subsystem is being rebuilt from scratch; this is the store, nothing reads it yet. Facts are atomic sentences extracted from an immutable episode, scoped private/circle/workspace, labelled with the security rank and classes of their origin conversation, and retired by a supersede pointer, a forget timestamp, or a valid_until, never by deletion. The reader's clearance is one SQL filter shared by search, lookup and forget, so there is no second gate in Go. Search fuses a pgvector cosine list and a pg_trgm word_similarity list with reciprocal rank fusion, then adjusts deterministically: episodes decay with age, reinforced preferences rank higher. The substring re-ranking that undid the old sidecar's ordering has no successor. Vectors sit in a side table the migration creates only when the vector extension is available, because two databases this runtime boots on have none: the bookworm guest rootfs (Debian ships no pgvector before trixie) and the embedded-postgres binaries behind the standalone command. Both apply the migration and answer lexically; the search result names the mode so a lexical answer is never mistaken for a hybrid one. memory_job is an outbox with FOR UPDATE SKIP LOCKED claims, a lease, exponential backoff and a partial unique index that collapses repeated enqueues of one subject into one pending row. The worker takes handlers per kind and claims only kinds it can run; extraction and profile handlers come with the next change. The capability embedding client gains inputType, batch input and outputDimensions, the fields the sidecar already sent and Qwen3-style models need. Not done here: tool wiring, launch-time recall, extraction, and the removal of the Graphiti and markdown stores. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Writing memory and extracting it from finished work were going to be two mechanisms with two schemas. They are now one: an episode (a task transcript, or the sentence a person asked the assistant to keep) is embedded, the nearest live facts the requester may read become candidates, and a low-tier model returns the facts the memory should hold afterwards, each related to a candidate as new, supersedes, or reinforces. The runtime rejects a relation to any fact it did not offer, narrows a circle fact to private when no active circle is at hand, and strips the security label from private facts because their scope is already one person. No similarity threshold ever merges two facts by itself; that judgment stays with the model, and the runtime only feeds it what exists. The agent sees three verbs: memory_search returns facts with IDs, memory_remember takes one sentence and reports what was created, superseded, or reinforced, and memory_forget takes IDs that memory_search returned in the same task, failing closed on any other. The write path carries no kind, scope, or ID, so a model does not have to look a fact up before correcting it. Every finished, failed, or cancelled task run is enqueued for extraction by a transition observer; the worker reads the label, requester name, active circle, and platform from a memory.extraction_context event the launcher wrote, so it never guesses who could read what. Launch loads the requester's profile and a hybrid recall of the prompt under character budgets and records what it spent in memory.recall_injected. The live check against the real low-tier model and embedding model caught a ranking fault: a mild multiplier for reinforced preferences flipped adjacent reciprocal ranks, whose gap is under two percent, and put a repeated preference ahead of the fact that answered the question. Reinforcement is now a tie-breaker. The old Graphiti and markdown path stays wired only where no database is configured and for the simulate harness's virtual sessions; its removal, migration 031, and the virtual-session store on the in-memory repository are the next change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Graphiti sidecar, the Kuzu graph, the per-person MEMORY.md and the in-process update queue are gone, with the admin graph endpoints, the health probe, the healthd flag, the backup root and the configuration that named them. Migration 031 drops graphiti_namespace and graphiti_episode; the current-schema probe now looks for memory_fact. Person canonicalisation and the direct-conversation reset follow the memory tables, and a reset forgets the facts that came out of the conversation with a stated reason instead of deleting rows. The namespace vocabulary went with the sidecar. A launch carries one memory.SecurityLabel, resolved by the connector from the channel policy (or the requester's own clearance in a direct message), and the tool catalog reads that label; the five-scope namespace list every caller had to assemble is not needed by a store whose reader filter is one SQL predicate. The simulate harness's virtual sessions run on the in-memory repository with a passthrough ingest model, so scenarios that remember and search still read back what they wrote. The default embedding model is qwen/qwen3-embedding-4b: a first benchmark on Korean facts separated the 8b, the 4b and pplx-embed-v1-4b on nothing but latency, and the Qwen3 models are the ones a self-hosting company can serve from llama.cpp with vendor-published GGUF files. The benchmark stays in the tree as an on-demand test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The web memory page read the Graphiti graph endpoints, which the previous change removed. It now has GET /admin/api/memory/facts, which returns a person's profile and the live facts that person may read, newest first, under the same reader filter the agent uses, and POST /admin/api/memory/facts/forget, which soft-forgets facts with a reason and refuses when nothing readable was forgotten. Both take the person as readerPersonID because the device web app resolves the signed-in member before it proxies. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The hybrid query joins memory_fact_embedding, which the migration creates only where the vector extension exists. A caller that already had a query embedding (ingest, on every host) hit a missing-table error on a plain Postgres, the extraction job retried forever, and no fact was written. CI runs plain Postgres, which is how it surfaced. The repository now checks for the table before choosing the hybrid query. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The memory store becomes its own module, github.com/yeomyeonggeori/bluememo, vendored at .dependency/bluememo the way the harness is. It depends on nothing but pgx; this host adapts its task, model and identity types in internal/memory (a reader from PersonAccess, a label from a channel policy, a transcript from a task run, the harness's structured-output provider behind bluememo's one-method interface). Migration 030 is a verbatim copy of bluememo's schema and a test fails when it drifts. Circle facts name one or more circles, and circles nest. A circle policy gains memberCircles, the projection turns that into a containment map, and a reader's readable set is their own circles plus everything those contain, transitively: a fact shared with platform is readable from engineering when engineering contains platform. Writing to a circle still needs direct membership, so the model may name several circles for one fact but never one the requester is not in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bluememo dropped the scope field: every fact has an owner who always reads it, and zero or more circles whose members read it too. There is no company-wide scope; sharing with everyone is sharing with the member circle everyone is in. memory_search returns each fact's circles instead of a scope, the admin listing carries the owner and circles, and the loop's memory context groups a fact under "user" when it names no circle and "circle" otherwise. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bluememo's ingest instruction now says memory is for what a different conversation will need once the transcript is gone, and that an empty list is the normal answer. The live check gains a mundane task run, a file rename with its shell step and reply, and asserts it yields no fact, alongside the transcripts that must. It passed once on 2026-09-03 against the real low-tier model and embedding model. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bluememo's default moves to qwen3-embedding-8b because OpenRouter serves the 4b through one provider only; the example configuration and the tests that spell the name follow it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bluememo's default moves to bge-m3, the model the device already serves locally, so local and remote embeddings share one vector space. The example configuration and the tests that spell the name follow it, and the on-demand benchmark measures bge-m3 beside the models it replaced. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bluememo now ranks vectors only from the store's own model and carries a reembed job for the rest. The application registers that handler and enqueues one reembed each time the worker starts, so a host that changed its embedding model brings its memory back into vector search on its own; the job finishes at once when nothing needs moving. The memory_guided_followup scenario used to finish both turns from a script and proved nothing about memory. It now calls memory_remember in the first turn and, in the second, requires the memory.recall_injected event and the remembered sentence in the model's context. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
eastriverlee
force-pushed
the
feat/memory-store
branch
from
September 3, 2026 04:35
a4779f9 to
331c2e2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What to look at
Seven commits, in order:
c55737dbthe store: migration 030 (memory_episode,memory_fact,memory_profile,memory_job,pg_trgm, and amemory_fact_embeddingside table created only where pgvector exists), the repositories, the hybrid search (pgvector cosine +pg_trgmword similarity, reciprocal rank fusion), and the job outbox withFOR UPDATE SKIP LOCKEDclaims.0daf5dd4ingest: one pipeline for what the agent writes (memory_remember) and what it did (a finished task run). The runtime embeds the episode, offers the nearest live facts the requester may read as candidates, and a low-tier model returns facts related asnew/supersedes/reinforces. The runtime rejects any relation to a fact it did not offer. Three tools for the model:memory_search,memory_remember {content},memory_forget {factIDs}(IDs only from a search in the same task).b6bc9cferemoval: Graphiti sidecar, Kuzu,MEMORY.md, the in-process queue, admin graph endpoints, health probe, config; migration 031 drops the graphiti tables; the namespace list becomes onememory.SecurityLabelper launch.7c408acaadmin:GET /admin/api/memory/facts(a person's profile and the live facts they may read) andPOST /admin/api/memory/facts/forget, for the device web page.922d1abbfix: lexical search when the vector table is absent (CI runs plain Postgres).27326c18the store moves to its own module, bluememo, vendored at.dependency/bluememolike the harness;internal/memorykeeps only the host adapters. Circles nest: a circle fact names one or more circles,memberCircleson a circle policy says which circles belong to it, and a member of a containing circle reads the contained circle's facts, transitively. Writing to a circle still needs direct membership.migrations/030_memory_store.sqlis a verbatim copy of bluememo's schema with a drift test.d45e13bcno company-wide scope: a fact with no circles is read by its owner alone; a fact that names circles is read by their members (and containing circles) too; sharing with everyone is sharing with themembercircle.memory_searchreturns each fact's circles instead of a scope.Design:
docs/internal/memory-redesign.mdin the internkim repository (arrives with the submodule bump, internkim PRfeat/memory-store).Evidence
go vet ./...andgo test ./...green; catalogs regenerated. bluememo's own suite (unit + Postgres onpgvector/pgvector:pg16and plainpostgres:16) green, covering multi-circle facts, transitive containment with a cycle, and the reader predicate in SQL and Go saying the same thing.BLUECLAW_TEST_POSTGRES_URL, run onpgvector/pgvector:pg16and on plainpostgres:16): reader filter across scope, rank and classes; supersede hides the old row and keeps it; expiry; scoped forget; vector ranking; job dedupe/claim/lease/retry; a finished task run observed through the real transition observer ends as rows in all three tables andRecallreturns them.BLUECLAW_LIVE_LLM_TEST=1, real low-tier model + real embedding model): a Korean transcript yields a preference and a temporary fact with its expiry; a correcting transcript supersedes the team fact and reinforces the preference; recall ranks the corrected fact first; a profile builds. It passed once on 2026-09-02 after fixing a ranking fault it found (a multiplier for reinforced preferences flipped adjacent reciprocal ranks; reinforcement is now a tie-breaker). Not run in CI.BLUECLAW_EMBEDDING_BENCHMARK=1, 30 Korean facts, 20 hand-marked queries, 1,024 dims): qwen3-embedding-8b, qwen3-embedding-4b and pplx-embed-v1-4b all at R@1 1.000; pplx-embed-v1-0.6b 0.950; bge-m3 R@1 0.950 and R@3 1.000. bluememo's library default isbaai/bge-m3; the host names the model it actually uses (a device: bge-m3 from its own llama.cpp; a direct-execution host: pplx-embed-v1-4b on OpenRouter). Vectors from two models do not compare, so search ranks vectors only from the store's own model and areembedjob, enqueued at every worker start, moves live facts onto it; thememory_guided_followupfleet scenario now calls memory_remember and requires the remembered sentence in the next task's model context. The live model check passed again against bge-m3 on 2026-09-03.What the model now sees
memory_search217 bytes,memory_remember428 bytes,memory_forget179 bytes of tool description (was 599 bytes for two tools). The ingest and profile instructions are sent only on the background low-tier calls, never in the agent turn.Not done here
MEMORY.mdfiles: none exist on a device this ships to first; the design keeps theimportjob kind for when one does.🤖 Generated with Claude Code