feat(memory): catch contradictions in other languages and long sentences

changes/95.fixed.md

@@ -0,0 +1,3 @@
+**Contradictions are caught in other languages and long sentences too.** A newer memory in Hindi or another
+language, or a long sentence that names the person a few words in, now replaces the older memory it contradicts
+instead of sitting next to it.

docs/API.md

@@ -3181,13 +3181,19 @@ A device ("node") is a phone, a pair of smart glasses, a watch, or the desktop a
   message for `dreaming.require_idle_minutes`. Steps: merge near-duplicates (similarity ≥ `merge_similarity` and word
   overlap ≥ `merge_min_overlap`, never facts whose names/places/numbers conflict; the kept fact keeps its id), settle
   contradictions: facts are paired when they are about the same subject (name, possessive relation like "Maya's sister
-  Riya", or the user as "I"/"the user") and share an attribute family or differ in names/places/numbers or negation;
-  place, job, relationship and diet pairs are always checked, broader families (routine, ownership, health) need
-  similarity ≥ `contradiction_similarity`; the model confirms and names the current fact (the newer one wins unless the
-  model picks the older and only the older has explicit recent wording such as "now" or "this month"); the loser is
-  deleted and its text kept as the winner's `previous_content`; pairs the model cleared are not asked again until either
-  fact changes. Between the two (`dreaming.merge_languages`, default on, #353), the same memory saved in two languages
-  is kept once: a fact led by Devanagari is paired with up to 3 facts without Devanagari that are clearly about the same
+  Riya", or the user as "I"/"the user" - a fact led by Devanagari gets its subject from `scripts.devanagari_subjects`
+  instead, since it has no capital letters to read, #95) and share an attribute family or differ in names/places/numbers
+  or negation; families come from English keywords, and a fact they miss (often one that isn't in English) gets
+  whichever family's short description its embedding is closest to, as long as it's ahead of the runner-up by at least
+  `memory.family_margin` (default 0.02) - a fixed similarity floor doesn't travel between embedding models, so this
+  compares the best match against the next-best instead of an absolute score, and reuses the fact's own stored
+  embedding rather than computing a new one; place, job, relationship and diet pairs are always checked, broader
+  families (routine, ownership, health) need similarity ≥ `contradiction_similarity`; the model confirms and names the
+  current fact (the newer one wins unless the model picks the older and only the older has explicit recent wording
+  such as "now" or "this month"); the loser is deleted and its text kept as the winner's `previous_content`; pairs the
+  model cleared are not asked again until either fact changes. Between the two (`dreaming.merge_languages`, default on,
+  #353), the same memory saved in two languages is kept once: a fact led by Devanagari is paired with up to 3 facts
+  without Devanagari that are clearly about the same
   person (both have a subject and the names match across scripts, #351), have the same numbers, and have no name or
   place the Hindi one lacks; with a multilingual embedding model (bge-m3) also similarity ≥
   `dreaming.language_merge_similarity` (0.7), otherwise the similarity only orders them. The `fast`-role model (the

sentient/config/schema.py

@@ -518,6 +518,14 @@ class MemoryConfig(BaseModel):
         6000, ge=500, description="Max characters injected from each workspace markdown file."
     )
     graph_link_similarity: float = Field(0.80, ge=0.0, le=1.0, description="Memory graph edge threshold.")
+    family_margin: float = Field(
+        0.02, ge=0.0, le=1.0,
+        description="When the English keyword attribute families (residence, job, ...) find nothing in a fact - "
+        "often because it isn't English - a family is assigned instead, whichever embeds closest to a short "
+        "description of it, as long as it's ahead of the next closest by at least this much. A fixed similarity "
+        "floor doesn't travel between embedding models (one scored an unrelated Hindi fact 0.44-0.49 against "
+        "every family alike), so this compares the best match against the runner-up instead of an absolute score.",
+    )
     summarize_after_minutes: int = Field(
         60, ge=5, description="Conversation chunks older than this are summarized into episodic memory."
     )

sentient/memory/dreaming.py

@@ -36,7 +36,7 @@
     TIME_WORDING_RE,
     conflict_strength,
     new_details,
-    profile_fact,
+    profile_fact_async,
     word_overlap,
 )
 from sentient.memory.schema import ensure_memory_schema
@@ -556,15 +556,25 @@ def _clearer(self, hindi: dict, other: dict, facts: list[dict]) -> tuple[dict, d
     def contradiction_candidate(a: str, b: str, user_name: str = "") -> bool:
         return conflict_strength(a, b, user_name) > 0
 
-    def contradiction_pairs(self, facts: list[dict]) -> list[tuple[int, float, int, int]]:
+    async def contradiction_pairs(self, facts: list[dict]) -> list[tuple[int, float, int, int]]:
         """Candidate pairs ``(strength, similarity, i, j)``, strongest first.
 
         Facts are grouped by subject (the person or entity a fact is about, including possessive relations
         and the user's own name), so "Maya moved to Bengaluru" meets "Maya lives in Pune" without sharing
         a leading word or a high embedding score. Pairs sharing only a broad attribute also need
-        ``contradiction_similarity``."""
+        ``contradiction_similarity``. A fact's attribute family falls back to embedding similarity when the
+        English keyword families miss it, e.g. non-English text (``profile_fact_async``)."""
         user = self.app.config.assistant.user_name
-        profiles = [profile_fact(f["content"], user) for f in facts]
+        mem = self.app.memory
+        embed_one = mem.embed_query if mem is not None else None
+        model_key = mem.llm.model_for("embedding") if mem is not None else ""
+        margin = self.app.config.memory.family_margin
+        # every active fact already has a stored vector (it was embedded on insert): reusing it here, instead
+        # of embedding the same text again, was the difference between 241 s and 14.5 s over 500 facts (#95)
+        profiles = [
+            await profile_fact_async(f["content"], embed_one, user, model_key, margin, vector=f.get("vector"))
+            for f in facts
+        ]
         groups: dict[str, list[int]] = {}
         for n, prof in enumerate(profiles):
             for s in prof.subjects:
@@ -600,7 +610,7 @@ async def _contradiction_step(self, stats: dict, notes: dict, budget: list[int])
         checked = set(checked_list)
         gone: set[int] = set()
         name = self._name()
-        for _strength, _sim, i, j in self.contradiction_pairs(facts):
+        for _strength, _sim, i, j in await self.contradiction_pairs(facts):
             if budget[0] <= 0:
                 break
             older, newer = sorted((facts[i], facts[j]), key=lambda f: (f["updated_at"], f["id"]))