Keep EquivalentTo axioms and pair text changes (#2)

.gitignore

@@ -64,3 +64,5 @@ Thumbs.db
 # uv
 .uv/
 uv.lock.bak
+
+runs/CL_9999999

clara_workflow/agent_instructions.md

@@ -41,6 +41,9 @@ Normalise ids as follows:
 The routing payload contains `targets`. Each target has one of these routes:
 
 - `ntr` — new term bundle; contains decomposable text plus structural context
+- `text_revision` — revised prose on an existing term; decomposable
+- `refs_added` — prose on an existing term that did **not** change, with one or
+  more references newly attached to it
 - `relationship` — a single atomic structural axiom to check
 - `synonym` — a single atomic synonym axiom to check
 
@@ -56,6 +59,8 @@ Relevant target fields:
 - `term_level_candidate_refs`
 - for `ntr`: `textual_changes` (legacy alias `definition_changes`) and
   `relationship_changes`
+- for `text_revision`: `textual_changes`, plus `prior_value` on each change
+- for `refs_added`: `textual_changes` and `refs_added`
 - for `relationship` / `synonym`: `change`
 
 ### Contract rules
@@ -108,6 +113,32 @@ For `ntr` targets:
 4. Also take each `added` change from `relationship_changes` and convert it
    into one atomic `core` assertion.
 
+### Route: `text_revision`
+
+The prose changed on an existing term. Treat it like the textual half of an
+`ntr` target:
+
+1. Decompose the `added` change's `value` into atomic assertions.
+2. Tag each `core` or `background` as above.
+
+`prior_value` is the previous wording, provided as context only. Do not verify
+it, and do not report assertions that are unchanged between the two wordings as
+new claims — the reviewable content is the current `value`.
+
+### Route: `refs_added`
+
+The prose did **not** change; a reference was attached to it. The claim under
+review is that the *newly added* reference supports the existing text.
+
+1. Decompose the `value` into atomic assertions as usual.
+2. Verify them **only** against `refs_added` — not the full ref list on the
+   axiom. The pre-existing refs were justified when they were added.
+3. If every assertion is `uncertain` against the new refs, that is a real
+   result: the added reference does not visibly support the definition. Say so
+   in the report rather than falling back to the older refs.
+
+Do not re-verify the definition against its whole reference set on this route.
+
 ### Route: `relationship`
 
 For `relationship` targets:
@@ -144,14 +175,28 @@ Examples:
 - `synonym_related`:
   - "`{value}` is a related synonym of `{term_label}`."
 
-For `equivalent_class` routed as a structural target:
+For `equivalent_class` routed as a structural target, `value` carries the bare
+axiom in Manchester syntax, e.g.
 
-- If the routed change does not expose enough semantics to verbalise the
-  logical definition faithfully, create one `core` assertion noting that the
-  equivalent-class axiom for the term requires support.
-- If you cannot verify it from the exposed routing data, it is acceptable to
-  leave that assertion `uncertain` with a note explaining that the routed
-  payload does not provide enough structure for faithful verbalisation.
+```text
+buccal mucosa cell EquivalentTo keratinocyte and (part of some mouth mucosa)
+```
+
+`EquivalentTo` states necessary **and sufficient** conditions, so it makes two
+claims. Emit **both** as separate `core` assertions:
+
+- necessary — "Every `{term_label}` is a `{genus}` that `{differentia}`."
+- sufficient — "Every `{genus}` that `{differentia}` is a `{term_label}`."
+
+The sufficient direction is the one editors get wrong and the one literature
+rarely states outright. Do not mark it `pass` on evidence that only supports
+the necessary direction, and do not collapse the two into a single assertion.
+If the axiom nests operators too deeply to verbalise faithfully, verbalise what
+you can and note the residue — do not silently drop a conjunct.
+
+Only if `value` is absent or empty may you fall back to a single `core`
+assertion noting that the equivalent-class axiom requires support, left
+`uncertain` with a note that the payload exposed no axiom text.
 
 ## Reference scope per assertion
 
@@ -165,7 +210,16 @@ Reference selection rules:
   `term_level_candidate_refs`.
 - For `relationship` targets, prefer `candidate_refs`; if empty, fall back to
   `term_level_candidate_refs`.
+- For `equivalent_class` changes, the refs are the **text definition's** refs.
+  A logical axiom carries no dbxrefs of its own, so the producer fills
+  `candidate_refs` from the definition — the logical definition formalises the
+  prose, so the prose's evidence is what has to support it. Treat an empty list
+  here as "the definition itself is uncited", not as a routing error.
 - For `synonym` targets, use `candidate_refs`.
+- For `text_revision` targets, use `candidate_refs`; if empty, fall back to
+  `term_level_candidate_refs`.
+- For `refs_added` targets, use `refs_added` only. Never widen to the axiom's
+  pre-existing refs.
 - Assume these ref lists were prefiltered upstream; only normalize case and
   slash formatting needed by the search tools.
 

clara_workflow/stage1/README.md

@@ -37,6 +37,38 @@ Any of the following, on either side (`added` or `removed`):
 Each of these carries its own `hasDbXref` list in the parser output; stage 2/3
 check that list when deciding whether the change is justified.
 
+`definition_refs()` reads each term's definition dbxrefs out of the **head** edit
+file rather than the diff, because `robot diff` reports only changed axioms: a PR
+that adds a logical definition to an untouched term carries no definition axiom,
+yet the text definition it formalises is what justifies it. These land in
+`by_term[*].definition_refs`.
+
+`equivalent_class` changes carry the axiom itself in `value`, as bare Manchester
+syntax (`buccal mucosa cell EquivalentTo keratinocyte and (part of some mouth
+mucosa)`). ROBOT's markdown diff already emits Manchester syntax; the parser just
+strips the markdown links off the labels.
+
+## Paired text changes (`text_deltas()`)
+
+ROBOT reports an annotated axiom as an independent `Removed` + `Added` bullet,
+and an OWL axiom's identity includes its annotations. So attaching one dbxref to
+an untouched definition is indistinguishable from a rewrite unless the two sides
+are paired and their values compared.
+
+`text_deltas()` does that pairing for `text_def` / `comment`, giving each a
+status:
+
+| `status`    | Meaning | What stages 2/3 should check |
+|-------------|---------|---|
+| `revised`   | The prose changed. | The new prose, against the axiom's refs. |
+| `refs_only` | The prose is byte-identical; only refs changed. | The existing prose, against `refs_added` **only**. |
+| `added`     | No matching removed side. | The prose, as a new claim. |
+| `removed`   | No matching added side. | Nothing — removals aren't justified by refs. |
+
+Without this split, a ref-only edit re-verifies an unchanged definition against
+its entire reference list, and the actual claim — "this newly added paper
+supports the existing definition" — never gets asked.
+
 ## Not reviewed (kept in output for traceability, but skipped by `reviewable_changes()`)
 
 | `kind`               | Why ignored |

clara_workflow/stage1/__init__.py

@@ -11,17 +11,23 @@
     CONTENT_KINDS,
     DECOMPOSABLE_KINDS,
     decomposable_changes,
+    definition_refs,
     parse_diff_markdown,
     reviewable_changes,
     summarise_by_term,
+    text_deltas,
+    TextDelta,
 )
 
 __all__ = [
     "Change",
     "CONTENT_KINDS",
     "DECOMPOSABLE_KINDS",
     "decomposable_changes",
+    "definition_refs",
     "parse_diff_markdown",
     "reviewable_changes",
     "summarise_by_term",
+    "text_deltas",
+    "TextDelta",
 ]

clara_workflow/stage1/extract.py

@@ -27,9 +27,11 @@
 from clara_workflow.stage1.parse import (
     Change,
     decomposable_changes,
+    definition_refs,
     parse_diff_markdown,
     reviewable_changes,
     summarise_by_term,
+    text_deltas,
 )
 
 
@@ -67,8 +69,13 @@ def extract(
     right_ref: str,
     edit_file: str,
     robot: str = "robot",
-) -> list[Change]:
-    """Resolve two refs against `edit_file`, run robot diff, parse, return changes."""
+) -> tuple[list[Change], dict[str, list[str]]]:
+    """Resolve two refs against `edit_file`, run robot diff, parse.
+
+    Returns the parsed changes plus the head-state definition refs per term,
+    which is where a logical definition's justification lives when the text
+    definition itself wasn't touched by the PR.
+    """
     with tempfile.TemporaryDirectory() as td:
         tdp = Path(td)
         left = tdp / "left.owl"
@@ -77,7 +84,10 @@ def extract(
         _git_show(repo, left_ref, edit_file, left)
         _git_show(repo, right_ref, edit_file, right)
         _robot_diff(left, right, diff_md, robot=robot)
-        return parse_diff_markdown(diff_md.read_text())
+        return (
+            parse_diff_markdown(diff_md.read_text()),
+            definition_refs(right.read_text()),
+        )
 
 
 def main(argv: list[str] | None = None) -> int:
@@ -94,7 +104,7 @@ def main(argv: list[str] | None = None) -> int:
         print(f"robot executable not found: {args.robot}", file=sys.stderr)
         return 2
 
-    changes = extract(
+    changes, head_definition_refs = extract(
         repo=args.repo,
         left_ref=args.left,
         right_ref=args.right,
@@ -109,10 +119,16 @@ def main(argv: list[str] | None = None) -> int:
         "changes": [_change_to_dict(c) for c in changes],
         "reviewable": [_change_to_dict(c) for c in reviewable],
         "decomposable": [_change_to_dict(c) for c in decomposable],
+        # Removed/added text axioms paired so a ref-only edit is distinguishable
+        # from a rewrite; consumed by the router to pick a route.
+        "text_deltas": [dataclasses.asdict(d) for d in text_deltas(changes)],
         "by_term": {
             tid: {
                 **{k: v for k, v in entry.items() if k != "changes"},
                 "changes": [_change_to_dict(c) for c in entry["changes"]],
+                # Refs on the term's definition as it stands at the head ref —
+                # present whether or not this PR touched the definition.
+                "definition_refs": head_definition_refs.get(tid, []),
             }
             for tid, entry in summarise_by_term(changes).items()
         },

clara_workflow/stage1/parse.py

@@ -120,6 +120,18 @@ def _iter_links(s: str) -> Iterator[tuple[str, str]]:
         yield m.group("text"), m.group("url")
 
 
+def _bare_axiom(body: str) -> str:
+    """Reduce a diff bullet to a bare Manchester-syntax axiom.
+
+    `robot diff --format markdown --labels true` already emits class expressions
+    in Manchester syntax; the labels just arrive wrapped in markdown links.
+    Dropping the links leaves the axiom as an editor would write it, e.g.
+
+        buccal mucosa cell EquivalentTo keratinocyte and (part of some mouth mucosa)
+    """
+    return re.sub(r"\s+", " ", _LINK_RE.sub(lambda m: m.group("text"), body)).strip()
+
+
 def _parse_bullet(line: str, term_id: str, term_label: str, side: str) -> Change:
     """Parse one top-level `- ...` bullet into a Change.
 
@@ -165,9 +177,11 @@ def _parse_bullet(line: str, term_id: str, term_label: str, side: str) -> Change
 
     # EquivalentClasses.
     if " EquivalentClasses " in body or "EquivalentTo" in body:
+        # The logical definition is the claim to verify, so carry the axiom
+        # itself rather than dropping it (issue #2).
         return Change(
             term_id=term_id, term_label=term_label, side=side,
-            kind="equivalent_class", raw=raw,
+            kind="equivalent_class", value=_bare_axiom(body), raw=raw,
         )
 
     # Annotation-style: [subj] [pred](IRI) <value>
@@ -326,3 +340,124 @@ def decomposable_changes(changes: list[Change]) -> list[Change]:
     without an intermediate decomposition step.
     """
     return [c for c in reviewable_changes(changes) if c.kind in DECOMPOSABLE_KINDS]
+
+
+# --- text change pairing ---------------------------------------------------
+
+@dataclass
+class TextDelta:
+    """One decomposable text axiom, paired across the removed/added sides.
+
+    `robot diff` reports an annotated axiom as an independent removed + added
+    bullet, and an OWL axiom's identity includes its annotations. So adding a
+    dbxref to an untouched definition looks exactly like a definition rewrite
+    unless the two sides are paired and their values compared.
+
+    status:
+      revised    — the prose changed (verify the new prose)
+      refs_only  — the prose is byte-identical; only the refs changed (verify
+                   the existing prose against `refs_added`, nothing else)
+      added      — no matching removed side (new axiom on an existing term)
+      removed    — no matching added side (axiom deleted)
+    """
+    term_id: str
+    term_label: str
+    kind: str                        # text_def | comment
+    status: str                      # revised | refs_only | added | removed
+    value: str | None = None         # the added (current) prose
+    prior_value: str | None = None   # the removed (previous) prose
+    refs: list[str] = field(default_factory=list)          # refs on the added side
+    refs_added: list[str] = field(default_factory=list)
+    refs_removed: list[str] = field(default_factory=list)
+
+
+def text_deltas(changes: list[Change]) -> list[TextDelta]:
+    """Pair removed/added decomposable text axioms per (term, kind).
+
+    Obsoleted terms and non-content kinds are excluded, matching
+    `reviewable_changes`.
+    """
+    grouped: dict[tuple[str, str], dict[str, list[Change]]] = {}
+    for c in reviewable_changes(changes):
+        if c.kind not in DECOMPOSABLE_KINDS:
+            continue
+        sides = grouped.setdefault((c.term_id, c.kind), {"added": [], "removed": []})
+        sides[c.side].append(c)
+
+    def _delta(kind: str, added: Change | None, removed: Change | None) -> TextDelta:
+        src = added or removed
+        assert src is not None
+        if added is not None and removed is not None:
+            status = "refs_only" if added.value == removed.value else "revised"
+        else:
+            status = "added" if added is not None else "removed"
+        before = set(removed.refs) if removed is not None else set()
+        after = set(added.refs) if added is not None else set()
+        return TextDelta(
+            term_id=src.term_id,
+            term_label=src.term_label,
+            kind=kind,
+            status=status,
+            value=added.value if added is not None else None,
+            prior_value=removed.value if removed is not None else None,
+            refs=list(added.refs) if added is not None else [],
+            refs_added=sorted(after - before),
+            refs_removed=sorted(before - after),
+        )
+
+    out: list[TextDelta] = []
+    for (_term_id, kind), sides in grouped.items():
+        added = list(sides["added"])
+        removed = list(sides["removed"])
+
+        # Pair identical prose first — those are ref-only edits, and matching
+        # them by value keeps a genuine rewrite from stealing the partner.
+        for a in list(added):
+            match = next((r for r in removed if r.value == a.value), None)
+            if match is not None:
+                added.remove(a)
+                removed.remove(match)
+                out.append(_delta(kind, a, match))
+
+        # Then pair what's left positionally; a term rarely carries more than
+        # one definition, so this only matters for multi-comment edits.
+        while added and removed:
+            out.append(_delta(kind, added.pop(0), removed.pop(0)))
+
+        out.extend(_delta(kind, a, None) for a in added)
+        out.extend(_delta(kind, None, r) for r in removed)
+
+    return out
+
+# --- head-state definition refs -------------------------------------------
+
+# One axiom per line in ROBOT's functional syntax, so a definition's dbxrefs are
+# whatever `hasDbXref` annotations sit to the left of the IAO_0000115 predicate.
+_DEF_PREDICATE_RE = re.compile(r"obo:IAO_0000115 obo:([A-Za-z]+_\d+)")
+_XREF_VALUE_RE = re.compile(r'oboInOwl:hasDbXref "([^"]+)"')
+
+
+def definition_refs(owl_text: str) -> dict[str, list[str]]:
+    """Map term CURIE -> refs on that term's definition, from an edit file.
+
+    `robot diff` only reports axioms that changed, so a PR that adds a logical
+    definition to an untouched term carries no definition axiom at all. The
+    references that justify a logical definition are the ones on the text
+    definition it formalises, which means reading them out of the head state
+    rather than the diff.
+
+    Parsed line-wise on purpose: a nested-paren regex misses definitions whose
+    dbxref values contain brackets (e.g. Wikipedia URLs with parenthesised
+    disambiguators).
+    """
+    out: dict[str, list[str]] = {}
+    for line in owl_text.splitlines():
+        stripped = line.strip()
+        if not stripped.startswith("AnnotationAssertion("):
+            continue
+        match = _DEF_PREDICATE_RE.search(stripped)
+        if match is None:
+            continue
+        term_id = match.group(1).replace("_", ":", 1)
+        out[term_id] = _XREF_VALUE_RE.findall(stripped[: match.start()])
+    return out