Skip to content

Keep EquivalentTo axioms and pair text changes (#2) - #4

Merged
ubyndr merged 3 commits into
mainfrom
issue-2-equivalent-to-and-ref-deltas
Sep 28, 2026
Merged

ubyndr merged 3 commits into
mainfrom
issue-2-equivalent-to-and-ref-deltas

Conversation

@ubyndr

@ubyndr ubyndr commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2 (consumer half).

Two gaps found while working out why CLARA reviewed almost nothing on cell-ontology#3717.

equivalent_class kept its axiom

parse.py matched an equivalence axiom and then discarded it, returning kind="equivalent_class" with value, predicate_iri and value_label all None — so the agent could only emit "the equivalent-class axiom requires support".

ROBOT's markdown diff already renders these in Manchester syntax; the labels just arrive wrapped in markdown links. Stripping the links leaves a bare axiom:

buccal mucosa cell EquivalentTo keratinocyte and (part of some mouth mucosa)

text_deltas() — pairing removed/added text axioms

An OWL axiom's identity includes its annotations, so attaching one dbxref to an untouched definition is reported by ROBOT as a removed/added pair, indistinguishable from a rewrite. text_deltas() pairs the two sides per term and classifies each revised / refs_only / added / removed. extract.py emits it in changes.json for the router.

Without this, routing existing-term prose would re-verify unchanged definitions against their entire ref list on every ref-only edit.

Agent instructions

  • New text_revision and refs_added routes. refs_added verifies the existing prose against the delta refs only — the pre-existing refs were justified when they landed.
  • EquivalentTo asserts necessary and sufficient conditions, so both directions are now required as separate core assertions, with an explicit instruction not to pass the sufficient direction on evidence that only supports the necessary one. That direction is the one literature rarely states and the one that would otherwise pass silently.

Testing

New fixture 3717-buccal-mucosa-cell-edits, generated with ROBOT from the exact commits the CLARA run reviewed (d7ff69b9 → 51ef2001). It reproduces that run's routing summary: 12 reviewable, 7 decomposable.

39 tests pass, up from 28. The new assertions were mutation-checked — reverting either fix fails them:

test_3717_equivalent_class_carries_manchester_axiom   FAILED (axiom discarded)
test_3717_ref_only_def_edit_is_not_a_revision         FAILED (pairing removed)

Merge order

Land this before cell-ontology#3720. The router tolerates payloads without text_deltas, so either order is safe, but this one activates nothing on its own.

🤖 Generated with Claude Code

Two gaps found while reviewing why CLARA checked almost nothing on
obophenotype/cell-ontology#3717.

parse.py:
- equivalent_class changes carried kind only, with value/predicate_iri/
  value_label all None, so the agent could only emit a placeholder. ROBOT's
  markdown diff already renders these in Manchester syntax with the labels
  wrapped in markdown links, so stripping the links yields a bare axiom to
  verify directly.
- add text_deltas(), pairing removed/added text_def and comment axioms per
  term so a ref-only edit (prose byte-identical, one dbxref added) is
  distinguishable from a genuine revision. An OWL axiom's identity includes
  its annotations, so ROBOT reports both as a removed/added pair.

extract.py exposes text_deltas in changes.json for the router to consume.

agent_instructions.md:
- add text_revision and refs_added routes; refs_added verifies the existing
  prose against the delta refs only.
- EquivalentTo asserts necessary AND sufficient conditions, so require both
  directions as separate core assertions. The sufficient direction is the one
  literature rarely supports and the one that was silently passing.

Fixture 3717-buccal-mucosa-cell-edits is generated from the commits the CLARA
run actually reviewed (base d7ff69b9, head 51ef2001) and reproduces its
routing summary: 12 reviewable, 7 decomposable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ubyndr and others added 2 commits September 8, 2026 11:46
…tions

An EquivalentTo axiom carries no dbxrefs of its own, so a logical definition
reached the agent with an empty ref list and could never be verified. Its
justification is the text definition it formalises, so the definition's refs
are the right scope.

The refs cannot come from the diff: robot diff reports only changed axioms, so
a PR adding a logical definition to an untouched term carries no definition
axiom at all. definition_refs() reads them from the head edit file instead.

Parsed line-wise rather than with a nested-paren regex, which missed 3 of 2926
CL definitions whose dbxref values contain brackets (Wikipedia URLs with
parenthesised disambiguators).

extract() now returns (changes, definition_refs) and changes.json carries
by_term[*].definition_refs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
runs/CL_0000312, runs/CL_0002170 and runs/CL_0002336 are the bundles from
reviewing obophenotype/cell-ontology#3717 with the routes this branch adds.
Kept as worked examples of the new routes, the way runs/ already holds
reference output from the previous CLARA implementation:

- CL_0000312  refs_added   5 of 12 core assertions pass on the newly added
              PMID:21316034 alone; the other five refs on the axiom were
              deliberately out of scope
- CL_0002170  text_revision  PMID:12014572 was in scope but returned 0 Asta
              hits and is inEPMC=N, so nothing could be read
- CL_0002336  text_revision + equivalent_class  definition cites only GOC:tfm
              and MESH:D009061, so nothing was searchable

Stage B was a real snippet_search run; Stage C could not be reached (issue #5).

Also ignore runs/CL_9999999, the hand-written synthetic fixture used to
exercise the equivalence route against a real reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ubyndr

ubyndr commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Ran the stage-2/3 agent against the routing this PR produces for cell-ontology#3717 — full reports below, also recorded in runs/CL_0000312, runs/CL_0002170, runs/CL_0002336.

All three land on term-level UNCERTAIN, but for three distinct, correctly-diagnosed reasons rather than the old "axiom discarded" dead end. Stage C (full-text escalation) is blocked repo-wide by #5, independent of this PR.

CL:0000312 — keratinocyte (refs_added) — 5 pass · 0 fail · 7 uncertain

Verifies the delta ref (PMID:21316034) only, per the new route — 5/12 assertions pass directly off it (oral AMP expression); the rest are pre-existing prose the new ref was never meant to cover.

Stage B ran against Asta snippet_search, scoped by paper_ids. Stage C (get_europepmc_full_text) returned empty output for every identifier tried (see issue #5), so unresolved core assertions could not be escalated to full text.

Routed targets

target_id route refs in scope where those refs come from
refs_added:CL:0000312:1 refs_added PMID:21316034 refs newly attached to the definition by this PR (delta only)

Assertions

# Assertion Cat Verdict
a1 Keratinocytes are epithelial cells of stratified squamous tissues. core uncertain
a2 Keratinocytes are present in skin. core pass
a3 Keratinocytes are present in oral mucosa. core pass
a4 Keratinocytes are present in esophagus. core uncertain
a5 Keratinocytes produce keratin proteins. core uncertain
a6 Keratinocytes secrete antimicrobial peptides. core pass
a7 Keratinocytes form a barrier against environmental damage. core uncertain
a8 Keratinocytes form a barrier against dehydration. core uncertain
a9 Keratinocytes form a barrier against pathogens and microbial invasion. core pass
a10 Keratinocytes undergo successive stages of differentiation marked by changes in keratin expression. core uncertain
a11 Keratinocyte differentiation supports tissue integrity and wound repair. core uncertain
a12 Keratinocytes contribute to immune defense. core pass

Supporting evidence

a2, a3, a6, a9, a12 — PMID:21316034:

In the oral cavity, mucosal keratinocytes resist bacterial infection, in part, by producing broad-spectrum antimicrobial peptides (AMPs) including defensin, adrenomedullin and calprotectin... HGKs from three subjects expressed eleven AMPs, including S100A7, S100A8, S100A9, S100A12, secretory leukocyte protease inhibitor, lipocalin 2 (LCN2), cystatin C and β-defensin 2.

Unresolved: a1, a4, a5, a7, a8, a11 — no snippet in the new ref addresses these claims. a10 — snippet says IL-1α regulates cell differentiation, but nothing about keratin-marked stages.

CL:0002170 — keratinized cell of the oral mucosa (text_revision) — 0 pass · 0 fail · 3 uncertain

Correctly scoped to PMID:12014572, but that ref is inEPMC=N and absent from Asta, so all 3 assertions land uncertain — a retrieval gap, not a routing bug.

Routed targets

target_id route refs in scope where those refs come from
text_revision:CL:0002170:1 text_revision PMID:12014572 refs on the revised definition axiom

Assertions

# Assertion Cat Verdict
a1 A keratinized cell of the oral mucosa is a keratinized buccal mucosa cell. core uncertain
a2 Keratinized cells of the oral mucosa are mostly located in the hard palate. core uncertain
a3 Keratinized cells of the oral mucosa are mostly located in the gingiva. core uncertain

Unresolved (not retrievable, 3): snippet_search returned 0 hits because PMID:12014572 is absent from the Asta corpus, and the paper is inEPMC=N so Stage C can't reach it either. The reference was in scope but could not be read.

CL:0002336 — buccal mucosa cell (text_revision + equivalent_class) — 0 pass · 0 fail · 4 uncertain (core)

The EquivalentTo axiom now survives (buccal mucosa cell EquivalentTo keratinocyte and (part of some mouth mucosa)) and is split into NECESSARY/SUFFICIENT assertions as designed, but the definition only carries GOC:tfm/MESH:D009061 — no searchable ref exists, so both directions are uncited.

Definition xrefs at head: GOC:tfm, MESH:D009061 — searchable: none

Assertions

# Assertion Cat Verdict
a1 Buccal mucosa cells are epithelial cells. core uncertain
a2 Buccal mucosa cells line the oral cavity, including the mucosa of the gums, palate, lip and cheek. core uncertain
a3 The term 'buccal mucosa cell' is sometimes used only for the lining of the cheeks, as part of the lining mucosa. background uncertain
a4 NECESSARY: Every buccal mucosa cell is a keratinocyte that is part of the mouth mucosa. core uncertain
a5 SUFFICIENT: Every keratinocyte that is part of the mouth mucosa is a buccal mucosa cell. core uncertain

Unresolved (uncited, all 5): no PMID:/DOI: id on the definition axiom to search against; the comment's only xref is a Scielo URL, not an accepted id format.

cc @dosumis

@ubyndr
ubyndr merged commit 2f5552c into main Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Route prose edits on existing terms; pass EquivalentTo axioms as reviewable assertions

1 participant