From 4638cd56c42029a1a6e883b1085846f56caaec10 Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Thu, 29 Oct 2026 16:33:50 +0000 Subject: [PATCH] tasks.md: T020 merged as #79, which replaced #78 Co-Authored-By: Claude Opus 5.5 --- specs/001-biopax-from-graph/differences.md | 13 +++++++++++-- specs/001-biopax-from-graph/tasks.md | 2 +- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/specs/001-biopax-from-graph/differences.md b/specs/001-biopax-from-graph/differences.md index eb2db15..842edd5 100644 --- a/specs/001-biopax-from-graph/differences.md +++ b/specs/001-biopax-from-graph/differences.md @@ -105,7 +105,7 @@ Measured 29 Oct 2026 on _P. falciparum_, from a private copy of release 98's gra - **Every event agrees in shape:** each of _P. falciparum_'s 1,630 events has the published file's PubMed and ISBN ids, curation comments by kind, summation comments and Evidence count. Its 5,750 PublicationXrefs have the published file's properties: 5,689 by PubMed id, 42 by ISBN, 11 web pages and 8 with no id. The fixture's 57 human events agree in the same four respects with release 98's human file, where 15,712 of 16,159 reactions have literature of their own and none is inferred. There is one Provenance, as in the published files. - **What still differs at the root, here:** - _Evidence comments (128):_ white space at either end, which the release's writer trims (T018). One of them also differs within, which trimming does not explain: T018 looks at it with the writer. - - _Authors (60 values in 23 of 5,750 PublicationXrefs):_ the release's form of a person's name is fuller than the graph's display name. Each writes the name its own database holds; the data, not the code, differs, so it is recorded, not pursued. + - _Authors (60 values, in 4 of 5,750 PublicationXrefs):_ the release's form of a person's name is fuller than the graph's display name. Each writes the name its own database holds; the data, not the code, differs ("Authors' names", below). The 23 PublicationXrefs first recorded here are not what `ci/compare.py` reports now: 4, each with its own db, id, year, title and source. - **Repeated values:** the published file writes an author value twice in some lists (two people of one name, its converter's comment says); Paxtools keeps each value once, which `ci/compare.py` counts once too. - **Counts:** triples only in the published file fall from 146,281 to 72,683, and only in the converted file from 9,842 to 7,959. The validator gives 0 errors on the fixture's output, with 501 warnings for human and 28 for _P. falciparum_ (558 and 32 at T015). - **The empty Evidence** every event had is gone, with `BioPAX3ReferenceUtils`, which stripped the HTML. @@ -183,11 +183,20 @@ Measured 29 Oct 2026: every species converted from a private copy of release 98' - `conversionDirection`, written as Paxtools writes it (95,355; T012's decision). - A name equal to the display name (28,267, nearly all on proteins of inferred species, whose graph name lists repeat a name): the published files list it among the names too, Paxtools writes it only as the display name (T011). In _Gallus gallus_ all 5,921 are this. - Cross-references the published files lack (entities 36,717, events 379; their RelationshipXrefs 4,427), nearly all of two causes: the graph's later data (Mondo, COSMIC and ClinGen, added after the relational database the release read: T011), and those the release's converter skipped after the first one it had already written (DOID, SYMP, ChEBI, COMPOUND, EMBL: T011, T014). In human, 269 on reactions and 110 on pathways, T014's 379. A few more of other databases, whose cause was not traced one by one: PubChem Substance (24, in 14 species), and in human NCBI Nucleotide (3), RNAcentral (2), ENSEMBL, NCBI Gene and EC (1 each). Every published cross-reference is among the converted ones. - - The form of an author's name (872 values; T016). + - The form of an author's name (872 values, all of one kind: "Authors' names", below). - An empty value the release wrote as an empty element, which Paxtools does not write: a ProteinReference's comment (38), a DNA or RNA reference's name (3, in human). - No `eCNumber` difference: the one reactome/graph-importer#9 records is not in release 98 (T012). - **The validator:** no file has an error its published one lacks (SC-003). The errors are the same, `multi.organism.pathway` (3 in _Gallus gallus_, 207 in human), and so are the warnings, rule by rule, but one more `unknown.db` in human. +## Authors' names + +Measured 29 Oct 2026 on all 15 species, from a private copy of release 98's graph, against release 98's published files: each PublicationXref matched to its graph publication (by PubMed id, else ISBN, else title) and its authors compared in order with that publication's, Person by Person. Names were read only to classify them; none is recorded here. + +- **One kind, 872 values:** in every one, the published value is the Person's surname and first name, and the graph's display name its surname and initials, both of the same Person, whose own `firstname` and `initial` hold each form. No other kind: no accent, order, missing initial or different person. +- **Few people:** 62 Persons, authors of 10 publications. The values repeat because those publications are in many species' files: 60 in _P. falciparum_, 67 in human, 20 in _Danio rerio_, 53 in _Caenorhabditis elegans_, 60 or 63 in each of the rest. The other authors of those publications agree. Six of the 10 publications were created in the relational database in July and August 2026, in release 98's curation, and 60 of the 62 Persons have database ids among theirs. +- **Why they differ:** the graph's display name is the relational `_displayName` as the graph importer read it (`ReactomeBatchImporter` copies `getDisplayName()` and builds nothing); the release's converter wrote `_displayName` too. So the two read the same field at different states, and for these 62 people it had changed between them. The graph alone cannot say which change, or when. +- **Which form is meant:** in the graph, every Person but one (a script's) has `surname, initial` as its display name, 145,658 of them with a first name too; and in the published files every other author has that form. But the curator tool's own rule (`InstanceDisplayNameGenerator.generatePersonName`) puts the first name first, and the initials only when there is none, so a Person it named and nothing renamed has the first-name form. Neither the exporter nor the graph importer is wrong: the exporter writes the graph's display name, as it should, and the importer copies the field. The two forms are the data's, and which one Reactome means is a question for those who own it (T022). + ## Paxtools 6, Kotlin 2.4, JUnit 6 (#66) Measured 29 Oct 2026. The fixture's output is unchanged to the byte, and _P. falciparum_ from a private copy of release 98's graph is unchanged by `ci/compare.py`. Its bytes differ in one way only: Paxtools 4.3.1 wrote a character outside ASCII as a character reference (`−`, 8 in the file), and Paxtools 6.0.0 writes it as itself, as release 98's published file does (937 lines with such characters, no references). The two behaviours recorded above as Paxtools 4.3.1's, the underscored `conversionDirection` and a product added to the participants, are 6.0.0's too. `Control.controlled` is deprecated (in Paxtools 5.1's javadoc, and by annotation in 6.0) with no replacement yet; each control here has one process, as the release writes it. Paxtools 6 brings slf4j-api 2, which the library leaves out, as it does graph-core's logging binding: graph-core brings slf4j-api 1.7, and the command line's logback 1.2 binds to that. diff --git a/specs/001-biopax-from-graph/tasks.md b/specs/001-biopax-from-graph/tasks.md index 87d8fa1..4e9cd80 100644 --- a/specs/001-biopax-from-graph/tasks.md +++ b/specs/001-biopax-from-graph/tasks.md @@ -318,7 +318,7 @@ description: 'Tasks for spec 001: BioPAX Level 3 from the graph, as the release ## Phase 5: Proof (User Stories 2 and 3) -- [x] T020 [US2] The full run: every species from a private copy of release 98's graph, the same 15 file names as the published zip, each compared by T005. `differences.md` records each remaining difference by kind, count and reason (order and ids; the data's, with its ticket, the EC numbers first, reactome/graph-importer#9; an obvious bug fixed), and nothing else. Validated by T004 against each published file's own baseline, made in this task for all 15 (human with enough memory for its size, measured). The private copy removed after. — #78. +- [x] T020 [US2] The full run: every species from a private copy of release 98's graph, the same 15 file names as the published zip, each compared by T005. `differences.md` records each remaining difference by kind, count and reason (order and ids; the data's, with its ticket, the EC numbers first, reactome/graph-importer#9; an obvious bug fixed), and nothing else. Validated by T004 against each published file's own baseline, made in this task for all 15 (human with enough memory for its size, measured). The private copy removed after. — #79, replacing #78 after its review. - **Found on the way (#76):** the first full run spent its time reading every object of the model for the one Provenance each individual names (`getObjects` filters the whole model), quadratic in a species' size; the Provenance is now made once per file and kept in the context, and the helpers that scanned are removed. Also, from #75's review: an output directory that is a file, or cannot be made, is said so; it is made only when the arguments are sound; a parsing failure exits 1. - **Found on the way (#77):** human then ran out of a 4 GB heap. Two causes, measured with class histograms: the conversion held every event it had written (the steps kept their events, and the species' top-level events were held in one list), so all that was loaded beneath them stayed; and the file was made in memory, then copied three times. A step now keeps the ids of the events before it, the top-level events are loaded one at a time, and the file is written as it is made, carriage returns escaped on the way. Human converts in 15 min 44 s, at most 1.7 GB resident; _P. falciparum_'s peak falls from 983 MB to 460 MB; the output is the same to the byte. - **Measured** (`differences.md`, the full run): every species in 1 h 55 min; the 15 published files by name and a sixteenth, _M. tuberculosis_, which the release step's species configuration leaves out (T023 decides what the release command writes). What differs at the root falls in five kinds, each recorded with its reason; no file has a validator error its published one lacks. The validator's baseline is made for all 15. Two of the task's conditions are carried: the private copy is kept for T021 and removed after it, and the sixteenth file is T023's.