Skip to content

Commit 9e04b54

Browse files
lgruen-cpglgruen-cpg
authored
Render a BITS book-part-wrapper (one NCBI Bookshelf chapter) (#10)
Europe PMC serves a Bookshelf chapter (a GeneReviews entry, say) as BITS Book Interchange XML: a <book-part-wrapper> holding <book-meta> and one unit of the book, normally a <book-part> whose <book-part-meta> carries the chapter's title and abstracts and whose <body>/<back> share the JATS Archiving content model. convert() dispatches that root to jats.render_book_part_wrapper, which renders the part's labelled title and subtitles as the H1 and each abstract through render_abstract in place of an article's <front>, then hands the body and back to the article code path. A wrapper holding anything but a single <book-part> (a <book-app>, a <preface>, a <book-part> beside a sibling unit, or nothing) raises ValueError naming what it holds. Both entry points share _render_document, so the rule separators and the split-superscript collapse apply alike; a section that renders nothing leaves no rule. BITS lets a part's <body> end in further <book-part>s (a Bookshelf part whose chapters nest). render_body renders them through the same pieces, one heading level down per nesting: every heading is built by _heading, which clamps at Markdown's six, and render_body, render_abstract, render_back, render_ref_list, render_fn_group, render_glossary, render_p and render_boxed_text take the level they head at (default 2, the article layout). <subtitle> joins the title after a colon in an article's and a book part's H1 alike, through one title-group helper. Bookshelf chapters nest their <ref-list> inside a body <sec> (References > Literature Cited), which render_sec dropped along with every reference. JATS Archiving permits <ref-list> in <sec>, so render_sec renders it: the enclosing section's title heads the list, and the list's own <title> adds a heading one level down only when it differs from the section's. A <ref> without a <label> is numbered from its id only when the id is a bare number or a PMC-style B-number; an opaque id (Springer's CR45, Elsevier's bib7, Bookshelf's brca1.REF.doe.2020) leaves the citation unlabelled behind its anchor instead of heading it with the id itself. Bookshelf table cells carry raw newlines and indentation. inline_to_md already flattens ASCII whitespace for every input and md_escape_cell trims it, so no book-specific handling is needed; a test pins the invariant for that cell shape. Bump version to 0.6.0 and refresh uv.lock for it.
1 parent 5f0102d commit 9e04b54

7 files changed

Lines changed: 614 additions & 99 deletions

File tree

‎CLAUDE.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# litdown development notes
22

3-
litdown converts scholarly full-text XML (JATS, Elsevier) to Markdown. See
4-
[`README.md`](README.md) for usage and the supported dialects.
3+
litdown converts scholarly full-text XML (JATS — journal articles and BITS book
4+
parts — and Elsevier) to Markdown. See [`README.md`](README.md) for usage and the
5+
supported dialects.
56

67
## Working norms
78

‎README.md‎

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,10 @@ Convert scholarly full-text XML to Markdown with embedded LaTeX for inline and
66
display math. Two dialects are supported behind a single `convert` entry point
77
that sniffs the document root and dispatches:
88

9-
- **JATS** (`<article>`) — the format PubMed Central distributes.
9+
- **JATS** — a journal article (`<article>`, the format PubMed Central
10+
distributes) or a BITS book part (`<book-part-wrapper>`, one NCBI Bookshelf
11+
chapter as Europe PMC's `bookXML` endpoint serves it; a GeneReviews entry,
12+
say). The chapter's `<book-part-meta>` stands in for `<front>`.
1013
- **Elsevier** (`<full-text-retrieval-response>`) — the ScienceDirect Article
1114
Retrieval API's `xocs`/`ja`/`ce` schema.
1215

@@ -26,6 +29,15 @@ This is **not** the Article Authoring tag set (more restrictive; intended as
2629
an authoring target, not a corpus). Article-Authoring-only content is a
2730
subset of Archiving content and works without code changes.
2831

32+
**BITS** (Book Interchange Tag Suite 2.x; content models checked against the
33+
2.1 Tag Library) is the book counterpart: a `<book-part>`'s `<body>` and
34+
`<back>` share the Archiving content model, so the same renderer handles them
35+
and reads the chapter's title and abstracts from `<book-part-meta>`. A `<book-part-wrapper>` holds one unit of a
36+
book, and only a `<book-part>` is rendered — a wrapper around a `<book-app>`,
37+
`<preface>`, `<glossary>` or other unit raises `ValueError`. A part whose body
38+
nests further parts renders them as sections, one heading level down per
39+
nesting.
40+
2941
The **Elsevier** dialect targets the `ce:`/`ja:`/`xocs:` schema returned by
3042
the ScienceDirect Article Retrieval API. Math is standard W3C MathML (shared
3143
with the JATS math path); tables are CALS (`tgroup`/`row`/`entry`); references
@@ -57,7 +69,7 @@ Library:
5769
```python
5870
from litdown import convert, mml_to_tex, render_mathml
5971

60-
md = convert("article.xml") # JATS or Elsevier XML path → markdown
72+
md = convert("article.xml") # JATS (article or book part) or Elsevier → markdown
6173
latex = mml_to_tex(math_element) # MathML Element → LaTeX
6274
fragment = render_mathml(math_element, display=True) # → "$$...$$"
6375
```
@@ -84,7 +96,7 @@ the regression suite re-runs the converter over them on every test run.
8496
pytest # full suite
8597
```
8698

87-
Three test files:
99+
Six test files:
88100

89101
- `tests/test_mml_unit.py` — exhaustive per-element MathML cases.
90102
- `tests/test_jats_articles.py` — structural assertions over real PMC
@@ -98,6 +110,15 @@ Three test files:
98110
(math not dropped, CALS tables rendered, every cross-ref/float/reference
99111
anchored). Vendor only CC-BY (`by/4.0`) articles; see
100112
`docs/elsevier-dialect-plan.md` for how to harvest fixtures.
113+
- `tests/test_book_part_unit.py` — hand-written BITS `<book-part-wrapper>`
114+
documents (Bookshelf prose is not redistributable): the title group,
115+
abstracts, nested parts, refusal of other wrapper units, reference labels,
116+
and a `<ref-list>` inside a `<sec>`.
117+
- `tests/test_convert_inputs.py` — `convert` accepts a path, raw bytes or a
118+
binary stream and yields the same Markdown, for both dialects.
119+
- `tests/test_whitespace_unit.py` — single-line Markdown constructs (headings,
120+
table rows, list items) survive pretty-printed source, one hand-written
121+
document per construct.
101122

102123
### Fetching test fixtures
103124

@@ -160,9 +181,10 @@ real defect the eval surfaces should be added to
160181
- The consortium author rendering for papers like gnomAD (PMC7334197)
161182
emits the consortium *name* only; individual members listed in nested
162183
`<contrib-group>` are dropped.
163-
- Some end-of-article metadata sections (Author contributions, Competing
164-
interests, Funding, Data availability) live inside `<fn-group>` or
165-
`<notes>` in `<back>`; these aren't currently rendered.
166184
- Soft hyphens / line-break artefacts in source XML are not normalised,
167185
so words split across lines in the JATS source can render with stray
168186
spaces ("si milarity").
187+
- A BITS `<book-part-meta>` yields the labelled title, its subtitles and the
188+
abstracts only; contributors, `<alt-title>`, translated titles, keyword
189+
groups, funding and notes are not rendered, nor are the enclosing
190+
`<book-meta>` and a part's own `<front-matter>`.

‎litdown/__init__.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@
33
Ships two dialects behind a single :func:`convert` entry point, which
44
sniffs the document root and dispatches:
55
6-
* **JATS** (``<article>``) — PMC / NLM full text, via :mod:`litdown.jats`.
6+
* **JATS** — a journal ``<article>`` (PMC / NLM full text) or a BITS
7+
``<book-part-wrapper>`` (one NCBI Bookshelf chapter, as Europe PMC serves
8+
it), via :mod:`litdown.jats`.
79
* **Elsevier** (``<full-text-retrieval-response>``) — the ScienceDirect
810
Article Retrieval API's ``xocs``/``ja``/``ce`` schema, via
911
:mod:`litdown.elsevier`.
@@ -59,6 +61,8 @@ def convert(source: str | pathlib.Path | bytes | IO[bytes]) -> str:
5961
name = common.get_tag(root)
6062
if name == 'article':
6163
return jats.render(root)
64+
if name == 'book-part-wrapper':
65+
return jats.render_book_part_wrapper(root)
6266
if name == 'full-text-retrieval-response':
6367
return elsevier.render(root)
6468
raise ValueError(f'unrecognized root element: {root.tag}')

0 commit comments

Comments
 (0)