Commit 01d73f2
committed
Extend the VLM scrub: image placeholders, LaTeX, footer logos
Three more markup classes the model emits, on top of the HTML/fence scrub:
- Image placeholders. Every figure the model cannot read becomes
, or an alt that reads the chart data itself
("Pie chart ... Malware (19,224)"). Both are the data-within-an-image
we skip; the real caption is separate prose, so the whole marker is
dropped. Present in 15 of 20 outputs.
- LaTeX. The model writes superscripts, subscripts and math as LaTeX:
$^{1,2}$ (95+ occurrences) are the author AFFILIATION markers and must
survive as their digits, so $^{1,2}$ -> 1,2 keeps the author-to-
institution mapping and makes the byline readable ("Aydan Gasimova1,
Sanjay Soundarajan1, Nayoon Gim2,3,4 ..."). Figure math ($\mu_1$,
$\alpha$, $\lambda 5007$) is mapped to unicode/plain text. Currency
($600) is safe: it has no closing $, so the paired patterns skip it.
- Footer logos. A poster's foot carries sponsor/institution logos, which
come out as trailing bold or ALL-CAPS name lines ("**FAIR DATA
INNOVATIONS HUB**", "**AI-READI**"). They are branding, absent from the
human transcription, and only dilute the tail, so they are peeled off
the END only -- a bold heading mid-document is untouched.
Corpus effect (scrubbed vs raw VLM): rGlobal 0.775 -> 0.816, rField 0.765
-> 0.784. Against the current pipeline the scrubbed VLM now wins rField
14 of 19 (was 13). gasimova's byline is clean and its four footer logo
lines are gone (+0.016 on that poster). No new per-poster regressions;
8228568 remains the single data-table casualty from the earlier commit.1 parent 1b216a6 commit 01d73f2
4 files changed
Lines changed: 222 additions & 75 deletions
0 commit comments