Skip to content

Commit 01d73f2

Browse files
committed
Extend the VLM scrub: image placeholders, LaTeX, footer logos
Three more markup classes the model emits, on top of the HTML/fence scrub: - Image placeholders. Every figure the model cannot read becomes ![image](image_N.png), or an alt that reads the chart data itself ("Pie chart ... Malware (19,224)"). Both are the data-within-an-image we skip; the real caption is separate prose, so the whole marker is dropped. Present in 15 of 20 outputs. - LaTeX. The model writes superscripts, subscripts and math as LaTeX: $^{1,2}$ (95+ occurrences) are the author AFFILIATION markers and must survive as their digits, so $^{1,2}$ -> 1,2 keeps the author-to- institution mapping and makes the byline readable ("Aydan Gasimova1, Sanjay Soundarajan1, Nayoon Gim2,3,4 ..."). Figure math ($\mu_1$, $\alpha$, $\lambda 5007$) is mapped to unicode/plain text. Currency ($600) is safe: it has no closing $, so the paired patterns skip it. - Footer logos. A poster's foot carries sponsor/institution logos, which come out as trailing bold or ALL-CAPS name lines ("**FAIR DATA INNOVATIONS HUB**", "**AI-READI**"). They are branding, absent from the human transcription, and only dilute the tail, so they are peeled off the END only -- a bold heading mid-document is untouched. Corpus effect (scrubbed vs raw VLM): rGlobal 0.775 -> 0.816, rField 0.765 -> 0.784. Against the current pipeline the scrubbed VLM now wins rField 14 of 19 (was 13). gasimova's byline is clean and its four footer logo lines are gone (+0.016 on that poster). No new per-poster regressions; 8228568 remains the single data-table casualty from the earlier commit.
1 parent 1b216a6 commit 01d73f2

4 files changed

Lines changed: 222 additions & 75 deletions

File tree

0 commit comments

Comments
 (0)