Skip to content
BranchesTags

Repository files navigation

poster-repo-to-json

Part of the Machine-Actionable Scientific Poster Initiative

Full pipeline for converting scientific posters into machine-actionable JSON. Uses poster2json for extraction (pdfplumber + Llama 3.1 8B), then enriches with Zenodo/Figshare repository metadata. Output conforms to the poster-json-schema (DataCite 4.7 with poster extensions).

This staging branch contains the refactored, production-hardened pipeline used to process the full Zenodo+Figshare corpus (~24K posters) on a multi-GPU node.

Pipeline Position

  1. Collect posters (poster-repo-scraper)
  2. Validate and classify (poster-repo-qc)
  3. Extract and enrich to machine-actionable JSON (this package)

How It Works

poster.pdf ──► poster2json ──► scaffold JSON ──► merge ──► final JSON
                                                   ▲
                                                   │
Zenodo/Figshare metadata ──► SchemaConverter ──────┘
                                          (curated fields win)

poster2json owns the poster content; the depositor owns the curated metadata. Extracted poster content (title, sections, captions, research field) is gospel. For fields the depositor curated on the repository record — the author list and ordering, the deposit description, funding, publication year, dates, rights, conference — repository metadata wins when present, and poster2json only enriches it (e.g. grafting resolved ORCID/ROR identifiers onto the curated authors). When the repository has nothing for a field, the extraction value is kept as a backfill.

Merge Rules

Field Source of truth
titles, content, imageCaptions, tableCaptions, researchField Extraction only (never overwritten)
conference Metadata supersedes extraction (LLM hallucinates conferences; repositories have authoritative data)
identifiers Metadata only (real poster DOI + record ID). DOIs found in poster text get moved to relatedIdentifiers with relationType: "References"
publicationYear, dates, fundingReferences Metadata wins when present; extraction backfills only when the deposit has nothing. Dates are normalized: publicationYear to a valid 4-digit year (garbage like 9999/future dropped), dates[].date to ISO 8601 (free-text conference dates parsed, junk like Not specified dropped)
rightsList Deposit-only. The license comes solely from the repository deposit (normalized: mit-license→MIT, cc-by-4.0→CC-BY-4.0, Zenodo other-* kept). If the deposit declares no license, any extraction value is dropped — the LLM must never set a license
creators Metadata wins for names and ordering (depositor-curated). Extraction enriches each creator with resolved ORCID (nameIdentifiers), ROR (affiliationIdentifier) and nameType — never overwriting a curated value
descriptions Repository deposit description is the primary Abstract. The poster2json LLM summary is retained but demoted to a secondary description of type Other. If the deposit has no description, the extraction summary stays as the Abstract
language, types, publisher Extraction base, metadata backfill if missing
subjects Union of both (deduped, case-insensitive)

nameType on converted Zenodo creators reflects the record's person_or_org.type when present (organizational → Organizational), defaulting to Personal.

Quality Enforcement

  • Post-batch QC (scripts/post_batch_qc.py) writes a TSV of failed extractions for retry (no-content, corrupt JSON, OCR failures).

Installation

# Clone with submodules
git clone https://github.com/fairdataihub/poster-repo-to-json.git
cd poster-repo-to-json
git checkout staging

# Install this package
pip install -e .

# Install poster2json as a sibling directory (patched version)
git clone https://github.com/fairdataihub/poster2json.git ../poster2json
cd ../poster2json && pip install -e .

Prerequisites

  • CUDA GPU with 16GB+ VRAM (or 6GB+ with 4-bit quantization)

PDF text extraction uses pdfplumber, which is installed with the Python dependencies and needs no separate system package.

Single-Poster Usage

# One-shot extraction via poster2json + merge with metadata
python run_extraction.py \
    --posters /path/to/poster/pdfs \
    --metadata /path/to/repo/metadata \
    --output ./output

Multi-GPU Batch Pipeline

The scripts/ directory contains the production batch pipeline used for large corpora:

scripts/
├── batch_extract_v2.py      # Two-phase extraction (text first, LLM second)
├── run_2gpu.sh              # Launcher for 2-GPU parallel processing
├── post_batch_qc.py         # Quality check + failed extraction log
├── run_merge.py             # Convert metadata + merge incrementally
└── clean_stale_errors.py    # Remove stale error files with recoverable cached text

Architecture: Two-Phase Extraction

Phase 1 (CPU+GPU): Extract raw text from all posters.

  • pdfplumber for text-based PDFs (fast, CPU-only)
  • Vision-model OCR fallback for image-based PDFs (GPU)
  • Raw text cached to disk (_raw_text/) so it survives crashes

Phase 2 (GPU): Load Llama 3.1 8B once, process all texts sequentially.

  • Model stays resident (no reload overhead between posters)
  • 4-bit NF4 quantization by default (~5GB VRAM, quality preserved)
  • Individual saves per poster (crash-safe resume)

Running on 2 GPUs

# Set poster2json location
export POSTER2JSON_PATH=/path/to/poster2json

# Split input across GPUs (zenodo first, figshare second)
# Place symlinks in ./gpu_splits/gpu0, gpu1, gpu2, gpu3

# Launch (runs gpu0+gpu1 first, then gpu2+gpu3 after they finish)
bash scripts/run_2gpu.sh

Running on 3 GPUs (with 4-bit quantization)

With the JSON model in 4-bit NF4 (~5GB VRAM, default in poster2json 0.4.x), three RTX 3090s at 250W each pull ~600W combined under load — within the same circuit budget as the old 2-GPU bfloat16 setup. The third GPU adds ~50% throughput.

# Cap all four GPUs to 250W (idempotent, runs from Windows side on WSL)
nvidia-smi -i 0 -pl 250
nvidia-smi -i 1 -pl 250
nvidia-smi -i 2 -pl 250
nvidia-smi -i 3 -pl 250

# Launch: 3 GPUs in parallel on splits 0,1,2; then GPU 0 picks up split 3
bash scripts/run_3gpu.sh

Why not 4 GPUs? Running all four RTX 3090s at full load (350W each) on a single consumer circuit trips overcurrent protection. Three at 250W stays within a standard 15A/20A circuit. Four at 250W is right at the edge — possible, not advised. nvidia-smi -pl 250 limits draw.

Post-Batch Quality Check (default)

QC runs automatically after every batch in both run_2gpu.sh and run_3gpu.sh — no manual step required. Each run writes the failure list to failed_extractions.tsv so you can retry the bad ones.

You can also invoke it standalone:

python scripts/post_batch_qc.py
# Writes: /path/to/output/extractions/../failed_extractions.tsv

Checks for:

  • multi_description — LLM dumped section content into descriptions instead of content.sections
  • no_content — no sections extracted
  • corrupt_json — mid-write crash
  • extraction_error — Phase 1 failures (OCR couldn't recover)

Backfilling New poster2json Features

When poster2json ships new enrichment features (e.g. SPDX license normalization, ROR affiliation IDs, heuristic language detection, researchField on the OpenAlex 4 domains), existing extraction JSONs can be updated in-place without re-running the LLM:

export POSTER2JSON_PATH=/path/to/poster2json
python scripts/backfill_features.py --extractions /path/to/output/extractions

The backfill is idempotent and reuses cached raw text from _raw_text/ for language detection. It applies:

  • SPDX license normalization (rightsList entries get canonical SPDX IDs + URIs)
  • Subject dedupe + NFKC cleanup
  • ROR enrichment on creators.affiliation and publisher (canonical names + ROR IDs)
  • Heuristic language detection from raw text (overrides LLM-hallucinated language)
  • researchField placeholder strip (drops "Other", "Unknown", etc. → null)
  • NFKC normalization on titles and descriptions

The feature modules (normalize.py, ror.py, language.py) are vendored in vendor/poster2json_features/ for reference.

Stale Error Recovery

If earlier runs had bugs (OOM crashes, extraction failures, etc.), those stale error JSONs block re-processing. This script removes them if the raw text is cached (Phase 1 succeeded but Phase 2 failed):

python scripts/clean_stale_errors.py
# Re-running the pipeline will reprocess them with instant Phase 1 (cache hit)

Incremental Merging

python scripts/run_merge.py
# Converts repository metadata + merges with available extractions.
# Safe to re-run — only processes new files.

Entity normalization & synonym clustering

Free-text organizational fields (publisher, fundingReferences.funderName, creators.affiliation, subjects) arrive in dozens of spelling and formatting variants for the same real entity. A corpus-wide pass collapses those variants to a canonical form so the same institution, funder, or subject reads consistently across records. The full settings live in docs/SYNONYM_NORMALIZATION.md.

The approach ("synonym-lustre", scripts/post_processing/build_synlustre.py) embeds the distinct terms of a field with gte-large, clusters them with HDBSCAN, and maps every cluster member to the most frequent variant (embedding-centroid distance breaks ties). Two guards keep the merge conservative:

  • ROR split. A semantic cluster that spans two or more distinct RORs (e.g. University of Washington vs Washington University) is partitioned by ROR before a canonical is chosen, so distinct institutions never collapse together. ROR-less members attach to the nearest ROR sub-centroid.
  • Acronym / short-token holdout. Terms under 4 characters and all-caps single tokens (ZHAW, LUH, GTC) carry too little signal to place reliably, so they are held out of clustering and map to themselves — only distinguishable multi-word / mixed-case names are merged.

Conference locations are normalized separately by geocoding (conference_location_geocode.py, Nominatim/OSM): variant strings that resolve to the same real place collapse, while genuinely different cities stay apart (a string embedding would wrongly merge "Graz" and "Vienna" as both "city in Austria"). Clustering quality is guarded by a V-measure validation harness (validate_vmeasure.py), which scores homogeneity/completeness against a gold set to pick and regression-check the epsilon per field.

Per-field HDBSCAN cluster_selection_epsilon settings (default 0.35; the V-measure harness sweeps 0.20–0.50 to select each field's value — see the doc for the tuned numbers):

  • publisher — semantic cluster + most-frequent canonical, plus an LLM junk-cleaning pass on the resulting names.
  • funder — semantic cluster; duplicate funderName/awardNumber pairs collapse after mapping.
  • affiliation — semantic cluster with ROR split and acronym holdout; the strongest guard against cross-institution merges.
  • subject — semantic cluster, deduped case-insensitively; PCA is an opt-in --pca-dims flag for very large fields (e.g. subject), kept at many dims to preserve granularity.

CLI Commands

# Single-step commands (wraps the Python API)
poster-to-json extract --input ./posters --output ./extractions
poster-to-json convert --input ./metadata/zenodo --output ./converted --source zenodo
poster-to-json merge --extractions ./extractions --metadata ./converted --output ./merged
poster-to-json pipeline --posters ./posters --metadata ./metadata --output ./output

Python API

from poster_to_json import PosterExtractor, SchemaConverter, MetadataMerger

extractor = PosterExtractor()
extraction = extractor.extract("poster.pdf")

converter = SchemaConverter()
metadata = converter.convert_zenodo(raw_zenodo_record)

merger = MetadataMerger()
complete = merger.merge(extraction, metadata)

Output Schema

Conforms to the poster-json-schema (DataCite 4.6 with poster extensions):

{
  "$schema": "https://posters.science/schema/v0.1/poster_schema.json",
  "identifiers": [{"identifier": "10.5281/zenodo.12345678", "identifierType": "DOI"}],
  "titles": [{"title": "..."}],
  "creators": [{"name": "Smith, John", "nameIdentifiers": [...], "affiliation": [...]}],
  "descriptions": [{"descriptionType": "Abstract", "description": "..."}],
  "content": {
    "sections": [
      {"sectionTitle": "Introduction", "sectionContent": "..."},
      {"sectionTitle": "Methods", "sectionContent": "..."},
      {"sectionTitle": "Results", "sectionContent": "..."}
    ]
  },
  "imageCaptions": [{"caption": "Figure 1: ..."}],
  "tableCaptions": [{"caption": "Table 1: ..."}],
  "conference": {"conferenceName": "...", "conferenceYear": 2025},
  "relatedIdentifiers": [{"relatedIdentifier": "10.1234/...", "relatedIdentifierType": "DOI", "relationType": "References"}]
}

Directory Structure

poster-repo-to-json/
├── src/poster_to_json/
│   ├── extractor.py         # Thin wrapper around poster2json
│   ├── schema_converter.py  # Zenodo/Figshare → poster_schema.json
│   ├── merger.py             # Extraction base + metadata backfill (conference superseding, single description)
│   ├── cli.py                # CLI commands
│   └── poster_schema.json    # Bundled schema
├── scripts/                  # Production batch pipeline
│   ├── batch_extract_v2.py
│   ├── run_2gpu.sh
│   ├── post_batch_qc.py
│   ├── run_merge.py
│   └── clean_stale_errors.py
├── run_extraction.py         # Single-instance batch extraction
├── posters/                  # Input: classified poster PDFs
├── metadata/                 # Input: per-record JSON metadata
└── output/
    ├── extractions/          # poster2json raw output (+ _raw_text/ cache)
    ├── converted/            # Metadata converted to schema
    └── merged/               # Final merged records

Operational Notes

  • Windows Task Scheduler is used to launch run_2gpu.sh so it survives SSH disconnects: schtasks /create /tn PosterExtraction /tr "wsl.exe -d Ubuntu-24.04 -e bash /path/to/run_2gpu.sh" /sc once /st 00:00 /f /rl highest && schtasks /run /tn PosterExtraction
  • Power limits (nvidia-smi -pl 250) reset on reboot — re-apply after any power event.
  • Vision OCR needs torchvision and cuDNN is disabled at the module level to avoid a driver incompatibility on CUDA 12.9 + cuDNN 9.19.

Related Packages

Package Purpose
poster2json Core extraction engine (pdfplumber + Llama 3.1)
poster-repo-scraper Scrape poster metadata from Zenodo/Figshare
poster-repo-qc Validate and classify posters with PosterSentry
poster-json-schema DataCite 4.6-based schema for scientific posters

License

MIT License - See LICENSE for details.

Citation

@software{poster_repo_to_json,
  title = {poster-repo-to-json: Machine-Actionable Scientific Poster Pipeline},
  author = {{FAIR Data Innovations Hub}},
  year = {2026},
  url = {https://github.com/fairdataihub/poster-repo-to-json}
}

About

Extract scientific poster content to machine-actionable JSON using LLMs. Part of the Machine-Actionable Poster Initiative (Beta).

Resources

Readme
MIT license
Activity
Custom properties

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages