Impresso NER EDA

Tools for preprocessing, evaluating, and analysing Named Entity Recognition (NER) and Entity Linking (EL) baseline outputs for the Impresso HIPE corpora.

This package is focused on the analysis workflow used in the impresso-named-entity-data-augmentation project. It prepares HIPE gold data, normalises baseline prediction runs, computes entity-level metrics across multiple seeds, and powers the notebooks used for profiling and error analysis.

What this folder contains

• scripts/ for the end-to-end preprocessing and evaluation pipeline.
• src/eda/ for reusable code used by the scripts and notebooks.
• notebooks/ for exploratory analysis, dataset profiling, and error pattern studies.
• data/ for cached intermediate artefacts and processed outputs.

Installation

From this directory:

pip install -e .

For notebook and visualisation support:

pip install -e ".[dev,viz]"

The project targets Python 3.11 or newer.

Required data layout

The scripts expect the following sibling repositories to be available next to this folder unless you override them with environment variables:

• HIPE-2022-data/
• CLEF-HIPE-2020-eval/

By default, the scripts resolve data from these paths:

• HIPE_DATA_DIR=/path/to/HIPE-2022-data/data/v2.1/hipe2020
• CLEF_HIPE_DIR=/path/to/CLEF-HIPE-2020-eval/data/release/v1.3

Set either variable explicitly when your checkout is stored elsewhere.

Workflow

The preprocessing pipeline is split into three steps.

1. Enrich HIPE v2.1 files with LED scores

The first script copies the LED column from HIPE v1.3 metadata into the v2.1 TSV files in place.

python scripts/00_enrich_hipe_with_led.py --lang fr
python scripts/00_enrich_hipe_with_led.py --lang de

The command is idempotent: files that already contain LED are skipped.

2. Normalise baseline prediction runs and build pair files

This step creates canonical symlinks for baseline runs, loads gold data, enriches predictions, and writes processed parquet files for the selected annotation schemes.

python scripts/01_setup_baseline_predictions.py --lang fr
python scripts/02_preprocess_baseline_predictions.py --lang fr
python scripts/02_preprocess_baseline_predictions.py --lang de --with-ocr

Useful options:

• --models to process only selected model keys.
• --tasks to restrict the annotation schemes.
• --with-ocr to compute OCR quality scores for gold data only.
• --force to recompute cached gold and pair files.
• --dry-run to inspect the planned actions without writing files.

3. Aggregate metrics across seeds

Once pair files exist, compute the final summary metrics:

python scripts/03_compute_baseline_metrics.py --lang fr
python scripts/03_compute_baseline_metrics.py --lang de

Output structure

The main generated artefacts are written under data/processed/:

• data/processed/gold/{lang}/{model_type}/ for cached gold splits and corpus statistics.
• data/processed/baseline_results/{lang}/{model_key}/seed{seed}/ for enriched predictions and pair files.
• data/processed/baseline_metrics_{lang}.parquet for aggregated metrics across seeds.

The preprocessing step produces three pair files per run:

• pairs.parquet for coarse literal evaluation.
• ne_fine_lit_pairs.parquet for fine-grained literal evaluation.
• ne_coarse_meto_pairs.parquet for metonymic evaluation.

Notebooks

The notebooks are organised by analysis stage:

• 1_baseline_selection/ for configuration comparison.
• 2_hipe_profiling/ for dataset and entity-level profiling.
• 3_error_patterns/ for multi-seed performance analysis and error pattern exploration.

Notes

• The package name is impresso-ner-eda.
• The code is designed around the French and German HIPE 2020 tracks.
• Heavy artefacts are generated locally and are not meant to be edited by hand.
• The OCR quality model used by --with-ocr is impresso-project/ocr-quality-assessor-unigram-light.

---

License

This work is licensed under the GNU Affero General Public License v3.0.

About

This work stems from the Master thesis of Léa Blinière (Fall 2026). It was conducted in the context of Impresso - Media Monitoring of the Past, a interdisciplinary research project that aims to develop and consolidate tools for processing and exploring large collections of media archives across modalities, time, languages and national borders. The first project (2017-2021) was funded by the Swiss National Science Foundation under grant No. CRSII5_173719 and the second project (2023-2027) by the SNSF under grant No. CRSII5_213585) and the Luxembourg National Research Fund under grant No. 17498891.

Copyrights

Copyright (C) 2026 Léa Blinière (Master's Student, EPFL) and the Impresso Team.

---