This repo contains the Benchmark of Loanwords and Anglicisms in Spanish (BLAS).
BLAS is a collection of 1,836 annotated journalistic sentences in Spanish (37,344 tokens) that contains 2,076 spans labeled as anglicisms.
Each sentence in BLAS is characterized by the linguistic attributes of the span it contains (in terms of shape, the context it appears in, its position within the sentence, its casing, etc.).
When models are evaluated on these sentences, scores can be computed over subsets of the data that share the same span attributes, which enables ascribing the resulting score to concrete linguistic attributes.
BLAS was the test set used at the second edition of ADoBo (Automatic Detection of Borrowings), the shared task on anglicism identification held at IberLEF 2025. See competition site and paper with overview and results of the task.
More information about the dataset, model experimentation and diagnostic capabilities of the dataset can be found in the paper: Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling Tasks.
If you use this dataset, please cite the following references:
@inproceedings{alvarezmellado-etal-2026-diagnostic,
title = {Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling Tasks},
author = {Alvarez-Mellado, Elena and Gonzalo, Julio},
booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
month = {May},
year = {2026},
pages = {5938--5959},
address = {Palma, Mallorca, Spain},
publisher = {European Language Resources Association (ELRA)},
editor = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
doi = {10.63317/3f4yxedpym6a},
abstract = {Standard evaluation in NLP typically indicates that system A is better on average than system B, but it provides little info on how to improve performance and, what is worse, it should not come as a surprise if B ends up being better than A on outside data. We propose an evaluation methodology for sequence labeling tasks grounded on error analysis that provides both quantitative and qualitative information on where systems must be improved and predicts how models will perform on a different distribution. The key is to create test sets that, contrary to common practice, do not rely on gathering large amounts of real-world in-distribution scraped data, but consists in handcrafting a small set of linguistically motivated examples that exhaustively cover the range of span attributes (such as shape, length, casing, sentence position, etc.) a system may encounter in the wild. We demonstrate this methodology on a benchmark for anglicism identification in Spanish. Our methodology provides results that are diagnostic (because they help identify systematic weaknesses in performance), actionable (because they can inform which model is better suited for a given scenario) and predictive: our method predicts model performance on external datasets with a median correlation of 0.85.}
}