Issues

• EvaluateRetrieval.evaluate() averages NDCG/MAP/Recall/P only over queries present in results, so a judged query with no results entry is dropped instead of scored 0

  beir-cellar/beir#222 · Filter by author shaurya416 opened on Sep 25, 2026

• Optional EvalPort interop for BEIR corpus/queries/qrels and evaluation results

  beir-cellar/beir#221 · Filter by author adhabnr-ux opened on Aug 22, 2026

• search/dense/exact_search failes at num_queries = 1

  beir-cellar/beir#217 · Filter by author Monolitho opened on Jun 26, 2026

• Allow sampling test query

  beir-cellar/beir#216 · Filter by author Monolitho opened on Jun 16, 2026

• [Docs] Add WFGY 16-mode RAG failure checklist as a BEIR-aligned debugging resource

  beir-cellar/beir#210 · Filter by author onestardao opened on Feb 20, 2026
  2

• HFDataLoader returns HuggingFace Dataset, but EvaluateRetrieval expects dict (AttributeError: 'Dataset' object has no attribute 'keys')

  beir-cellar/beir#208 · Filter by author kcambrek opened on Dec 31, 2025

• Different results by retriever.encode_and_retrieve and retriever.retrieve

  beir-cellar/beir#206 · Filter by author liyongkang123 opened on Sep 7, 2025
  6

• Possibility of including TREC DL 2019 and TREC DL 2020

  beir-cellar/beir#205 · Filter by author liyongkang123 opened on Aug 12, 2025
  3

• Importlib.util needs to be directly imported

  beir-cellar/beir#204 · Filter by author amanchoudhri opened on Jul 28, 2025

• BUG - EvaluateRetrieval's evaluate doesn't consider results dict order

  beir-cellar/beir#184 · Filter by author vd-omermazig opened on Sep 12, 2024
  1

• mt5-base model (unicamp-dl/mt5-base-en-msmarco) throws 'nan' outputs

  beir-cellar/beir#173 · Filter by author cramraj8 opened on May 31, 2024
  1

• retriever.retrieve(corpus, queries) with len(queries)==1 errors

  beir-cellar/beir#170 · Filter by author manestay opened on Apr 17, 2024
  1

• Rename Accuracy@k Metric to Success@k metric

  beir-cellar/beir#166 · Filter by author root-goksenin opened on Mar 10, 2024

• Corpus encoding times for hotpotqa on A100 GPU

  beir-cellar/beir#165 · Filter by author jeyendranbalakrishnan opened on Feb 27, 2024

• example file, train_msmarco_v2.py is not working

  beir-cellar/beir#163 · Filter by author baekingeol opened on Feb 13, 2024

• Malformed Chinese mMARCO dataset

  beir-cellar/beir#161 · Filter by author sissilab opened on Jan 10, 2024

• Extras-require, base package without Faiss-CPU requirement

  beir-cellar/beir#160 · Filter by author raphaelsty opened on Oct 30, 2023
  1

• Fine-tuned Cross-encoder scores are lower than ms-marco zero-shot scores ?

  beir-cellar/beir#156 · Filter by author cramraj8 opened on Sep 15, 2023
  1

• Typing for retriever is too strict: implement abstract retriever base class

  beir-cellar/beir#152 · Filter by author jon-chuang opened on Jul 10, 2023

• org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024

  beir-cellar/beir#151 · Filter by author 0ddAstronaut opened on Jul 2, 2023

• Redirect Anserini BM25 baselines in BEIR to Pyserini

  beir-cellar/beir#149 · Filter by author thakur-nandan opened on Jun 20, 2023
  

• evaluate_anserini_bm25.py retrieves 1000 documents for each query no matter which k I set in payload

  beir-cellar/beir#146 · Filter by author zhiyuanpeng opened on Jun 16, 2023
  2

• Training script/configuration of T5 generator for GenQ

  beir-cellar/beir#141 · Filter by author jihyukkim-nlp opened on Apr 20, 2023
  2

• AssertionError: Elastic-Search Window too large, Max-Size = 10000

  beir-cellar/beir#140 · Filter by author zulle09 opened on Apr 18, 2023

• [Question] Using T5 as a retriveal model in BeIR

  beir-cellar/beir#139 · Filter by author rahmanidashti opened on Apr 3, 2023
  2