MMTEB: Massive Multilingual Text Embedding Benchmark - Kenneth C. Enevoldsen; Isaac Chung; Imene Kerboua; Márton Kardos; Ashwin Mathur; David Stap; Jay Gala; Wissam Siblini; Dominik Krzeminski; Genta Indra Winata; Saba Sturua; Saiteja Utpala; et al. (CoRR, 2025)

"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation - Nandan Thakur; Luiz Bonifacio; Crystina Zhang; Odunayo Ogundepo; Ehsan Kamalloo; David Alfonso-Hermelo; Xiaoguang Li; Qun Liu; Boxing Chen; Mehdi Rezagholizadeh; Jimmy Lin (Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, 2024)

Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval - Nandan Thakur; Jianmo Ni; Gustavo Hernández Ábrego; John Wieting; Jimmy Lin; Daniel Cer (Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, 2024)

Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR - Nandan Thakur; Luiz Bonifacio; Maik Fröbe; Alexander Bondarenko; Ehsan Kamalloo; Martin Potthast; Matthias Hagen; Jimmy Lin (Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, 2024)

Resources for Brewing BEIR: Reproducible Reference Models and Statistical Analyses - Ehsan Kamalloo; Nandan Thakur; Carlos Lassance; Xueguang Ma; Jheng-Hong Yang; Jimmy Lin (Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, 2024)

UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor - Shivani Upadhyay; Ronak Pradeep; Nandan Thakur; Nick Craswell; Jimmy Lin (CoRR, 2024)

Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track - Ronak Pradeep; Nandan Thakur; Sahel Sharifymoghaddam; Eric Zhang; Ryan Nguyen; Daniel Campos; Nick Craswell; Jimmy Lin (CoRR, 2024)

Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR - Nandan Thakur; Luiz Bonifacio; Maik Fröbe; Alexander Bondarenko; Ehsan Kamalloo; Martin Potthast; Matthias Hagen; Jimmy Lin (CoRR, 2024)

MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems - Nandan Thakur; Suleman Kazi; Ge Luo; Jimmy Lin; Amin Ahmad (CoRR, 2024)

A Large-Scale Study of Relevance Assessments with Large Language Models: An Initial Look - Shivani Upadhyay; Ronak Pradeep; Nandan Thakur; Daniel Campos; Nick Craswell; Ian Soboroff; Hoa Trang Dang; Jimmy Lin (CoRR, 2024)

Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework - Ronak Pradeep; Nandan Thakur; Shivani Upadhyay; Daniel Campos; Nick Craswell; Jimmy Lin (CoRR, 2024)

MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages - Xinyu Zhang; Nandan Thakur; Odunayo Ogundepo; Ehsan Kamalloo; David Alfonso-Hermelo; Xiaoguang Li; Qun Liu; Mehdi Rezagholizadeh; Jimmy Lin (Trans. Assoc. Comput. Linguistics, 2023)

Evaluating Embedding APIs for Information Retrieval - Ehsan Kamalloo; Xinyu Zhang; Odunayo Ogundepo; Nandan Thakur; David Alfonso-Hermelo; Mehdi Rezagholizadeh; Jimmy Lin (Proceedings of the The 61st Annual Meeting of the Association for Computational Linguistics: Industry Track, ACL 2023, Toronto, Canada, July 9-14, 2023, 2023)

SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval - Nandan Thakur; Kexin Wang; Iryna Gurevych; Jimmy Lin (Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023, 2023)

Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval - Jimmy Lin; David Alfonso-Hermelo; Vitor Jeronymo; Ehsan Kamalloo; Carlos Lassance; Rodrigo Nogueira; Odunayo Ogundepo; Mehdi Rezagholizadeh; Nandan Thakur; Jheng-Hong Yang; Xinyu Zhang (CoRR, 2023)

Evaluating Embedding APIs for Information Retrieval - Ehsan Kamalloo; Xinyu Zhang; Odunayo Ogundepo; Nandan Thakur; David Alfonso-Hermelo; Mehdi Rezagholizadeh; Jimmy Lin (CoRR, 2023)

Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard - Ehsan Kamalloo; Nandan Thakur; Carlos Lassance; Xueguang Ma; Jheng-Hong Yang; Jimmy Lin (CoRR, 2023)

SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval - Nandan Thakur; Kexin Wang; Iryna Gurevych; Jimmy Lin (CoRR, 2023)

HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution - Ehsan Kamalloo; Aref Jafari; Xinyu Zhang; Nandan Thakur; Jimmy Lin (CoRR, 2023)

Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval - Nandan Thakur; Jianmo Ni; Gustavo Hernández Ábrego; John Wieting; Jimmy Lin; Daniel Cer (CoRR, 2023)

NoMIRACL: Knowing When You Don't Know for Robust Multilingual Retrieval-Augmented Generation - Nandan Thakur; Luiz Bonifacio; Xinyu Zhang; Odunayo Ogundepo; Ehsan Kamalloo; David Alfonso-Hermelo; Xiaoguang Li; Qun Liu; Boxing Chen; Mehdi Rezagholizadeh; Jimmy Lin (CoRR, 2023)

GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval - Kexin Wang; Nandan Thakur; Nils Reimers; Iryna Gurevych (Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, 2022)

Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval - Jimmy Lin; David Alfonso-Hermelo; Vitor Jeronymo; Ehsan Kamalloo; Carlos Lassance; Rodrigo Nogueira; Odunayo Ogundepo; Mehdi Rezagholizadeh; Nandan Thakur; Jheng-Hong Yang; Xinyu Zhang (Proceedings of the Thirty-First Text REtrieval Conference, TREC 2022, online, November 15-19, 2022, 2022)

Domain Adaptation for Memory-Efficient Dense Retrieval - Nandan Thakur; Nils Reimers; Jimmy Lin (CoRR, 2022)

Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages - Xinyu Zhang; Nandan Thakur; Odunayo Ogundepo; Ehsan Kamalloo; David Alfonso-Hermelo; Xiaoguang Li; Qun Liu; Mehdi Rezagholizadeh; Jimmy Lin (CoRR, 2022)

Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks - Nandan Thakur; Nils Reimers; Johannes Daxenberger; Iryna Gurevych (Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, 2021)

BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models - Nandan Thakur; Nils Reimers; Andreas Rücklé; Abhishek Srivastava; Iryna Gurevych (Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021)

BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models - Nandan Thakur; Nils Reimers; Andreas Rücklé; Abhishek Srivastava; Iryna Gurevych (CoRR, 2021)

GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval - Kexin Wang; Nandan Thakur; Nils Reimers; Iryna Gurevych (CoRR, 2021)

Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks - Nandan Thakur; Nils Reimers; Johannes Daxenberger; Iryna Gurevych (CoRR, 2020)