ForTune: Running Offline Scenarios to Estimate Impact on Business Metrics - Georges Dupret; Konstantin Sozinov; Carmen Barcena Gonzalez; Ziggy Zacks; Amber Yuan; Ben Carterette; Manuel Mai; Andrey Gatash; Gwo Liang Lien; Shubham Bansal; Roberto Sanchis-Ojeda; Mounia Lalmas (Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.1, KDD 2025, Toronto, ON, Canada, August 3-7, 2025, 2025)

Distributionally-Informed Recommender System Evaluation - Michael D. Ekstrand; Ben Carterette; Fernando Diaz (Trans. Recomm. Syst., 2024)

PODTILE: Facilitating Podcast Episode Browsing with Auto-generated Chapters - Azin Ghazimatin; Ekaterina Garmash; Gustavo Penha; Kristen Sheets; Martin Achenbach; Oguz Semerci; Remi Galvez; Marcus Tannenberg; Sahitya Mantravadi; Divya Narayanan; Ofeliya Kalaydzhyan; Douglas Cole; et al. (Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024, 2024)

Long-term Off-Policy Evaluation and Learning - Yuta Saito; Himan Abdollahpouri; Jesse Anderton; Ben Carterette; Mounia Lalmas (Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, 2024)

ForTune: Running Offline Scenarios to Estimate Impact on Business Metrics - Georges Dupret; Konstantin Sozinov; Carmen Barcena Gonzalez; Ziggy Zacks; Amber Yuan; Benjamin A. Carterette; Manuel Mai; Shubham Bansal; Gwo Liang Lien; Andrey Gatash; Roberto Sanchis-Ojeda; Mounia Lalmas (CoRR, 2024)

Long-term Off-Policy Evaluation and Learning - Yuta Saito; Himan Abdollahpouri; Jesse Anderton; Ben Carterette; Mounia Lalmas (CoRR, 2024)

PODTILE: Facilitating Podcast Episode Browsing with Auto-generated Chapters - Azin Ghazimatin; Ekaterina Garmash; Gustavo Penha; Kristen Sheets; Martin Achenbach; Oguz Semerci; Remi Galvez; Marcus Tannenberg; Sahitya Mantravadi; Divya Narayanan; Ofeliya Kalaydzhyan; Douglas Cole; et al. (CoRR, 2024)

A Comparison of Methods for Treatment Assignment with an Application to Playlist Generation - Carlos Fernández-Loría; Foster J. Provost; Jesse Anderton; Benjamin A. Carterette; Praveen Chandar (Inf. Syst. Res., 2023)

Report on the Dagstuhl Seminar on Frontiers of Information Access Experimentation for Research and Education - Christine Bauer; Ben Carterette; Nicola Ferro; Norbert Fuhr; Joeran Beel; Timo Breuer; Charles L. A. Clarke; Anita Crescenzi; Gianluca Demartini; Giorgio Maria Di Nunzio; Laura Dietz; Guglielmo Faggioli; et al. (SIGIR Forum, 2023)

Graph Learning for Exploratory Query Suggestions in an Instant Search System - Enrico Palumbo; Andreas Damianou; Alice Wang; Alva Liu; Ghazal Fazelnia; Francesco Fabbri; Rui Ferreira; Fabrizio Silvestri; Hugues Bouchard; Claudia Hauff; Mounia Lalmas; Ben Carterette; et al. (Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM 2023, Birmingham, United Kingdom, October 21-25, 2023, 2023)

A is for Adele: An Offline Evaluation Metric for Instant Search - Negar Arabzadeh; Oleksandra Kmet; Ben Carterette; Charles L. A. Clarke; Claudia Hauff; Praveen Chandar (Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2023, Taipei, Taiwan, 23 July 2023, 2023)

Calibrated Recommendations as a Minimum-Cost Flow Problem - Himan Abdollahpouri; Zahra Nazari; Alex Gain; Clay Gibson; Maria Dimakopoulou; Jesse Anderton; Benjamin A. Carterette; Mounia Lalmas; Tony Jebara (Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM 2023, Singapore, 27 February 2023 - 3 March 2023, 2023)

Report from Dagstuhl Seminar 23031: Frontiers of Information Access Experimentation for Research and Education - Christine Bauer; Ben Carterette; Nicola Ferro; Norbert Fuhr (CoRR, 2023)

Distributionally-Informed Recommender System Evaluation - Michael D. Ekstrand; Ben Carterette; Fernando Diaz (CoRR, 2023)

Frontiers of Information Access Experimentation for Research and Education (Dagstuhl Seminar 23031) - Christine Bauer; Ben Carterette; Nicola Ferro; Norbert Fuhr; Guglielmo Faggioli (Dagstuhl Reports, 2023)

Chair's Letter - Ben Carterette (SIGIR Forum, 2022)

Report on the 1st Workshop on Audio Collection Human Interaction (AudioCHI 2022) at CHIIR 2022 - Gareth J. F. Jones; Maria Eskevich; Ben Carterette; Joana Correia; Rosie Jones; Jussi Karlgren; Ian Soboroff (SIGIR Forum, 2022)

CHIIR Workshop on Audio Collection Human Interaction (AudioCHI 2022): http: //speechretrievalworkshop.github.io - Gareth J. F. Jones; Maria Eskevich; Ben Carterette; Joana Correia; Rosie Jones; Jussi Karlgren; Ian Soboroff (CHIIR '22: ACM SIGIR Conference on Human Information Interaction and Retrieval, Regensburg, Germany, March 14 - 18, 2022, 2022)

Variational User Modeling with Slow and Fast Features - Ghazal Fazelnia; Eric Simon; Ian Anderson; Benjamin A. Carterette; Mounia Lalmas (WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 - 25, 2022, 2022)

Choice of Implicit Signal Matters: Accounting for User Aspirations in Podcast Recommendations - Zahra Nazari; Praveen Chandar; Ghazal Fazelnia; Catherine M. Edwards; Benjamin A. Carterette; Mounia Lalmas (WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022, 2022)

Using Survival Models to Estimate User Engagement in Online Experiments - Praveen Chandar; Brian St. Thomas; Lucas Maystre; Vijay Pappu; Roberto Sanchis-Ojeda; Tiffany Wu; Ben Carterette; Mounia Lalmas; Tony Jebara (WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022, 2022)

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022 - Enrique Amigó; Pablo Castells; Julio Gonzalo; Ben Carterette; J. Shane Culpepper; Gabriella Kazai (ACM, 2022)

Chair's letter - Ben Carterette (SIGIR Forum, 2021)

How Am I Doing?: Evaluating Conversational Search Systems Offline - Aldo Lipani; Ben Carterette; Emine Yilmaz (ACM Trans. Inf. Syst., 2021)

Leveraging Semantic Information to Facilitate the Discovery of Underserved Podcasts - Maryam Aziz; Alice Wang; Aasish Pappu; Hugues Bouchard; Yu Zhao; Benjamin A. Carterette; Mounia Lalmas (CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, 2021)

Neural Instant Search for Music and Podcast - Helia Hashemi; Aasish Pappu; Mi Tian; Praveen Chandar; Mounia Lalmas; Benjamin A. Carterette (KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, 2021)

Current Challenges and Future Directions in Podcast Information Access - Rosie Jones; Hamed Zamani; Markus Schedl; Ching-Wei Chen; Sravana Reddy; Ann Clifton; Jussi Karlgren; Helia Hashemi; Aasish Pappu; Zahra Nazari; Longqi Yang; Oguz Semerci; et al. (SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, 2021)

Podcast Metadata and Content: Episode Relevance and Attractiveness in Ad Hoc Search - Ben Carterette; Rosie Jones; Gareth J. F. Jones; Maria Eskevich; Sravana Reddy; Ann Clifton; Yongze Yu; Jussi Karlgren; Ian Soboroff (SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, 2021)

TREC 2021 Podcasts Track Overview - Jussi Karlgren; Rosie Jones; Ben Carterette; Ann Clifton; Edgar Tanaka; Maria Eskevich; Gareth J. F. Jones; Sravana Reddy (Proceedings of the Thirtieth Text REtrieval Conference, TREC 2021, online, November 15-19, 2021, 2021)

Target-aware Aggregate Diversification in Recommendation - Himan Abdollahpouri; Jesse Anderton; Zahra Nazari; Ben Carterette (Adjunct Publication of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2021, Utrecht, The Netherlands, June 21-25, 2021, 2021)

Estimation of Fair Ranking Metrics with Incomplete Judgments - Ömer Kirnap; Fernando Diaz; Asia Biega; Michael D. Ekstrand; Ben Carterette; Emine Yilmaz (WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, 2021)

TREC 2020 Podcasts Track Overview - Rosie Jones; Ben Carterette; Ann Clifton; Maria Eskevich; Gareth J. F. Jones; Jussi Karlgren; Aasish Pappu; Sravana Reddy; Yongze Yu (CoRR, 2021)

Model Selection for Production System via Automated Online Experiments - Zhenwen Dai; Praveen Chandar; Ghazal Fazelnia; Ben Carterette; Mounia Lalmas-Roelleke (CoRR, 2021)

Current Challenges and Future Directions in Podcast Information Access - Rosie Jones; Hamed Zamani; Markus Schedl; Ching-Wei Chen; Sravana Reddy; Ann Clifton; Jussi Karlgren; Helia Hashemi; Aasish Pappu; Zahra Nazari; Longqi Yang; Oguz Semerci; et al. (CoRR, 2021)

Estimation of Fair Ranking Metrics with Incomplete Judgments - Ömer Kirnap; Fernando Diaz; Asia Biega; Michael D. Ekstrand; Ben Carterette; Emine Yilmaz (CoRR, 2021)

Podcast Metadata and Content: Episode Relevance andAttractiveness in Ad Hoc Search - Ben Carterette; Rosie Jones; Gareth J. F. Jones; Maria Eskevich; Sravana Reddy; Ann Clifton; Yongze Yu; Jussi Karlgren; Ian Soboroff (CoRR, 2021)

Report on the 2nd ACM SIGIR/SIGKDD Africa school on machine learning for data mining and search - Tanya Y. Berger-Wolf; Ben Carterette; Tamer Elsayed; C. Maria Keet; Fabrizio Sebastiani; Hussein Suleman (SIGIR Forum, 2020)

Evaluating Stochastic Rankings with Expected Exposure - Fernando Diaz; Bhaskar Mitra; Michael D. Ekstrand; Asia J. Biega; Ben Carterette (CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020, 2020)

100, 000 Podcasts: A Spoken English Document Corpus - Ann Clifton; Sravana Reddy; Yongze Yu; Aasish Pappu; Rezvaneh Rezapour; Hamed R. Bonab; Maria Eskevich; Gareth J. F. Jones; Jussi Karlgren; Ben Carterette; Rosie Jones (Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, 2020)

Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions - James McInerney; Brian Brost; Praveen Chandar; Rishabh Mehrotra; Benjamin A. Carterette (KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, 2020)

Advances in Recommender Systems: From Multi-stakeholder Marketplaces to Automated RecSys - Rishabh Mehrotra; Ben Carterette; Yong Li; Quanming Yao; Chen Gao; James T. Kwok; Qiang Yang; Isabelle Guyon (KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, 2020)

Model Selection for Production System via Automated Online Experiments - Zhenwen Dai; Praveen Chandar; Ghazal Fazelnia; Benjamin A. Carterette; Mounia Lalmas (Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020)

From Offline to Online Experimentation: Considerations from Experiences at Spotify - Ben Carterette (Proceedings of the 15th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR 2020, Tokyo, Japan, December 8-11, 2020, 2020)

Investigating Listeners' Responses to Divergent Recommendations - Rishabh Mehrotra; Chirag Shah; Benjamin A. Carterette (RecSys 2020: Fourteenth ACM Conference on Recommender Systems, Virtual Event, Brazil, September 22-26, 2020, 2020)

The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify - David Holtz; Ben Carterette; Praveen Chandar; Zahra Nazari; Henriette Cramer; Sinan Aral (EC '20: The 21st ACM Conference on Economics and Computation, Virtual Event, Hungary, July 13-17, 2020, 2020)

Bayesian Inferential Risk Evaluation On Multiple IR Systems - Rodger Benham; Ben Carterette; J. Shane Culpepper; Alistair Moffat (Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, 2020)

Recommending Podcasts for Cold-Start Users Based on Music Listening and Taste - Zahra Nazari; Christophe Charbuillet; Johan Pages; Martin Laurent; Denis Charrier; Briana Vecchione; Ben Carterette (Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, 2020)

TREC 2020 Podcasts Track Overview - Rosie Jones; Ben Carterette; Ann Clifton; Jussi Karlgren; Aasish Pappu; Sravana Reddy; Yongze Yu; Maria Eskevich; Gareth J. F. Jones (Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020, Virtual Event [Gaithersburg, Maryland, USA], November 16-20, 2020, 2020)

Leveraging Behavioral Heterogeneity Across Markets for Cross-Market Training of Recommender Systems - Kevin Roitero; Ben Carterette; Rishabh Mehrotra; Mounia Lalmas (Companion of The 2020 Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, 2020)

Do podcasts and music compete with one another? Understanding users' audio streaming habits - Ang Li; Alice Wang; Zahra Nazari; Praveen Chandar; Benjamin A. Carterette (WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, 2020)

The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify - David Holtz; Benjamin A. Carterette; Praveen Chandar; Zahra Nazari; Henriette Cramer; Sinan Aral (CoRR, 2020)

The Spotify Podcasts Dataset - Ann Clifton; Aasish Pappu; Sravana Reddy; Yongze Yu; Jussi Karlgren; Ben Carterette; Rosie Jones (CoRR, 2020)

Methods for Individual Treatment Assignment: An Application and Comparison for Playlist Generation - Carlos Fernandez; Foster J. Provost; Jesse Anderton; Benjamin A. Carterette; Praveen Chandar (CoRR, 2020)

Evaluating Stochastic Rankings with Expected Exposure - Fernando Diaz; Bhaskar Mitra; Michael D. Ekstrand; Asia J. Biega; Ben Carterette (CoRR, 2020)

Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions - James McInerney; Brian Brost; Praveen Chandar; Rishabh Mehrotra; Ben Carterette (CoRR, 2020)

Recommending Podcasts for Cold-Start Users Based on Music Listening and Taste - Zahra Nazari; Christophe Charbuillet; Johan Pages; Martin Laurent; Denis Charrier; Briana Vecchione; Ben Carterette (CoRR, 2020)

Trajectory Based Podcast Recommendation - Greg Benton; Ghazal Fazelnia; Alice Wang; Ben Carterette (CoRR, 2020)

A Topological Method for Comparing Document Semantics - Yuqi Kong; Fanchao Meng; Benjamin A. Carterette (CoRR, 2020)

Report on the 1st ACM SIGIR/SIGKDD Africa School on Machine Learning for Data Mining and Search - Ben Carterette; Hussein Suleman; Douglas W. Oard (SIGIR Forum, 2019)

Taking Risks with Confidence - Rodger Benham; Ben Carterette; Alistair Moffat; J. Shane Culpepper (Proceedings of the 24th Australasian Document Computing Symposium, ADCS 2019, Sydney, Australia, December 5-6, 2019, 2019)

From a User Model for Query Sessions to Session Rank Biased Precision (sRBP) - Aldo Lipani; Ben Carterette; Emine Yilmaz (Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2019, Santa Clara, CA, USA, October 2-5, 2019, 2019)

Statistical Significance Testing in Theory and in Practice - Ben Carterette (Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2019, Santa Clara, CA, USA, October 2-5, 2019, 2019)

Recommendations in a marketplace - Rishabh Mehrotra; Benjamin A. Carterette (Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, Copenhagen, Denmark, September 16-20, 2019, 2019)

Offline Evaluation to Make Decisions About PlaylistRecommendation Algorithms - Alois Gruson; Praveen Chandar; Christophe Charbuillet; James McInerney; Samantha Hansen; Damien Tardieu; Ben Carterette (Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019, 2019)

Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR 2019, Santa Clara, CA, USA, October 2-5, 2019 - Yi Fang; Yi Zhang; James Allan; Krisztian Balog; Ben Carterette; Jiafeng Guo (ACM, 2019)

Improving medical search tasks using learning to rank - Mohammad Alsulmi; Ben Carterette (2018 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology, CIBCB 2018, Saint Louis, MO, USA, May 30 - June 2, 2018, 2018)

Estimating Clickthrough Bias in the Cascade Model - Praveen Chandar; Ben Carterette (Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22-26, 2018, 2018)

Mixed methods for evaluating user satisfaction - Jean Garcia-Gathright; Christine Hosey; Brian St. Thomas; Ben Carterette; Fernando Diaz (Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, Vancouver, BC, Canada, October 2-7, 2018, 2018)

Offline Comparative Evaluation with Incremental, Minimally-Invasive Online Feedback - Ben Carterette; Praveen Chandar (The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018, 2018)

Precision and Recall - Ben Carterette (Encyclopedia of Database Systems, Second Edition, 2018)

Search Engine Metrics - Ben Carterette (Encyclopedia of Database Systems, Second Edition, 2018)

Test Collection - Ben Carterette (Encyclopedia of Database Systems, Second Edition, 2018)

User Click Detection in Ideal Sessions - Mustafa Zengin; Ben Carterette (Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, CHIIR 2017, Oslo, Norway, March 7-11, 2017, 2017)

Learning to Rate Clinical Concepts Using Simulated Clinician Feedback - Mohammad R. Alsulmi; Benjamin A. Carterette (Proceedings of the 22nd International Conference on Intelligent User Interfaces, IUI 2017, Limassol, Cyprus, March 13-16, 2017, 2017)

But Is It Statistically Significant?: Statistical Significance in IR Research, 1995-2014 - Ben Carterette (Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017, 2017)

Statistical Significance Testing in Information Retrieval: Theory and Practice - Ben Carterette (Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7-11, 2017, 2017)

TREC 2017 Tasks Track Overview - Evangelos Kanoulas; Emine Yilmaz; Rishabh Mehrotra; Ben Carterette; Nick Craswell; Peter Bailey (Proceedings of The Twenty-Sixth Text REtrieval Conference, TREC 2017, Gaithersburg, Maryland, USA, November 15-17, 2017, 2017)

Information retrieval evaluation using test collections - Falk Scholer; Diane Kelly; Ben Carterette (Inf. Retr. J., 2016)

Reformulate or Quit: Predicting User Abandonment in Ideal Sessions - Mustafa Zengin; Ben Carterette (Information Retrieval Technology - 12th Asia Information Retrieval Societies Conference, AIRS 2016, Beijing, China, November 30 - December 2, 2016, Proceedings, 2016)

Improving clinical case search using semantic based query reformulations - Mohammad Alsulmi; Ben Carterette (IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2016, Shenzhen, China, December 15-18, 2016, 2016)

Learning to predict the performance of clinical queries using an integrated approach - Mohammad Alsulmi; Ben Carterette (IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2016, Shenzhen, China, December 15-18, 2016, 2016)

Generating Pseudo Search History Data in the Absence of Real Search History - Ashraf Bah; Ben Carterette (Database and Expert Systems Applications - 27th International Conference, DEXA 2016, Porto, Portugal, September 5-8, 2016, Proceedings, Part II, 2016)

PDF: A Probabilistic Data Fusion Framework for Retrieval and Ranking - Ashraf Bah Rabiou; Ben Carterette (Proceedings of the 2016 ACM on International Conference on the Theory of Information Retrieval, ICTIR 2016, Newark, DE, USA, September 12- 6, 2016, 2016)

Evaluating Retrieval over Sessions: The TREC Session Track 2011-2014 - Ben Carterette; Paul D. Clough; Mark M. Hall; Evangelos Kanoulas; Mark Sanderson (Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 17-21, 2016, 2016)

Overview of the TREC Tasks Track 2016 - Manisha Verma; Emine Yilmaz; Rishabh Mehrotra; Evangelos Kanoulas; Ben Carterette; Nick Craswell; Peter Bailey (Proceedings of The Twenty-Fifth Text REtrieval Conference, TREC 2016, Gaithersburg, Maryland, USA, November 15-18, 2016, 2016)

Fusing Search Results from Possible Alternative Queries - Ashraf Bah; Ben Carterette (2016 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2016, Omaha, NE, USA, October 13-16, 2016, 2016)

Proceedings of the 2016 ACM on International Conference on the Theory of Information Retrieval, ICTIR 2016, Newark, DE, USA, September 12- 6, 2016 - Ben Carterette; Hui Fang; Mounia Lalmas; Jian-Yun Nie (ACM, 2016)

Improving Ranking and Robustness of Search Systems by Exploiting the Popularity of Documents - Ashraf Bah; Ben Carterette (Information Retrieval Technology - 11th Asia Information Retrieval Societies Conference, AIRS 2015, Brisbane, QLD, Australia, December 2-4, 2015. Proceedings, 2015)

Learning User Preferences for Topically Similar Documents - Mustafa Zengin; Ben Carterette (Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM 2015, Melbourne, VIC, Australia, October 19 - 23, 2015, 2015)

Statistical Significance Testing in Information Retrieval: Theory and Practice - Ben Carterette (Proceedings of the 2015 International Conference on The Theory of Information Retrieval, ICTIR 2015, Northampton, Massachusetts, USA, September 27-30, 2015, 2015)

Bayesian Inference for Information Retrieval Evaluation - Ben Carterette (Proceedings of the 2015 International Conference on The Theory of Information Retrieval, ICTIR 2015, Northampton, Massachusetts, USA, September 27-30, 2015, 2015)

Dynamic Test Collections for Retrieval Evaluation - Ben Carterette; Ashraf Bah; Mustafa Zengin (Proceedings of the 2015 International Conference on The Theory of Information Retrieval, ICTIR 2015, Northampton, Massachusetts, USA, September 27-30, 2015, 2015)

Document Comprehensiveness and User Preferences in Novelty Search Tasks - Ashraf Bah; Praveen Chandar; Ben Carterette (Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015, 2015)

The Best Published Result is Random: Sequential Testing and its Effect on Reported Effectiveness - Ben Carterette (Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015, 2015)

Overview of the TREC 2015 Tasks Track - Emine Yilmaz; Manisha Verma; Rishabh Mehrotra; Evangelos Kanoulas; Ben Carterette; Nick Craswell (Proceedings of The Twenty-Fourth Text REtrieval Conference, TREC 2015, Gaithersburg, Maryland, USA, November 17-20, 2015, 2015)

Integrating information retrieval with distant supervision for Gene Ontology annotation - Dongqing Zhu; Dingcheng Li; Ben Carterette; Hongfang Liu (Database J. Biol. Databases Curation, 2014)

Using large clinical corpora for query expansion in text-based cohort identification - Dongqing Zhu; Stephen T. Wu; Ben Carterette; Hongfang Liu (J. Biomed. Informatics, 2014)

Aggregating Results from Multiple Related Queries to Improve Web Search over Sessions - Ashraf Bah; Ben Carterette (Information Retrieval Technology - 10th Asia Information Retrieval Societies Conference, AIRS 2014, Kuching, Malaysia, December 3-5, 2014. Proceedings, 2014)

Udel @ NTCIR-11 IMine Track - Ashraf Bah; Ben Carterette; Praveen Chandar (Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9-12, 2014, 2014)

Udel @ NTCIR-11 MobileClick Track - Ashraf Bah Rabiou; Ben Carterette (Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9-12, 2014, 2014)

Statistical significance testing in information retrieval: theory and practice - Ben Carterette (The 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '14, Gold Coast , QLD, Australia - July 06 - 11, 2014, 2014)

University of Delaware at TREC 2014 - Ashraf Bah; Karankumar Sabhnani; Mustafa Zengin; Ben Carterette (Proceedings of The Twenty-Third Text REtrieval Conference, TREC 2014, Gaithersburg, Maryland, USA, November 19-21, 2014, 2014)

Overview of the TREC 2014 Session Track - Ben Carterette; Evangelos Kanoulas; Mark M. Hall; Paul D. Clough (Proceedings of The Twenty-Third Text REtrieval Conference, TREC 2014, Gaithersburg, Maryland, USA, November 19-21, 2014, 2014)

Seventh ACM International Conference on Web Search and Data Mining, WSDM 2014, New York, NY, USA, February 24-28, 2014 - Ben Carterette; Fernando Diaz; Carlos Castillo; Donald Metzler (ACM, 2014)

An overview of the BioCreative 2012 Workshop Track III: interactive text mining task - Cecilia N. Arighi; Ben Carterette; K. Bretonnel Cohen; Martin Krallinger; W. John Wilbur; Petra Fey; Robert J. Dodson; Laurel Cooper; Ceri E. Van Slyke; Wasila M. Dahdul; Paula M. Mabee; Donghui Li; et al. (Database J. Biol. Databases Curation, 2013)

Joint search in text and concept spaces for EMR-based cohort identification - Dongqing Zhu; Ben Carterette (2013 IEEE International Conference on Bioinformatics and Biomedicine, Shanghai, China, December 18-21, 2013, 2013)

An Incremental Approach for MEDLINE MeSH Indexing - Dongqing Zhu; Dingcheng Li; Ben Carterette; Hongfang Liu (Proceedings of the first Workshop on Bio-Medical Semantic Indexing and Question Answering, a Post-Conference Workshop of Conference and Labs of the Evaluation Forum 2013 (CLEF 2013) , Valencia, Spain, September 27th, 2013, 2013)

Using Discharge Summaries to Improve Information Retrieval in Clinical Domain - Dongqing Zhu; Stephen T. Wu; James J. Masanz; Ben Carterette; Hongfang Liu (Working Notes for CLEF 2013 Conference , Valencia, Spain, September 23-26, 2013, 2013)

Time Based Feedback and Query Expansion for Twitter Search - Naveen Kumar; Benjamin A. Carterette (Advances in Information Retrieval - 35th European Conference on IR Research, ECIR 2013, Moscow, Russia, March 24-27, 2013. Proceedings, 2013)

Statistical Significance Testing in Information Retrieval: Theory and Practice - Ben Carterette (International Conference on the Theory of Information Retrieval, ICTIR '13, Copenhagen, Denmark, September 29 - October 02, 2013, 2013)

Preference based evaluation measures for novelty and diversity - Praveen Chandar; Ben Carterette (The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '13, Dublin, Ireland - July 28 - August 01, 2013, 2013)

Document features predicting assessor disagreement - Praveen Chandar; William Webber; Ben Carterette (The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '13, Dublin, Ireland - July 28 - August 01, 2013, 2013)

An adaptive evidence weighting method for medical record search - Dongqing Zhu; Ben Carterette (The 36th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '13, Dublin, Ireland - July 28 - August 01, 2013, 2013)

MayoClinicNLP-CORE: Semantic representations for textual similarity - Stephen T. Wu; Dongqing Zhu; Ben Carterette; Hongfang Liu (Proceedings of the Second Joint Conference on Lexical and Computational Semantics, *SEM 2013, June 13-14, 2013, Atlanta, Georgia, USA, 2013)

Overview of the TREC 2013 Session Track - Ben Carterette; Ashraf Bah; Evangelos Kanoulas; Mark M. Hall; Paul D. Clough (Proceedings of The Twenty-Second Text REtrieval Conference, TREC 2013, Gaithersburg, Maryland, USA, November 19-22, 2013, 2013)

Multiple testing in statistical analysis of systems-based information retrieval experiments - Benjamin A. Carterette (ACM Trans. Inf. Syst., 2012)

Improving health records search using multiple query expansion collections - Dongqing Zhu; Ben Carterette (2012 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2012, Philadelphia, PA, USA, October 4-7, 2012, 2012)

Predicting baby feeding method from unstructured electronic health record data - Ashwani Rao; Kristin Maiden; Ben Carterette; Deb Ehrenthal (Proceedings of the ACM sixth international workshop on Data and text mining in biomedical informatics, DTMBIO@CIKM 2012, Maui, Hawaii, USA, October 29, 2012, 2012)

Combining multi-level evidence for medical record retrieval - Dongqing Zhu; Ben Carterette (Proceedings of the 2012 International Workshop on Smart Health and Wellbeing, SHB 2012, October 29, 2012, Maui, HI, USA, 2012)

Alternative assessor disagreement and retrieval depth - William Webber; Praveen Chandar; Ben Carterette (21st ACM International Conference on Information and Knowledge Management, CIKM'12, Maui, HI, USA, October 29 - November 02, 2012, 2012)

Incorporating variability in user behavior into systems based evaluation - Ben Carterette; Evangelos Kanoulas; Emine Yilmaz (21st ACM International Conference on Information and Knowledge Management, CIKM'12, Maui, HI, USA, October 29 - November 02, 2012, 2012)

Using preference judgments for novel document retrieval - Praveen Chandar; Ben Carterette (The 35th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '12, Portland, OR, USA, August 12-16, 2012, 2012)

Using PageRank to infer user preferences - Praveen Chandar; Ben Carterette (The 35th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '12, Portland, OR, USA, August 12-16, 2012, 2012)

Advances on the development of evaluation measures - Ben Carterette; Evangelos Kanoulas; Emine Yilmaz (The 35th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '12, Portland, OR, USA, August 12-16, 2012, 2012)

Overview of the TREC 2012 Session Track - Evangelos Kanoulas; Ben Carterette; Mark M. Hall; Paul D. Clough; Mark Sanderson (Proceedings of The Twenty-First Text REtrieval Conference, TREC 2012, Gaithersburg, Maryland, USA, November 6-9, 2012, 2012)

Exploring Evidence Aggregation Methods and External Expansion Sources for Medical Record Search - Dongqing Zhu; Ben Carterette (Proceedings of The Twenty-First Text REtrieval Conference, TREC 2012, Gaithersburg, Maryland, USA, November 6-9, 2012, 2012)

An analysis of NP-completeness in novelty and diversity ranking - Ben Carterette (Inf. Retr., 2011)

Report on the ECIR 2011 workshop on information retrieval over query sessions - Ben Carterette; Paul D. Clough; Evangelos Kanoulas; Mark Sanderson (SIGIR Forum, 2011)

Simulating simple user behavior for system effectiveness evaluation - Ben Carterette; Evangelos Kanoulas; Emine Yilmaz (Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011, Glasgow, United Kingdom, October 24-28, 2011, 2011)

A Methodology for Evaluating Aggregated Search Results - Jaime Arguello; Fernando Diaz; Jamie Callan; Ben Carterette (Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Dublin, Ireland, April 18-21, 2011. Proceedings, 2011)

Within-Document Term-Based Index Pruning with Statistical Hypothesis Testing - Sree Lekha Thota; Ben Carterette (Advances in Information Retrieval - 33rd European Conference on IR Research, ECIR 2011, Dublin, Ireland, April 18-21, 2011. Proceedings, 2011)

Model-Based Inference about IR Systems - Ben Carterette (Advances in Information Retrieval Theory - Third International Conference, ICTIR 2011, Bertinoro, Italy, September 12-14, 2011. Proceedings, 2011)

System effectiveness, user models, and user utility: a conceptual framework for investigation - Ben Carterette (Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, Beijing, China, July 25-29, 2011, 2011)

Evaluating multi-query sessions - Evangelos Kanoulas; Ben Carterette; Paul D. Clough; Mark Sanderson (Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, Beijing, China, July 25-29, 2011, 2011)

Implicit Feedback and Document Filtering for Retrieval Over Query Sessions - Ben Carterette; Praveen Chandar (Proceedings of The Twentieth Text REtrieval Conference, TREC 2011, Gaithersburg, Maryland, USA, November 15-18, 2011, 2011)

Simple Rank-Based Filtering for Microblog Retrieval: Implications for Evaluation and Test Collections - Ben Carterette; Naveen Kumar; Ashwani Rao; Dongqing Zhu (Proceedings of The Twentieth Text REtrieval Conference, TREC 2011, Gaithersburg, Maryland, USA, November 15-18, 2011, 2011)

Overview of the TREC 2011 Session Track - Evangelos Kanoulas; Mark M. Hall; Paul D. Clough; Ben Carterette; Mark Sanderson (Proceedings of The Twentieth Text REtrieval Conference, TREC 2011, Gaithersburg, Maryland, USA, November 15-18, 2011, 2011)

Using Multiple External Collections for Query Expansion - Dongqing Zhu; Ben Carterette (Proceedings of The Twentieth Text REtrieval Conference, TREC 2011, Gaithersburg, Maryland, USA, November 15-18, 2011, 2011)

Overview of Information Retrieval Evaluation - Ben Carterette; Ellen M. Voorhees (Current Challenges in Patent Information Retrieval, 2011)

The effect of assessor error on IR system evaluation - Ben Carterette; Ian Soboroff (Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, July 19-23, 2010, 2010)

Reusable test collections through experimental design - Ben Carterette; Evangelos Kanoulas; Virgiliu Pavlu; Hui Fang (Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, July 19-23, 2010, 2010)

Diversification of search results using webgraphs - Praveen Chandar; Ben Carterette (Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, July 19-23, 2010, 2010)

Low cost evaluation in information retrieval - Ben Carterette; Evangelos Kanoulas; Emine Yilmaz (Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, July 19-23, 2010, 2010)

Overview of the TREC 2010 Session Track - Evangelos Kanoulas; Paul D. Clough; Ben Carterette; Mark Sanderson (Proceedings of The Nineteenth Text REtrieval Conference, TREC 2010, Gaithersburg, Maryland, USA, November 16-19, 2010, 2010)

Measuring the reusability of test collections - Ben Carterette; Evgeniy Gabrilovich; Vanja Josifovski; Donald Metzler (Proceedings of the Third International Conference on Web Search and Web Data Mining, WSDM 2010, New York, NY, USA, February 4-6, 2010, 2010)

Redundancy, diversity and interdependent document relevance - Filip Radlinski; Paul N. Bennett; Ben Carterette; Thorsten Joachims (SIGIR Forum, 2009)

Probabilistic models of ranking novel documents for faceted topic retrieval - Ben Carterette; Praveen Chandar (Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009, Hong Kong, China, November 2-6, 2009, 2009)

If I Had a Million Queries - Ben Carterette; Virgiliu Pavlu; Evangelos Kanoulas; Javed A. Aslam; James Allan (Advances in Information Retrieval, 31th European Conference on IR Research, ECIR 2009, Toulouse, France, April 6-9, 2009. Proceedings, 2009)

An Analysis of NP-Completeness in Novelty and Diversity Ranking - Ben Carterette (Advances in Information Retrieval Theory, Second International Conference on the Theory of Information Retrieval, ICTIR 2009, Cambridge, UK, September 10-12, 2009, Proceedings, 2009)

On rank correlation and the distance between rankings - Ben Carterette (Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009, Boston, MA, USA, July 19-23, 2009, 2009)

Agreement among statistical significance tests for information retrieval evaluation at varying sample sizes - Mark D. Smucker; James Allan; Ben Carterette (Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009, Boston, MA, USA, July 19-23, 2009, 2009)

Minimal Test Collections for Relevance Feedback - Ben Carterette; Praveen Chandar; Aparna Kailasam; Divya Muppaneni; Sree Lekha Thota (Proceedings of The Eighteenth Text REtrieval Conference, TREC 2009, Gaithersburg, Maryland, USA, November 17-20, 2009, 2009)

Million Query Track 2009 Overview - Ben Carterette; Virgiliu Pavlu; Hui Fang; Evangelos Kanoulas (Proceedings of The Eighteenth Text REtrieval Conference, TREC 2009, Gaithersburg, Maryland, USA, November 17-20, 2009, 2009)

Ad Hoc and Diversity Retrieval at the University of Delaware - Praveen Chandar; Aparna Kailasam; Divya Muppaneni; Sree Lekha Thota; Ben Carterette (Proceedings of The Eighteenth Text REtrieval Conference, TREC 2009, Gaithersburg, Maryland, USA, November 17-20, 2009, 2009)

Precision and Recall - Ben Carterette (Encyclopedia of Database Systems, 2009)

Search Engine Metrics - Ben Carterette (Encyclopedia of Database Systems, 2009)

Test Collection - Ben Carterette (Encyclopedia of Database Systems, 2009)

Beyond binary relevance: preferences, diversity, and set-level judgments - Paul N. Bennett; Ben Carterette; Olivier Chapelle; Thorsten Joachims (SIGIR Forum, 2008)

Low-cost and robust evaluation of information retrieval systems - Benjamin A. Carterette (SIGIR Forum, 2008)

Here or There - Ben Carterette; Paul N. Bennett; David Maxwell Chickering; Susan T. Dumais (Advances in Information Retrieval , 30th European Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3, 2008. Proceedings, 2008)

Evaluation over thousands of queries - Ben Carterette; Virgiliu Pavlu; Evangelos Kanoulas; Javed A. Aslam; James Allan (Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20-24, 2008, 2008)

Evaluation measures for preference judgments - Ben Carterette; Paul N. Bennett (Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20-24, 2008, 2008)

Million Query Track 2008 Overview - James Allan; Javed A. Aslam; Virgil Pavlu; Evangelos Kanoulas; Ben Carterette (Proceedings of The Seventeenth Text REtrieval Conference, TREC 2008, Gaithersburg, Maryland, USA, November 18-21, 2008, 2008)

A comparison of statistical significance tests for information retrieval evaluation - Mark D. Smucker; James Allan; Ben Carterette (Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, CIKM 2007, Lisbon, Portugal, November 6-10, 2007, 2007)

Hypothesis testing with incomplete relevance judgments - Ben Carterette; Mark D. Smucker (Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, CIKM 2007, Lisbon, Portugal, November 6-10, 2007, 2007)

Semiautomatic evaluation of retrieval systems using document similarities - Ben Carterette; James Allan (Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, CIKM 2007, Lisbon, Portugal, November 6-10, 2007, 2007)

Evaluating Search Engines by Modeling the Relationship Between Relevance and Clicks - Ben Carterette; Rosie Jones (Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007, 2007)

Research Methodology in Studies of Information Retrieval Evaluation - Ben Carterette; James Allan (Computer-Assisted Information Retrieval (Recherche d'Information et ses Applications) - RIAO 2007, 8th International Conference, Carnegie Mellon University, Pittsburgh, PA, USA, May 30 - June 1, 2007. Proceedings, CD-ROM, 2007)

Robust test collections for retrieval evaluation - Ben Carterette (SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23-27, 2007, 2007)

Million Query Track 2007 Overview - James Allan; Ben Carterette; Blagovest Dachev; Javed A. Aslam; Virgiliu Pavlu; Evangelos Kanoulas (Proceedings of The Sixteenth Text REtrieval Conference, TREC 2007, Gaithersburg, Maryland, USA, November 5-9, 2007, 2007)

N Semantic Classes are Harder than Two - Ben Carterette; Rosie Jones; Wiley Greiner; Cory Barr (ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006, 2006)

Minimal test collections for retrieval evaluation - Ben Carterette; James Allan; Ramesh K. Sitaraman (SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA, August 6-11, 2006, 2006)

Learning a ranking from pairwise preferences - Ben Carterette; Desislava Petkova (SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA, August 6-11, 2006, 2006)

Comparing inverted files and signature files for searching a large lexicon - Ben Carterette; Fazli Can (Inf. Process. Manag., 2005)

Incremental test collections - Ben Carterette; James Allan (Proceedings of the 2005 ACM CIKM International Conference on Information and Knowledge Management, Bremen, Germany, October 31 - November 5, 2005, 2005)

When will information retrieval be "good enough"? - James Allan; Ben Carterette; Joshua Lewis (SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, August 15-19, 2005, 2005)