Four Principles of Explainable AI as Applied to Biometrics and Facial Forensic Algorithms - P. Jonathon Phillips; Mark A. Przybocki (CoRR, 2020)

A new data science research program: evaluation, metrology, standards, and community outreach - Bonnie J. Dorr; Craig S. Greenberg; Peter C. Fontana; Mark A. Przybocki; Marion Le Bras; Cathryn A. Ploehn; Oleg Aulov; Martial Michel; E. Jim Golden; Wo Chang (Int. J. Data Sci. Anal., 2016)

Evaluation-driven research in data science: Leveraging cross-field methodologies - Bonnie J. Dorr; Peter C. Fontana; Craig S. Greenberg; Marion Le Bras; Mark A. Przybocki (2016 IEEE International Conference on Big Data (IEEE BigData 2016), Washington DC, USA, December 5-8, 2016, 2016)

The NIST data science evaluation series: Part of the NIST information access division data science initiative - Bonnie J. Dorr; Craig S. Greenberg; Peter C. Fontana; Mark A. Przybocki; Marion Le Bras; Cathryn A. Ploehn; Oleg Aulov; Wo Chang (2015 IEEE International Conference on Big Data (IEEE BigData 2015), Santa Clara, CA, USA, October 29 - November 1, 2015, 2015)

The NIST data science initiative - Bonnie J. Dorr; Craig S. Greenberg; Peter C. Fontana; Mark A. Przybocki; Marion Le Bras; Cathryn A. Ploehn; Oleg Aulov; Martial Michel; E. Jim Golden; Wo Chang (2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015, Campus des Cordeliers, Paris, France, October 19-21, 2015, 2015)

NIST 2013 Open Handwriting Recognition and Translation (Open HaRT'13) Evaluation - Audrey Tong; Mark A. Przybocki; Volker Märgner; Haikal El Abed (11th IAPR International Workshop on Document Analysis Systems, DAS 2014, Tours, France, April 7-10, 2014, 2014)

Summary and initial results of the 2013-2014 speaker recognition i-vector machine learning challenge - Désiré Bansé; George R. Doddington; Daniel Garcia-Romero; John J. Godfrey; Craig S. Greenberg; Alvin F. Martin; Alan McCree; Mark A. Przybocki; Douglas A. Reynolds (15th Annual Conference of the International Speech Communication Association, INTERSPEECH 2014, Singapore, September 14-18, 2014, 2014)

The NIST 2014 Speaker Recognition i-vector Machine Learning Challenge - Alan McCree; Douglas A. Reynolds; Daniel Garcia-Romero; Tomi Kinnunen; Craig S. Greenberg; Désiré Bansé; George R. Doddington; John J. Godfrey; Alvin F. Martin; Mark A. Przybocki (Odyssey 2014: The Speaker and Language Recognition Workshop, Joensuu, Finland, June 16-19, 2014, 2014)

The 2011 NIST Language Recognition Evaluation - Craig S. Greenberg; Alvin F. Martin; Mark A. Przybocki (13th Annual Conference of the International Speech Communication Association, INTERSPEECH 2012, Portland, Oregon, USA, September 9-13, 2012, 2012)

The 2011 BEST speaker recognition interim assessment - Craig S. Greenberg; Alvin F. Martin; Mark A. Przybocki (Odyssey 2012: The Speaker and Language Recognition Workshop, Singapore, June 25-28, 2012, 2012)

Document Image Collection Using Amazon's Mechanical Turk - Audrey N. Le; Jerome Ajot; Mark A. Przybocki; Stephanie M. Strassel (Proceedings of the 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, Los Angeles, USA, June 6, 2010, 2010)

Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation - Chris Callison-Burch; Philipp Koehn; Christof Monz; Kay Peterson; Mark A. Przybocki; Omar Zaidan (Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, WMT@ACL 2010, Uppsala, Sweden, July 15-16, 2010, 2010)

Introduction to the special issue on "Automated Metrics for Machine Translation Evaluation" - Alon Lavie; Mark A. Przybocki (Mach. Transl., 2009)

The NIST 2008 Metrics for machine translation challenge - overview, methodology, metrics, and results - Mark A. Przybocki; Kay Peterson; Sebastien Bronsart; Gregory A. Sanders (Mach. Transl., 2009)

Translation Adequacy and Preference Evaluation Tool (TAP-ET) - Mark A. Przybocki; Kay Peterson; Sebastien Bronsart (Proceedings of the International Conference on Language Resources and Evaluation, LREC 2008, 26 May - 1 June 2008, Marrakech, Morocco, 2008)

Linguistic Resources and Evaluation Techniques for Evaluation of Cross-Document Automatic Content Extraction - Stephanie M. Strassel; Mark A. Przybocki; Kay Peterson; Zhiyi Song; Kazuaki Maeda (Proceedings of the International Conference on Language Resources and Evaluation, LREC 2008, 26 May - 1 June 2008, Marrakech, Morocco, 2008)

NIST Speaker Recognition Evaluations Utilizing the Mixer Corpora - 2004, 2005, 2006 - Mark A. Przybocki; Alvin F. Martin; Audrey N. Le (IEEE Trans. Speech Audio Process., 2007)

NIST and NFI-TNO evaluations of automatic speaker recognition - David A. van Leeuwen; Alvin F. Martin; Mark A. Przybocki; Jos S. Bouten (Comput. Speech Lang., 2006)

The Mixer and Transcript Reading Corpora: Resources for Multilingual, Crosschannel Speaker Recognition Research - Christopher Cieri; Walter D. Andrews; Joseph P. Campbell; George R. Doddington; John J. Godfrey; Shudong Huang; Mark Y. Liberman; Alvin F. Martin; Hirotaka Nakasone; Mark A. Przybocki; Kevin Walker (Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC 2006, Genoa, Italy, May 22-28, 2006, 2006)

Edit Distance: A Metric for Machine Translation Evaluation - Mark A. Przybocki; Gregory A. Sanders; Audrey N. Le (Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC 2006, Genoa, Italy, May 22-28, 2006, 2006)

NIST Speaker Recognition Evaluation Chronicles - Part 2 - Mark A. Przybocki; Alvin F. Martin; Audrey N. Le (Odyssey 2006: The Speaker and Language Recognition Workshop, San Juan, Puerto Rico, June 28-30, 2006, 2006)

The Automatic Content Extraction (ACE) Program - Tasks, Data, and Evaluation - George R. Doddington; Alexis Mitchell; Mark A. Przybocki; Lance A. Ramshaw; Stephanie M. Strassel; Ralph M. Weischedel (Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal, 2004)

NIST Language Technology Evaluation Cookbook - Alvin F. Martin; John S. Garofolo; Jonathan G. Fiscus; Audrey N. Le; David S. Pallett; Mark A. Przybocki; Gregory A. Sanders (Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal, 2004)

Conversational Telephone Speech Corpus Collection for the NIST Speaker Recognition Evaluation 2004 - Alvin F. Martin; David Miller; Mark A. Przybocki; Joseph P. Campbell; Hirotaka Nakasone (Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC 2004, May 26-28, 2004, Lisbon, Portugal, 2004)

NIST speaker recognition evaluation chronicles - Mark A. Przybocki; Alvin F. Martin (Odyssey 2004: The Speaker and Language Recognition Workshop, Toledo, Spain, May 31 - June 3, 2004, 2004)

The MMSR bilingual and crosschannel corpora for speaker recognition research and evaluation - Joseph P. Campbell; Hirotaka Nakasone; Christopher Cieri; David Miller; Kevin Walker; Alvin F. Martin; Mark A. Przybocki (Odyssey 2004: The Speaker and Language Recognition Workshop, Toledo, Spain, May 31 - June 3, 2004, 2004)

NIST 2003 language recognition evaluation - Alvin F. Martin; Mark A. Przybocki (8th European Conference on Speech Communication and Technology, EUROSPEECH 2003 - INTERSPEECH 2003, Geneva, Switzerland, September 1-4, 2003, 2003)

NIST Rich Transcription 2002 Evaluation: A Preview - John S. Garofolo; Jonathan G. Fiscus; Alvin F. Martin; David S. Pallett; Mark A. Przybocki (Proceedings of the Third International Conference on Language Resources and Evaluation, LREC 2002, May 29-31, 2002, Las Palmas, Canary Islands, Spain, 2002)

Speaker recognition in a multi-speaker environment - Alvin F. Martin; Mark A. Przybocki (EUROSPEECH 2001 Scandinavia, 7th European Conference on Speech Communication and Technology, 2nd INTERSPEECH Event, Aalborg, Denmark, September 3-7, 2001, 2001)

Odyssey text independent evaluation data - Mark A. Przybocki; Alvin F. Martin (2001: A Speaker Odyssey - The Speaker Recognition Workshop, Crete, Greece, June 18-22, 2001, 2001)

The NIST Speaker Recognition Evaluations: 1996-2001 - Alvin F. Martin; Mark A. Przybocki (2001: A Speaker Odyssey - The Speaker Recognition Workshop, Crete, Greece, June 18-22, 2001, 2001)

An Introduction to Evaluating Biometric Systems - P. Jonathon Phillips; Alvin F. Martin; Charles L. Wilson; Mark A. Przybocki (Computer, 2000)

The NIST 1999 Speaker Recognition Evaluation - An Overview - Alvin F. Martin; Mark A. Przybocki (Digit. Signal Process., 2000)

The NIST speaker recognition evaluation - Overview, methodology, systems, results, perspective - George R. Doddington; Mark A. Przybocki; Alvin F. Martin; Douglas A. Reynolds (Speech Commun., 2000)

Design Issues in Text-Independent Speaker Recognition Evaluation - Alvin F. Martin; Mark A. Przybocki (Proceedings of the Second International Conference on Language Resources and Evaluation, LREC 2000, 31 May - June 2, 2000, Athens, Greece, 2000)

The 1999 NIST speaker recognition evaluation, using summed two-channel telephone data for speaker detection and speaker tracking - Mark A. Przybocki; Alvin F. Martin (Sixth European Conference on Speech Communication and Technology, EUROSPEECH 1999, Budapest, Hungary, September 5-9, 1999, 1999)

SHEEP, GOATS, LAMBS and WOLVES: a statistical analysis of speaker performance in the NIST 1998 speaker recognition evaluation - George R. Doddington; Walter Liggett; Alvin F. Martin; Mark A. Przybocki; Douglas A. Reynolds (The 5th International Conference on Spoken Language Processing, Incorporating The 7th Australian International Speech Science and Technology Conference, Sydney Convention Centre, Sydney, Australia, 30th November - 4th December 1998, 1998)

NIST speaker recognition benchmark tests - Mark A. Przybocki; Alvin F. Martin (Proceedings of the First International Conference on Language Resources and Evaluation, LREC 1998, May 28-30, 1998, Granada, Spain, 1998)

The DET curve in assessment of detection task performance - Alvin F. Martin; George R. Doddington; Terri Kamm; Mark Ordowski; Mark A. Przybocki (Fifth European Conference on Speech Communication and Technology, EUROSPEECH 1997, Rhodes, Greece, September 22-25, 1997, 1997)

1993 Benchmark Tests for the ARPA Spoken Language Program - David S. Pallett; Jonathan G. Fiscus; William M. Fisher; John S. Garofolo; Bruce A. Lund; Mark A. Przybocki (Human Language Technology, Proceedings of a Workshop held at Plainsboro, New Jerey, USA, March 8-11, 1994, 1994)