Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting - Nicholas Carlini; Jorge Chávez-Saab; Anna Hambitzer; Francisco Rodríguez-Henríquez; Adi Shamir (Advances in Cryptology - EUROCRYPT 2025 - 44th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Madrid, Spain, May 4-8, 2025, Proceedings, Part I, 2025)

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models - Michael Aerni; Javier Rando; Edoardo Debenedetti; Nicholas Carlini; Daphne Ippolito; Florian Tramèr (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI - Robert Hönig; Javier Rando; Nicholas Carlini; Florian Tramèr (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

Scalable Extraction of Training Data from Aligned, Production Language Models - Milad Nasr; Javier Rando; Nicholas Carlini; Jonathan Hayase; Matthew Jagielski; A. Feder Cooper; Daphne Ippolito; Christopher A. Choquette-Choo; Florian Tramèr; Katherine Lee (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

On Evaluating the Durability of Safeguards for Open-Weight LLMs - Xiangyu Qi; Boyi Wei; Nicholas Carlini; Yangsibo Huang; Tinghao Xie; Luxi He; Matthew Jagielski; Milad Nasr; Prateek Mittal; Peter Henderson (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

Persistent Pre-training Poisoning of LLMs - Yiming Zhang; Javier Rando; Ivan Evtimov; Jianfeng Chi; Eric Michael Smith; Nicholas Carlini; Florian Tramèr; Daphne Ippolito (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

SoK: Watermarking for AI-Generated Content - Xuandong Zhao; Sam Gunn; Miranda Christ; Jaiden Fairoze; Andres Fabrega; Nicholas Carlini; Sanjam Garg; Sanghyun Hong; Milad Nasr; Florian Tramèr; Somesh Jha; Lei Li; et al. (IEEE Symposium on Security and Privacy, SP 2025, San Francisco, CA, USA, May 12-15, 2025, 2025)

Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards - Yangsibo Huang; Milad Nasr; Anastasios Angelopoulos; Nicholas Carlini; Wei-Lin Chiang; Christopher A. Choquette-Choo; Daphne Ippolito; Matthew Jagielski; Katherine Lee; Ken Ziyu Liu; Ion Stoica; Florian Tramèr; et al. (CoRR, 2025)

Adversarial ML Problems Are Getting Harder to Solve and to Evaluate - Javier Rando; Jie Zhang; Nicholas Carlini; Florian Tramèr (CoRR, 2025)

AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses - Nicholas Carlini; Javier Rando; Edoardo Debenedetti; Milad Nasr; Florian Tramèr (CoRR, 2025)

In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI - Shayne Longpre; Kevin Klyman; Ruth E. Appel; Sayash Kapoor; Rishi Bommasani; Michelle Sahar; Sean McGregor; Avijit Ghosh; Borhane Blili-Hamelin; Nathan Butters; Alondra Nelson; Amit Elazari; et al. (CoRR, 2025)

Defeating Prompt Injections by Design - Edoardo Debenedetti; Ilia Shumailov; Tianqi Fan; Jamie Hayes; Nicholas Carlini; Daniel Fabian; Christoph Kern; Chongyang Shi; Andreas Terzis; Florian Tramèr (CoRR, 2025)

LLMs unlock new paths to monetizing exploits - Nicholas Carlini; Milad Nasr; Edoardo Debenedetti; Barry Wang; Christopher A. Choquette-Choo; Daphne Ippolito; Florian Tramèr; Matthew Jagielski (CoRR, 2025)

IF-GUIDE: Influence Function-Guided Detoxification of LLMs - Zachary Coalson; Juhan Bae; Nicholas Carlini; Sanghyun Hong (CoRR, 2025)

Defending Against Prompt Injection With a Few DefensiveTokens - Sizhe Chen; Yizhu Wang; Nicholas Carlini; Chawin Sitawarin; David A. Wagner (CoRR, 2025)

Initialization Matters for Adversarial Transfer Learning - Andong Hua; Jindong Gu; Zhiyu Xue; Nicholas Carlini; Eric Wong; Yao Qin (IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, 2024)

Stealing part of a production language model - Nicholas Carlini; Daniel Paleka; Krishnamurthy Dj Dvijotham; Thomas Steinke; Jonathan Hayase; A. Feder Cooper; Katherine Lee; Matthew Jagielski; Milad Nasr; Arthur Conmy; Eric Wallace; David Rolnick; et al. (Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024)

Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining - Florian Tramèr; Gautam Kamath; Nicholas Carlini (Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, 2024)

Query-Based Adversarial Prompt Generation - Jonathan Hayase; Ema Borevkovic; Nicholas Carlini; Florian Tramèr; Milad Nasr (Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024)

Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models - Yuxin Wen; Leo Marchyok; Sanghyun Hong; Jonas Geiping; Tom Goldstein; Nicholas Carlini (Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024)

Evading Black-box Classifiers Without Breaking Eggs - Edoardo Debenedetti; Nicholas Carlini; Florian Tramèr (IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2024, Toronto, ON, Canada, April 9-11, 2024, 2024)

Poisoning Web-Scale Training Datasets is Practical - Nicholas Carlini; Matthew Jagielski; Christopher A. Choquette-Choo; Daniel Paleka; Will Pearce; Hyrum S. Anderson; Andreas Terzis; Kurt Thomas; Florian Tramèr (IEEE Symposium on Security and Privacy, SP 2024, San Francisco, CA, USA, May 19-23, 2024, 2024)

Privacy Side Channels in Machine Learning Systems - Edoardo Debenedetti; Giorgio Severi; Milad Nasr; Christopher A. Choquette-Choo; Matthew Jagielski; Eric Wallace; Nicholas Carlini; Florian Tramèr (33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024, 2024)

Query-Based Adversarial Prompt Generation - Jonathan Hayase; Ema Borevkovic; Nicholas Carlini; Florian Tramèr; Milad Nasr (CoRR, 2024)

Stealing Part of a Production Language Model - Nicholas Carlini; Daniel Paleka; Krishnamurthy (Dj) Dvijotham; Thomas Steinke; Jonathan Hayase; A. Feder Cooper; Katherine Lee; Matthew Jagielski; Milad Nasr; Arthur Conmy; Eric Wallace; David Rolnick; et al. (CoRR, 2024)

Diffusion Denoising as a Certified Defense against Clean-label Poisoning - Sanghyun Hong; Nicholas Carlini; Alexey Kurakin (CoRR, 2024)

Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models - Yuxin Wen; Leo Marchyok; Sanghyun Hong; Jonas Geiping; Tom Goldstein; Nicholas Carlini (CoRR, 2024)

Forcing Diffuse Distributions out of Language Models - Yiming Zhang; Avi Schwarzschild; Nicholas Carlini; Zico Kolter; Daphne Ippolito (CoRR, 2024)

Cutting through buggy adversarial example defenses: fixing 1 line of code breaks Sabre - Nicholas Carlini (CoRR, 2024)

Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI - Robert Hönig; Javier Rando; Nicholas Carlini; Florian Tramèr (CoRR, 2024)

Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting - Nicholas Carlini; Jorge Chávez-Saab; Anna Hambitzer; Francisco Rodríguez-Henríquez; Adi Shamir (CoRR, 2024)

Persistent Pre-Training Poisoning of LLMs - Yiming Zhang; Javier Rando; Ivan Evtimov; Jianfeng Chi; Eric Michael Smith; Nicholas Carlini; Florian Tramèr; Daphne Ippolito (CoRR, 2024)

Remote Timing Attacks on Efficient Language Model Inference - Nicholas Carlini; Milad Nasr (CoRR, 2024)

Stealing User Prompts from Mixture of Experts - Itay Yona; Ilia Shumailov; Jamie Hayes; Nicholas Carlini (CoRR, 2024)

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models - Michael Aerni; Javier Rando; Edoardo Debenedetti; Nicholas Carlini; Daphne Ippolito; Florian Tramèr (CoRR, 2024)

Gradient Masking All-at-Once: Ensemble Everything Everywhere Is Not Robust - Jie Zhang; Kristina Nikolic; Nicholas Carlini; Florian Tramèr (CoRR, 2024)

SoK: Watermarking for AI-Generated Content - Xuandong Zhao; Sam Gunn; Miranda Christ; Jaiden Fairoze; Andres Fabrega; Nicholas Carlini; Sanjam Garg; Sanghyun Hong; Milad Nasr; Florian Tramèr; Somesh Jha; Lei Li; et al. (CoRR, 2024)

On Evaluating the Durability of Safeguards for Open-Weight LLMs - Xiangyu Qi; Boyi Wei; Nicholas Carlini; Yangsibo Huang; Tinghao Xie; Luxi He; Matthew Jagielski; Milad Nasr; Prateek Mittal; Peter Henderson (CoRR, 2024)

Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting - Nicholas Carlini; Jorge Chávez-Saab; Anna Hambitzer; Francisco Rodríguez-Henríquez; Adi Shamir (IACR Cryptol. ePrint Arch., 2024)

Identifying and Mitigating the Security Risks of Generative AI - Clark W. Barrett; Brad Boyd; Elie Bursztein; Nicholas Carlini; Brad Chen; Jihye Choi; Amrita Roy Chowdhury; Mihai Christodorescu; Anupam Datta; Soheil Feizi; Kathleen Fisher; Tatsunori Hashimoto; et al. (Found. Trends Priv. Secur., 2023)

Quantifying Memorization Across Neural Language Models - Nicholas Carlini; Daphne Ippolito; Matthew Jagielski; Katherine Lee; Florian Tramèr; Chiyuan Zhang (The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023)

(Certified!!) Adversarial Robustness for Free! - Nicholas Carlini; Florian Tramèr; Krishnamurthy (Dj) Dvijotham; Leslie Rice; Mingjie Sun; J. Zico Kolter (The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023)

Measuring Forgetting of Memorized Training Examples - Matthew Jagielski; Om Thakkar; Florian Tramèr; Daphne Ippolito; Katherine Lee; Nicholas Carlini; Eric Wallace; Shuang Song; Abhradeep Guha Thakurta; Nicolas Papernot; Chiyuan Zhang (The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023)

Part-Based Models Improve Adversarial Robustness - Chawin Sitawarin; Kornrapat Pongmala; Yizheng Chen; Nicholas Carlini; David A. Wagner (The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 2023)

Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems - Chawin Sitawarin; Florian Tramèr; Nicholas Carlini (International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, 2023)

Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy - Daphne Ippolito; Florian Tramèr; Milad Nasr; Chiyuan Zhang; Matthew Jagielski; Katherine Lee; Christopher A. Choquette-Choo; Nicholas Carlini (Proceedings of the 16th International Natural Language Generation Conference, INLG 2023, Prague, Czechia, September 11 - 15, 2023, 2023)

Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System - Daphne Ippolito; Nicholas Carlini; Katherine Lee; Milad Nasr; Yun William Yu (Proceedings of the 16th International Natural Language Generation Conference, INLG 2023, Prague, Czechia, September 11 - 15, 2023, 2023)

Are aligned neural networks adversarially aligned? - Nicholas Carlini; Milad Nasr; Christopher A. Choquette-Choo; Matthew Jagielski; Irena Gao; Pang Wei Koh; Daphne Ippolito; Florian Tramèr; Ludwig Schmidt (Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023)

Students Parrot Their Teachers: Membership Inference on Model Distillation - Matthew Jagielski; Milad Nasr; Katherine Lee; Christopher A. Choquette-Choo; Nicholas Carlini; Florian Tramèr (Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023)

Effective Robustness against Natural Distribution Shifts for Models with Different Training Data - Zhouxing Shi; Nicholas Carlini; Ananth Balashankar; Ludwig Schmidt; Cho-Jui Hsieh; Alex Beutel; Yao Qin (Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023)

Counterfactual Memorization in Neural Language Models - Chiyuan Zhang; Daphne Ippolito; Katherine Lee; Matthew Jagielski; Florian Tramèr; Nicholas Carlini (Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023)

Publishing Efficient On-device Models Increases Adversarial Vulnerability - Sanghyun Hong; Nicholas Carlini; Alexey Kurakin (2023 IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2023, Raleigh, NC, USA, February 8-10, 2023, 2023)

Tight Auditing of Differentially Private Machine Learning - Milad Nasr; Jamie Hayes; Thomas Steinke; Borja Balle; Florian Tramèr; Matthew Jagielski; Nicholas Carlini; Andreas Terzis (32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023, 2023)

Extracting Training Data from Diffusion Models - Nicholas Carlini; Jamie Hayes; Milad Nasr; Matthew Jagielski; Vikash Sehwag; Florian Tramèr; Borja Balle; Daphne Ippolito; Eric Wallace (32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023, 2023)

Extracting Training Data from Diffusion Models - Nicholas Carlini; Jamie Hayes; Milad Nasr; Matthew Jagielski; Vikash Sehwag; Florian Tramèr; Borja Balle; Daphne Ippolito; Eric Wallace (CoRR, 2023)

Effective Robustness against Natural Distribution Shifts for Models with Different Training Data - Zhouxing Shi; Nicholas Carlini; Ananth Balashankar; Ludwig Schmidt; Cho-Jui Hsieh; Alex Beutel; Yao Qin (CoRR, 2023)

Tight Auditing of Differentially Private Machine Learning - Milad Nasr; Jamie Hayes; Thomas Steinke; Borja Balle; Florian Tramèr; Matthew Jagielski; Nicholas Carlini; Andreas Terzis (CoRR, 2023)

Poisoning Web-Scale Training Datasets is Practical - Nicholas Carlini; Matthew Jagielski; Christopher A. Choquette-Choo; Daniel Paleka; Will Pearce; Hyrum S. Anderson; Andreas Terzis; Kurt Thomas; Florian Tramèr (CoRR, 2023)

Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators - Keane Lucas; Matthew Jagielski; Florian Tramèr; Lujo Bauer; Nicholas Carlini (CoRR, 2023)

Students Parrot Their Teachers: Membership Inference on Model Distillation - Matthew Jagielski; Milad Nasr; Christopher A. Choquette-Choo; Katherine Lee; Nicholas Carlini (CoRR, 2023)

Evading Black-box Classifiers Without Breaking Eggs - Edoardo Debenedetti; Nicholas Carlini; Florian Tramèr (CoRR, 2023)

Are aligned neural networks adversarially aligned? - Nicholas Carlini; Milad Nasr; Christopher A. Choquette-Choo; Matthew Jagielski; Irena Gao; Anas Awadalla; Pang Wei Koh; Daphne Ippolito; Katherine Lee; Florian Tramèr; Ludwig Schmidt (CoRR, 2023)

Backdoor Attacks for In-Context Learning with Language Models - Nikhil Kandpal; Matthew Jagielski; Florian Tramèr; Nicholas Carlini (CoRR, 2023)

A LLM Assisted Exploitation of AI-Guardian - Nicholas Carlini (CoRR, 2023)

Identifying and Mitigating the Security Risks of Generative AI - Clark W. Barrett; Brad Boyd; Ellie Burzstein; Nicholas Carlini; Brad Chen; Jihye Choi; Amrita Roy Chowdhury; Mihai Christodorescu; Anupam Datta; Soheil Feizi; Kathleen Fisher; Tatsunori Hashimoto; et al. (CoRR, 2023)

Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System - Daphne Ippolito; Nicholas Carlini; Katherine Lee; Milad Nasr; Yun William Yu (CoRR, 2023)

Privacy Side Channels in Machine Learning Systems - Edoardo Debenedetti; Giorgio Severi; Nicholas Carlini; Christopher A. Choquette-Choo; Matthew Jagielski; Milad Nasr; Eric Wallace; Florian Tramèr (CoRR, 2023)

Report of the 1st Workshop on Generative AI and Law - A. Feder Cooper; Katherine Lee; James Grimmelmann; Daphne Ippolito; Christopher Callison-Burch; Christopher A. Choquette-Choo; Niloofar Mireshghallah; Miles Brundage; David Mimno; Madiha Zahrah Choksi; Jack M. Balkin; Nicholas Carlini; et al. (CoRR, 2023)

Scalable Extraction of Training Data from (Production) Language Models - Milad Nasr; Nicholas Carlini; Jonathan Hayase; Matthew Jagielski; A. Feder Cooper; Daphne Ippolito; Christopher A. Choquette-Choo; Eric Wallace; Florian Tramèr; Katherine Lee (CoRR, 2023)

Initialization Matters for Adversarial Transfer Learning - Andong Hua; Jindong Gu; Zhiyu Xue; Nicholas Carlini; Eric Wong; Yao Qin (CoRR, 2023)

Deduplicating Training Data Makes Language Models Better - Katherine Lee; Daphne Ippolito; Andrew Nystrom; Chiyuan Zhang; Douglas Eck; Chris Callison-Burch; Nicholas Carlini (Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, 2022)

Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets - Florian Tramèr; Reza Shokri; Ayrton San Joaquin; Hoang Le; Matthew Jagielski; Sanghyun Hong; Nicholas Carlini (Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS 2022, Los Angeles, CA, USA, November 7-11, 2022, 2022)

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation - David Berthelot; Rebecca Roelofs; Kihyuk Sohn; Nicholas Carlini; Alexey Kurakin (The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022)

Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent - Oliver Bryniarski; Nabeel Hingun; Pedro Pachuca; Vincent Wang; Nicholas Carlini (The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022)

Poisoning and Backdooring Contrastive Learning - Nicholas Carlini; Andreas Terzis (The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022)

Data Poisoning Won't Save You From Facial Recognition - Evani Radiya-Dixit; Sanghyun Hong; Nicholas Carlini; Florian Tramèr (The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022)

Handcrafted Backdoors in Deep Neural Networks - Sanghyun Hong; Nicholas Carlini; Alexey Kurakin (Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022)

The Privacy Onion Effect: Memorization is Relative - Nicholas Carlini; Matthew Jagielski; Chiyuan Zhang; Nicolas Papernot; Andreas Terzis; Florian Tramèr (Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022)

Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples - Maura Pintor; Luca Demetrio; Angelo Sotgiu; Ambra Demontis; Nicholas Carlini; Battista Biggio; Fabio Roli (Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022)

Increasing Confidence in Adversarial Robustness Evaluations - Roland S. Zimmermann; Wieland Brendel; Florian Tramèr; Nicholas Carlini (Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022)

Membership Inference Attacks From First Principles - Nicholas Carlini; Steve Chien; Milad Nasr; Shuang Song; Andreas Terzis; Florian Tramèr (43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022, 2022)

Quantifying Memorization Across Neural Language Models - Nicholas Carlini; Daphne Ippolito; Matthew Jagielski; Katherine Lee; Florian Tramèr; Chiyuan Zhang (CoRR, 2022)

Debugging Differential Privacy: A Case Study for Privacy Auditing - Florian Tramèr; Andreas Terzis; Thomas Steinke; Shuang Song; Matthew Jagielski; Nicholas Carlini (CoRR, 2022)

Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets - Florian Tramèr; Reza Shokri; Ayrton San Joaquin; Hoang Le; Matthew Jagielski; Sanghyun Hong; Nicholas Carlini (CoRR, 2022)

The Privacy Onion Effect: Memorization is Relative - Nicholas Carlini; Matthew Jagielski; Chiyuan Zhang; Nicolas Papernot; Andreas Terzis; Florian Tramèr (CoRR, 2022)

(Certified!!) Adversarial Robustness for Free! - Nicholas Carlini; Florian Tramèr; Krishnamurthy Dvijotham; J. Zico Kolter (CoRR, 2022)

Increasing Confidence in Adversarial Robustness Evaluations - Roland S. Zimmermann; Wieland Brendel; Florian Tramèr; Nicholas Carlini (CoRR, 2022)

Measuring Forgetting of Memorized Training Examples - Matthew Jagielski; Om Thakkar; Florian Tramèr; Daphne Ippolito; Katherine Lee; Nicholas Carlini; Eric Wallace; Shuang Song; Abhradeep Thakurta; Nicolas Papernot; Chiyuan Zhang (CoRR, 2022)

Part-Based Models Improve Adversarial Robustness - Chawin Sitawarin; Kornrapat Pongmala; Yizheng Chen; Nicholas Carlini; David A. Wagner (CoRR, 2022)

No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy" - Nicholas Carlini; Vitaly Feldman; Milad Nasr (CoRR, 2022)

Preprocessors Matter! Realistic Decision-Based Attacks on Machine Learning Systems - Chawin Sitawarin; Florian Tramèr; Nicholas Carlini (CoRR, 2022)

Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy - Daphne Ippolito; Florian Tramèr; Milad Nasr; Chiyuan Zhang; Matthew Jagielski; Katherine Lee; Christopher A. Choquette-Choo; Nicholas Carlini (CoRR, 2022)

Considerations for Differentially Private Learning with Large-Scale Public Pretraining - Florian Tramèr; Gautam Kamath; Nicholas Carlini (CoRR, 2022)

Publishing Efficient On-device Models Increases Adversarial Vulnerability - Sanghyun Hong; Nicholas Carlini; Alexey Kurakin (CoRR, 2022)

Security of Machine Learning (Dagstuhl Seminar 22281) - Battista Biggio; Nicholas Carlini; Pavel Laskov; Konrad Rieck; Antonio Emanuele Cinà (Dagstuhl Reports, 2022)

Session details: Session 1: Adversarial Machine Learning - Nicholas Carlini (AISec@CCS 2021: Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, Virtual Event, Republic of Korea, 15 November 2021, 2021)

Session details: Session 2A: Machine Learning for Cybersecurity - Nicholas Carlini (AISec@CCS 2021: Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, Virtual Event, Republic of Korea, 15 November 2021, 2021)

Label-Only Membership Inference Attacks - Christopher A. Choquette-Choo; Florian Tramèr; Nicholas Carlini; Nicolas Papernot (Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, 2021)

How Private is Machine Learning? - Nicholas Carlini (IH&MMSec '21: ACM Workshop on Information Hiding and Multimedia Security, Virtual Event, Belgium, June, 22-25, 2021, 2021)

Is Private Learning Possible with Instance Encoding? - Nicholas Carlini; Samuel Deng; Sanjam Garg; Somesh Jha; Saeed Mahloujifar; Mohammad Mahmoody; Abhradeep Thakurta; Florian Tramèr (42nd IEEE Symposium on Security and Privacy, SP 2021, San Francisco, CA, USA, 24-27 May 2021, 2021)

Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning - Milad Nasr; Shuang Song; Abhradeep Thakurta; Nicolas Papernot; Nicholas Carlini (42nd IEEE Symposium on Security and Privacy, SP 2021, San Francisco, CA, USA, 24-27 May 2021, 2021)

Poisoning the Unlabeled Dataset of Semi-Supervised Learning - Nicholas Carlini (30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021, 2021)

Extracting Training Data from Large Language Models - Nicholas Carlini; Florian Tramèr; Eric Wallace; Matthew Jagielski; Ariel Herbert-Voss; Katherine Lee; Adam Roberts; Tom B. Brown; Dawn Song; Úlfar Erlingsson; Alina Oprea; Colin Raffel (30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021, 2021)

AISec@CCS 2021: Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, Virtual Event, Republic of Korea, 15 November 2021 - Nicholas Carlini; Ambra Demontis; Yizheng Chen (ACM, 2021)

Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning - Milad Nasr; Shuang Song; Abhradeep Thakurta; Nicolas Papernot; Nicholas Carlini (CoRR, 2021)

Poisoning the Unlabeled Dataset of Semi-Supervised Learning - Nicholas Carlini (CoRR, 2021)

Handcrafted Backdoors in Deep Neural Networks - Sanghyun Hong; Nicholas Carlini; Alexey Kurakin (CoRR, 2021)

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation - David Berthelot; Rebecca Roelofs; Kihyuk Sohn; Nicholas Carlini; Alex Kurakin (CoRR, 2021)

Poisoning and Backdooring Contrastive Learning - Nicholas Carlini; Andreas Terzis (CoRR, 2021)

Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples - Maura Pintor; Luca Demetrio; Angelo Sotgiu; Giovanni Manca; Ambra Demontis; Nicholas Carlini; Battista Biggio; Fabio Roli (CoRR, 2021)

Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent - Oliver Bryniarski; Nabeel Hingun; Pedro Pachuca; Vincent Wang; Nicholas Carlini (CoRR, 2021)

Deduplicating Training Data Makes Language Models Better - Katherine Lee; Daphne Ippolito; Andrew Nystrom; Chiyuan Zhang; Douglas Eck; Chris Callison-Burch; Nicholas Carlini (CoRR, 2021)

NeuraCrypt is not private - Nicholas Carlini; Sanjam Garg; Somesh Jha; Saeed Mahloujifar; Mohammad Mahmoody; Florian Tramèr (CoRR, 2021)

Unsolved Problems in ML Safety - Dan Hendrycks; Nicholas Carlini; John Schulman; Jacob Steinhardt (CoRR, 2021)

Membership Inference Attacks From First Principles - Nicholas Carlini; Steve Chien; Milad Nasr; Shuang Song; Andreas Terzis; Florian Tramèr (CoRR, 2021)

Counterfactual Memorization in Neural Language Models - Chiyuan Zhang; Daphne Ippolito; Katherine Lee; Matthew Jagielski; Florian Tramèr; Nicholas Carlini (CoRR, 2021)

AISec'20: 13th Workshop on Artificial Intelligence and Security - Sadia Afroz; Nicholas Carlini; Ambra Demontis (CCS '20: 2020 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, USA, November 9-13, 2020, 2020)

Cryptanalytic Extraction of Neural Network Models - Nicholas Carlini; Matthew Jagielski; Ilya Mironov (Advances in Cryptology - CRYPTO 2020 - 40th Annual International Cryptology Conference, CRYPTO 2020, Santa Barbara, CA, USA, August 17-21, 2020, Proceedings, Part III, 2020)

Evading Deepfake-Image Detectors with White- and Black-Box Attacks - Nicholas Carlini; Hany Farid (2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, June 14-19, 2020, 2020)

ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring - David Berthelot; Nicholas Carlini; Ekin D. Cubuk; Alex Kurakin; Kihyuk Sohn; Han Zhang; Colin Raffel (8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020)

Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations - Florian Tramèr; Jens Behrmann; Nicholas Carlini; Nicolas Papernot; Jörn-Henrik Jacobsen (Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, 2020)

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence - Kihyuk Sohn; David Berthelot; Nicholas Carlini; Zizhao Zhang; Han Zhang; Colin Raffel; Ekin Dogus Cubuk; Alexey Kurakin; Chun-Liang Li (Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020)

Measuring Robustness to Natural Distribution Shifts in Image Classification - Rohan Taori; Achal Dave; Vaishaal Shankar; Nicholas Carlini; Benjamin Recht; Ludwig Schmidt (Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020)

On Adaptive Attacks to Adversarial Example Defenses - Florian Tramèr; Nicholas Carlini; Wieland Brendel; Aleksander Madry (Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020)

High Accuracy and High Fidelity Extraction of Neural Networks - Matthew Jagielski; Nicholas Carlini; David Berthelot; Alex Kurakin; Nicolas Papernot (29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, 2020)

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence - Kihyuk Sohn; David Berthelot; Chun-Liang Li; Zizhao Zhang; Nicholas Carlini; Ekin D. Cubuk; Alex Kurakin; Han Zhang; Colin Raffel (CoRR, 2020)

Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations - Florian Tramèr; Jens Behrmann; Nicholas Carlini; Nicolas Papernot; Jörn-Henrik Jacobsen (CoRR, 2020)

On Adaptive Attacks to Adversarial Example Defenses - Florian Tramèr; Nicholas Carlini; Wieland Brendel; Aleksander Madry (CoRR, 2020)

Cryptanalytic Extraction of Neural Network Models - Nicholas Carlini; Matthew Jagielski; Ilya Mironov (CoRR, 2020)

Evading Deepfake-Image Detectors with White- and Black-Box Attacks - Nicholas Carlini; Hany Farid (CoRR, 2020)

Measuring Robustness to Natural Distribution Shifts in Image Classification - Rohan Taori; Achal Dave; Vaishaal Shankar; Nicholas Carlini; Benjamin Recht; Ludwig Schmidt (CoRR, 2020)

Label-Only Membership Inference Attacks - Christopher A. Choquette-Choo; Florian Tramèr; Nicholas Carlini; Nicolas Papernot (CoRR, 2020)

A Partial Break of the Honeypots Defense to Catch Adversarial Attacks - Nicholas Carlini (CoRR, 2020)

Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning - Guneet S. Dhillon; Nicholas Carlini (CoRR, 2020)

An Attack on InstaHide: Is Private Learning Possible with Instance Encoding? - Nicholas Carlini; Samuel Deng; Sanjam Garg; Somesh Jha; Saeed Mahloujifar; Mohammad Mahmoody; Shuang Song; Abhradeep Thakurta; Florian Tramèr (CoRR, 2020)

Extracting Training Data from Large Language Models - Nicholas Carlini; Florian Tramèr; Eric Wallace; Matthew Jagielski; Ariel Herbert-Voss; Katherine Lee; Adam Roberts; Tom B. Brown; Dawn Song; Úlfar Erlingsson; Alina Oprea; Colin Raffel (CoRR, 2020)

AISec'19: 12th ACM Workshop on Artificial Intelligence and Security - Sadia Afroz; Battista Biggio; Nicholas Carlini; Yuval Elovici; Asaf Shabtai (Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019, 2019)

Adversarial Examples Are a Natural Consequence of Test Error in Noise - Justin Gilmer; Nicolas Ford; Nicholas Carlini; Ekin D. Cubuk (Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, 2019)

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition - Yao Qin; Nicholas Carlini; Garrison W. Cottrell; Ian J. Goodfellow; Colin Raffel (Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, 2019)

MixMatch: A Holistic Approach to Semi-Supervised Learning - David Berthelot; Nicholas Carlini; Ian J. Goodfellow; Nicolas Papernot; Avital Oliver; Colin Raffel (Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019)

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks - Nicholas Carlini; Chang Liu; Úlfar Erlingsson; Jernej Kos; Dawn Song (28th USENIX Security Symposium, USENIX Security 2019, Santa Clara, CA, USA, August 14-16, 2019, 2019)

Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, AISec@CCS 2019, London, UK, November 15, 2019 - Lorenzo Cavallaro; Johannes Kinder; Sadia Afroz; Battista Biggio; Nicholas Carlini; Yuval Elovici; Asaf Shabtai (ACM, 2019)

Adversarial Examples Are a Natural Consequence of Test Error in Noise - Nic Ford; Justin Gilmer; Nicholas Carlini; Ekin Dogus Cubuk (CoRR, 2019)

Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples? - Nicholas Carlini (CoRR, 2019)

On Evaluating Adversarial Robustness - Nicholas Carlini; Anish Athalye; Nicolas Papernot; Wieland Brendel; Jonas Rauber; Dimitris Tsipras; Ian J. Goodfellow; Aleksander Madry; Alexey Kurakin (CoRR, 2019)

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition - Yao Qin; Nicholas Carlini; Ian J. Goodfellow; Garrison W. Cottrell; Colin Raffel (CoRR, 2019)

Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness - Jörn-Henrik Jacobsen; Jens Behrmann; Nicholas Carlini; Florian Tramèr; Nicolas Papernot (CoRR, 2019)

SysML: The New Frontier of Machine Learning Systems - Alexander Ratner; Dan Alistarh; Gustavo Alonso; David G. Andersen; Peter Bailis; Sarah Bird; Nicholas Carlini; Bryan Catanzaro; Eric S. Chung; Bill Dally; Jeff Dean; Inderjit S. Dhillon; et al. (CoRR, 2019)

MixMatch: A Holistic Approach to Semi-Supervised Learning - David Berthelot; Nicholas Carlini; Ian J. Goodfellow; Nicolas Papernot; Avital Oliver; Colin Raffel (CoRR, 2019)

A critique of the DeepSec Platform for Security Analysis of Deep Learning Models - Nicholas Carlini (CoRR, 2019)

Stateful Detection of Black-Box Adversarial Attacks - Steven Chen; Nicholas Carlini; David A. Wagner (CoRR, 2019)

High-Fidelity Extraction of Neural Network Models - Matthew Jagielski; Nicholas Carlini; David Berthelot; Alex Kurakin; Nicolas Papernot (CoRR, 2019)

Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications - Nicholas Carlini; Úlfar Erlingsson; Nicolas Papernot (CoRR, 2019)

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring - David Berthelot; Nicholas Carlini; Ekin D. Cubuk; Alex Kurakin; Kihyuk Sohn; Han Zhang; Colin Raffel (CoRR, 2019)

Evaluation and Design of Robust Neural Network Defenses - Nicholas Carlini (University of California, Berkeley, USA, 2018)

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples - Anish Athalye; Nicholas Carlini; David A. Wagner (Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, 2018)

Audio Adversarial Examples: Targeted Attacks on Speech-to-Text - Nicholas Carlini; David A. Wagner (2018 IEEE Security and Privacy Workshops, SP Workshops 2018, San Francisco, CA, USA, May 24, 2018, 2018)

Audio Adversarial Examples: Targeted Attacks on Speech-to-Text - Nicholas Carlini; David A. Wagner (CoRR, 2018)

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples - Anish Athalye; Nicholas Carlini; David A. Wagner (CoRR, 2018)

The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets - Nicholas Carlini; Chang Liu; Jernej Kos; Úlfar Erlingsson; Dawn Song (CoRR, 2018)

On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses - Anish Athalye; Nicholas Carlini (CoRR, 2018)

Unrestricted Adversarial Examples - Tom B. Brown; Nicholas Carlini; Chiyuan Zhang; Catherine Olsson; Paul F. Christiano; Ian J. Goodfellow (CoRR, 2018)

Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods - Nicholas Carlini; David A. Wagner (Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec@CCS 2017, Dallas, TX, USA, November 3, 2017, 2017)

Towards Evaluating the Robustness of Neural Networks - Nicholas Carlini; David A. Wagner (2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017, 2017)

Adversarial Example Defense: Ensembles of Weak Defenses are not Strong - Warren He; James Wei; Xinyun Chen; Nicholas Carlini; Dawn Song (11th USENIX Workshop on Offensive Technologies, WOOT 2017, Vancouver, BC, Canada, August 14-15, 2017, 2017)

Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods - Nicholas Carlini; David A. Wagner (CoRR, 2017)

Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong - Warren He; James Wei; Xinyun Chen; Nicholas Carlini; Dawn Song (CoRR, 2017)

Ground-Truth Adversarial Examples - Nicholas Carlini; Guy Katz; Clark W. Barrett; David L. Dill (CoRR, 2017)

MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples - Nicholas Carlini; David A. Wagner (CoRR, 2017)

Hidden Voice Commands - Nicholas Carlini; Pratyush Mishra; Tavish Vaidya; Yuankai Zhang; Micah Sherr; Clay Shields; David A. Wagner; Wenchao Zhou (25th USENIX Security Symposium, USENIX Security 16, Austin, TX, USA, August 10-12, 2016, 2016)

Defensive Distillation is Not Robust to Adversarial Examples - Nicholas Carlini; David A. Wagner (CoRR, 2016)

Towards Evaluating the Robustness of Neural Networks - Nicholas Carlini; David A. Wagner (CoRR, 2016)

Control-Flow Bending: On the Effectiveness of Control-Flow Integrity - Nicholas Carlini; Antonio Barresi; Mathias Payer; David A. Wagner; Thomas R. Gross (24th USENIX Security Symposium, USENIX Security 15, Washington, D.C., USA, August 12-14, 2015, 2015)

ROP is Still Dangerous: Breaking Modern Defenses - Nicholas Carlini; David A. Wagner (Proceedings of the 23rd USENIX Security Symposium, San Diego, CA, USA, August 20-22, 2014, 2014)

Improved Support for Machine-assisted Ballot-level Audits - Eric Kim; Nicholas Carlini; Andrew Chang; George Yiu; Kai Wang; David A. Wagner (2013 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections, EVT/WOTE '13, Washington, D.C., USA, August 12-13, 2013, 2013)

An Evaluation of the Google Chrome Extension Security Architecture - Nicholas Carlini; Adrienne Porter Felt; David A. Wagner (Proceedings of the 21th USENIX Security Symposium, Bellevue, WA, USA, August 8-10, 2012, 2012)

Operator-Assisted Tabulation of Optical Scan Ballots - Kai Wang; Nicholas Carlini; Eric Kim; Ivan Motyashov; Daniel Nguyen; David A. Wagner (2012 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections, EVT/WOTE '12, Bellevue, WA, USA, August 6-7, 2012, 2012)