On the shortcut learning in multilingual neural machine translation - Wenxuan Wang; Wenxiang Jiao; Jen-tse Huang; Zhaopeng Tu; Michael R. Lyu (Neurocomputing, 2025)

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang; Jen-tse Huang; Jiahao Xu; Tian Liang; Pinjia He; Zhaopeng Tu (Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025, 2025)

Chain-of-Jailbreak Attack for Image Generation Models via Step by Step Editing - Wenxuan Wang; Kuiyi Gao; Youliang Yuan; Jen-tse Huang; Qiuzhi Liu; Shuai Wang; Wenxiang Jiao; Zhaopeng Tu (Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, 2025)

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs - Wenxuan Wang; Xiaoyuan Liu; Kuiyi Gao; Jen-tse Huang; Youliang Yuan; Pinjia He; Shuai Wang; Zhaopeng Tu (Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025, 2025)

Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs - Xiaoyuan Liu; Wenxuan Wang; Youliang Yuan; Jen-tse Huang; Qiuzhi Liu; Pinjia He; Zhaopeng Tu (Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025, 2025)

Competing Large Language Models in Multi-Agent Gaming Environments - Jen-tse Huang; Eric John Li; Man Ho Lam; Tian Liang; Wenxuan Wang; Youliang Yuan; Wenxiang Jiao; Xing Wang; Zhaopeng Tu; Michael R. Lyu (The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025)

FairCode: Evaluating Social Bias of LLMs in Code Generation - Yongkang Du; Jen-tse Huang; Jieyu Zhao; Lu Lin (CoRR, 2025)

Fact-or-Fair: A Checklist for Behavioral Testing of AI Models on Fairness-Related Queries - Jen-tse Huang; Yuhang Yan; Linqi Liu; Yixin Wan; Wenxuan Wang; Kai-Wei Chang; Michael R. Lyu (CoRR, 2025)

CoSER: Coordinating LLM-Based Persona Simulation of Established Roles - Xintao Wang; Heng Wang; Yifei Zhang; Xinfeng Yuan; Rui Xu; Jen-tse Huang; Siyu Yuan; Haoran Guo; Jiangjie Chen; Wei Wang; Yanghua Xiao; Shuchang Zhou (CoRR, 2025)

VLMs as GeoGuessr Masters: Exceptional Performance, Hidden Biases, and Privacy Risks - Jingyuan Huang; Jen-tse Huang; Ziyi Liu; Xiaoyuan Liu; Wenxuan Wang; Jieyu Zhao (CoRR, 2025)

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs - Wenxuan Wang; Xiaoyuan Liu; Kuiyi Gao; Jen-tse Huang; Youliang Yuan; Pinjia He; Shuai Wang; Zhaopeng Tu (CoRR, 2025)

VisFactor: Benchmarking Fundamental Visual Cognition in Multimodal Large Language Models - Jen-tse Huang; Dasen Dai; Jen-Yuan Huang; Youliang Yuan; Xiaoyuan Liu; Wenxuan Wang; Wenxiang Jiao; Pinjia He; Zhaopeng Tu (CoRR, 2025)

VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models - Jen-tse Huang; Jiantong Qin; Jianping Zhang; Youliang Yuan; Wenxuan Wang; Jieyu Zhao (CoRR, 2025)

Can LLMs Grasp Implicit Cultural Values? Benchmarking LLMs' Metacognitive Cultural Intelligence with CQ-Bench - Ziyi Liu; Priyanka Dey; Zhenyu Zhao; Jen-tse Huang; Rahul Gupta; Yang Liu; Jieyu Zhao (CoRR, 2025)

BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents - Haoxuan Li; Mingyu Derek Ma; Jen-tse Huang; Zhaotian Weng; Wei Wang; Jieyu Zhao (CoRR, 2025)

CODECRASH: Stress Testing LLM Reasoning under Structural and Semantic Perturbations - Man Ho Lam; Chaozheng Wang; Jen-tse Huang; Michael R. Lyu (CoRR, 2025)

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment - Kun Wang; Guibin Zhang; Zhenhong Zhou; Jiahao Wu; Miao Yu; Shiqian Zhao; Chenlong Yin; Jinhu Fu; Yibo Yan; Hanjun Luo; Liang Lin; Zhihao Xu; et al. (CoRR, 2025)

SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation - Xuhui Zhou; Zhe Su; Sophie Feng; Jiaxu Zhou; Jen-tse Huang; Hsien-Te Kao; Spencer Lynch; Svitlana Volkova; Tongshuang Sherry Wu; Anita Woolley; Hao Zhu; Maarten Sap (CoRR, 2025)

A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron? - Ada Chen; Yongjiang Wu; Junyuan Zhang; Jingyu Xiao; Shu Yang; Jen-tse Huang; Kun Wang; Wenxuan Wang; Shuai Wang (CoRR, 2025)

Towards Evaluating Proactive Risk Awareness of Multimodal Language Models - Youliang Yuan; Wenxiang Jiao; Yuejin Xie; Chihao Shen; Menghan Tian; Wenxuan Wang; Jen-tse Huang; Pinjia He (CoRR, 2025)

InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews - Xintao Wang; Yunze Xiao; Jen-tse Huang; Siyu Yuan; Rui Xu; Haoran Guo; Quan Tu; Yaying Fei; Ziang Leng; Wei Wang; Jiangjie Chen; Cheng Li; et al. (Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, 2024)

All Languages Matter: On the Multilingual Safety of LLMs - Wenxuan Wang; Zhaopeng Tu; Chang Chen; Youliang Yuan; Jen-tse Huang; Wenxiang Jiao; Michael R. Lyu (Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, 2024)

Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models - Wenxuan Wang; Wenxiang Jiao; Jingyuan Huang; Ruyi Dai; Jen-tse Huang; Zhaopeng Tu; Michael R. Lyu (Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, 2024)

LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models - Yuxuan Wan; Wenxuan Wang; Yiliu Yang; Youliang Yuan; Jen-tse Huang; Pinjia He; Wenxiang Jiao; Michael R. Lyu (Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, 2024)

On the Reliability of Psychological Scales on Large Language Models - Jen-tse Huang; Wenxiang Jiao; Man Ho Lam; Eric John Li; Wenxuan Wang; Michael R. Lyu (Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, 2024)

InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context - Ziyi Liu; Abhishek Anand; Pei Zhou; Jen-tse Huang; Jieyu Zhao (Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, 2024)

On the Humanity of Conversational AI: Evaluating the Psychological Portrayal of LLMs - Jen-tse Huang; Wenxuan Wang; Eric John Li; Man Ho Lam; Shujie Ren; Youliang Yuan; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024)

GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang; Jen-tse Huang; Pinjia He; Shuming Shi; Zhaopeng Tu (The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024)

New Job, New Gender? Measuring the Social Bias in Image Generation Models - Wenxuan Wang; Haonan Bai; Jen-tse Huang; Yuxuan Wan; Youliang Yuan; Haoyi Qiu; Nanyun Peng; Michael R. Lyu (Proceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 - 1 November 2024, 2024)

Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans - Jen-tse Huang; Man Ho Lam; Eric John Li; Shujie Ren; Wenxuan Wang; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024)

A & B == B & A: Triggering Logical Reasoning Failures in Large Language Models - Yuxuan Wan; Wenxuan Wang; Yiliu Yang; Youliang Yuan; Jen-tse Huang; Pinjia He; Wenxiang Jiao; Michael R. Lyu (CoRR, 2024)

The Earth is Flat? Unveiling Factual Errors in Large Language Models - Wenxuan Wang; Juluan Shi; Zhaopeng Tu; Youliang Yuan; Jen-tse Huang; Wenxiang Jiao; Michael R. Lyu (CoRR, 2024)

New Job, New Gender? Measuring the Social Bias in Image Generation Models - Wenxuan Wang; Haonan Bai; Jen-tse Huang; Yuxuan Wan; Youliang Yuan; Haoyi Qiu; Nanyun Peng; Michael R. Lyu (CoRR, 2024)

How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments - Jen-tse Huang; Eric John Li; Man Ho Lam; Tian Liang; Wenxuan Wang; Youliang Yuan; Wenxiang Jiao; Xing Wang; Zhaopeng Tu; Michael R. Lyu (CoRR, 2024)

How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO - Man Tik Ng; Hui Tung Tse; Jen-tse Huang; Jingjing Li; Wenxuan Wang; Michael R. Lyu (CoRR, 2024)

A Unified Debugging Approach via LLM-Based Multi-Agent Synergy - Cheryl Lee; Chunqiu Steven Xia; Jen-tse Huang; Zhouruixin Zhu; Lingming Zhang; Michael R. Lyu (CoRR, 2024)

InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context - Ziyi Liu; Abhishek Anand; Pei Zhou; Jen-tse Huang; Jieyu Zhao (CoRR, 2024)

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang; Jen-tse Huang; Jiahao Xu; Tian Liang; Pinjia He; Zhaopeng Tu (CoRR, 2024)

On the Resilience of Multi-Agent Systems with Malicious Agents - Jen-tse Huang; Jiaxu Zhou; Tailin Jin; Xuhui Zhou; Zixi Chen; Wenxuan Wang; Youliang Yuan; Maarten Sap; Michael R. Lyu (CoRR, 2024)

Learning to Ask: When LLMs Meet Unclear Instruction - Wenxuan Wang; Juluan Shi; Chaozheng Wang; Cheryl Lee; Youliang Yuan; Jen-tse Huang; Michael R. Lyu (CoRR, 2024)

Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step - Wenxuan Wang; Kuiyi Gao; Zihan Jia; Youliang Yuan; Jen-tse Huang; Qiuzhi Liu; Shuai Wang; Wenxiang Jiao; Zhaopeng Tu (CoRR, 2024)

InstantIR: Blind Image Restoration with Instant Generative Reference - Jen-Yuan Huang; Haofan Wang; Qixun Wang; Xu Bai; Hao Ai; Peng Xing; Jen-Tse Huang (CoRR, 2024)

Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs - Xiaoyuan Liu; Wenxuan Wang; Youliang Yuan; Jen-tse Huang; Qiuzhi Liu; Pinjia He; Zhaopeng Tu (CoRR, 2024)

On the Shortcut Learning in Multilingual Neural Machine Translation - Wenxuan Wang; Wenxiang Jiao; Jen-tse Huang; Zhaopeng Tu; Michael R. Lyu (CoRR, 2024)

Improving the Transferability of Adversarial Samples by Path-Augmented Method - Jianping Zhang; Jen-tse Huang; Wenxuan Wang; Yichen Li; Weibin Wu; Xiaosen Wang; Yuxin Su; Michael R. Lyu (IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, 2023)

ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback - Wenxiang Jiao; Jen-tse Huang; Wenxuan Wang; Zhiwei He; Tian Liang; Xing Wang; Shuming Shi; Zhaopeng Tu (Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, 2023)

MTTM: Metamorphic Testing for Textual Content Moderation Software - Wenxuan Wang; Jen-tse Huang; Weibin Wu; Jianping Zhang; Yizhan Huang; Shuqing Li; Pinjia He; Michael R. Lyu (45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023, 2023)

An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software - Wenxuan Wang; Jingyuan Huang; Jen-tse Huang; Chang Chen; Jiazhen Gu; Pinjia He; Michael R. Lyu (38th IEEE/ACM International Conference on Automated Software Engineering, ASE 2023, Luxembourg, September 11-15, 2023, 2023)

Is ChatGPT A Good Translator? A Preliminary Study - Wenxiang Jiao; Wenxuan Wang; Jen-tse Huang; Xing Wang; Zhaopeng Tu (CoRR, 2023)

MTTM: Metamorphic Testing for Textual Content Moderation Software - Wenxuan Wang; Jen-tse Huang; Weibin Wu; Jianping Zhang; Yizhan Huang; Shuqing Li; Pinjia He; Michael R. Lyu (CoRR, 2023)

Improving the Transferability of Adversarial Samples by Path-Augmented Method - Jianping Zhang; Jen-tse Huang; Wenxuan Wang; Yichen Li; Weibin Wu; Xiaosen Wang; Yuxin Su; Michael R. Lyu (CoRR, 2023)

ParroT: Translating During Chat Using Large Language Models - Wenxiang Jiao; Jen-tse Huang; Wenxuan Wang; Xing Wang; Shuming Shi; Zhaopeng Tu (CoRR, 2023)

ChatGPT an ENFJ, Bard an ISTJ: Empirical Study on Personalities of Large Language Models - Jen-tse Huang; Wenxuan Wang; Man Ho Lam; Eric John Li; Wenxiang Jiao; Michael R. Lyu (CoRR, 2023)

Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench - Jen-tse Huang; Man Ho Lam; Eric John Li; Shujie Ren; Wenxuan Wang; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (CoRR, 2023)

GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher - Youliang Yuan; Wenxiang Jiao; Wenxuan Wang; Jen-tse Huang; Pinjia He; Shuming Shi; Zhaopeng Tu (CoRR, 2023)

An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software - Wenxuan Wang; Jingyuan Huang; Jen-tse Huang; Chang Chen; Jiazhen Gu; Pinjia He; Michael R. Lyu (CoRR, 2023)

All Languages Matter: On the Multilingual Safety of Large Language Models - Wenxuan Wang; Zhaopeng Tu; Chang Chen; Youliang Yuan; Jen-tse Huang; Wenxiang Jiao; Michael R. Lyu (CoRR, 2023)

Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench - Jen-tse Huang; Wenxuan Wang; Eric John Li; Man Ho Lam; Shujie Ren; Youliang Yuan; Wenxiang Jiao; Zhaopeng Tu; Michael R. Lyu (CoRR, 2023)

Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models - Wenxuan Wang; Wenxiang Jiao; Jingyuan Huang; Ruyi Dai; Jen-tse Huang; Zhaopeng Tu; Michael R. Lyu (CoRR, 2023)

Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models - Tian Liang; Zhiwei He; Jen-tse Huang; Wenxuan Wang; Wenxiang Jiao; Rui Wang; Yujiu Yang; Zhaopeng Tu; Shuming Shi; Xing Wang (CoRR, 2023)

Improving Adversarial Transferability via Neuron Attribution-based Attacks - Jianping Zhang; Weibin Wu; Jen-tse Huang; Yizhan Huang; Wenxuan Wang; Yuxin Su; Michael R. Lyu (IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 2022)

AEON: a method for automatic evaluation of NLP test cases - Jen-tse Huang; Jianping Zhang; Wenxuan Wang; Pinjia He; Yuxin Su; Michael R. Lyu (ISSTA '22: 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, South Korea, July 18 - 22, 2022, 2022)

Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages - Wenxiang Jiao; Zhaopeng Tu; Jiarui Li; Wenxuan Wang; Jen-tse Huang; Shuming Shi (Proceedings of the Seventh Conference on Machine Translation, WMT 2022, Abu Dhabi, United Arab Emirates (Hybrid), December 7-8, 2022, 2022)

Improving Adversarial Transferability via Neuron Attribution-Based Attacks - Jianping Zhang; Weibin Wu; Jen-tse Huang; Yizhan Huang; Wenxuan Wang; Yuxin Su; Michael R. Lyu (CoRR, 2022)

AEON: A Method for Automatic Evaluation of NLP Test Cases - Jen-tse Huang; Jianping Zhang; Wenxuan Wang; Pinjia He; Yuxin Su; Michael R. Lyu (CoRR, 2022)

Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages - Wenxiang Jiao; Zhaopeng Tu; Jiarui Li; Wenxuan Wang; Jen-tse Huang; Shuming Shi (CoRR, 2022)