Jiayang Song

Avenida Wai Long

Taipa, Macao SAR

About Me

I am currently an Assistant Professor in the School of Computer Science and Engineering at Macau University of Science and Technology. I received my Ph.D. (Software Engineering and Intelligent System) at the University of Alberta in 2025, M.Eng at the University of Toronto in 2021, and B.Eng at Western University in 2019.

Research Interest

My primary research direction revolves around Quality Assurance for Trustworthy AI systems such as AI-enabled Cyber-Physical systems (AI-CPS), Large Language Models (LLMs), and Multimodal AI Agents.

My research is devoted to exploring these directions in three stages: (1) testing and evaluating the performance and reliability of generative AI in dynamic, real-world environments, (2) developing methods to enhance the safety and resilience of complex AI systems, and (3) investigating the potential of multimodal foundation models to revolutionize the capabilities of AI applications.

My long-term vision is to create a cohesive research program that pushes the boundaries of AI systems by combining foundational models and cutting-edge quality assurance techniques.

Prospective Students

I am looking for self-motivated Ph.D. and MSc. students with strong programming skills and relevant research experience. Please send me an email with your CV and transcripts if you are interested.

News

Jan 05, 2026	Our paper Antidote or Placebo? Unraveling the Efficacy of Neuron Coverage Criteria on Testing Transformer-based Language Models is accepted at ACM Transactions on Software Engineering and Methodology (TOSEM).
May 08, 2025	Our paper Towards Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation: An Empirical Study is accepted at The ACM International Conference on the Foundations of Software Engineering (FSE).

Selected Publications

LLM

Look before you leap: An exploratory study of uncertainty measurement for large language models

Yuheng Huang, Jiayang Song, Zhijie Wang, and 4 more authors

IEEE Transactions on Software Engineering., 2024

Bib PDF

@article{huang2023look,
  title = {Look before you leap: An exploratory study of uncertainty measurement for large language models},
  author = {Huang, Yuheng and Song, Jiayang and Wang, Zhijie and Zhao, Shengming and Chen, Huaming and Juefei-Xu, Felix and Ma, Lei},
  journal = {IEEE Transactions on Software Engineering.},
  year = {2024},
  organization = {IEEE},
}

AI-CPS

When cyber-physical systems meet AI: A benchmark, an evaluation, and a way forward

Jiayang Song, Deyun Lyu, Zhenya Zhang, and 3 more authors

In Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice, 2022

Bib PDF

@inproceedings{song2022cyber,
  title = {When cyber-physical systems meet AI: A benchmark, an evaluation, and a way forward},
  author = {Song, Jiayang and Lyu, Deyun and Zhang, Zhenya and Wang, Zhijie and Zhang, Tianyi and Ma, Lei},
  booktitle = {Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice},
  pages = {343--352},
  year = {2022},
}

AI-CPS

SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems

Jiayang Song, Xuan Xie, and Lei Ma

IEEE Transactions on Software Engineering, 2023

Bib PDF

@article{song2023mathtt,
  title = {SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled Cyber-Physical Systems},
  author = {Song, Jiayang and Xie, Xuan and Ma, Lei},
  journal = {IEEE Transactions on Software Engineering},
  volume = {49},
  number = {8},
  pages = {4058--4080},
  year = {2023},
  publisher = {IEEE},
}

LLM

LUNA: A Model-Based Universal Analysis Framework for Large Language Models

Da Song, Xuan Xie, Jiayang Song, and 4 more authors

IEEE Transactions on Software Engineering, 2024

Bib PDF

@article{song2024luna,
  title = {LUNA: A Model-Based Universal Analysis Framework for Large Language Models},
  author = {Song, Da and Xie, Xuan and Song, Jiayang and Zhu, Derui and Huang, Yuheng and Juefei-Xu, Felix and Ma, Lei},
  journal = {IEEE Transactions on Software Engineering},
  year = {2024},
  publisher = {IEEE},
}

Robotics-RL

GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model

Zhehua Zhou, Xuan Xie, Jiayang Song, and 2 more authors

IEEE Transactions on Neural Networks and Learning Systems, 2024

Bib PDF

@article{10766903,
  author = {Zhou, Zhehua and Xie, Xuan and Song, Jiayang and Shu, Zhan and Ma, Lei},
  journal = {IEEE Transactions on Neural Networks and Learning Systems},
  title = {GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model},
  year = {2024},
  publisher = {IEEE},
  pages = {1-15},
}

LLM

Antidote or Placebo? Unraveling the Efficacy of Neuron Coverage Criteria on Testing Transformer-based Language Models

Xiaoning Ren, Jiayang Song, Chongyang Liu, and 3 more authors

TOSEM, 2026

Bib ArXiv

@article{10.1145/3787101,
  author = {Ren, Xiaoning and Song, Jiayang and Liu, Chongyang and Li, Jie and Ma, Lei and Xue, Yinxing},
  title = {Antidote or Placebo? Unraveling the Efficacy of Neuron Coverage Criteria on Testing Transformer-based Language Models},
  year = {2026},
  journal = {TOSEM},
}