Scaling Speech Technology to 1, 000+ Languages. - Pratap, Vineel; Tjandra, Andros; Shi, Bowen; Tomasello, Paden; Babu, Arun; Kundu, Sayani; Elkahky, Ali; Ni, Zhaoheng; Vyas, Apoorv; Fazel-Zarandi, Maryam; Baevski, Alexei; Adi, Yossi; et al. (J. Mach. Learn. Res., 2024)

Stack-and-Delay: A New Codebook Pattern for Music Generation. - Lan, Gaël Le; Nagaraja, Varun; Chang, Ernie; Kant, David; Ni, Zhaoheng; Shi, Yangyang; Iandola, Forrest N.; Chandra, Vikas (2024)

An Empirical Study on the Impact of Positional Encoding in Transformer-Based Monaural Speech Enhancement. - Zhang, Qiquan; Ge, Meng; Zhu, Hongxu; Ambikairajah, Eliathamby; Song, Qi; Ni, Zhaoheng; Li, Haizhou (2024)

On the Open Prompt Challenge in Conditional Audio Generation. - Chang, Ernie; Srinivasan, Sidd; Luthra, Mahi; Lin, Pin-Jie; Nagaraja, Varun; Iandola, Forrest N.; Liu, Zechun; Ni, Zhaoheng; Zhao, Changsheng; Shi, Yangyang; Chandra, Vikas (2024)

Less Peaky and More Accurate CTC Forced Alignment by Label Priors. - Huang, Ruizhe; Zhang, Xiaohui; Ni, Zhaoheng; Sun, Li; Hira, Moto; Hwang, Jeff; Manohar, Vimal; Pratap, Vineel; Wiesner, Matthew; Watanabe, Shinji; Povey, Daniel; Khudanpur, Sanjeev (2024)

Folding Attention: Memory and Power Optimization for On-Device Transformer-Based Streaming Speech Recognition. - Li, Yang; Lai, Liangzhen; Shangguan, Yuan; Iandola, Forrest N.; Ni, Zhaoheng; Chang, Ernie; Shi, Yangyang; Chandra, Vikas (2024)

Foleygen: Visually-Guided Audio Generation. - Mei, Xinhao; Nagaraja, Varun; Lan, Gaël Le; Ni, Zhaoheng; Chang, Ernie; Shi, Yangyang; Chandra, Vikas (2024)

Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition. - Shi, Hao; Gao, Yuan; Ni, Zhaoheng; Kawahara, Tatsuya (2024)

Data Efficient Reflow for Few Step Audio Generation. - Wu, Lemeng; Ni, Zhaoheng; Shi, Bowen; Lan, Gaël Le; Kumar, Anurag; Nagaraja, Varun; Mei, Xinhao; Xiong, Yunyang; Soran, Bilge; Krishnamoorthi, Raghuraman; Hsu, Wei-Ning; Shi, Yangyang; et al. (2024)

Massively Multilingual Forced Aligner Leveraging Self-Supervised Discrete Units. - Inaguma, Hirofumi; Kulikov, Ilia; Ni, Zhaoheng; Popuri, Sravya; Tomasello, Paden (2024)

An Empirical Study on the Impact of Positional Encoding in Transformer-based Monaural Speech Enhancement. - Zhang, Qiquan; Ge, Meng; Zhu, Hongxu; Ambikairajah, Eliathamby; Song, Qi; Ni, Zhaoheng; Li, Haizhou (CoRR, 2024)

Less Peaky and More Accurate CTC Forced Alignment by Label Priors. - Huang, Ruizhe; Zhang, Xiaohui; Ni, Zhaoheng; Sun, Li; Hira, Moto; Hwang, Jeff; Manohar, Vimal; Pratap, Vineel; Wiesner, Matthew; Watanabe, Shinji; Povey, Daniel; Khudanpur, Sanjeev (CoRR, 2024)

URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement. - Zhang, Wangyou; Scheibler, Robin; Saijo, Kohei; Cornell, Samuele; Li, Chenda; Ni, Zhaoheng; Kumar, Anurag; Pirklbauer, Jan; Sach, Marvin; Watanabe, Shinji; Fingscheidt, Tim; Qian, Yanmin (CoRR, 2024)

High Fidelity Text-Guided Music Generation and Editing via Single-Stage Flow Matching. - Lan, Gaël Le; Shi, Bowen; Ni, Zhaoheng; Srinivasan, Sidd; Kumar, Anurag; Ellis, Brian; Kant, David; Nagaraja, Varun; Chang, Ernie; Hsu, Wei-Ning; Shi, Yangyang; Chandra, Vikas (CoRR, 2024)

Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition. - Shi, Hao; Gao, Yuan; Ni, Zhaoheng; Kawahara, Tatsuya (CoRR, 2024)

SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text. - Liu, Haohe; Lan, Gaël Le; Mei, Xinhao; Ni, Zhaoheng; Kumar, Anurag; Nagaraja, Varun; Wang, Wenwu; Plumbley, Mark D.; Shi, Yangyang; Chandra, Vikas (CoRR, 2024)

Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding. - Zhao, Jiahui; Shi, Hao; Cui, Chenrui; Wang, Tianrui; Liu, Hexin; Ni, Zhaoheng; Ye, Lingxuan; Wang, Longbiao (CoRR, 2024)

Software Design and User Interface of ESPnet-SE++: Speech Enhancement for Robust Speech Processing. - Lu, Yen-Ju; Chang, Xuankai; Li, Chenda; Zhang, Wangyou; Cornell, Samuele; Ni, Zhaoheng; Masuyama, Yoshiki; Yan, Brian; Scheibler, Robin; Wang, Zhong-Qiu; Tsao, Yu; Qian, Yanmin; et al. (J. Open Source Softw., 2023)

A Time-Frequency Attention Module for Neural Speech Enhancement. - Zhang, Qiquan; Qian, Xinyuan; Ni, Zhaoheng; Nicolson, Aaron; Ambikairajah, Eliathamby; Li, Haizhou (IEEE ACM Trans. Audio Speech Lang. Process., 2023)

ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit. - Yan, Brian; Shi, Jiatong; Tang, Yun; Inaguma, Hirofumi; Peng, Yifan; Dalmia, Siddharth; Polak, Peter; Fernandes, Patrick; Berrebbi, Dan; Hayashi, Tomoki; Zhang, Xiaohui; Ni, Zhaoheng; et al. (2023)

TorchAudio 2.1: Advancing Speech Recognition, Self-Supervised Learning, and Audio Processing Components for Pytorch. - Hwang, Jeff; Hira, Moto; Chen, Caroline; Zhang, Xiaohui; Ni, Zhaoheng; Sun, Guangzhi; Ma, Pingchuan; Huang, Ruizhe; Pratap, Vineel; Zhang, Yuekai; Kumar, Anurag; Yu, Chin-Yun; et al. (2023)

Torchaudio-Squim: Reference-Less Speech Quality and Intelligibility Measures in Torchaudio. - Kumar, Anurag; Tan, Ke; Ni, Zhaoheng; Manocha, Pranay; Zhang, Xiaohui; Henderson, Ethan; Xu, Buye (2023)

Ripple Sparse Self-Attention for Monaural Speech Enhancement. - Zhang, Qiquan; Zhu, Hongxu; Song, Qi; Qian, Xinyuan; Ni, Zhaoheng; Li, Haizhou (2023)

Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute. - Chen, William; Chang, Xuankai; Peng, Yifan; Ni, Zhaoheng; Maiti, Soumi; Watanabe, Shinji (2023)

Software Design and User Interface of ESPnet-SE++: Speech Enhancement for Robust Speech Processing (espnet-v.202310). - Lu, Yen-Ju; Chang, Xuankai; Li, Chenda; Zhang, Wangyou; Cornell, Samuele; Ni, Zhaoheng; Masuyama, Yoshiki; Yan, Brian; Scheibler, Robin; Wang, Zhong-Qiu; Tsao, Yu; Qian, Yanmin; et al. (Zenodo, 2023)

Ripple sparse self-attention for monaural speech enhancement. - Zhang, Qiquan; Zhu, Hongxu; Song, Qi; Qian, Xinyuan; Ni, Zhaoheng; Li, Haizhou (CoRR, 2023)

Scaling Speech Technology to 1, 000+ Languages. - Pratap, Vineel; Tjandra, Andros; Shi, Bowen; Tomasello, Paden; Babu, Arun; Kundu, Sayani; Elkahky, Ali; Ni, Zhaoheng; Vyas, Apoorv; Fazel-Zarandi, Maryam; Baevski, Alexei; Adi, Yossi; et al. (CoRR, 2023)

Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute. - Chen, William; Chang, Xuankai; Peng, Yifan; Ni, Zhaoheng; Maiti, Soumi; Watanabe, Shinji (CoRR, 2023)

Enhance audio generation controllability through representation similarity regularization. - Shi, Yangyang; Lan, Gaël Le; Nagaraja, Varun; Ni, Zhaoheng; Mei, Xinhao; Chang, Ernie; Iandola, Forrest N.; Liu, Yang; Chandra, Vikas (CoRR, 2023)

Stack-and-Delay: a new codebook pattern for music generation. - Lan, Gaël Le; Nagaraja, Varun; Chang, Ernie; Kant, David; Ni, Zhaoheng; Shi, Yangyang; Iandola, Forrest N.; Chandra, Vikas (CoRR, 2023)

FoleyGen: Visually-Guided Audio Generation. - Mei, Xinhao; Nagaraja, Varun; Lan, Gaël Le; Ni, Zhaoheng; Chang, Ernie; Shi, Yangyang; Chandra, Vikas (CoRR, 2023)

TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch. - Hwang, Jeff; Hira, Moto; Chen, Caroline; Zhang, Xiaohui; Ni, Zhaoheng; Sun, Guangzhi; Ma, Pingchuan; Huang, Ruizhe; Pratap, Vineel; Zhang, Yuekai; Kumar, Anurag; Yu, Chin-Yun; et al. (CoRR, 2023)

On The Open Prompt Challenge In Conditional Audio Generation. - Chang, Ernie; Srinivasan, Sidd; Luthra, Mahi; Lin, Pin-Jie; Nagaraja, Varun; Iandola, Forrest N.; Liu, Zechun; Ni, Zhaoheng; Zhao, Changsheng; Shi, Yangyang; Chandra, Vikas (CoRR, 2023)

Torchaudio: Building Blocks for Audio and Speech Processing. - Yang, Yao-Yuan; Hira, Moto; Ni, Zhaoheng; Astafurov, Artyom; Chen, Caroline; Puhrsch, Christian; Pollack, David; Genzel, Dmitriy; Greenberg, Donny; Yang, Edward Z.; Lian, Jason; Hwang, Jeff; et al. (2022)

Time-Frequency Attention for Monaural Speech Enhancement. - Zhang, Qiquan; Song, Qi; Ni, Zhaoheng; Nicolson, Aaron; Li, Haizhou (2022)

Towards Low-Distortion Multi-Channel Speech Enhancement: The ESPNET-Se Submission to the L3DAS22 Challenge. - Lu, Yen-Ju; Cornell, Samuele; Chang, Xuankai; Zhang, Wangyou; Li, Chenda; Ni, Zhaoheng; Wang, Zhong-Qiu; Watanabe, Shinji (2022)

ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding. - Lu, Yen-Ju; Chang, Xuankai; Li, Chenda; Zhang, Wangyou; Cornell, Samuele; Ni, Zhaoheng; Masuyama, Yoshiki; Yan, Brian; Scheibler, Robin; Wang, Zhong-Qiu; Tsao, Yu; Qian, Yanmin; et al. (2022)

Towards Low-distortion Multi-channel Speech Enhancement: The ESPNet-SE Submission to The L3DAS22 Challenge. - Lu, Yen-Ju; Cornell, Samuele; Chang, Xuankai; Zhang, Wangyou; Li, Chenda; Ni, Zhaoheng; Wang, Zhong-Qiu; Watanabe, Shinji (CoRR, 2022)

ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding. - Lu, Yen-Ju; Chang, Xuankai; Li, Chenda; Zhang, Wangyou; Cornell, Samuele; Ni, Zhaoheng; Masuyama, Yoshiki; Yan, Brian; Scheibler, Robin; Wang, Zhong-Qiu; Tsao, Yu; Qian, Yanmin; et al. (CoRR, 2022)

WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation. - Ni, Zhaoheng; Xu, Yong; Yu, Meng; Wu, Bo; Zhang, Shi-Xiong; Yu, Dong; Mandel, Michael I. (2021)

TorchAudio: Building Blocks for Audio and Speech Processing. - Yang, Yao-Yuan; Hira, Moto; Ni, Zhaoheng; Chourdia, Anjali; Astafurov, Artyom; Chen, Caroline; Yeh, Ching-Feng; Puhrsch, Christian; Pollack, David; Genzel, Dmitriy; Greenberg, Donny; Yang, Edward Z.; et al. (CoRR, 2021)

Time-Frequency Attention for Monaural Speech Enhancement. - Zhang, Qiquan; Song, Qi; Ni, Zhaoheng; Nicolson, Aaron; Li, Haizhou (CoRR, 2021)

Mask-Dependent Phase Estimation for Monaural Speaker Separation. - Ni, Zhaoheng; Mandel, Michael I. (2020)

WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation. - Ni, Zhaoheng; Xu, Yong; Yu, Meng; Wu, Bo; Zhang, Shi-Xiong; Yu, Dong; Mandel, Michael I. (CoRR, 2020)

Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks. - Grèzes, Félix; Ni, Zhaoheng; Trinh, Viet Anh; Mandel, Michael I. (CoRR, 2020)

Improved MVDR Beamforming Using LSTM Speech Models to Clean Spatial Clustering Masks. - Ni, Zhaoheng; Grèzes, Félix; Trinh, Viet Anh; Mandel, Michael I. (CoRR, 2020)

Combining Spatial Clustering with LSTM Speech Models for Multichannel Speech Enhancement. - Grèzes, Félix; Ni, Zhaoheng; Trinh, Viet Anh; Mandel, Michael I. (CoRR, 2020)

Onssen: an open-source speech separation and enhancement library. - Ni, Zhaoheng; Mandel, Michael I. (CoRR, 2019)

Unusable Spoken Response Detection with BLSTM Neural Networks. - Ni, Zhaoheng; Ubale, Rutuja; Qian, Yao; Mandel, Michael I.; Yoon, Su-Youn; Misra, Abhinav; Suendermann-Oeft, David (2018)

Sound Signal Processing with Seq2Tree Network. - Ma, Weicheng; Cao, Kai; Ni, Zhaoheng; Chin, Peter; Li, Xiang (2018)

Confused or not Confused?: Disentangling Brain Activity from EEG Data Using Bidirectional LSTM Recurrent Neural Networks. - Ni, Zhaoheng; Yuksel, Ahmet Cem; Ni, Xiuyan; Mandel, Michael I.; Xie, Lei (2017)

A Sep2Tree Model for Recognizing Synthetic Bach Chorales. - Ni, Xiuyan; Ma, Weicheng; Ni, Zhaoheng; Haralick, Robert M. (2017)