Electrical Engineering and Systems Science > Audio and Speech Processing
[Submitted on 27 Nov 2017 (this version), latest version 23 Jan 2018 (v2)]
Title:Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Model
View PDFAbstract:Phoneme-based multilingual training and different cross-lingual adaptation techniques for Automatic Speech Recognition (ASR) are explored in Connectionist Temporal Classification (CTC)-based systems. The multilingual model is trained to model a universal IPA-based phone set using CTC loss function. While the same IPA symbol may not correspond to acoustic similarity, Learning Hidden Unit Contribution (LHUC) is investigated. Given the multilingual model, different approaches are exploited and compared to adapt the multilingual model to a target language with limited adaptation data. In addition, dropout during cross-lingual adaptation is also studied and tested in order to mitigate the overfitting problem.
Experiments show that the performance of the universal phoneme-based CTC system can be improve by apply LHUC and it is extensible to new phonemes during cross-lingual adaptation. Updating all the parameters shows consistently improvement on limited data. Applying dropout during adaptation can further improve the system and achieve competitive performance with Deep Neural Network (DNN)/ Hidden Markov Model (HMM) systems even on 21 hours data.
Submission history
From: Sibo Tong [view email][v1] Mon, 27 Nov 2017 22:23:52 UTC (93 KB)
[v2] Tue, 23 Jan 2018 14:35:03 UTC (94 KB)
Current browse context:
eess.AS
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.