Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task
Abstract
:1. Introduction
Contributions
2. Vowel-to-Vowel Imitation Task
2.1. Articulatory Speech Synthesis Model
2.2. Vocal Tract State and Action Spaces
3. Method
3.1. Acoustic Representation
3.2. Learning Algorithm
- 1.
- Collect a set of state-action-state transitions using the policy and synthesizer f.
- 2.
- Train the forward-model to minimize the prediction error on the collected data.
- 3.
- Use the learned forward model to compute the policy loss function and backpropagate the error into the policy through the forward model.
- 4.
- Update policy parameters.
- 5.
- Go to 1.
3.3. Reference Dataset
3.4. Evaluation Metrics
4. Experiments
- Acoustic reference (ac) (original scenario). In this case, the policy had access only to the acoustic features of the reference samples
- Acoustic + articulatory reference (ac + vt). The policy had access to the full reference state meaning both acoustic and articulatory states
- Acoustic + partially observable articulatory reference (ac + partial vt). The policy had access to the acoustic features of the reference as well as six articulatory parameters defining the positioning of lips, jaw, and hyoid bone. Namely, the parameters were: HX, HY, JX, JA, LP, LP. The idea behind selecting these parameters is that they are visually accessible and can be derived by analysis of the facial movements of the speaker uttering the reference sound.
- Articulatory reference (vt). The policy observed only the articulatory state of the reference
5. Results and Discussion
6. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
References
- Mermelstein, P. Articulatory model for the study of speech production. J. Acoust. Soc. Am. 1973, 53, 1070–1082. [Google Scholar] [CrossRef] [PubMed]
- Maeda, S. Compensatory Articulation During Speech: Evidence from the Analysis and Synthesis of Vocal-Tract Shapes Using an Articulatory Model. In Speech Production and Speech Modelling; Springer: Dordrecht, The Netherlands, 1990; pp. 131–149. [Google Scholar] [CrossRef]
- Payan, Y.; Perrier, P. Synthesis of V-V sequences with a 2D biomechanical tongue model controlled by the Equilibrium Point Hypothesis. Speech Commun. 1997, 22, 185–205. [Google Scholar] [CrossRef]
- Badin, P.; Bailly, G.; Revéret, L.; Baciu, M.; Segebarth, C.; Savariaux, C. Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images. J. Phon. 2002, 30, 533–553. [Google Scholar] [CrossRef]
- Dang, J.; Honda, K. Construction and control of a physiological articulatory model. J. Acoust. Soc. Am. 2004, 115, 853–870. [Google Scholar] [CrossRef] [PubMed]
- van den Doel, K.; Vogt, F.; Elliot English, R.; Fels, S. Towards articulatory speech synthesis with a dynamic 3D finite element tongue model. In Proceedings of the ISSP 2006—7th International Seminar on Speech Production, Ubatuba, Brazil, 13–15 December 2006. [Google Scholar]
- Birkholz, P. Modeling Consonant-Vowel Coarticulation for Articulatory Speech Synthesis. PLoS ONE 2013, 8, e0060603. [Google Scholar] [CrossRef] [PubMed]
- Ohman, S.E.G. Numerical Model of Coarticulation. J. Acoust. Soc. Am. 1967, 41, 310–320. [Google Scholar] [CrossRef] [PubMed]
- Birkholz, P.; Kröger, B. Vocal tract model adaptation using magnetic resonance imaging. In Proceedings of the ISSP 2006—7th International Seminar on Speech Production, Ubatuba, Brazil, 13–15 December 2006. [Google Scholar]
- Lindblom, B.; Sussman, H.M. Dissecting coarticulation: How locus equations happen. J. Phon. 2012, 40, 1–19. [Google Scholar] [CrossRef]
- Saltzman, E.L. Dynamics and Coordinate Systems in Skilled Sensorimotor Activity. In Mind as Motion: Explorations in the Dynamics of Cognition; van Gelder, T., Port, R., Eds.; MIT Press: Cambridge, MA, USA, 1995; pp. 149–173. [Google Scholar]
- Wei, J.; lu, X.; Dang, J. A Model-Based Learning Process for Modeling Coarticulation of Human Speech. IEICE Trans. 2007, 90, 1582–1591. [Google Scholar] [CrossRef]
- Philippsen, A.K.; Reinhart, R.F.; Wrede, B. Goal babbling of acoustic-articulatory models with adaptive exploration noise. In Proceedings of the 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Cergy-Pontoise, France, 19–22 September 2016; pp. 72–78. [Google Scholar] [CrossRef]
- Gao, Y.; Steiner, P.; Birkholz, P. Articulatory Copy Synthesis using Long-Short Term Memory Networks. Stud. Zur Sprachkommun. Elektron. Sprachsignalverarb. 2020, 2020, 52–59. [Google Scholar]
- Igl, M.; Zintgraf, L.M.; Le, T.A.; Wood, F.; Whiteson, S. Deep Variational Reinforcement Learning for POMDPs. arXiv 2018, arXiv:1806.02426. [Google Scholar]
- Ha, D.; Schmidhuber, J. World Models. arXiv 2018, arXiv:1803.10122. [Google Scholar]
- Liang, X.; Wang, Q.; Feng, Y.; Liu, Z.; Huang, J. VMAV-C: A Deep Attention-based Reinforcement Learning Algorithm for Model-based Control. arXiv 2018, arXiv:1812.09968. [Google Scholar]
- Hafner, D.; Lillicrap, T.P.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; Davidson, J. Learning Latent Dynamics for Planning from Pixels. arXiv 2018, arXiv:1811.04551. [Google Scholar]
- Ha, D.; Schmidhuber, J. Recurrent World Models Facilitate Policy Evolution. In Advances in Neural Information Processing Systems 31; Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2018; pp. 2450–2462. [Google Scholar]
- Deisenroth, M.; Rasmussen, C. PILCO: A Model-Based and Data-Efficient Approach to Policy Search. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), Bellevue, WA, USA, 28 June–2 July 2011; pp. 465–472. [Google Scholar]
- Peng, X.B.; Abbeel, P.; Levine, S.; van de Panne, M. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. arXiv 2018, arXiv:1804.02717. [Google Scholar] [CrossRef]
- Shitov, D.; Pirogova, E.; Wysocki, T.; Lech, M. Learning Acoustic Word Embeddings with Dynamic Time Warping Triplet Networks. IEEE Access 2020, 8, 103327–103338. [Google Scholar] [CrossRef]
- Lloyd, J.E.; Stavness, I.; Fels, S. ARTISYNTH: A fast interactive biomechanical modeling toolkit combining multibody and finite element simulation. In Soft Tissue Biomechanical Modeling for Computer Assisted Surgery; Springer: Berlin/Heidelberg, Germany, 2012; pp. 355–394. [Google Scholar]
- Iskarous, K.; Goldstein, L.M.; Whalen, D.H.; Tiede, M.K.; Rubin, P.E. CASY: The Haskins Configurable Articulatory Synthesizer; International Congress of Phonetic Sciences: Barcelona, Spain, 2003. [Google Scholar]
- Tourville, J.; Guenther, F. The DIVA model: A neural theory of speech acquisition and production. Lang. Cogn. Process. 2011, 26, 952–981. [Google Scholar] [CrossRef] [PubMed]
- Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A. Automatic Differentiation in PyTorch. 2017. Available online: https://openreview.net/forum?id=BJJsrmfCZ (accessed on 1 January 2023).
- Jordan, M.I.; Rumelhart, D.E. Forward models: Supervised learning with a distal teacher. Cogn. Sci. 1992, 16, 307–354. [Google Scholar] [CrossRef]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32; Curran Associates, Inc.: Red Hook, NY, USA, 2019; pp. 8024–8035. [Google Scholar]
Index | Name | Description | Min | Max | Unit |
---|---|---|---|---|---|
0 | HX | Horiz. hyoid position | 0.0 | 1.0 | |
1 | HY | Vert. hyoid position | −6.0 | −3.5 | cm |
2 | JX | Horizontal jaw displacement | −0.5 | 0.0 | cm |
3 | JA | Jaw angle | −7.0 | 0.0 | deg |
4 | LP | Lip protrusion | −1.0 | 1.0 | |
5 | LD | Vert. lip distance | −2.0 | 4.0 | cm |
6 | VS | Velum shape | 0.0 | 1.0 | |
7 | VO | Velic opening | −0.1 | 1.0 | |
8 | TCX | Tongue body center X | −3.0 | 4.0 | cm |
9 | TCY | Tongue body center Y | −3.0 | 1.0 | cm |
10 | TTX | Tongue tip X | 1.5 | 5.5 | cm |
11 | TTY | Tongue tip Y | −3.0 | 2.5 | cm |
12 | TBX | Tongue blade X | −3.0 | 4.0 | cm |
13 | TBY | Tongue blade Y | −3.0 | 5.0 | cm |
14 | TRX | Tongue root X | −4.0 | 2.0 | cm |
15 | TRY | Tongue root Y | −6.0 | 0.0 | cm |
16 | TS1 | Tongue side elevation 1 | −1.4 | 1.4 | cm |
17 | TS2 | Tongue side elevation 2 | −1.4 | 1.4 | cm |
18 | TS3 | Tongue side elevation 3 | −1.4 | 1.4 | cm |
19 | TS4 | Tongue side elevation 4 | −1.4 | 1.4 | cm |
20 | MA1 | Min. area tongue back region | 0.0 | 0.3 | cm |
21 | MA2 | Min. area tongue tip region | 0.0 | 0.3 | cm |
22 | MA3 | Min. area lip region | 0.0 | 0.3 | cm |
Metric | Matching Classes | Non-Matching Classes | ||
---|---|---|---|---|
Mean | Std | Mean | Std | |
0.22 | 0.02 | 0.30 | 0.02 | |
0.14 | 0.02 | 0.22 | 0.03 | |
0.89 | 0.01 | 0.85 | 0.02 | |
0.88 | 0.02 | 0.78 | 0.05 |
Training Scenario | Metrics | |||
---|---|---|---|---|
vt | 0.21 | 0.14 | 0.89 | 0.88 |
ac + vt | 0.22 | 0.10 | 0.89 | 0.91 |
ac + partial vt | 0.24 | 0.15 | 0.88 | 0.88 |
ac | 0.23 | 0.18 | 0.88 | 0.84 |
optimal |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shitov, D.; Pirogova, E.; Wysocki, T.A.; Lech, M. Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors 2023, 23, 3437. https://doi.org/10.3390/s23073437
Shitov D, Pirogova E, Wysocki TA, Lech M. Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task. Sensors. 2023; 23(7):3437. https://doi.org/10.3390/s23073437
Chicago/Turabian StyleShitov, Denis, Elena Pirogova, Tadeusz A. Wysocki, and Margaret Lech. 2023. "Deep Reinforcement Learning for Articulatory Synthesis in a Vowel-to-Vowel Imitation Task" Sensors 23, no. 7: 3437. https://doi.org/10.3390/s23073437