Comparison of Word Embeddings of Unaligned Audio and Text Data Using Persistent Homology

Z Yessenbayev, Z Kozhirbayev - International Conference on Speech and …, 2022 - Springer
International Conference on Speech and Computer, 2022Springer
We have performed preliminary work on topological analysis of audio and text data for
unsupervised speech processing. The work is based on the assumption that phoneme
frequencies and contextual relationships are similar in the acoustic and text domains for the
same language. Accordingly, this allowed the creation of a mapping between these spaces
that takes into account their geometric structure. As a first step, generative methods based
on variational autoencoders were chosen to map audio and text data into two latent vector …
Abstract
We have performed preliminary work on topological analysis of audio and text data for unsupervised speech processing. The work is based on the assumption that phoneme frequencies and contextual relationships are similar in the acoustic and text domains for the same language. Accordingly, this allowed the creation of a mapping between these spaces that takes into account their geometric structure. As a first step, generative methods based on variational autoencoders were chosen to map audio and text data into two latent vector spaces. In the next stage, persistent homology methods are used to analyze the topological structure of two spaces. Although the results obtained support the idea of the similarity of the two spaces, further research is needed to correctly map acoustic and text spaces, as well as to evaluate the real effect of including topological information in the autoencoder training process.
Springer
Showing the best result for this search. See all results