Learning Topic Models by Neighborhood Aggregation
Learning Topic Models by Neighborhood Aggregation
Ryohei Hisano
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Main track. Pages 2498-2505.
https://doi.org/10.24963/ijcai.2019/347
Topic models are frequently used in machine learning owing to their high interpretability and modular structure. However, extending a topic model to include a supervisory signal, to incorporate pre-trained word embedding vectors and to include a nonlinear output function is not an easy task because one has to resort to a highly intricate approximate inference procedure. The present paper shows that topic modeling with pre-trained word embedding vectors can be viewed as implementing a neighborhood aggregation algorithm where messages are passed through a network defined over words. From the network view of topic models, nodes correspond to words in a document and edges correspond to either a relationship describing co-occurring words in a document or a relationship describing the same word in the corpus. The network view allows us to extend the model to include supervisory signals, incorporate pre-trained word embedding vectors and include a nonlinear output function in a simple manner. In experiments, we show that our approach outperforms the state-of-the-art supervised Latent Dirichlet Allocation implementation in terms of held-out document classification tasks.
Keywords:
Machine Learning: Classification
Machine Learning: Semi-Supervised Learning