Reliability-aware Dynamic Feature Composition for Name Tagging

Ying Lin, Liyuan Liu, Heng Ji, Dong Yu, Jiawei Han


Abstract
Word embeddings are widely used on a variety of tasks and can substantially improve the performance. However, their quality is not consistent throughout the vocabulary due to the long-tail distribution of word frequency. Without sufficient contexts, rare word embeddings are usually less reliable than those of common words. However, current models typically trust all word embeddings equally regardless of their reliability and thus may introduce noise and hurt the performance. Since names often contain rare and uncommon words, this problem is particularly critical for name tagging. In this paper, we propose a novel reliability-aware name tagging model to tackle this issue. We design a set of word frequency-based reliability signals to indicate the quality of each word embedding. Guided by the reliability signals, the model is able to dynamically select and compose features such as word embedding and character-level representation using gating mechanisms. For example, if an input word is rare, the model relies less on its word embedding and assigns higher weights to its character and contextual features. Experiments on OntoNotes 5.0 show that our model outperforms the baseline model by 2.7% absolute gain in F-score. In cross-genre experiments on five genres in OntoNotes, our model improves the performance for most genre pairs and obtains up to 5% absolute F-score gain.
Anthology ID:
P19-1016
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
165–174
Language:
URL:
https://aclanthology.org/P19-1016
DOI:
10.18653/v1/P19-1016
Bibkey:
Cite (ACL):
Ying Lin, Liyuan Liu, Heng Ji, Dong Yu, and Jiawei Han. 2019. Reliability-aware Dynamic Feature Composition for Name Tagging. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 165–174, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Reliability-aware Dynamic Feature Composition for Name Tagging (Lin et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1016.pdf
Video:
 https://aclanthology.org/P19-1016.mp4
Data
OntoNotes 5.0