Модификация архитектуры RNN, сфокусированная на извлечении признаков из триграмм (символьных сочетаний длины 3). Такой подход позволяет ИИ-модели лучше справляться с редкими словами и опечатками, значительно повышая качество обработки естественного языка.
This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.