Эта нейросеть использует инновационный подход к созданию эмбеддингов слов на основе символьных n-грамм. Комбинируя классические векторы с информацией о символах, ИИ достигает высокой точности в задачах языкового моделирования, лучше понимая структуру слов.
This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.