Продвинутая рекуррентная нейросеть (RNN), которая переосмысляет классическое языковое моделирование за счет использования данных из промежуточных слоев. Благодаря методу матричной факторизации, эта AI-модель на 37 млн параметров демонстрирует высокую выразительность и точность предсказания текстов.
This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers. Our proposed method raises the expressive power of a language model based on the matrix factorization interpretation of language modeling introduced by Yang et al. (2018). The proposed method improves the current state-of-the-art language model and achieves the best score on the Penn Treebank and WikiText-2, which are the standard benchmark datasets. Moreover, we indicate our proposed method contributes to two application tasks: machine translation and headline generation. Our code is publicly available at: this https URL.