Иерархическая лог-билинейная модель (HLBL) эффективно решает проблему медленного обучения нейросетевых языковых моделей. Используя структуру бинарного дерева, этот ИИ обрабатывает данные в разы быстрее классических n-gram моделей, сохраняя высокую точность прогнозирования.
Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models. The main drawback of NPLMs is their extremely long training and testing times. Morin and Bengio have proposed a hierarchical language model built around a binary tree of words, which was two orders of magnitude faster than the non-hierarchical model it was based on. However, it performed considerably worse than its non-hierarchical counterpart in spite of using a word tree created using expert knowledge. We introduce a fast hierarchical language model along with a simple feature-based algorithm for automatic construction of word trees from the data. We then show that the resulting models can outperform non-hierarchical neural models as well as the best n-gram models.