SB-LM — это классическая языковая модель от Google, использующая эффективный алгоритм «глупого отката» (Stupid Backoff) для машинного перевода. Она оптимизирует работу с n-граммами в огромных текстовых корпусах, обеспечивая высокую скорость обработки без потери качества контекста.
Systems, methods, and computer program products for machine translation are provided. In some implementations a system is provided. The system includes a language model including a collection of n-grams from a corpus, each n-gram having a corresponding relative frequency in the corpus and an order n corresponding to a number of tokens in the n-gram, each n-gram corresponding to a backoff n-gram having an order of n-1 and a collection of backoff scores, each backoff score associated with an n-gram, the backoff score determined as a function of a backoff factor and a relative frequency of a corresponding backoff n-gram in the corpus.