nekomata-14b — это продвинутая языковая модель, дообученная на базе Qwen для безупречной работы с японским и английским языками. Этот ИИ демонстрирует высокую эффективность в обработке текстов благодаря расширенному словарю и обучению на 66 миллиардах токенов.
We conduct continual pre-training of qwen-14b on 66B tokens from a mixture of Japanese and English datasets. The continual pre-training significantly improves the model's performance on Japanese tasks. It also enjoys the following great features provided by the original Qwen model. * The inclusive Qwen vocabulary (vocab size > 150k) enables the model to processs Japanese texts much more efficiently than the previously released youri series. * The model supports a maximum sequence length of 8192. The name nekomata comes from the Japanese word 猫又/ねこまた/Nekomata, which is a kind of Japanese mythical creature (妖怪/ようかい/Youkai).