Открытая языковая модель от университета Фудань, поддерживающая работу с внешними плагинами. Благодаря оптимизации, этот ИИ можно запускать на потребительских видеокартах, получая высокую производительность в генерации кода и ведении диалогов.
MOSS is an open-sourced plugin-augmented conversational language model. moss-moon models have 16B parameters, allowing users to perform inference on a single A100 GPU or 2 NVIDIA 3090 GPUs with FP16 precision, and on a single NVIDIA 3090 GPU with INT-4/8 precision. The base language model of MOSS was pre-trained on ~700B English, Chinese, and code tokens, including the PILE, BigQuery, BigPython, and our private Chinese corpus. The base model was then fine-tuned on multi-turn plugin-augmented conversational data. Finally, we performed preference-aware training to further improve the model.