XuanYuan 2.0 представляет собой крупнейшую китайскую чат-модель с открытым исходным кодом, оптимизированную для финансовой сферы. Этот ИИ-гигант на 176 миллиардов параметров устанавливает новые стандарты в обработке естественного языка и аналитике сложных рыночных данных.
In recent years, pre-trained language models have undergone rapid development with the emergence of large-scale models. However, there is a lack of open-sourced chat models specifically designed for the Chinese language, especially in the field of Chinese finance, at the scale of hundreds of billions. To address this gap, we introduce XuanYuan 2.0, the largest Chinese chat model to date, built upon the BLOOM-176B architecture. Additionally, we propose a novel training method called hybrid-tuning to mitigate catastrophic forgetting. By combining general-domain with domain-specific knowledge and integrating the stages of pre-training and fine-tuning, XuanYuan 2.0 is capable of providing accurate and contextually appropriate responses in the Chinese financial domain.