Fuyu-Heavy от Adept — мультимодальный гигант, созданный специально для управления цифровыми агентами и понимания пользовательских интерфейсов. Несмотря на компактность, этот ИИ уступает в визуальном анализе лишь лидерам рынка вроде GPT-4V, мастерски справляясь с чтением UI-элементов.
We’re excited to introduce Adept Fuyu-Heavy, a new multimodal model designed specifically for digital agents. Fuyu-Heavy is the world’s third-most-capable multimodal model, behind only GPT4-V and Gemini Ultra, which are 10-20 times bigger. We’re excited about this model because: It excels at multimodal reasoning. To us the killer feature is UI understanding, but it also performs well on more traditional multimodal benchmarks. In particular, Fuyu-Heavy scores higher on the MMMU benchmark than even Gemini Pro. On standard text-based benchmarks, it matches or exceeds the performance of models in the same compute class despite having to devote some of its capacity to image modeling. It demonstrates that (with some modifications) we can scale up the Fuyu architecture and reap all of the associated benefits, including handling arbitrary size/shape images and efficiently re-using existing transformer optimizations.