The Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), in partnership with Petuum and LLM360, proudly announces the global launch of K2-65B, a revolutionary 65-billion parameter large language model (LLM). K2-65B establishes new benchmarks for sustainable, transparent, and high-performing open-source artificial intelligence (AI).
K2-65B is designed to propel knowledge sharing, fundamental research, and technological transfer in generative artificial intelligence (AGI). Using LLM360’s framework, K2-65B fosters a collaborative environment for developing AGI through peer-reviewed, transparent, and reproducible open-source research. This groundbreaking model is available under the Apache 2.0 license, promoting global accessibility and innovation.
Trained on 1.4 trillion tokens with 480 A100 GPUs in NVIDIA’s DGX Cloud, K2-65B performs better using 35 per cent fewer resources than comparable models like Llama 2 70B. This efficiency makes K2-65B one of the most sustainable LLMs in its class. It excels in strategic areas such as mathematical and logical reasoning, competing with larger models like GPT-4.
K2-65B continues the UAE’s legacy of innovation in natural language processing (NLP), building on successes like the Arabic LLM Jais, developed in partnership with Core42, MBZUAI, and Cerebras Systems. The UAE is solidifying its reputation as a leader in AI by continuously advancing superior LLM development.
President and University Professor Eric Xing at MBZUAI emphasized the importance of K2-65B’s launch: “The UAE is demonstrating its growing prowess in superior LLM development. K2-65B showcases the power of open, collaborative approaches to creating high-performance, efficient models that can transform various sectors.”
K2-65B underwent extensive evaluation across 22 multidisciplinary assessments, outperforming Llama 2 70B in math, coding, and medicine. In competitive settings like the Open LLM Leaderboard, K2-65B and its chat model variant, K2-Chat, have consistently demonstrated superior performance.
Hector Liu, Head of Engineering at Petuum and lead developer of K2-65B, highlighted the significance of transparency: “Providing a reproducible blueprint for K2-65B will advance global research capabilities and development options for LLMs. Our detailed documentation will benefit the open-source ecosystem and encourage community engagement.”
K2-65B stands out for its transparency, supported by LLM360’s Pretraining and Developer Suites. These tools include comprehensive training guides, intermediate checkpoints, and evaluation results, ensuring full reproducibility and auditability. Additionally, K2-65B’s efficient resource use promotes sustainable computing practices worldwide.
Looking ahead, the development team plans to incorporate image understanding capabilities and pursue ongoing enhancements to K2-65B’s performance and versatility. The LLM360 Research Suite will continue to provide valuable insights into training dynamics, supporting further exploration by researchers and developers.
K2-65B’s launch marks a significant milestone in AI development, reinforcing the UAE’s commitment to advancing global AI research and innovation.
