Artificial intelligence (AI) has been a part of existence since the 1950s. But what makes it so unique and different now is generative AI (gen AI). What GenAI has done is truly democratise the use of technology. This was made possible when Open AI used large language models (LLMs), that were processed by graphic processing units (GPUs). These GPUs made days and hours’ worth of tasks within minutes.
There was no need to be a coder or developer to use AI, anybody can now use AI. So, when ChatGPT came out, everyone simply loved experimenting and playing with it. But as these models saw more usage, there were issues, from AI hallucinations and disinformation to biased responses.
Adding to the data pool
With more users of gen AI, there has been a growing need for more startups and innovation to set new Large Language Models that are all-encompassing and have a larger data pool. What this does is make the data sets that the LLMs use to generate information larger, thus reducing bias.
At the forefront are the Gulf Corporation Council (GCC) countries. The region has already created their LLMs. Notably so, Arabic currently is spoken by over 310 million people and boasts numerous dialects. The linguistic richness presents unique challenges for AI models.
While startups and private companies have built their LLMs in the region, there are commendable strides in building bigger and larger models that can accommodate more data and inputs.
Saudi Arabia and the UAE have already made commendable strides. The UAE for example had acquired high-performance GPUs and NVIDIA chips. Here is a look at the LLMs of the region and what they are doing
Falcon by UAE
The Abu Dhabi government’s Technology Innovation Insititute (TII) in 2023 had released an LLM – Falcon 180B, it was the largest openly available LLM with over 180 billion parameters and had even outperformed the GPT 3.5 on the Massive Multitask Language Understanding (MMLU) benchmark.
This benchmark has been designed to measure knowledge that is gained during pretraining. This is done by evaluating models exclusive in zero-shot and few-shot settings. What this does is make the benchmark more challenging and like how humans are evaluated.
The Falcon 180B has trained on 3.5 trillion tokens of text, which was over 4x the amount of data used to train Llama 2. The team that worked on developing the model believes that innovation should be allowed to flourish and have open access.
Jais in UAE by Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Touted as the world’s highest-quality Arabic AI software, Jais is an open-source bilingual model built on both Arabic and English-language data. Built by Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), and Cerebras, the California-based AI company, Vicuna – a sustainable model in partnership with other universities.
In March, the MBZUAI Institute of Foundational Models (IFM) announced that it has been breaking new ground in GenAI tools by deploying specialised language and multimodal models.
The five models have been designed to make a real-world impact on healthcare, multimodal reasoning for geo-spatial sector, efficient LLMs for mobile devices, detailed visual reasoning etc. Each of these models have been developed from e
Each were developed from extensive research from the university’s faculty, students, and researchers. These have been tailored for specific needs across sectors.
Professor Timothy Baldwin, MBZUAI’s Acting Provost and Professor of Natural Language Processing, said, “These models show the ability of IFM to transform research into applications of single-modality models and offer numerous applications across different industries.”
The five new models are:
- BiMediX is the bilingual medical mixture of expert LLMs, which outperforms many existing LLMs on medical benchmarks in English and Arabic including medical board exams. The potential use cases include – virtual healthcare assistants, telemedicine, clinical symptom diagnosis, medical research, mental health support and counselling, or even lifestyle enhancements and diet plans.
- PALO is the multilingual LMM to offer visual reasoning capabilities in 10 major languages, namely English, Chinese, Hindu, Spanish, French, Arabic, Bengali, Russian, Urdu, and Japanese. The solution ensures high linguistic fidelity even for low resource languages such as Urdu or Bengali, covering two-thirds of the world’s population and helping to bring the benefits of AI to more people. The model has broad potential applications, including everything from monitoring crops, to recording wildlife, and aiding with search and rescue missions.
- GLaMM is a one-of-its-kind LMM capable of generating natural language responses related to objects in an image at the pixel level. This offers an enhanced version of automated image captioning, reasoning, and the ability to switch objects in images.
- GeoChat is a grounded LMM, tailored to remote sensing scenarios (RS). GeoChat handles high-resolution RS imagery, employing region-level reasoning for comprehensive scene interpretation. GeoChat leverages a newly created RS multimodal dataset with zero-shot performance across different RS tasks. This includes- visual question answering, scene classification, image and region captioning, visual grounded conversations, and referring object detection.
- MobiLLaMa is a transparent, open-source lightweight and efficient small language model (SLM) used specifically for resource-constrained devices like mobile phones that can be readily deployed and used on a smartphone e or tablet. MobiLLaMa uses novel parameter-sharing scheme to reduce the pre-training computing and deployment costs, and memory footprint along with having multimodal capabilities. It is a fully transparent model with complete training data A fully transparent model with complete training data available as part of the LLM360 initiative, it has intermediate checkpoints, evaluation and training code, and a mobile deployment environment.
Saudi Arabia not far behind
While UAE is making its strides in the right direction, Saudi Arabia isn’t far behind. The kingdom has been collaborating with the likes of Huawei and Tonomus, and building projects within NEOM and PIF.
The region already has companies like Mozn that is building its LLM. In a conversation with edge/ Malik Al Yousef Co-founder and Chief Operating Officer, said, “Mozn is one of the leading enterprise AI companies in the region. Our mission is to use technology to improve digital humanity. We focus on basically two areas. One is on building the best-performing Arabic LLM model and technology. And the second area is around risk and compliance for financial services. So, through these platforms, we have empowered more than a hundred customers across the region from a range of industries including leading government organisations, banks, financial services, and fintechs.”
Having localised models helps, as these increase and diversify the data pools. Most of these LLMs in the region work best with collaboration between private and public entities. This can increase the speed and scale of development for the language models. With diversified language models the scope of GenAI becoming more open and inclusive increases.
