It is no secret large language models (LLMs) can come at a significantly high cost for organisations. In the rush to harness the power of LLMs and build generative AI (gen AI) models, the costs for enterprises significantly increase. To combat this problem, Dataiku today announced the launch of its dedicated cost monitoring solution, LLM Cost Guard, a new component of the Dataiku LLM Mesh.
The LLM Cost Guard enables effective tracing and monitoring of enterprise LLM usage to better anticipate and control gen AI costs. It provides visibility into costs attributed to specific applications, providers, and users for a fine-grained understanding of which LLM use cases are driving what costs.
“AI is a critical strategy for every enterprise’s innovation and growth, yet we constantly hear business leaders’ concerns not only about the potential costs of Generative AI projects but the inherent variability of that cost,” said Florian Douetteau, co-founder and CEO, Dataiku in a note shared by the company. “With LLM Cost Guard, we’re aiming to demystify these expenses. IT leaders will now not only have controlled, governed LLM access within the Dataiku LLM Mesh, but detailed, real-time control over spend, so that they can focus their time on building and innovating. By distinguishing between various costs and setting early warnings, we empower leaders to move forward with their Generative AI initiatives confidently.”
LLM Cost Guard is a feature within the Dataiku LLM Mesh, which provides a secure LLM gateway and allows customers to be agnostic when it comes to LLM providers, including integration of LLMs provided by OpenAI, Microsoft Azure, Amazon Web Services, Google Cloud Platform, Databricks, Anthropic, AI21 Labs, and Cohere.
Unpacking the True Cost of Generative AI
With gen AI forecasted to capture 55 per cent of AI spend by 2030, ensuring that Generative AI initiatives are backed by a strong financial justification presents a critical area of investment.
“Dataiku’s innovation in Generative AI cost monitoring is pivotal, meeting a crucial market demand. As enterprises weave Generative AI more deeply into their operational fabric, the necessity for a solution that not only clarifies costs but also integrates governance into its framework becomes essential. Dataiku is empowering companies to pursue sustainable growth with a comprehensive understanding and oversight of their Generative AI investments,” said Ritu Jyoti, Group VP, AI and Automation Research, IDC.
The Dataiku LLM Mesh offers a centralized solution with an unparalleled breadth of connectivity to today’s most in-demand LLM providers. Today’s enterprises prefer to use an array of LLM vendors for different use cases, and LLM Cost Guard monitors and reports on insights across providers so that organizations have global visibility over their projects.
LLM Cost Guard enables enterprises to:
- Tag and Track LLM Expenditures: Assign costs to specific projects for better financial clarity and accountability.
- Differentiate Between Costs: Clearly distinguish between production and development expenses to aid strategic financial planning.
- Implement Early Warning Systems: Identify potential cost overruns early to mitigate financial risks, including those related to governance failures.
- Gain Detailed Insights: Access a dashboard of LLM usage and spending to make informed decisions and optimize investment.
