Galileo, the AI Evaluation company, announced an integration with NVIDIA NeMo™, enabling customers to continuously improve their custom generative AI models. Now, customers can evaluate models comprehensively across the development lifecycle, curating feedback into datasets that power additional fine-tuning. As a result, customers ship GenAI apps that are more reliable, trusted, and cost-effective.
Data Flywheel for AI
The majority of enterprises are developing GenAI applications – including agents and RAG-based chatbots – but it can be challenging to ship and scale these applications due to the non-deterministic outputs of Large Language Models (LLMs). There’s even more complexity when AI teams wish to test new LLMs, which are constantly evolving in capability and price point. The solution is to build an AI data flywheel, enabling continuous testing and refinement, collecting data about user interactions for subsequent improvement. When AI teams use data to improve outcomes (whether by fine-tuning, prompt engineering, or in-context learning), they gain a competitive advantage.
Also Read: BreachRx Launches Rex AI™ for Automated Incident Response
Galileo and NVIDIA accelerate a data flywheel by collecting and curating better data about the interactions of an AI application. Galileo complements NVIDIA NeMo Evaluator with customizable evaluators that empower teams to iterate and better measure the success of their AI applications. For example, Galileo can evaluate whether an AI agent is calling the right tools and if RAG-based outputs are hallucinations or pulled from provided context. Galileo evaluators include explanations for the score as well as recommendations about how to improve it. Galileo also serves as a control plane for guardrails, extending NeMo Guardrails with low-latency, low-cost models that intercept risky outputs before they are presented to users.
“NVIDIA NeMo is a powerful toolchain for end-to-end generative AI development,” said Yash Sheth, COO and Co-founder of Galileo. “The Galileo platform, combined with NVIDIA NeMo, can accelerate an AI data flywheel from data curation, fine-tuning, generative AI development into production, for a continuously improving system. We’re giving developers the tools they need to find the best models, fine-tune quickly, and respond to any drift that arises in the real world.”
“The Galileo platform, integrated with NVIDIA NeMo, can turbo-charge an AI data flywheel,” said Santiago Pombo, Group Product Manager at NVIDIA. “Customers can leverage the best datasets and metrics to customize, evaluate and scale their LLMs with confidence, and use NeMo Guardrails within Galileo Protect to build safe, secure and robust solutions. Galileo‘s real-time observability also instills trust in production by continuously evaluating systems running on top of NVIDIA NIM, sending alerts if something goes wrong or interactions drift from the training data.”
Source: PRNewswire