Cloudera’s AI Inference Service uses NVIDIA accelerated computing and NVIDIA NIM microservices to accelerate LLM performance by 36x, providing enterprises with superior performance, robust security and scalable flexibility
Cloudera , the only true hybrid platform for data, analytics and AI , announced Cloudera AI Inference with NVIDIA NIM microservices, part of the NVIDIA AI Enterprise platform. One of the industry’s first AI inference services to offer built-in NIM microservices capabilities, Cloudera AI Inference uniquely streamlines the deployment and management of AI models at scale, enabling enterprises to harness the true potential of their data to move generative AI from pilot to production.
Recent data from Deloitte reveals that compliance risks and governance concerns are the biggest barriers to enterprises adopting generative AI, yet generative AI adoption is growing rapidly, with more than two-thirds of enterprises increasing their generative AI budgets in the third quarter of this year. To mitigate these concerns, enterprises must switch to privately running their AI models and applications, whether on-premise or in the public cloud. Such a transition requires secure, scalable solutions that avoid complex DIY approaches.
Cloudera AI Inference provides enterprises with self-managed, secure development and deployment to prevent sensitive data from leaking to non-private, vendor-hosted AI model services. Powered by NVIDIA technology, the service accelerates the creation of trusted data for trusted AI, enabling efficient development of AI-driven chatbots, virtual assistants and agent-like applications, impacting both productivity and new business growth.
The announcement of Cloudera AI Inference follows the company’s collaboration with NVIDIA and strengthens Cloudera’s commitment to powering enterprise AI innovation at a critical moment as industries address the complex challenges of digital transformation and AI integration.
Developers can build, customize, and deploy enterprise-grade LLMs with up to 36x faster performance using NVIDIA Tensor Core GPUs, delivering nearly 4x the throughput compared to CPUs. A seamless user experience integrates UI and APIs directly into NVIDIA NIM microservices containers , eliminating the need for command line interfaces (CLIs) or separate monitoring systems. Service integration with Cloudera’s AI model registry also enhances security and governance by managing access control for both model endpoints and operations. Users benefit from a unified platform where all models, whether LLM-deployed or traditional, are seamlessly managed in a single service.
Also Read: AuditBoard Launches Advanced Analytics in the UK
Other key features of Cloudera AI Inference include:
- Advanced AI Capabilities: Leverages NVIDIA NIM microservices to optimize open source LLMs such as LLama and Mistral to deliver cutting-edge advancements in natural language processing (NLP), computer vision and other AI domains.
- Hybrid Cloud and Privacy: Run workloads on-premise or in the cloud, with enhanced security and regulatory compliance through VPC deployment.
- Scalability and monitoring: Auto-scaling, high availability (HA), and real-time performance tracking to detect and fix problems and ensure efficient resource management.
- Open APIs and CI/CD Integration: Access standards-compliant APIs for model deployment, management, and monitoring, enabling seamless integration with CI/CD pipelines and ML operations workflows.
- Enterprise Security: Enforce access to your models with Service Accounts, Access Control, Lineage and Auditing capabilities.
- Risk-managed deployment: Conduct A/B testing and canary rollouts to control model updates.
“Enterprises want to invest in generative AI, but they need not only scalable data, but also data that is secure, compliant and well-governed,” said industry analyst Sanjeev Mohan. “Practicalizing AI at scale and privately introduces complexities that are difficult to address with a DIY approach. Cloudera AI Inference bridges this gap by integrating advanced data management with NVIDIA’s AI expertise to secure data while unlocking its full potential. Enterprise-grade security features like service accounts, access controls and auditing enable organizations to confidently protect their data, run workloads on-premise or in the cloud, and efficiently deploy AI models with the flexibility and governance they need.”
“We are excited to work with NVIDIA to bring Cloudera AI Inference to market. This product provides a single AI/ML platform that supports nearly any model and use case, enabling companies to create powerful AI applications on our software and then run those high-performance AI applications on Cloudera,” said Dipto Chakravarty, chief product officer at Cloudera. “NVIDIA AI integration enables smarter decisions through advanced performance, so Cloudera is innovating for our customers by building trusted AI apps with trusted data at scale.”
“Enterprises need to seamlessly integrate generative AI into their existing data infrastructure to drive business outcomes,” said Kari Briski, vice president of AI Software, Models and Services at NVIDIA. “By incorporating NVIDIA NIM microservices into the Cloudera AI Inference Platform, we are making it easier for developers to create reliable generative AI applications, fueling a self-sustaining AI data flywheel.”
Source: GlobeNewsWire