Retrieval-Augmented Generation (RAG) has become one of the most common architectures for enterprise generative AI. Instead of relying only on knowledge embedded in a large language model, RAG enables organizations to ground responses using their own documents, data, and knowledge repositories.
But deploying a production RAG environment involves much more than launching an LLM. A complete RAG stack can require embedding models, reranking models, document ingestion pipelines, search infrastructure, vision-language models, GPU scheduling, networking, ingress, model caches, and application endpoints.
For organizations operating GPU infrastructure, the challenge is straightforward:
How do you make this complete RAG stack available as a simple, repeatable service without requiring every user to become an expert in Kubernetes, NVIDIA NIM, Helm, networking, and GPU infrastructure?
One approach is to package NVIDIA's RAG Blueprint as a self-service offering on top of a neocloud's GPU infrastructure. The result looks much more like a cloud service than a traditional Kubernetes deployment.
At a high level, the architecture has three layers:
Users → Rafay GPU PaaS → GPU Kubernetes Cluster
Users consume the NVIDIA RAG Blueprint through a self-service catalog. Rafay handles deployment of all the RAG blueprint components and lifecycle automation, while Kubernetes provides the runtime environment for the RAG stack.
Depending on infrastructure and isolation requirements, the RAG environment can run on shared Kubernetes infrastructure or on a Kubernetes cluster dedicated to a specific tenant.
The experience begins with the user.
Instead of receiving Kubernetes credentials and a collection of Helm charts, users access a self-service catalog and select the NVIDIA RAG Blueprint.
The underlying SKU packages NVIDIA's RAG Blueprint as a deployable service. The blueprint includes a Llama 3.1 70B LLM, Nemotron embedding and reranking models, a vision-language model, and the nv-ingest document ingestion pipeline.
From the user's perspective, the deployment experience can remain intentionally simple.
For example, the service can expose only parameters that matter to the application owner, such as GPU allocations for individual model services. Infrastructure-specific configuration can remain hidden and centrally controlled by the platform team.
A typical user experience becomes:
The platform team handles:
The user therefore consumes RAG as a platform service rather than as a collection of infrastructure components.
Between the user and Kubernetes sits the Rafay GPU PaaS Platform. Rafay converts the NVIDIA RAG Blueprint into a reusable service through several platform capabilities.
The NVIDIA RAG Blueprint is registered as a billable SKU and exposed through the service catalog. Rather than exposing every Helm parameter, Kubernetes object, credential, and infrastructure dependency, the SKU presents a curated set of configuration options.
When a user deploys the service, Rafay executes a series of workflows that automate the deployment of the RAG blueprint on the Kubernetes cluster, create ingress resources and program DNS for the user to access the app via a https URL.
The workflow:
The service therefore encapsulates the operational logic required to deploy and expose the complete RAG environment.
A RAG deployment runs inside its own Kubernetes namespace.
For example:
This provides a clean deployment boundary for each RAG environment. It also gives the platform a natural place to apply Kubernetes and platform controls such as:
For environments running on shared Kubernetes infrastructure, namespaces provide logical separation between deployments.
Not every RAG deployment has the same infrastructure requirements. Some organizations may prefer to maximize GPU utilization by running multiple RAG environments on a shared Kubernetes cluster. Others may require a dedicated Kubernetes cluster because of:
A neocloud can support both models.
In the shared-cluster model, multiple RAG environments consume resources from the same Kubernetes and GPU infrastructure. In the dedicated-cluster model, the Kubernetes cluster itself becomes the isolation boundary.
Importantly, the user experience does not need to change. The same NVIDIA RAG service can be selected from the catalog and deployed through the same automation workflow, regardless of whether the target is shared or dedicated infrastructure.
For neoclouds, this enables multiple service tiers from the same platform:
This gives providers flexibility to align infrastructure with the requirements of each workload.
Each deployed instance contains the services required to implement the RAG pipeline. Conceptually, it looks like the following:
The NVIDIA Blueprint combines these capabilities into a complete multimodal RAG environment rather than requiring users to assemble them independently.
The Llama 3.1 70B model performs answer generation after relevant information has been retrieved.
The embedding service transforms document content and queries into representations that can be used during retrieval.
The reranking model evaluates candidate retrieval results and improves the context passed to the LLM.
The VLM enables processing of visual information contained in multimodal documents.
nv-ingest provides the document ingestion pipeline.
Documents can contain text, tables, and images. The ingestion pipeline processes those assets before they become available to the RAG system.
Elasticsearch provides the search backend required by the RAG deployment and is operated through the Elastic Cloud on Kubernetes operator.
Together, these components provide a complete RAG environment.
Once the instance is ready, users interact with the RAG UI using a web browser. A typical workflow looks like:
The ingestion pipeline processes content including text, tables, and images. The RAG pipeline then retrieves relevant information and supplies that context to the LLM before generating a response.
The infrastructure complexity remains hidden behind the service interface.
The value of this architecture is not simply that it can deploy NVIDIA's RAG Blueprint. The larger benefit is that it transforms a complex AI application stack into a repeatable service that can be monetized by a neocloud as a value added service.
Without Rafay's platform abstraction, application teams will need to understand Kubernetes, NVIDIA GPU Operator, NIM Operator, NIM Services, Helm, Elasticsearch, DNS etc. With a GPU PaaS service model, the experience becomes extremely streamlined for users. That is a significant operational simplification.
Click here if you are ready to go deeper and try this out?
Learn how NeoClouds can use NVIDIA Cluster Readiness Engine to certify GPU Kubernetes clusters before tenant handoff and validate AI workload readiness.
Read Now
Rafay's VMaaS offering is now NVIDIA-Certified for HGX and NVL72 systems, giving operators secure multi-tenancy at near bare-metal GPU performance.
Read Now
Rafay CEO Haseeb Budhani joins theCUBE to discuss sovereign AI, multi-tenant cloud platforms, GPU monetization, and the shift from infrastructure to AI services.
Read Now
