How NVIDIAOpenShell™, Protopia AI, and Rafay are enabling secure, multi-tenant, commercially viable Agent-as-a-Service and Tokens-as-a-Service
At NVIDIA GTC Taipei in June, Jensen Huang described what he called the application pattern of the next decade: AI agents operating inside software harnesses, using large language models to reason and tools to interact with the world.
In his words, “the large language model is used to do the thinking, and the harness connects everything together, just like an operating system.” Every company will run this pattern, with whichever harness and whichever models it prefers.
Until recently, GPU cloud providers primarily focused on delivering compute capacity. Enterprises consumed that capacity through virtual machines, Kubernetes clusters, dedicated GPU servers, or increasingly, serverless inference endpoints.
AI agents introduce an entirely new consumption model.
Instead of developers manually invoking models, autonomous agents continuously interact with models, retrieve information, execute tools, and make decisions. These agents may run for hours or days, executing thousands of inference requests while accessing sensitive enterprise information.
For AI clouds,, NVIDIA Cloud Partners (NCPs), and sovereign AI infrastructure providers, this creates an opportunity to deliver two complementary services:
But delivering these services requires solving three distinct challenges.
First, how can providers ensure that autonomous agents cannot perform unauthorized actions or compromise the infrastructure hosting them?
Second, how can providers protect sensitive enterprise information when inference workloads execute on shared infrastructure?
Third, how can providers operationalize these capabilities into scalable, multi-tenant services with automated provisioning, governance, usage metering, and monetization?
A provider has to answer all three for the same customer at the same time: that customer’s agent runs on the provider’s platform, calls a model on the same platform, and both show up on one bill. NVIDIA, Protopia AI, and Rafay each answer one of the three, inside a single deployment.
NVIDIA governs what agents can do. Protopia AI protects the sensitive information those agents process. Rafay operationalizes both capabilities as scalable, multi-tenant cloud services that infrastructure providers can deliver and monetize.
Figure 1 shows how they fit together. Each tenant’s agents run in NVIDIA OpenShell sandboxes on the operator’s platform, each tenant’s model calls pass through its own Stained Glass Transform at the factory ingress, and the model runs on multi-tenant GPUs that the operator meters through Rafay.
Traditional applications generally execute predefined operations within relatively predictable boundaries. AI agents, by contrast, dynamically determine which actions to take, which tools to invoke, and which information to retrieve.
An agent with access to enterprise systems could potentially execute unintended commands, access unauthorized resources, or transmit sensitive information to external services.
The challenge becomes even more significant when infrastructure providers host agents belonging to multiple independent customers.
In September 2026, NVIDIA introduced the Open Agent Safety Platform (OASP), establishing an architectural approach to securing autonomous agents.
Two important components are NVIDIA OpenShell and NVIDIA Sentry.
Rather than relying exclusively on instructions embedded within the agent, NVIDIA OpenShell enforces controls outside the agent's decision-making process.
The principle NVIDIA published OASP with: “An agent cannot act without its next thought. By controlling the path to the model, you own both the best observation point” and the kill switch. This is particularly important for AI cloudsplanning to offer managed agent services.
A provider hosting hundreds or thousands of customer agents needs to ensure that each agent operates within defined boundaries, regardless of the tools, frameworks, or models it uses.
There are two ends to that path. OASP owns the agent’s end. The other end is where the thinking happens, and even a perfectly governed agent must send information to a model somewhere. For most enterprises that model runs on multi-tenant capacity someone other than the data owner operates: a n AI cloud, an NCP, a sovereign AI factory, or the enterprise’s own AI factory, where the tenants are business units of the same company that do not, as a matter of course, see one another’s data.
Consider an enterprise deploying an AI agent to analyze customer records, proprietary source code, or confidential financial information. The agent may operate within a tightly controlled NVIDIA OpenShell environment. Its access to tools, networks, and external services may be explicitly governed.
But when the agent invokes a language model, its request must travel to inference infrastructure. In conventional deployments, that request contains plaintext information.
The model-serving infrastructure processes this information, and sensitive content may appear across multiple operational surfaces, including request logs, debugging traces, inference caches, observability systems, and operator tooling.
Even when the underlying infrastructure is properly configured, the operator must manage the risk associated with privileged access, software vulnerabilities, and configuration errors.
These concerns become particularly important in multi-tenant GPU environments.
An August 2026 SemiAnalysis security audit examined 25 GPU providers and 32 clusters over four months, and found enterprise data reachable on public inference endpoints through a misconfiguration, agent and coding-harness traffic exposed through a co-tenant’s token provider, and every tenant’s logs and metrics readable through a single provider-side mistake.
The findings reinforced a fundamental principle of cloud security: a single configuration error should not result in immediate exposure of another tenant's sensitive information.
Traditional defenses such as network isolation, access controls, encryption in transit, and dedicated infrastructure remain essential.
Every operator should make every fix SemiAnalysis recommends. Each fix lowers the odds of a mistake. None of them changes what a mistake exposes.
For enterprises handling highly sensitive information, the conventional answer has often been dedicated infrastructure. Even then, someone other than the tenant still operates the machine, so a dedicated box does not keep the tenant’s data away from the operator’s staff or from anyone who gets in through a misused credential or a misconfiguration. Tenants often keep their most sensitive agent workflows off it anyway, and it is expensive. For operators, dedicated deployments also fragment GPU capacity, reduce opportunities for pooling workloads, and make inference services more expensive to operate.
Agents raise the stakes. An agentic task is a loop of requests, and each pass carries the context the agent gathered on the way: the customer record it retrieved, the output of the tool it just called, the reasoning it is about to act on. OASP’s design depends on that reasoning being visible on the path to the model, which is why it is valuable to the data owner and exactly what nobody else in the factory should be able to read.
The challenge is therefore both technical and economic.
How can an AI infrastructure provider securely serve sensitive enterprise workloads while preserving the utilization and economic benefits of multi-tenant GPU infrastructure?
Protopia AI answers it at the point where requests enter the factory.
Protopia AI's Stained Glass Transform addresses the challenge of protecting sensitive information during model inference. Rather than sending the original plaintext request directly to the model-serving infrastructure, the Stained Glass Transform converts the input into a protected representation.
This representation is designed to preserve the information required for model inference while keeping the data owner’s plaintext out of the serving stack. Importantly, the target model operates directly on the transformed representation.
The approach does not require retraining the underlying model or deploying a corresponding decoder inside the inference-serving environment.
In July 2026, Rafay and Protopia AI introduced a reference architecture for a Private Token Factory that combined this technology with Rafay's shared inference infrastructure. In that architecture, the enterprise operated the Stained Glass Transform within its own trust boundary.
Only transformed representations were transmitted to the inference platform.
The underlying GPU infrastructure could therefore serve inference requests without directly receiving the original plaintext prompts.
This established an important capability: multiple enterprises could share model-serving infrastructure without the platform holding their plaintext.
The next architectural evolution makes this capability easier for infrastructure providers to deliver as a managed service.
Instead of requiring each enterprise to independently deploy and manage its transformation infrastructure, the Sovereign Token Factory introduces dedicated transformation ingress points for individual tenants.
Each tenant's requests pass through its designated Stained Glass Transform before reaching the shared inference-serving tier. Everything downstream of that point holds protected representations: the vLLM endpoint with prompt embeddings enabled, the model (Nemotron or any model with a Stained Glass Transform), the multi-tenant GPUs, and every operational surface in between, from logs and observability tools to the inference caches, scheduler state and operator tooling.
The architecture separates two concerns that have traditionally been coupled: protecting tenant information and dedicating GPU infrastructure. The transformation component is isolated per tenant, while the expensive model-serving infrastructure can remain shared.
Tenants are separated at these ingress points rather than by giving each customer its own model deployment, so far fewer GPUs are dedicated to any one customer.
The ingress is the one place on the provider’s side where a tenant’s plaintext exists: one proxy per tenant, isolated from other tenants and from the multi-tenant serving tier, rather than every log, cache and trace in the factory. That ingress is isolated and protected, with access controls and operational safeguards suited to the enterprise's security requirements.
The transform also complements rather than replaces conventional infrastructure security. Authentication, tenant isolation, secure logging, and access controls remain necessary.
With those safeguards in place, the Sovereign Token Factory creates an opportunity to expand the range of enterprise workloads that can consume shared inference services.
But protecting requests is only one part of delivering the service. Someone must deploy the models, provision the infrastructure, manage tenants, enforce quotas, meter consumption, and expose the entire capability through a usable cloud experience.
Rafay runs the agent sandboxes, the per-tenant transforms and the multi-tenant serving behind them as services the operator can sell.
Security technologies become broadly useful when enterprises can consume them without having to assemble and operate the underlying infrastructure themselves. For NeoClouds and sovereign AI infrastructure providers, this is a critical distinction.
An enterprise that runs its own inference endpoint can put the Stained Glass Transform in front of it directly, as the July reference architecture for private token factories showed, and a developer can run an agent in NVIDIA OpenShell on their own machine. An operator’s job is different: it has to offer protected inference and governed agents to many customers at once, as services each of them can discover, provision, consume, and pay for. It is the same problem operators already solve with Rafay for the GPU capacity they sell.
Rafay provides the cloud service orchestration, multi-tenancy, governance, and monetization capabilities that allow infrastructure providers to offer both to their own customers as commercial services.
The Rafay GPU platform enables infrastructure providers to transform GPU capacity into a catalog of consumable services, including GPU virtual machines, Kubernetes environments, developer environments, inference endpoints, and other AI workloads.
With Rafay Token Factory, providers can deliver inference as a managed, metered service rather than requiring every customer to provision and operate dedicated model infrastructure. This includes several capabilities essential to commercial AI infrastructure.
Rafay enables providers to deploy and operate inference services on GPU infrastructure across their data centers. Operators can register GPU capacity, configure model-serving environments, expose inference endpoints, and manage the lifecycle of the underlying services.
For example, an operator can deploy a compatible vLLM serving environment with prompt-embedding support and integrate it with Protopia's Stained Glass Transform.
The transformation technology protects the request, while Rafay manages the inference service that processes it.
Crucially, the provider does not need to deploy a separate model instance for every customer. Multiple tenants can consume shared inference capacity through their respective protected ingress paths, subject to model compatibility, isolation controls, and capacity policies. This allows the provider to pool GPU resources and improve opportunities for utilization.
A commercially viable Token Factory must support multiple independent customers, each with distinct users, projects, policies, and consumption limits.
Rafay provides the multi-tenant management layer required to organize and govern these services. Infrastructure providers can establish tenant boundaries, delegate administration, and apply quotas and access policies to GPU and inference resources.
This separation is particularly important when a single underlying inference deployment serves requests from multiple enterprises. Protopia ensures that deployment receives protected representations rather than each tenant’s plaintext.
Rafay governs which tenants can access the service, how much capacity they may consume, and how the underlying infrastructure is allocated and operated. These are complementary forms of protection and governance.
One of the largest differences between operating inference infrastructure and running a commercial Token Factory is the ability to measure and monetize consumption. Rafay Token Factory provides the usage-management foundation for token-based services.
Providers can meter inference consumption, configure pricing, establish budgets and quotas, and support chargeback or customer billing workflows. This enables commercial models that are difficult to implement efficiently with dedicated GPU allocations alone.
The tenants who pay the most are the ones whose security review runs longest and ends in a request for a dedicated box. With the Stained Glass Transform at the ingress, that review gets shorter, because the multi-tenant tier never holds their plaintext, and more of their workloads can move to multi-tenant capacity. Not every workload leaves the dedicated box, but the share that has to stay there shrinks.
An enterprise can consume protected inference through an endpoint without purchasing an entire GPU server or committing to a dedicated model deployment. The provider can monetize actual token consumption while serving multiple customers from pooled GPU capacity.
The commercial value is not simply higher GPU utilization. It is the ability to generate more revenue (i.e. yield) from available GPU capacity by serving more customers, more workloads, and more types of AI services.
The next evolution extends the same service model from inference to autonomous agents. Today, many AI agents run on developer laptops, manually provisioned virtual machines, or application-specific infrastructure.
These environments are useful for experimentation but create operational challenges when enterprises want agents to run continuously. An enterprise may need agents that execute scheduled tasks, monitor business processes, respond to events, or coordinate workflows over extended periods.
These agents need persistent execution environments, secure access to tools and services, resource governance, lifecycle management, and operational visibility. For infrastructure providers, they also represent a new category of billable workload.
The Rafay GPU platform provides a foundation for delivering hosted agent environments as managed services.
In this architecture, a NeoCloud operator uses Rafay to provision and manage agent execution environments incorporating NVIDIA OpenShell.
The infrastructure provider manages the underlying environment, including provisioning, lifecycle operations, and resource consumption.
Agents can run continuously or on schedules, independent of a developer's laptop. Every model call NVIDIA OpenShell allows the agent to make passes through the tenant’s Stained Glass Transform at the factory ingress, and Rafay Token Factory serves it from multi-tenant capacity that holds only protected representations.
This establishes two complementary consumption models on the same infrastructure platform.
Agent-as-a-Service provides the managed execution environment for autonomous agents.
Tokens-as-a-Service provides the inference capacity those agents consume.
Rafay enables the infrastructure provider to manage these as complementary services, creating opportunities to monetize both agent execution resources and inference consumption.
Follow one model call through it. A tenant’s agent, running in its OpenShell governed agent sandbox on capacity the operator provisioned through Rafay, needs the model. OpenShell’s supervisor checks the outbound request against the agent’s policy and its approved endpoint. The request leaves the sandbox and reaches the tenant’s Stained Glass Transform at the factory ingress, where it becomes a protected representation. Rafay Token Factory serves it on multi-tenant GPUs and meters the tokens against the tenant’s quota. The tenant is the same unit at every step: the OpenShell policies for its agents, one Stained Glass Transform at its ingress, and one set of quotas and one meter in Rafay.
NVIDIA Sentry, where it is deployed, is the last element on the agent’s side of the path to the model. The Stained Glass Transform is the first thing on the factory’s side. What a governed agent is permitted to send, and what the factory receives, are decided at adjacent points on the same path.
r operator hosting agents now has to answer two questions: what do you see of my data, and what can my agent do on your platform? With NVIDIA OpenShell and the Stained Glass Transform on the same path, they can answer both, and meter it like any other endpoint.”
Mohan Atreya, Chief Product Officer, Rafay
For NeoCloud operators, this represents an opportunity to move beyond selling raw GPU capacity toward delivering higher-value managed AI services.
As NVIDIA GTC Berlin brings together the ecosystem developing sovereign and agentic AI infrastructure, Rafay and Protopia AI are advancing a reference architecture that combines governed agents with protected inference on shared GPU infrastructure.
The architecture brings together NVIDIA's Open Agent Safety Platform, Protopia AI's Stained Glass Transform, and Rafay's GPU PaaS and Token Factory capabilities.
For the enterprise, regulated or not, this should be the default architecture: governed agents contained by OASP, whose every request reaches the factory as a protected representation. Until now an enterprise that wanted this had to assemble the pieces itself. Here the provider hosts the agent in NVIDIA OpenShell and runs each tenant’s transform at the ingress, and the enterprise brings only its agents and its data.
Over the coming weeks, Rafay and Protopia AI plan to further develop and document this architecture for infrastructure providers seeking to deliver secure, multi-tenant agent and inference services.
The objective is straightforward.
Enable NeoClouds and sovereign AI infrastructure providers to offer enterprises the security and data protection they require without sacrificing the operational efficiency and commercial advantages of shared GPU infrastructure.
As autonomous agents become a primary way enterprises consume AI, infrastructure providers will need to support both the execution environments where agents operate and the inference services where those agents perform their reasoning.
For every model call a hosted agent makes, NVIDIA OpenShell governs what it may send, Protopia’s Stained Glass Transform decides what the factory receives, and Rafay runs the service that the operator meters and sells.
A governed agent may be well behaved, but it still has to call a model somewhere. Operators treated TLS as optional for two decades, until browsers made it the default. In hindsight, this pairing will look just as obvious.
At scale, AI training jobs fail every few hours, and each failure burns GPU-hours. Clockwork AI Fault Tolerance and AI Fabric Observability are now a turnkey option on the Rafay Platform. Turn them on once with a blueprint, and they apply across every GPU cluster in the fleet.
Read Now
A production RAG stack spans LLMs, embedding and reranking models, ingestion, and search. See how Rafay GPU PaaS turns the NVIDIA RAG Blueprint into a repeatable, self-service catalog offering for neoclouds.
Read Now
At the AI Infrastructure Summit, leaders from NVIDIA and Rafay discussed the financial shift in AI buildouts. Explore the four tiers of AI infrastructure monetization and discover why scaling up the value chain efficiently is critical for operators.
Read Now
