Private Token Factories: How Rafay and Protopia AI Let Sensitive Workloads Run on Shared GPU Capacity

Rafay’s Token Factory makes it brutally simple for neoclouds to publish AI models as OpenAI-compatible, token-metered endpoints on shared GPU capacity. Developers love it. Finance loves the chargeback. 

But, regulated enterprises with clinical records, financial transactions, or proprietary source code may struggle to leverage this opportunity. Typical inference endpoints receive inputs in plaintext. For a healthcare system, a bank, or a government agency, plaintext on shared infrastructure can be a non-starter. So the operator's only lever has been coarse-grained isolation: carve out dedicated GPU capacity, size it physically instead of logically, and let that customer run in a silo. 

Although this approach will work, it's expensive. Dedicated slices sit underutilized between jobs, scale separately from the rest of the platform, and erode the multi-tenant economics the operator built the platform to deliver in the first place. Someone has to pay for this. The operator in margin, or the customer in price. We call this the isolation tax, and it's exactly why the highest-value, highest-token workloads in most organizations never make it onto shared inference infrastructure at all.

We built this reference architecture with Protopia AI to close that gap.

‍Why plaintext is the real blocker, not multi-tenancy

Data is encrypted at rest and in transit, but inference itself requires the serving stack to work with usable representations of that data. At the model endpoint, that processing happens in the clear. Sensitive inputs show up across the full operational surface of a serving environment i.e. the request and response logs, debugging traces, prompt and KV caches, scheduler records, observability tooling, and memory residue on the host.

Multi-tenant scheduling decides who gets compute. It does nothing to stop the serving environment from seeing, logging, or retaining what that compute is processing. That's the actual reason regulated workloads end up on single-tenant infrastructure. Not because the GPUs are shared, but because plaintext is exposed the moment it lands.

Where Protopia’s Stained Glass Transform fits

Protopia's Stained Glass Transform is an inference privacy layer that removes plaintext from the equation entirely. Before a sensitive input ever leaves the data owner's environment, Stained Glass converts it into a protected, stochastic representation. The target model reasons directly over that representation and returns accurate results and the representation is the only thing that ever crosses into the shared serving stack.

The transform is model-specific and generated once, offline, by the Stained Glass Engine, producing an artifact called Stained Glass Transform (SGT). At runtime, the Stained Glass Proxy applies that artifact to inference data at the data owner's logically defined trust boundary. 

There's no decoder sitting on the Rafay side, no key that reverses the transformation, and no point in the pipeline where the original data can be reconstructed. The protection comes from where the transformation happens, not from a policy the serving stack has to be trusted to enforce.

Protopia maintains pre-built SGTs for the models operators are already running through Token Factory (Llama 3.1, Llama 4 Scout, Llama 3.3 70B, Mistral, Qwen, DeepSeek Distilled Llama, and NVIDIA Nemotron 3 -Super, Ultra, and Nano Omni) among them. An operator picks the SGT that matches the deployed model. No retraining required.

How the integration actually deploys

The integration touches three points in the request path, and the OpenAI-compatible interface holds at every one of them:

Input Protection

The Stained Glass Proxy runs at the data owner's trust boundary, in their own data center, as a managed Kubernetes workload, or on any host inside their network. It intercepts outbound requests, applies the transform locally, and forwards only protected representations to the Token Factory endpoint. From the calling application's point of view, nothing changes except the endpoint URL and API key.

‍Serving-side

On the Rafay platform, the operator points the model deployment at a Protopia-enabled vLLM image that accepts prompt embeddings as a native input format (upstream vLLM users can enable this directly via the `--enable-prompt-embeds` engine argument). 

Authentication, authorization, rate limiting, tiers, metering, quotas and chargeback all work exactly as they do for any other Token Factory deployment, because as far as the platform is concerned, it's serving a mode, it just never sees the underlying data.

‍Output Protection

An output-protection plugin runs in the same vLLM process, protecting responses on the way back out before they leave the serving environment. The full round trip in and out never touches plaintext.

What it looks like for the operator, in practice

This is a configuration change, not a re-architecture. The operator registers a compute cluster, configures the model source, and selects the prompt-embeds-enabled vLLM image in the deployment settings, essentially the same workflow used for any Token Factory deployment. Pricing, rate limits, and quotas get set the same way. The endpoint publishes to the user’s self service portal like every other endpoint on the platform.

Developers on the other end see no difference at all: an endpoint, an API key, standard OpenAI-compatible calls, token usage accumulating against their key for billing. That consistency is the point and the privacy layer sits underneath the developer experience rather than replacing it.

‍What Operators Get

The direct win is being able to serve workloads that used to require dedicated, single-tenant endpoints at multi-tenant utilization instead. In Protopia's modeled deployment economics, that shift meaningfully lowers annual infrastructure cost for those workloads, since they're no longer sitting in underutilized carve-outs (actual savings depend on workload mix and scale). Because the protection happens at the data owner's boundary upstream from any chosen serving stack, the same workload can also move between on-prem, sovereign region, and burst capacity based on cost and availability, not on where the data is allowed to go.

The opportunity compounds with the agentic workflows that have made AI useful in the enterprise. A single agent task is not one model call: the harness reasons, calls a tool, reads the result, and reasons again, and every pass is another LLM request for Token Factory to serve. Each pass is also another chance for sensitive context to land in a log or cache on the backend infrastructure. Protopia's SafeCLAW extends the Stained Glass pattern into agent harnesses, so every call in the loop, not just the first, routes through the local transform before it reaches Token Factory. The operator captures the full LLM dependency of enterprise agents at the economics the factory was built for, without the isolation tax eating the return. 

‍Who is this for? 

Neoclouds, telcos, sovereign cloud operators, enterprise platform teams, GPU providers, and NVIDIA Cloud Partners looking to monetize shared GPU capacity without turning away regulated data.

For sovereign operators specifically, this covers what geography alone doesn't: sovereignty controls where compute runs, not what the serving stack can see once data arrives. For enterprise platform teams, it means sensitive business-unit workloads can finally share the same governed infrastructure as everything else, instead of each one needing its own carve-out. 

For neoclouds and NCPs, it opens up a customer segment (regulated enterprise and government buyers) that data policy has been quietly excluding from shared inference altogether.

For federal agencies and the operators that serve them, the opportunity carries over directly. Because Stained Glass ensures sensitive and controlled agency data reaches the serving stack only as protected representations, operators can offer token-metered inference at multi-tenant economics, with the metering, governance, and access control that public sector procurement expects. Agencies get AI at scale with the cost efficiencies of a shared token factory, and NVIDIA Cloud Partners, GovCloud operators, and systems integrators get to serve them without the coarse-grained carve-outs that sensitive data used to require.

The Stained Glass Proxy runs at the data owner's boundary. The serving stack holds only protected representations. The endpoint stays OpenAI-compatible, and Token Factory still handles metering, quotas, governance, and chargeback because none of that changes. What changes is that the workloads carrying the isolation tax today can finally run at the utilization the platform was built for, billed the same way as everything else on it.

‍See it at RAISE

Both teams will be at the RAISE Summit in Paris, July 8 and 9 at the Carrousel du Louvre. If you operate shared GPU capacity and want to see what it takes to serve regulated workloads at multi-tenant economics, come find us on the floor, or reach out ahead of time to set up a working session: protopia.ai/contact and rafay.co/contact.

About Rafay Systems

Rafay Systems is a leading platform provider for modern infrastructure and AI workloads, delivering Platform-as-a-Service (PaaS) capabilities that enable organizations to operationalize compute infrastructure with self-service automation, governance and multi-tenancy. The Rafay Platform helps enterprises, cloud providers and sovereign AI cloud operators transform raw infrastructure into fully operational platforms for AI, Kubernetes and cloud-native applications. Rafay's Token Factory extends the platform to AI monetization, letting operators publish models as token-metered, OpenAI-compatible endpoints on shared GPU capacity. By simplifying infrastructure orchestration and lifecycle management, Rafay enables organizations to accelerate innovation while maintaining security, consistency and operational control. For more information, visit rafay.co.

About Protopia AI

Protopia AI delivers inference privacy for AI. Its Stained Glass technology converts sensitive data into protected representations before it leaves the data owner's trust boundary, so models reason over data that never appears in plaintext anywhere on the serving stack. Protopia's Stained Glass Transform and SafeCLAW anchor the inference privacy layer in zero-trust AI factory architectures, spanning confidential inference, agentic workloads, and multi-tenant token factories across on-prem, sovereign, and cloud infrastructure. By protecting the inference data path wherever AI runs, Protopia lets organizations put their most sensitive data to work with AI. Learn more at protopia.ai