.png)
AI infrastructure is moving from the server to the rack. NVIDIA GB200 NVL72 connects 72 NVIDIA Blackwell GPUs and 36 NVIDIA Grace™ CPUs in a single NVIDIA NVLink™ domain, so the whole rack behaves like one very large accelerator, which raises the bar for the software that runs it. The control plane now has to understand the rack: verify hardware before handoff, carve fabrics into isolated tenant slices, place workloads with topology in mind, and return every node to a clean state between tenants.
That's the bar Rafay is now testing against at rack scale. Rafay is recognized as an NVIDIA AI Cloud-Ready validation partner and implements NVIDIA Infra Controller (NICo) on a GB200 NVL72 rack in the NVIDIA validation lab, extending its validation work from single systems to full racks.
NVIDIA positions GB200 NVL72 for real-time inference and training of trillion-parameter and mixture-of-experts models, and GB300 NVL72 for AI reasoning. Three groups of operators are deploying these systems:
They share one business problem. A liquid-cooled rack drawing well over 100 kW is a large capital commitment, so every idle week is expensive. Rafay is already helping close that gap: in July 2026, Rafay and Singapore-based neocloud Aolani announced the NVIDIA DSX OS deployments on GB200 NVL72. The same rack-scale design carries forward to GB300 and NVIDIA Vera Rubin NVL72, so these capabilities apply to the racks operators are planning next.
NVIDIA AI Cloud-Ready does for AI cloud software what the NVIDIA Cloud Partner reference design does for hardware. Stacks are assessed against the NCP Software Reference Guide, and results are reviewed jointly with NVIDIA.
In May 2026, Rafay announced that the Rafay Platform, integrated with NICo, achieved AI Cloud-Ready validation, and Rafay is listed among the initiative's first-ever validated partners. This round extends that work to rack-scale systems.
NICo is NVIDIA's open-source, API-based microservice for site-local, zero-trust, bare-metal lifecycle management. Rafay builds on it to deliver self-service consumption, multi-tenancy with governance, and Kubernetes lifecycle management. Rafay deployed this stack directly on the rack's bare metal, so the tests exercise real NVLink switches, real fabrics, and real rack hardware.
What makes rack-scale multi-tenancy hard is isolation. The traditional approach configures tenant boundaries at the switch, with VPCs and subnets carved through switch-level APIs, and that complexity compounds with every new tenant and every new rack. NICo moves the boundary onto the host itself, enforced by the NVIDIA BlueField™ DPU on each server. The DPU runs in zero-trust mode: the host OS can't reconfigure it, so it stays owned by the operator even after the host is handed to a tenant. If a tenant's workload or OS is ever compromised, the network and management planes stay isolated.
Rafay's multi-tenancy, RBAC, and quota enforcement sit directly on this host-level model, which is why the validation treats attestation, per-tenant isolation, and sanitization as first-class checks.
The validation focuses on network management, compute provisioning, and workload orchestration. The rack-scale tests probe these areas:
Validation on the same class of hardware shortens the distance between racking a system and selling it.
The goal is the same at every scale: let operators run AI infrastructure, from a single server to a full NVL72 rack, as a governed, self-service platform from day one.
To learn how Rafay can help you bring rack-scale capacity to market, talk to Rafay.
Inference now defines AI infrastructure economics. See how NVIDIA Dynamo splits prefill and decode across GPU pools, and how Rafay Token Factory turns it into a priced, multi-tenant service.
Read Now
From workload scheduling and inference to GPU fleet health, the Rafay Platform integrates with NVIDIA DSX OS across the layers operators run every day, and turns the reference stack into services operators can sell.
Read Now
Hosted AI agents raise two security questions: what the agent can do, and what the platform sees of its data. NVIDIA OpenShell, Protopia AI, and Rafay together let NeoClouds and sovereign AI providers sell governed Agent-as-a-Service and protected Tokens-as-a-Service on shared GPUs.
Read Now
