Securing inference without slowing it down: Rafay and LuminAI

Open-weight models bring huge efficiencies but also carry enhanced security risks

Open-weight models now carry a rapidly growing share of production AI, chosen for cost, latency, privacy, and sovereignty. However, open-weight models also bring growing and unmanaged security risks, ranging from adversarial exploitation of guardrails and data extraction to unauthorized behavioral overreach in agentic systems.

Securing AI inference has become a must

The intelligence of AI models is embedded in their weights. While the weights of the model are accessible, the system's decision-making process is opaque and obscure, requiring a dedicated security solution for AI inference. Furthermore, the non-deterministic nature of AI, increases the ease and potential of adversaries to cause major damage. For neoclouds and enterprises serving inference at scale, that leaves a gap at exactly the layer where requests are processed. Operators who want to run open-weight models in production need security that works inside the execution path, not bolted on around it. LuminAI Security enables organizations to securely use and deploy open-weight models. 

Rafay governs the token path, LuminAI secures it

The Rafay Platform is the operating layer between GPU infrastructure and revenue. Its control plane already governs authentication, multi-tenancy, token metering, and monitoring across the inference path, exposing inference as a service that tenants consume and operators price. LuminAI extends that control plane with runtime inference protection. LuminAI measures the model's intrinsic intent during inference, and detects malicious and unintended patterns even when the final output appears benign. The guardrail operates where the risk actually lives, governed through the same control plane operators already run.

Security that reads activations without waiting for inference to finish

The reason this matters to operators is performance. Conventional approaches wrap the model in a proxy or a rule-based check, which can double the latency while adding compute overhead on the serving path. LuminAI takes a streaming approach that reads the numerical nature of neural activations and analyzes them without waiting for inference to complete. That keeps added overhead and latency low, and keeps the footprint light enough to run from cloud to edge. The security layer does its work in line with serving, not on top of it.

Demonstrated end to end inside Rafay's environment

Rafay and LuminAI validated the integration on the Rafay Platform directly. Example open-weight models were onboarded, LuminAI enforced runtime guardrails within the execution path, and serving latency stayed effectively unchanged. All of it was observable inside Rafay's environment, alongside the token and inference telemetry operators already track. Coverage extends across any model and any modality, including text, image, speech, and agentic workloads, so one security layer follows the full catalog rather than one model at a time.

What operators get

For a neocloud, sovereign AI cloud operator, or enterprise running inference as a service, the integration turns on protection for the token path without re-architecting the platform or giving up serving performance. Open-weight models move into production with enabling higher visibility and guardrails against prompt injection, data extraction, and behavioral anomalies in place. Because the layer is governed and monitored through Rafay's control plane, security, metering, and operations stay on one pane of glass.

About LuminAI‍

LuminAI secures AI during the inference process. Its runtime protection operates inside the execution path of open-weight models, using mechanistic interpretability to measure a model's intrinsic intent and detect adversarial manipulation, data extraction, and behavioral anomalies in real time. Because the approach analyzes neural activations in stream rather than wrapping the model in a proxy, it adds minimal latency and runs from cloud to edge. LuminAI is an early-stage company based in Tel Aviv.

About Rafay Systems

Rafay Systems is a leading software provider powering the operators building the AI cloud, including neoclouds, telecommunications providers, enterprises, and sovereign AI operators. The Rafay Platform lets these organizations operationalize GPU and compute infrastructure with self-service automation, governance, and multi-tenancy, spanning bare metal provisioning, infrastructure lifecycle management, virtual machines, Kubernetes, GPU PaaS, AI development environments, and Token Factory  for publishing AI models as token-metered inference services. By simplifying orchestration and operations across this stack, Rafay helps operators increase GPU utilization and turn raw infrastructure into monetizable, production-ready AI services, all while maintaining security, consistency, and control. For more information, visit rafay.co.