AI is becoming a new class of network traffic, and it does not behave like the web the internet was built to carry. According to Cisco's 2026 analysis of live service provider networks, token consumption is growing close to 10x year over year, and McKinsey suggests that AI inference is on track to reach roughly a quarter of all network traffic by 2035.1 The shape of the traffic is changing, not only the volume. When a new kind of traffic gets large and latency sensitive, the internet has a well established response. It moves delivery closer to the user. It did this once for content, and it is about to do it again for inference.
The commercial web of the late 1990s ran into a wall. Websites depended on single origin servers that often sat far from users, so when traffic surged those servers overloaded and when distance grew pages felt slow. Akamai built the first commercial CDN in 1998, caching content on distributed servers and steering each user to a nearby copy.2 That set a pattern the internet has followed ever since. When an experience becomes distributed and latency sensitive, the architecture moves toward the edge. Inference delivery sits at that same early stage today.
The structural pressure behind a CDN and a Token Delivery Network is the same. Demand is distributed, and the experience is sensitive to how far a request has to travel. What changes is the object being delivered.
Traditional CDNs replicate static, pre-existing files to serve vast audiences with minimal operational friction. Conversely, a Token Delivery Network facilitates the real-time generation of tokens across distributed AI infrastructure, navigating complex layers of governance, data residency, and cost management that static content never encounters.
Inference workloads lack this static nature. While strategies like KV or prefix caching optimize input costs, the output remains a manufacturing effort constructed token by token on live hardware.
Therefore, proximity is only one factor; a functional endpoint must simultaneously negotiate GPU availability, model versions, and regional policy constraints. Ultimately, token delivery has evolved beyond simple distribution into the sophisticated orchestration of programmable compute at scale.
Inference is moving toward distribution because the workload itself has moved. McKinsey projects that inference will overtake training as the dominant AI workload by 2030, reaching more than half of all AI compute.3 Unlike training, it runs continuously and scales with every user, agent, and application. McKinsey notes that training gravitates toward large, high density campuses, while inference is driving build outs in metro and near metro sites optimized for low latency. That is the CDN edge pattern reappearing.
A single centralized cluster cannot deliver low response times everywhere, and inference increasingly involves user data, which pulls deployment toward regional and sovereign locations for reasons of regulation as much as speed. Sub-1MW power sites are also easier to secure across a geography than 100MW campuses, so for inference a distributed footprint becomes an advantage.
Telcos, neoclouds, CDNs, and sovereign clouds already hold the assets this era requires. What most of them lack is the software layer that turns those assets into programmable inference capacity.
Deploying a single model endpoint is straightforward. Deploying many across many locations, keeping them consistent, and governing, metering, and managing them from one control plane is what decides whether a distributed footprint becomes a real service.
This is the layer Rafay operates, deploying, governing, metering, and monetizing distributed AI services across many locations while integrating with the routing systems that connect users to the best endpoint. Each piece of the model has a clear role.
The Token Delivery Network is the use case, and Rafay Token Factory is the offer that gets providers there, turning GPU inference into governed, token metered AI services exposed through APIs. Underneath both sits the programmable edge, which lets those services deploy wherever compute is available.
The CDN era made distributed delivery a discipline, and inference now inherits that lineage, driven by the same forces of proximity, performance, and scale. Goldman Sachs projects that total token consumption will multiply roughly 24x between 2026 and 2030.4 Serving that volume well will not happen inside one giant data center. It will happen across programmable locations that deploy quickly, govern consistently, and meter cleanly. Token Delivery Networks describe that shift, and Rafay builds the layer that operates it.
A Token Delivery Network is a distributed inference network that brings AI model endpoints closer to users, applications, and agents. Learn how the model works and where the Rafay Platform operates it.
Read Now
