This interview with Haseeb Budhani, CEO and Co-Founder of Rafay Systems, was conducted by the Edge AI Summit in advance of their December 11, 2018 conference is San Francisco. The original link can be found here.

At the Edge AI Summit next month, you will be presenting a presentation titled ‘What is an Edge and what does it look like?’ – could you provide a preview of what we are to expect?

In addition to describing the edges that we deploy for Rafay’s Programmable Edge Network, I’ll be sharing some really interesting learnings based on Rafay’s initial customer experiences. There are a lot of ‘not so obvious’ aspects that we have encountered. For example, how will you securely deploy an application image to every edge in your network and run them as needed? Or, how will you remotely manage a large number of compute clusters, assuming each edge is some container specific environment? Many customers may not have the deep pockets, resources or time to work through these issues. But with Rafay, developers can deploy their microservices based applications close to user and endpoints in just a matter of minutes without building an in-house platform or developing any specialized compute distribution capabilities.

Machine learning at the edge is all about scalability. What are some of the barriers to scaling and how might we overcome them?

The biggest impediment to scaling the edge is “scale” itself. Building a global network of edges is hard. To bring up hundreds or thousands of worldwide edge locations is not just a real estate problem, but a software infrastructure and management problem that is not simply addressed by throwing bodies and money at the problem. Recognizing this, Rafay took the approach of creating an eco-system of edge partners around the world. We have partnered with a number of service providers and CDNs who partner us to provide the hardware infrastructure for which we deploy our programmable edge platform in hundreds (and soon thousands) of locations. We think this makes tremendous sense as no one vendor has the deep pockets to deploy hyperconverged infrastructure in thousands of locations.

What are you looking forward to most at the Edge AI Summit in December?

I’m especially looking forward to meeting attendees to learn about their applications, as well as speaking with other vendors and seeing their product demonstrations. Customers who attend shows like the Edge AI Summit tend be really smart and on the cutting edge. So, while I’m sure many will be interested in what Rafay is developing, we are equally interested to understand what others are doing and thinking.

Creating a network of edges is a much harder problem than it appears. I know because I talk with enterprise and service providers every day about this challenge.

I’ve seen numerous press stories recently on edge computing, and nearly all describe the drivers, players, use cases and economics of the edge. But why?

Why should you care about edge computing?

As the CEO of an edge computing platform company, I believe the primary reason why edge computing holds promise is that it enables you to run your apps closer to end users. That means latency can be reduced and the performance end users experience when accessing their applications will be snappier. It also means that certain interactive experiences that are currently technically challenging to deliver due to low network latency targets may now be possible to implement. Edge computing may soon make it practical for an entirely new generation of consumer and industrial applications to exist. And your favorite ride-sharing app, which may seem sluggish when it’s showing you real-time location information for the car that’s on its way to pick you up, could provide a much snappier end-user experience.

That brings us to the following question: If you have established that you need an edge, what does it take to build it? Perhaps you’re thinking: “Easy; just stand up a server or two in a number of far-flung data centers.” (More likely, it will be hundreds of distant data centers or colocations.) That’s a start, but it’s way more complicated than that. Below, I’ll discuss some of the obvious and not-so-obvious things you’ll need to do to build out a scalable application delivery platform that exists at the edge.

The Obvious

There are many elements of building out a physical edge that are self-evident to IT or cloud practitioners. Some of these are listed below, along with some topics of consideration for each.

• Physical infrastructure: Which hosting companies should you partner with and where? How do these providers bill for your power needs per edge?

• Servers: How many servers do you need in each location? What specifications (CPU, memory, disk and interfaces) do you need to plan for? How do you get these servers shipped to various countries? What happens when one of the servers (or sub-components) fails?

• Network and bandwidth: What type of network peering relationships are needed globally to ensure connectivity across disparate carrier and telecommunications networks?

• Storage: Is the data that is being saved on a local disk secure? Have you considered encryption requirements?

• People: Do you have access to personnel for on-site installation, maintenance and troubleshooting services?

Moreover, organizations need a certain level of knowledge about the distant places their edges will be installed. In many cases, it’s not a technology issue as much as it is about process and getting in front of the right providers in the right regions of the world. In my experience and opinion, this — more than the technical expertise — is frequently the most difficult and costly step.

I think we could all agree that executing the aforementioned in hundreds or thousands of remote locations is a huge challenge. If you want to do this, consider the not-so-obvious aspects of building and deploying edges. This is where things can get really hard.

The Not-So-Obvious

Since founding our company, my co-founder and I have been grappling with the hard problems of deploying and enabling edge platforms as a service. This has proven to be both challenging and exciting, as making complex systems easy to consume is clearly very hard.

Here are some things to consider:

• How will you securely deploy an application image (e.g., a container) to every edge in your network and run them as needed? How do you do this several times a day if the application developer needs to apply fixes or add enhancements?

• How will you ensure that your application is running in a location close to your end users without amassing a massive infrastructure bill? In other words, how do you maximize your application’s global footprint while minimizing your cost?

• How will you remotely manage a large number of computer clusters, assuming each edge is some container-specific environment?

• How will you implement network and resource isolation so no one workload impacts another?

• How will you debug your remote application when (not if) things break?

• How will you make sure everything works in concert, on a global basis?

• How will you hire the personnel with the necessary technical expertise (bandwidth and global peering networks, hosting contracts, supply chain and shipping of servers worldwide, in-region troubleshooting, and so on) to build and maintain all this?

This is just a partial list.

Readers who have attempted this effort know that even seemingly straightforward tasks, such as deploying application images across tens of global locations, can be akin to building a software distribution system from scratch.

And readers in the know are well aware that ensuring cryptographic artifacts (e.g., SSL private keys) are not accessible “in the clear” on a remote server in a distant country is a complex task. This will require developers skilled in security best practices and with knowledge of key vaulting methodologies.

Clearly, when it comes to globally scaling applications and edge deployments, the not-so-obvious issues are multi-layered and complex and may be an unattainable task for some organizations. Many can, and will, attempt this with varying degrees of success. But similar to the way organizations eventually shifted away from maintaining their own data center footprints, there may be room to cut out many of these complicated steps. The challenge for innovative organizations is to find ways to continue drawing on the benefits of edge computing while minimizing the resources — and steps — necessary to do so.

When you see or hear the word “serverless,” which of the following comes to mind:

  1. AWS Lambda@Edge
  2. Cloudflare Workers
  3. Physical servers whimsically flying out of a data center

If you picked (3), you may be on the wrong website. If you picked (1) or (2), you’re part of the problem: You are conflating serverless and functions.

As Faizan Bashir wrote in a recent Hackernoon article, “Serverless is a cloud computing execution model where the cloud provider dynamically manages the allocation and provisioning of servers.”

To date, the one technology trend that has adopted the serverless paradigm is “Functions.” Functions are usually event-driven code snippets written in JavaScript, C# and other languages, which may be deployed as handlers for HTTP calls received by an HTTP gateway. For example, when a request is received, a registered handler can be called to modify the request header or body in order to to augment the feature set exposed by the application.

Given the footprint for a Function and their interactions with end users, packaging them alongside CDN infrastructure makes a lot of sense, which is why Cloudflare launched Workers, and AWS launched Lambda@Edge. CDNs are finely tuned for the delivery of static content that can be cached across a CDN’s global data centers. CDN customers can improve the end user experience by implementing useful, static, simple logic as Functions, and run them closer to users — which is where CDNs already have footprint.

But as applications get more interactive and evolve to deliver hyper personalized experiences, and as application developers explore how to move larger portions of their applications closer to end users, the utility of Functions becomes limited. A number of large, tech savvy companies such as Netflix, Youtube, Facebook and Uber have built out specialized infrastructure to solve this problem for their containerized applications. And now the need is arising within developer circles more broadly. But it doesn’t make sense for all companies to build out their own complex infrastructure along with the ability to build out pipelines to deploy and manage applications running all over the place.

Specifically, there is growing demand from developers to run their containerized microservices close to end users without having to worry about allocation and provisioning of servers near end users. This is essentially a “serverless for containers” model, where a developer can focus on application logic and rely on the service provider to take care of hardware and software infrastructure whenever and wherever it is needed.

So if a “serverless for containers” platform existed, is there need for a “serverless for functions” type platform? Absolutely. Depending on the application’s level of interactivity and personalization, developers may deploy:

  1. A majority of their application logic in 1 to 3 public cloud regions
  2. Microservices responsible for delivering interactive/personalized experiences close to end users, and
  3. Functions to offload specific actions from the core app that run in the CDN context.

As an industry, we went from (1) to (3) because CDNs were able to easily deliver (3). (2) has, to date, been a club of less than 20 companies that have benefited from their investments in distributed infrastructure, the right tools to simplify application distribution, etc.

This must change.

Anyone should be able to deploy microservices closer to end users. Team Rafay has strong ideas around how this should work. Interested in learning more? Email us or sign up for periodic updates.