Using Kubernetes (and not VMs) for Edge Computing and Machine Learning

There is near-universal agreement in IT circles that containers and Kubernetes are de facto technologies used for managing modern applications both for on-premises and cloud deployments. But what about emerging use cases such as edge computing and machine learning?

Edge and machine learning use cases are growing in popularity. In order to remove network latency for high-performance applications and to take advantage of 5G and IoT, companies are moving applications to the edge of the network. The edge can be anything such as a cell tower, retail stores, cars, and even airplanes. Managing these numerous edge applications that are likely distributed across the globe is a challenge. Normally, virtual machines (VMs) would help but they don’t do the trick because they are much more resource-intensive on the limited capabilities of edge compute infrastructure. Containers, on the other hand, are lightweight and resource-efficient, making them the perfect application packaging framework at the edge. And Kubernetes is the industry’s tool of choice for container orchestration. Verizon recently launched an entirely new managed service based on this strategy.

Machine learning (ML) has been a technology trend for some time now, but rapidly deploying model iterations can be difficult. This is where containers and Kubernetes come in. With this combination, companies can deploy, test, and iterate ML models much more frequently using Kubeflow for example, and thus react to the market more quickly.

Gartner recently published a report: “Tech CEOs Can Expand the Container-Based Infrastructure Market by Supporting Edge Computing and Machine Learning” (Nov 2020), written by Wataru Katsurashima and Arun Chandrasekaran, which helps us understand these use cases better.

In the report, Gartner finds, “Leading enterprises and vendors are adopting containers and Kubernetes in their edge computing infrastructure because containers are often more suitable than virtual machines (VMs) to resource-constrained edge nodes and modular application architecture, which is often used at the edge.”

The report goes on to recommend that companies “Improve their offering’s strengths in edge computing use cases by implementing differentiable enhancements in these three key features: management of edge nodes, edge-optimized Kubernetes distributions and integrated edge networking/security.” And for ML, “Make their offerings easier to use in ML use cases by providing integration with a broad set of DataOps and MLOps tools, …and creating a software catalog of artificial intelligence (AI) independent software vendor (ISV) tools and software operators.”

Containers and Kubernetes help companies manage modern applications. But the moniker “modern application” can mean a lot of different applications using many different types of technologies. Now, we’ll need to have it include edge computing and machine learning technologies as well.