EC2 vs. Fargate for Amazon EKS: A Cost Comparison

When it comes to running workloads on Amazon Web Services (AWS), two popular choices are Amazon Elastic Compute Cloud (EC2) and AWS Fargate. Both have their merits, but understanding their cost implications is crucial for making an informed decision. In this blog, we’ll dive into a cost comparison of EC2 and Fargate configurations within an Amazon Elastic Kubernetes Service (EKS) cluster.

The Basics: Fargate with EKS

Fargate provides on-demand, right-sized compute capacity for containers. You don’t have to provision, configure, or scale groups of ec2 instances to run containers. You also don’t need to choose server types, decide when to scale your node groups, or optimize cluster packing.

Containers deployed to EKS via Fargate run inside dedicated virtual machines providing a pretty high level of security isolation. The fact that Fargate doesn’t support privileged mode also can help reduce the risk of running privileged containers.

But, what about costs? To understand costs, we need to first understand how resources are allocated in Fargate.

vCPU and Memory Combos

When Pods are scheduled on Fargate, the vCPU and memory reservations within the pod’s specs are mapped to one of the “available combinations” in Fargate (see table below).

Fargate Overhead

Running EKS on Fargate comes with its own set of challenges and technical limitations. For each Fargate node, Amazon EKS adds an overhead of 256 MB to each Pod’s memory reservation to accommodate critical Kubernetes components (kubelet, kube-proxy, and containerd). These are required for it to become part of the EKS cluster.

Note that this overhead makes it impossible to utilize the smallest possible node size for your workloads. So. instead of using the 0.5 GB memory configuration, you’ll need to upgrade to a Fargate node with 1 GB of memory, which costs more.

Limitations and Hidden Costs of Fargate

One of the significant limitations of Fargate is the inability to over-provision workloads. In EC2, you can “bin pack” workloads, allowing some to burst while others idle, maximizing resource utilization. With Fargate, this isn’t an option.

In addition, with Fargate, all your daemonsets need to run as sidecars. This adds to the complexity and resource consumption as well.

Cost Comparison: A Practical Example

With the basics out of the way, let us try and compare costs of an Amazon EKS cluster with EC2 and Fargate. Let’s make the following assumptions:

• Every node is running 10 workload pods
• We have 4 daemonsets (monitoring, logging, security, etc)
• Each workload and daemonset consumes 0.5 vCPU and 0.5 GB memory
• Our cluster has a total of 10 ec2 based nodes
• Our cluster is operating in the us-east-1 region.

Based on EC2

With 10 nodes in our cluster, we have a total of “100 workload pods” and “40 daemonsets”. The resources required per node is “7 vCPUs and 7 GB Memory”

vCPU

• 10 workload pods * 0.5 vCPUs
• 4 daemonsets * 0.5 vCPUs
• Total = 7 vCPUs

Memory

• 10 workload pods * 0.5 GB
• 4 daemonsets * 0.5 GB
• Total = 7 GB

The smallest ec2 instance type that can accommodate our resource requirements is “t3a.2xlarge” which has 8 vCPUs and 32 GB Memory.

Total Cost ($/month)

Using On Demand pricing, the official AWS calculator estimates the cost to be $2,195.84 ($0.3008/hr x 730 hrs x 10 nodes). Users can drive down the costs further by using reserved instances. This will bring down the costs to $1,517.38 ($137.68 + $0.189/hr * 730 hrs *10 nodes)

Based on Fargate

In Fargate, a node is spawned per pod. So, we can expect to have 100 nodes with 4 sidecars per node (to accommodate the daemonsets). The resources required per node are 2.5 vCPUs (0.5/pod + 4×0.5 for daemonsets) and 2.756 GB Memory (0.5/pod + 4×0.5 for daemonsets + 0.256 GB for required k8s resources).

The closest Fargate configuration for our resource requirements is “4 vCPU, 8 GB memory”. The AWS cost calculator for Fargate estimates the cost as $14,416/month. This is 9x the costs compared to EC2 with Reserved Instances and 6x the costs compared to EC2 with On-Demand.

Conclusion

While EKS with Fargate may seem appealing due to its serverless nature and ease of use, the cost implications are significant. In our simplistic example, we discovered that using Fargate is 6-10x more expensive than running the same workloads on EC2.

In addition, the lack of resource flexibility and the need to round up to larger configurations make Fargate less economical, particularly when you’re dealing with multiple small workloads within an EKS cluster.

From a cluster design perspective, it makes more sense to use Fargate “selectively” in an EKS cluster for resources that can actually benefit from it. For example, it is a good design pattern to deploy and operate “Karpenter” for auto scaling in a Fargate profile with the workloads and daemonsets operating in EC2 nodes.

If cost is a significant factor in your decision-making process, EC2 remains the more viable option for EKS. Fargate’s simplicity and scalability are appealing, but the hidden costs and limitations can quickly add up, making it a less attractive choice for cost-conscious users.