Securing Kubernetes: Applying Zero-Trust Principles to Your Kubernetes Environment

Controlling Kubernetes Access Becomes More Challenging by the Day

The footprint of Kubernetes is expanding rapidly as IT teams take advantage of its power, efficiency, and flexibility to develop modern applications quickly and deliver them at scale. You may already have a large number of development and production Kubernetes clusters spread across on-premises data centers, multiple public cloud providers, and edge locations.

When it comes to ensuring Kubernetes security, however, it really is the wild west out there. Most teams have multiple clusters in multiple locations—often running different distributions with different management interfaces—and teams of developers, operators, contractors, and partners who need varying levels of access.

Given this complexity, it shouldn’t be a surprise that a lot of mistakes are happening. And many of the companies getting hacked, like Microsoft, Docker, and Jenkins, aren’t exactly new to Kubernetes or security.

If your team is deploying Kubernetes at scale, you have to do everything possible to ensure access security. In this blog, we’ll look at how you can apply Kubernetes Zero Trust principles to secure your entire environment, providing zero-trust security for containers.

Applying Kubernetes Zero Trust to Cluster Access

Zero Trust is a security model that assumes all actors, systems, and services operating in and between networks cannot be trusted. It draws on technologies such as authentication, authorization, and encryption and is continuously validated for security configuration and posture to ensure trust.

The Kubernetes API Server is the core of the Kubernetes control plane for each cluster. API calls are used to query and manipulate the state of all Kubernetes objects (namespaces, pods, config maps, etc.) Therefore, controlling access to API use is the key to controlling Kubernetes access and achieving Kubernetes Zero Trust. The first step in securing access is to protect traffic to and from the API server with Transport Layer Security (TLS).

Image Source: kubernetes.io

Once transport is secure, Kubernetes includes all the necessary hooks to implement Kubernetes Zero Trust to control API Server access for each Kubernetes cluster. These hooks fall into four key areas:

• Authentication
• Authorization
• Admission Control
• Logging and Auditing

Kubernetes Authentication

With Kubernetes Zero Trust, all user and service accounts must be authenticated before they can execute an API call. There are a number of Kubernetes security modules and plugins to ensure that the platform will work with your preferred authentication system:

• Client certificates
• HTTP Basic Auth
• Bearer tokens
• OpenID Connect Tokens
• Authentication Proxy (to support LDAP, SAML, Kerberos, etc.)
• Webhook Token Authorization

It’s considered a best practice to use at least two authentication methods (multi-factor authentication or MFA).

Authorization

Obviously, you don’t want every user or service account accessing a Kubernetes cluster to be able to carry out every possible action. With Kubernetes Zero Trust, a request is only authorized if an authenticated user has permission to complete the requested action. Each request specifies the username of the requester, the action requested, and the objects affected.

As with authentication, Kubernetes supports a number of different ways to handle authorization, including:

• ABAC mode. Attribute-based access control or ABAC authorizes access dynamically based on a combination of user, environment, and resource attributes.
• RBAC mode. Role-based access control or RBAC authorizes access based on the user’s role in the organization such as admin, developer, security, etc.

Most organizations that implement access control choose RBAC as the more practical solution because it’s easier to manage and provides all the granularity necessary for most use cases. The additional granularity of ABAC can be time consuming to define and configure properly and troubleshooting can be challenging if issues arise.

Admission Control

implement business logic to refine your Kubernetes Zero Trust approach. Admission controllers provide methods to act on requests that create, modify, delete, or connect to Kubernetes objects. You can have multiple admission controllers, and if any admission controller rejects a request, then the request is immediately rejected.

There are a variety of built-in admission controllers you can enable to enforce policies and implement various actions. Dynamic controllers enable you to modify requests on the fly to adhere to your established rules. For example, the ResourceQuota admission controller observes incoming requests and ensures that they don’t violate the constraints you’ve enumerated in the ResourceQuota object for a Namespace. For more details, see Using Admission Controllers.

Logging and Auditing

Auditing is an essential part of Kubernetes security. Kubernetes auditing capabilities enable you to track the actions performed within a cluster, including actions by users, applications, and the control plane itself. Available audit levels include:

• None. Don’t log this event.
• Metadata. Log request metadata.
• Request. Log event metadata and the request.
• RequestResponse. Log event metadata, the request, and the response.

You can also control where audited events are logged. The log backend writes events to the cluster’s local filesystem, while the webhook backend sends audit events to an external logging system.

Kubernetes Zero Trust Best Practices

Here’s a summary of best practices for implementing Zero Trust for Kubernetes Access:

API Server

• Enable TLS everywhere
• Use a private endpoint for the API Server
• Use third-party authentication for the API Server
• Close firewall inbound rules to the API Server, ensuring it is cloaked and not directly accessible from the Internet

Authentication

• Use at least two authentication methods
• Rotate client certificates regularly

Authorization

• Enable RBAC with least privilege

Admission Control

• Enable appropriate admission controllers for your needs

Auditing

• Enable audit logs with central log aggregation

Zero Trust at Rafay

While the various methods and best practices just described give you the ability to create a Kubernetes Zero Trust environment, keeping all the individual elements correctly configured and aligned becomes a challenge once you have more than a few clusters, especially when multiple workloads and Kubernetes distributions are involved.

This is the exact challenge faced by many Rafay customers. One of Rafay’s fast-growing customers is a consumer security monitoring company. Because this company’s applications work with streams of data like video and audio, its Kubernetes clusters have to be geographically close to its global customer base. The company operates roughly 90 clusters ranging from pre-production to QA to staging to Prod.

This company faced three main security problems with regard to secure user access to Kubernetes clusters:

• Managing user accounts. With roughly 300-400 developers and 20 operations personnel, manually adding and removing users from each cluster was an operational and security nightmare.
• Slow access. When a severity 1 incident occurs, time to remediation is critical, but with “traditional” access methods it was taking troubleshooters 5 minutes just to get logged into the affected cluster.
• Complex compliance reporting. With log data spread across 90 clusters, centralized auditing and compliance reporting was nearly impossible.

To find a solution to these challenges, the company turned to Rafay. Rafay’s Zero-Trust Access Service, part of Rafay’s Kubernetes Operations Platform, enables the globally distributed team to manage user access across all its clusters from a central location; centralized audit logging makes compliance reporting much simpler. The customer achieves three clear benefits:

• Flexible role-based access control. If a developer changes roles, access permissions are updated automatically so they never have too much or too little access.
• Fast and streamlined access. No more 5-minute delays. An authorized user can get access to a cluster with a single click.
• Just-in-time credentials. Service accounts for authorized users are created on remote clusters “just in time” on access and automatically flushed after the user logs out, eliminating the chance of dangling, out-of-date credentials.

The Rafay Zero-Trust Access Service ensures that the Zero Trust Kubernetes best practices described earlier are applied and enforced to secure your entire Kubernetes environment, eliminating the need to manually apply best practices on every cluster. You can operate Kubernetes at scale with far less risk.

Rafay utilizes the latest Zero Trust Kubernetes principles and security best practices throughout our Kubernetes Operations Platform to guarantee security, and—because our management solution works with all major cloud providers and certified Kubernetes distributions—you can increase security everywhere, including enabling Zero Trust on AWS.

To find out more about Rafay’s approach to Kubernetes security, visit our Kubernetes Cluster & Application Security page. Ready to find out why so many enterprises and platform teams have partnered with Rafay to streamline Kubernetes operations? Sign up for a free trial today.