What’s New in Kubernetes v1.23

Many people already have eyes on the upcoming Kubernetes v1.24, which is scheduled for an official rollout on Tuesday, April 19, 2022. Indeed, the release schedule is already ongoing. Week one of the release cycle started on January 10, 2022, and Production Readiness Soft Freeze was just recently concluded.

However, as is the norm, it will take a few months (after the official release) before most developers even think about using K8s v1.24. So instead, the focus is still to get to grips with v1.23, officially released to the public on December 7, 2021.

So, we spared some time to look at some of the new features in v1.23, and it’s turning out to be a significant improvement from the previous version. Below is a roundup of what’s new.

What’s New in Kubernetes v1.23?

There are tons of changes. For instance, you can look forward to a host of new features, such as an events subcommand for Kubectl, gRPC probes, and expression language for custom resources.

Several features, such as horizontal pod auto-scaler v2 and IPv4/IPv6 dual-stack, have also reached general availability (GA). Meanwhile, others, including podsecurity and ephemeral containers, are in Beta, and CRD validation is in alpha.

Of course, we can’t discuss all the changes. However, it’s always helpful to understand the most significant highlights to plan. So, we’ve rounded up the most notable changes to give you a head start.  

Deprecated 

Deprecated items/features are those removed or discontinued on the Kubernetes platform. Only one item was deprecated in the recently released v1.23.

• Klog Specific Flags Deprecated 

You may recall that developers recently used the glog library for logging, a move initially taken to simplify logging on the platform. However, the library was moved to klog in recent versions. While this has been helpful, a more flexible, scalable, and high-performance solution is necessary, thus the decision to remove the flags.

Some of the flags have already been removed in v1.23. Meanwhile, the code implementing others will be removed in future releases. So, you need to plan accordingly and find alternative solutions if some of the flags you’ve always relied on are on the chopping board.  

Graduating to Stable

Stable or General Availability (GA) features are part of the Kubernetes environment, with a guarantee from the Kubernetes community that they will stay in place throughout the current major version. They cannot be disabled. At least eleven features have graduated to stable in v1.23.

• Dual-stack IPv4/IPv6 has graduated to GA

Developers have waited for IPv6 with excitement since it graduated to alpha in v1.9. But unfortunately, the GA graduation had to wait for various reasons. Now, IPv4/IPv6 networking modules have graduated to general availability, fixing many perennial issues. For instance, it supports native IPv4-to-IPv4 communication in parallel with IPv6-to-IPv6.

Beware, though, that users may be limited to their cloud providers and underlying infrastructure. For instance, since pods have dual-stack networking attached and nodes have routable IPv4/IPv6 network interfaces, you must use plugins that can process dual-stack networking to assign IPs to pods and services.

• Horizontal pod autoscaler v2 is now in GA

Another feature that has graduated to general availability is horizontal pod autoscaler v2 (HPA), the most popular tool for scaling applications in Kubernetes today. You may have noticed that HPA v1 only scaled applications based on pod memory and CPU usage. In contrast, HPA v2 considers other important custom metrics, such as queue sizes, which have proven invaluable for developers.

Unfortunately, HPA v2 has been in Beta for eternity. Now, this useful feature is moving to general availability, meaning its structure is finally stable and final. So, if you’ve been waiting for this stability guarantee to adopt HPA v2, the wait is over.

• Generic ephemeral volumes feature graduates to GA

The generic ephemeral volumes feature allows dynamic provisioning-supporting storage drivers as ephemeral volumes with the volume’s lifecycles dependent on the pod. In other words, they provide storage for pods for temporary data and become empty after provisioning, which is vital for flexibility.

So, you’ll also be excited that the generic ephemeral volumes feature is moving up to general availability in v1.23.

• Skip Volume Ownership change graduates to GA

The feature to configure volume permission and ownership change policy for Pods moved to GA in v1.23. This allows users to skip recursive permission changes on mount and speeds up the pod start up time

Others 

Other features that have graduated to General Availability in Kubernetes v1.23 are CronJobs (previously ScheduledJobs) and TTL after finish. Cronjobs which runs periodic tasks in Kubernetes clusters was introduced in Kubernetes 1.4 and has been in Beta since v1.8. Meanwhile, TTL after Finish helps clean up finished or complete jobs automatically.

Namespace Scoped Ingress Class Parameters, AWS EBS in-tree to CSI driver migration, new recursive volume ownership (FSGroup), and Config FSGroup Policy in CSI Driver Object are also graduating to general availability. Finally, the feature for defending against logging secrets via static analysis and HPA API also graduate to stable in v1.23.

Graduating to Beta

Beta features are enabled by default and can be disabled by the Google Kubernetes Engine (GKE) for a particular version. So, although you should assume the feature is enabled, you may need to contact Cloud Customer Care for assistance to verify if it is indeed enabled. At least 17 items have moved to Beta in v1.23.

• Scheduler Component Config API is now in Beta

Component configuration is used for the definition of a static configuration for components. It is available in the component metadata, and you can access it without creating component instances. 

Now the scheduler component of ComponentConfig is moving to Beta. Additionally, Kubernetes has removed multiple plugins from the component to make it more flexible. For instance, NodeResourcesLeastAllocated and NodeResourcesMostAllocated have been removed from the component. In addition, NodeLabel and ServiceAffinity have also been removed.

• Priority and Fairness for API Server Requests feature is in Beta

The API Priority and Fairness (APF) feature gate on Kubernetes improves on max-in-flight limitations. It classifies and isolates requests in a more fine-tuned manner and introduces a limited amount for queueing, such that requests are rejected in cases of very brief bursts. Doing so boosts responsiveness for admin and improves maintenance during high-load tasks.

Besides graduating to Beta, significant changes have been made to the feature in v123. In fact, the item nearly graduated to stable. However, the team couldn’t complete some of the changes needed to graduate it to general availability. So, there’s a great chance it will move to GA in the next version.

• Job tracking without lingering pods is in Beta

 Jobs create one or more pods in Kubernetes and continue to retry pod execution until a specified number of pods terminate successfully. Therefore, jobs track successfully terminated pods and only end when a predetermined number of completions is reached.

Job tracking without lingering pods is a new feature recently introduced to the platform to allow removal of completed pods earlier to free resources in the cluster. And now the feature has moved to Beta!

• Pod security admission is in Beta

Pod Security Policies (PSPs) were deprecated in v1.21 and will be removed entirely from Kubernetes in v1.25, due in 2022. The main reason is that pod security policies have significant usability problems. They are also confusing, which makes overall security on the platform even more complex.

The K8s community initiated a different security approach for validating admission webhooks. This new security model is known as Pod Security Admission (PSA). It’s similar to PSP and, like PSP, is installed in Kubernetes API. The new model just reached Beta in v1.23.

Others 

Other features and items graduating to Beta in Kubernetes v1.23 include structured logging and the Add minReadySeconds to Statefulsets feature, which brings to Statefulsets the optional minreadySeconds field you may have already used in deployments and Replication Controllers. Topology-aware hints, Kubernetes Container Runtime Interface (CRI) support, and the extend podresources API feature also move to Beta.

Also graduating to Beta is the feature that allows simplified multi-point plugin configuration for scheduler, JobMutableNodeSchedulingDirectives, which allows updating affinity constraints for suspended jobs, and support for windows privileged containers. Pod security admission and Kubetest2 CI migration are also finally in Beta.

Graduating to Alpha

Finally, alpha items on Kubernetes are disabled by default but can be enabled by GKE for a specified GKE control plane version. Similarly, an alpha cluster enables all features in the cluster by default. Alpha clusters are typically intended for experiments rather than production workloads. Nineteen items have graduated to alpha in Kubernetes v1.23.

• CRD Validations are in alpha

Developers who’ve written K8s operators to manage custom resources in cluster APIs understand the difficulty and challenges. Typically, you must implement a custom admission webhook that validates everything, often leaving developers wondering why Kubernetes can’t validate on your behalf.

You can now turn to CRD validations for this, though it’s still in alpha. To easily write custom Kubernetes controllers, enable your CustomResourceValidationExpessions feature gate to activate CRD validations. Then use it to define validation rules in your CRD OpenAPI schema.

• OpenAPI v3

OpenAPI v3 feature gate is enabled starting Kubernetes version v1.23 allowing users to request OpenAPI v3.0 spec for all K8s types. The main aim here is to make OpenAPI v3 as transparent as possible while including support for fieldsets dropped when publishing OpenAPI v2, including one-of, nullable, default, and any-of.

A separate spec is published for every K8s group version for improved performance and discovery. You can access the specs by going to $cluster/openapi/v3/apis/ and scrolling to <group> then navigating to <version>. You can also find specs for all versions at $cluster/openapi/v3/.

• Server-side field validation 

Also moved to alpha is server-side field validation. Up to now, developers have used kubectl- validate = true to indicate that requests should fail if they specify unknown fields on an object. Unfortunately, this means that validation is done on the client-side (kubectl) rather than the server-side (kube-apiserver). Client-side validation comes with several issues, including the need for each client to implement validations and slower bug fixes.

Now, this feature gate is enabled and in alpha. Enabling this feature in your Kubernetes lets you get warnings from your server whenever you send request objects containing unknown or duplicate fields.

• JobReadyPods 

Graduating JobReadyPods to alpha adds a job.status.ready field to easily count the number of job pods with a Ready status. It’s not dissimilar to the active, failed, and succeed fields. However, you may have noticed that active fields include pending and running jobs. Therefore, it’s easy to be misled on the progress since the jobs stay pending for quite a while.

The new ready field means you don’t have to listen to pod updates all the time to get an accurate picture of job statuses.

Others 

Other features graduated to Alpha in Kubernetes v1.23 are OpenAPI Enum Types, instrumentation (which simplifies logging), c-advisor-less pod stats, and CRI-full container. PriorityClassValueGracefulShutdown, add gRPC probe to pod (for configuring gRPC (HTTP/2 over TLS) liveness probes to pod), distribute-cpus-across-numa (a new CPUManager policy option), and recovervolumeexpansionfailure (for recovering from resize failures) have all graduated to alpha too.

What Else is New?

There are many other changes in Kubernetes v1.23. For instance, all Portworx Container Storage Interface (CSI) driver plugin operations are henceforth redirected to pxd.portwrx.com driver. Meanwhile, RBD CSI plugin operations are redirected to the rbd.csi.com driver. Both features are now in alpha.

You’ll also find an IdentifyPodsOS feature that adds the OS field to PodSpec. This makes it easier to define the operating system you want the pod to run on.

Overall, K8s v1.23 consists of 47 enhancements, 11 of which have graduated to stable, 17 moved to Beta, 19 entered into alpha, and one deprecated.

Summary 

The new release makes significant changes/improvements in the critical areas of storage, networking, and security in Kubernetes, aiming to make the containerization management service faster, more flexible, and future-proof. The goal is to make Kubernetes the leading container orchestration platform in the industry.