Release notes for extended support versions
This topic gives important changes to be aware of for each Kubernetes version in extended support. When upgrading, carefully review the changes that have occurred between the old and new versions for your cluster.
Kubernetes 1.24
Kubernetes 1.24
is now available in Amazon EKS. For more information about Kubernetes
1.24
, see the official release announcement
Important
-
Starting with Kubernetes
1.24
, new beta APIs aren't enabled in clusters by default. By default, existing beta APIs and new versions of existing beta APIs continue to be enabled. Amazon EKS follows the same behavior as upstream Kubernetes1.24
. The feature gates that control new features for both new and existing API operations are enabled by default. This is in alignment with upstream Kubernetes. For more information, see KEP-3136: Beta APIs Are Off by Defaulton GitHub. -
Support for Container Runtime Interface (CRI) for Docker (also known as Dockershim) is removed from Kubernetes
1.24
. Amazon EKS official AMIs have containerd as the only runtime. Before moving to Amazon EKS1.24
or higher, you must remove any reference to bootstrap script flags that aren't supported anymore. You must also make sure that IP forwarding is enabled for your worker nodes. For more information, see Amazon EKS ended support for Dockershim. -
If you already have Fluentd configured for Container Insights, then you must migrate Fluentd to Fluent Bit before updating your cluster. The Fluentd parsers are configured to only parse log messages in JSON format. Unlike
dockerd
, thecontainerd
container runtime has log messages that aren't in JSON format. If you don't migrate to Fluent Bit, some of the configured Fluentd's parsers will generate a massive amount of errors inside the Fluentd container. For more information on migrating, see Set up Fluent Bit as a DaemonSet to send logs to CloudWatch Logs. -
In Kubernetes
1.23
and earlier,kubelet
serving certificates with unverifiable IP and DNS Subject Alternative Names (SANs) are automatically issued with unverifiable SANs. These unverifiable SANs are omitted from the provisioned certificate. In version1.24
and later clusters,kubelet
serving certificates aren't issued if any SAN can't be verified. This preventskubectl
exec andkubectl
logs commands from working. For more information, see Certificate signing considerations before upgrading your cluster to Kubernetes 1.24. -
When upgrading an Amazon EKS
1.23
cluster that uses Fluent Bit, you must make sure that it's runningk8s/1.3.12
or later. You can do this by reapplying the latest applicable Fluent Bit YAML file from GitHub. For more information, see Setting up Fluent Bit in the Amazon CloudWatch User Guide.
-
You can use Topology Aware Hints to indicate your preference for keeping traffic in zone when cluster worker nodes are deployed across multiple availability zones. Routing traffic within a zone can help reduce costs and improve network performance. By default, Topology Aware Hints are enabled in Amazon EKS
1.24
. For more information, see Topology Aware Hintsin the Kubernetes documentation. -
The
PodSecurityPolicy
(PSP) is scheduled for removal in Kubernetes1.25
. PSPs are being replaced with Pod Security Admission (PSA). PSA is a built-in admission controller that uses the security controls that are outlined in the Pod Security Standards (PSS) . PSA and PSS are both beta features and are enabled in Amazon EKS by default. To address the removal of PSP in version 1.25
, we recommend that you implement PSS in Amazon EKS. For more information, see Implementing Pod Security Standards in Amazon EKSon the Amazon blog. -
The
client.authentication.k8s.io/v1alpha1
ExecCredential is removed in Kubernetes1.24
. The ExecCredential API was generally available in Kubernetes1.22
. If you use a client-go credential plugin that relies on thev1alpha1
API, contact the distributor of your plugin on how to migrate to thev1
API. -
For Kubernetes
1.24
, we contributed a feature to the upstream Cluster Autoscaler project that simplifies scaling Amazon EKS managed node groups to and from zero nodes. Previously, for the Cluster Autoscaler to understand the resources, labels, and taints of a managed node group that was scaled to zero nodes, you needed to tag the underlying Amazon EC2 Auto Scaling group with the details of the nodes that it was responsible for. Now, when there are no running nodes in the managed node group, the Cluster Autoscaler calls the Amazon EKSDescribeNodegroup
API operation. This API operation provides the information that the Cluster Autoscaler requires of the managed node group's resources, labels, and taints. This feature requires that you add theeks:DescribeNodegroup
permission to the Cluster Autoscaler service account IAM policy. When the value of a Cluster Autoscaler tag on the Auto Scaling group powering an Amazon EKS managed node group conflicts with the node group itself, the Cluster Autoscaler prefers the value of the Auto Scaling group tag. This is so that you can override values as needed. For more information, see Autoscaling. -
If you intend to use Inferentia or Trainium instance types with Amazon EKS
1.24
, you must upgrade to the Amazon Neuron device plugin version 1.9.3.0 or later. For more information, see Neuron K8 release [1.9.3.0]in the Amazon Neuron Documentation. -
Containerd
hasIPv6
enabled for Pods, by default. It applies node kernel settings to Pod network namespaces. Because of this, containers in a Pod bind to bothIPv4
(127.0.0.1
) andIPv6
(::1
) loopback addresses.IPv6
is the default protocol for communication. Before updating your cluster to version1.24
, we recommend that you test your multi-container Pods. Modify apps so that they can bind to all IP addresses on loopback interfaces. The majority of libraries enableIPv6
binding, which is backward compatible withIPv4
. When it's not possible to modify your application code, you have two options:-
Run an
init
container and setdisable ipv6
totrue
(sysctl -w net.ipv6.conf.all.disable_ipv6=1
). -
Configure a mutating admission webhook
to inject an init
container alongside your application Pods.
If you need to block
IPv6
for all Pods across all nodes, you might have to disableIPv6
on your instances. -
-
The
goaway-chance
option in the Kubernetes API server helps prevent HTTP/2
client connections from being stuck on a single API server instance, by randomly closing a connection. When the connection is closed, the client will try to reconnect, and will likely land on a different API server as a result of load balancing. Amazon EKS version1.24
has enabledgoaway-chance
flag. If your workload running on Amazon EKS cluster uses a client that is not compatible withHTTP GOAWAY
, we recommend that you update your client to handle GOAWAY
by reconnecting on connection termination.
For the complete Kubernetes 1.24
changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#changelog-since-v1230
Kubernetes 1.23
Kubernetes 1.23
is now available in Amazon EKS. For more information about Kubernetes
1.23
, see the official release announcement
Important
-
The Kubernetes in-tree to container storage interface (CSI) volume migration feature is enabled. This feature enables the replacement of existing Kubernetes in-tree storage plugins for Amazon EBS with a corresponding Amazon EBS CSI driver. For more information, see Kubernetes 1.17 Feature: Kubernetes In-Tree to CSI Volume Migration Moves to Beta
on the Kubernetes blog. The feature translates in-tree APIs to equivalent CSI APIs and delegates operations to a replacement CSI driver. With this feature, if you use existing
StorageClass
,PersistentVolume
, andPersistentVolumeClaim
objects that belong to these workloads, there likely won't be any noticeable change. The feature enables Kubernetes to delegate all storage management operations from the in-tree plugin to the CSI driver. If you use Amazon EBS volumes in an existing cluster, install the Amazon EBS CSI driver in your cluster before you update your cluster to version1.23
. If you don't install the driver before updating an existing cluster, interruptions to your workloads might occur. If you plan to deploy workloads that use Amazon EBS volumes in a new1.23
cluster, install the Amazon EBS CSI driver in your cluster before deploying the workloads your cluster. For instructions on how to install the Amazon EBS CSI driver on your cluster, see Amazon EBS CSI driver. For frequently asked questions about the migration feature, see Amazon EBS CSI migration frequently asked questions. -
Extended Support for Amazon EKS optimized Windows AMIs that are published by Amazon isn't available for Kubernetes version
1.23
but is available for Kubernetes version1.24
and higher.
-
Kubernetes stopped supporting
dockershim
in version1.20
and removeddockershim
in version1.24
. For more information, see Kubernetes is Moving on From Dockershim: Commitments and Next Stepsin the Kubernetes blog. Amazon EKS will end support for dockershim
starting in Amazon EKS version1.24
. Starting with Amazon EKS version1.24
, Amazon EKS official AMIs will havecontainerd
as the only runtime.Even though Amazon EKS version
1.23
continues to supportdockershim
, we recommend that you start testing your applications now to identify and remove any Docker dependencies. This way, you are prepared to update your cluster to version1.24
. For more information aboutdockershim
removal, see Amazon EKS ended support for Dockershim. -
Kubernetes graduated
IPv4
/IPv6
dual-stack networking for Pods, services, and nodes to general availability. However, Amazon EKS and the Amazon VPC CNI plugin for Kubernetes don't support dual-stack networking. Your clusters can assignIPv4
orIPv6
addresses to Pods and services, but can't assign both address types. -
Kubernetes graduated the Pod Security Admission (PSA) feature to beta. The feature is enabled by default. For more information, see Pod Security Admission
in the Kubernetes documentation. PSA replaces the Pod Security Policy (PSP) admission controller. The PSP admission controller isn't supported and is scheduled for removal in Kubernetes version 1.25
.The PSP admission controller enforces Pod security standards on Pods in a namespace based on specific namespace labels that set the enforcement level. For more information, see Pod Security Standards (PSS) and Pod Security Admission (PSA)
in the Amazon EKS best practices guide. -
The
kube-proxy
image deployed with clusters is now the minimal base imagemaintained by Amazon EKS Distro (EKS-D). The image contains minimal packages and doesn't have shells or package managers. -
Kubernetes graduated ephemeral containers to beta. Ephemeral containers are temporary containers that run in the same namespace as an existing Pod. You can use them to observe the state of Pods and containers for troubleshooting and debugging purposes. This is especially useful for interactive troubleshooting when
kubectl exec
is insufficient because either a container has crashed or a container image doesn't include debugging utilities. An example of a container that includes a debugging utility is distroless images. For more information, see Debugging with an ephemeral debug container in the Kubernetes documentation. -
Kubernetes graduated the
HorizontalPodAutoscaler
autoscaling/v2
stable API to general availability. TheHorizontalPodAutoscaler
autoscaling/v2beta2
API is deprecated. It will be unavailable in1.26
. -
The
goaway-chance
option in the Kubernetes API server helps prevent HTTP/2
client connections from being stuck on a single API server instance, by randomly closing a connection. When the connection is closed, the client will try to reconnect, and will likely land on a different API server as a result of load balancing. Amazon EKS version1.23
has enabledgoaway-chance
flag. If your workload running on Amazon EKS cluster uses a client that is not compatible withHTTP GOAWAY
, we recommend that you update your client to handle GOAWAY
by reconnecting on connection termination.
For the complete Kubernetes 1.23
changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#changelog-since-v1220