Release notes for standard support versions - Amazon EKS
Kubernetes 1.29Kubernetes 1.28Kubernetes 1.27Kubernetes 1.26Kubernetes 1.25
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Release notes for standard support versions

This topic gives important changes to be aware of for each Kubernetes version in standard support. When upgrading, carefully review the changes that have occurred between the old and new versions for your cluster.

Note

For 1.24 and later clusters, officially published Amazon EKS AMIs include containerd as the only runtime. Kubernetes versions earlier than 1.24 use Docker as the default runtime. These versions have a bootstrap flag option that you can use to test out your workloads on any supported cluster with containerd. For more information, see Amazon EKS ended support for Dockershim.

Kubernetes 1.29

Kubernetes 1.29 is now available in Amazon EKS. For more information about Kubernetes 1.29, see the official release announcement.

Important

  • The deprecated flowcontrol.apiserver.k8s.io/v1beta2 API version of FlowSchema and PriorityLevelConfiguration are no longer served in Kubernetes v1.29. If you have manifests or client software that uses the deprecated beta API group, you should change these before you upgrade to v1.29.

  • The .status.kubeProxyVersion field for Node objects is now deprecated, and the Kubernetes project is proposing to remove that field in a future release. The deprecated field is not accurate and has historically been managed by kubelet - which does not actually know the kube-proxy version, or even whether kube-proxy is running. If you've been using this field in client software, stop - the information isn't reliable and the field is now deprecated.

  • In Kubernetes 1.29 to reduce potential attack surface, the LegacyServiceAccountTokenCleanUp feature labels legacy auto-generated secret-based tokens as invalid if they have not been used for a long time (1 year by default), and automatically removes them if use is not attempted for a long time after being marked as invalid (1 additional year by default). To identify such tokens, a you can run: 

    kubectl get cm kube-apiserver-legacy-service-account-token-tracking -nkube-system

For the complete Kubernetes 1.29 changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.29.md#changelog-since-v1280.

Kubernetes 1.28

Kubernetes 1.28 is now available in Amazon EKS. For more information about Kubernetes 1.28, see the official release announcement.

  • Kubernetes v1.28 expanded the supported skew between core node and control plane components by one minor version, from n-2 to n-3, so that node components (kubelet and kube-proxy) for the oldest supported minor version can work with control plane components (kube-apiserver, kube-scheduler, kube-controller-manager, cloud-controller-manager) for the newest supported minor version.

  • Metrics force_delete_pods_total and force_delete_pod_errors_total in the Pod GC Controller are enhanced to account for all forceful pods deletion. A reason is added to the metric to indicate whether the pod is forcefully deleted because it's terminated, orphaned, terminating with the out-of-service taint, or terminating and unscheduled.

  • The PersistentVolume (PV) controller has been modified to automatically assign a default StorageClass to any unbound PersistentVolumeClaim with the storageClassName not set. Additionally, the PersistentVolumeClaim admission validation mechanism within the API server has been adjusted to allow changing values from an unset state to an actual StorageClass name.

For the complete Kubernetes 1.28 changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.28.md#changelog-since-v1270.

Kubernetes 1.27

Kubernetes 1.27 is now available in Amazon EKS. For more information about Kubernetes 1.27, see the official release announcement.

Important

  • The support for the alpha seccomp annotations seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io annotations was removed. The alpha seccomp annotations was deprecated in 1.19, and with their removal in 1.27, seccomp fields will no longer auto-populate for Pods with seccomp annotations. Instead, use the securityContext.seccompProfile field for Pods or containers to configure seccomp profiles. To check whether you are using the deprecated alpha seccomp annotations in your cluster, run the following command:

    kubectl get pods --all-namespaces -o json | grep -E 'seccomp.security.alpha.kubernetes.io/pod|container.seccomp.security.alpha.kubernetes.io'

  • The --container-runtime command line argument for the kubelet was removed. The default container runtime for Amazon EKS has been containerd since 1.24, which eliminates the need to specify the container runtime. From 1.27 onwards, Amazon EKS will ignore the --container-runtime argument passed to any bootstrap scripts. It is important that you don't pass this argument to --kubelet-extra-args in order to prevent errors during the node bootstrap process. You must remove the --container-runtime argument from all of your node creation workflows and build scripts.

  • The kubelet in Kubernetes 1.27 increased the default kubeAPIQPS to 50 and kubeAPIBurst to 100. These enhancements allow the kubelet to handle a higher volume of API queries, improving response times and performance. When the demands for Pods increase, due to scaling requirements, the revised defaults ensure that the kubelet can efficiently manage the increased workload. As a result, Pod launches are quicker and cluster operations are more effective.

  • You can use more fine grained Pod topology to spread policies such as minDomain. This parameter gives you the ability to specify the minimum number of domains your Pods should be spread across. nodeAffinityPolicy and nodeTaintPolicy allow for an extra level of granularity in governing Pod distribution. This is in accordance to node affinities, taints, and the matchLabelKeys field in the topologySpreadConstraints of your Pod's specification. This permits the selection of Pods for spreading calculations following a rolling upgrade.

  • Kubernetes1.27 promoted to beta a new policy mechanism for StatefulSets that controls the lifetime of their PersistentVolumeClaims(PVCs). The new PVC retention policy lets you specify if the PVCs generated from the StatefulSet spec template will be automatically deleted or retained when the StatefulSet is deleted or replicas in the StatefulSet are scaled down.

  • The goaway-chance option in the Kubernetes API server helps prevent HTTP/2 client connections from being stuck on a single API server instance, by randomly closing a connection. When the connection is closed, the client will try to reconnect, and will likely land on a different API server as a result of load balancing. Amazon EKS version 1.27 has enabled goaway-chance flag. If your workload running on Amazon EKS cluster uses a client that is not compatible with HTTP GOAWAY, we recommend that you update your client to handle GOAWAY by reconnecting on connection termination.

For the complete Kubernetes 1.27 changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.27.md#changelog-since-v1260.

Kubernetes 1.26

Kubernetes 1.26 is now available in Amazon EKS. For more information about Kubernetes 1.26, see the official release announcement.

Important

Kubernetes 1.26 no longer supports CRI v1alpha2. This results in the kubelet no longer registering the node if the container runtime doesn't support CRI v1. This also means that Kubernetes 1.26 doesn't support containerd minor version 1.5 and earlier. If you're using containerd, you need to upgrade to containerd version 1.6.0 or later before you upgrade any nodes to Kubernetes 1.26. You also need to upgrade any other container runtimes that only support the v1alpha2. For more information, defer to the container runtime vendor. By default, Amazon Linux and Bottlerocket AMIs include containerd version 1.6.6.

  • Before you upgrade to Kubernetes 1.26, upgrade your Amazon VPC CNI plugin for Kubernetes to version 1.12 or later. If you don't upgrade to Amazon VPC CNI plugin for Kubernetes version 1.12 or later, the Amazon VPC CNI plugin for Kubernetes will crash. For more information, see Working with the Amazon VPC CNI plugin for Kubernetes Amazon EKS add-on.

  • The goaway-chance option in the Kubernetes API server helps prevent HTTP/2 client connections from being stuck on a single API server instance, by randomly closing a connection. When the connection is closed, the client will try to reconnect, and will likely land on a different API server as a result of load balancing. Amazon EKS version 1.26 has enabled goaway-chance flag. If your workload running on Amazon EKS cluster uses a client that is not compatible with HTTP GOAWAY, we recommend that you update your client to handle GOAWAY by reconnecting on connection termination.

For the complete Kubernetes 1.26 changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md#changelog-since-v1250.

Kubernetes 1.25

Kubernetes 1.25 is now available in Amazon EKS. For more information about Kubernetes 1.25, see the official release announcement.

Important

  • Starting with Kubernetes version 1.25, you will no longer be able to use Amazon EC2 P2 instances with the Amazon EKS optimized accelerated Amazon Linux AMIs out of the box. These AMIs for Kubernetes versions 1.25 or later will support NVIDIA 525 series or later drivers, which are incompatible with the P2 instances. However, NVIDIA 525 series or later drivers are compatible with the P3, P4, and P5 instances, so you can use those instances with the AMIs for Kubernetes version 1.25 or later. Before your Amazon EKS clusters are upgraded to version 1.25, migrate any P2 instances to P3, P4, and P5 instances. You should also proactively upgrade your applications to work with the NVIDIA 525 series or later. We plan to back port the newer NVIDIA 525 series or later drivers to Kubernetes versions 1.23 and 1.24 in late January 2024.

  • PodSecurityPolicy (PSP) is removed in Kubernetes 1.25. PSPs are replaced with Pod Security Admission (PSA) and Pod Security Standards (PSS). PSA is a built-in admission controller that implements the security controls outlined in the PSS . PSA and PSS are graduated to stable in Kubernetes 1.25 and are enabled in Amazon EKS by default. If you have PSPs in your cluster, make sure to migrate from PSP to the built-in Kubernetes PSS or to a policy-as-code solution before upgrading your cluster to version 1.25. If you don't migrate from PSP, you might encounter interruptions to your workloads. For more information, see the Pod security policy (PSP) removal FAQ.

  • Kubernetes version 1.25 contains changes that alter the behavior of an existing feature known as API Priority and Fairness (APF). APF serves to shield the API server from potential overload during periods of heightened request volumes. It does this by placing restrictions on the number of concurrent requests that can be processed at any given time. This is achieved through the application of distinct priority levels and limits to requests originating from various workloads or users. This approach ensures that critical applications or high-priority requests receive preferential treatment, while simultaneously preventing lower priority requests from overwhelming the API server. For more information, see API Priority and Fairness in the Kubernetes documentation or API Priority and Fairness in the EKS Best Practices Guide.

    These updates were introduced in PR #10352 and PR #118601. Previously, APF treated all types of requests uniformly, with each request consuming a single unit of the concurrent request limit. The APF behavior change assigns higher units of concurrency to LIST requests due to the exceptionally heavy burden put on the API server by these requests. The API server estimates the number of objects that will be returned by a LIST request. It assigns a unit of concurrency that is proportional to the number of objects returned.

    Upon upgrading to Amazon EKS version 1.25 or higher, this updated behavior might cause workloads with heavy LIST requests (that previously functioned without issue) to encounter rate limiting. This would be indicated by an HTTP 429 response code. To avoid potential workload disruption due to LIST requests being rate limited, we strongly encourage you to restructure your workloads to reduce the rate of these requests. Alternatively, you can address this issue by adjusting the APF settings to allocate more capacity for essential requests while reducing the capacity allocated to non-essential ones. For more information about these mitigation techniques, see Preventing Dropped Requests in the EKS Best Practices Guide.

  • Amazon EKS 1.25 includes enhancements to cluster authentication that contain updated YAML libraries. If a YAML value in the aws-auth ConfigMap found in the kube-system namespace starts with a macro, where the first character is a curly brace, you should add quotation marks (“ ”) before and after the curly braces ({ }). This is required to ensure that aws-iam-authenticator version v0.6.3 accurately parses the aws-auth ConfigMap in Amazon EKS 1.25.

  • The beta API version (discovery.k8s.io/v1beta1) of EndpointSlice was deprecated in Kubernetes 1.21 and is no longer served as of Kubernetes 1.25. This API has been updated to discovery.k8s.io/v1. For more information, see EndpointSlice in the Kubernetes documentation. The Amazon Load Balancer Controller v2.4.6 and earlier used the v1beta1 endpoint to communicate with EndpointSlices. If you're using the EndpointSlices configuration for the Amazon Load Balancer Controller, you must upgrade to Amazon Load Balancer Controller v2.4.7 before upgrading your Amazon EKS cluster to 1.25. If you upgrade to 1.25 while using the EndpointSlices configuration for the Amazon Load Balancer Controller, the controller will crash and result in interruptions to your workloads. To upgrade the controller, see What is the Amazon Load Balancer Controller?.

  • SeccompDefault is promoted to beta in Kubernetes 1.25. By setting the --seccomp-default flag when you configure kubelet, the container runtime uses its RuntimeDefaultseccomp profile, rather than the unconfined (seccomp disabled) mode. The default profiles provide a strong set of security defaults, while preserving the functionality of the workload. Although this flag is available, Amazon EKS doesn't enable this flag by default, so Amazon EKS behavior is effectively unchanged. If you want to, you can start enabling this on your nodes. For more details, see the tutorial Restrict a Container's Syscalls with seccomp in the Kubernetes documentation.

  • Support for the Container Runtime Interface (CRI) for Docker (also known as Dockershim) was removed from Kubernetes 1.24 and later. The only container runtime in Amazon EKS official AMIs for Kubernetes 1.24 and later clusters is containerd. Before upgrading to Amazon EKS 1.24 or later, remove any reference to bootstrap script flags that aren't supported anymore. For more information, see Amazon EKS ended support for Dockershim.

  • The support for wildcard queries was deprecated in CoreDNS 1.8.7 and removed in CoreDNS 1.9. This was done as a security measure. Wildcard queries no longer work and return NXDOMAIN instead of an IP address.

  • The goaway-chance option in the Kubernetes API server helps prevent HTTP/2 client connections from being stuck on a single API server instance, by randomly closing a connection. When the connection is closed, the client will try to reconnect, and will likely land on a different API server as a result of load balancing. Amazon EKS version 1.25 has enabled goaway-chance flag. If your workload running on Amazon EKS cluster uses a client that is not compatible with HTTP GOAWAY, we recommend that you update your client to handle GOAWAY by reconnecting on connection termination.

For the complete Kubernetes 1.25 changelog, see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#changelog-since-v1240.