Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Kubernetes service accounts

A Kubernetes service account provides an identity for processes that run in a pod. For more information see Managing Service Accounts in the Kubernetes documentation. If your pod needs access to Amazon services, you can map the service account to an Amazon Identity and Access Management identity to grant that access. For more information, see IAM roles for service accounts.

Service account tokens

The BoundServiceAccountTokenVolume feature is enabled by default in Kubernetes version 1.21 and later. This feature improves the security of service account tokens by allowing workloads running on Kubernetes to request JSON web tokens that are audience, time, and key bound. Service account tokens have an expiration of one hour. In earlier Kubernetes versions, the tokens didn't have an expiration. This means that clients that rely on these tokens must refresh the tokens within an hour. The following Kubernetes client SDKs refresh tokens automatically within the required time frame:

If your workload is using an earlier client version, then you must update it. To enable a smooth migration of clients to the newer time-bound service account tokens, Kubernetes version 1.21 and later adds an extended expiry period to the service account token over the default one hour. For Amazon EKS clusters, the extended expiry period is 90 days. Your Amazon EKS cluster's Kubernetes API server rejects requests with tokens that are greater than 90 days old. We recommend that you check your applications and their dependencies to make sure that the Kubernetes client SDKs are the same or later than the versions listed previously.

When the API server receives requests with tokens that are greater than one hour old, it annotates the API audit log event with annotations.authentication.k8s.io/stale-token. The value of the annotation looks like the following example:

subject: system:serviceaccount:common:fluent-bit, seconds after warning threshold: 4185802.

If your cluster has control plane logging enabled, then the annotations are in the audit logs. You can use the following CloudWatch Logs Insights query to identify all the pods in your Amazon EKS cluster that are using stale tokens:

fields @timestamp
| filter @logStream like /kube-apiserver-audit/
| filter @message like /seconds after warning threshold/
| parse @message "subject: *, seconds after warning threshold:*\"" as subject, elapsedtime

The subject refers to the service account that the pod used. The elapsedtime indicates the elapsed time (in seconds) after reading the latest token. The requests to the API server are denied when the elapsedtime exceeds 90 days (7,776,000 seconds). You should proactively update your applications' Kubernetes client SDK to use one of the version listed previously that automatically refresh the token. If the service account token used is close to 90 days and you don't have sufficient time to update your client SDK versions before token expiration, then you can terminate existing pods and create new ones. This results in refetching of the service account token, giving you an additional 90 days to update your client version SDKs.

If the pod is part of a deployment, the suggested way to terminate pods while keeping high availability is to perform a roll out with the following command. Replace my-deployment with the name of your deployment.

kubectl rollout restart deployment/my-deployment

Cluster add-ons

The following cluster add-ons have been updated to use the Kubernetes client SDKs that automatically refetch service account tokens. We recommend making sure that the listed versions, or later versions, are installed on your 1.21 or later cluster.