Launching self-managed Amazon Linux nodes
This topic describes how you can launch Auto Scaling groups of Linux nodes that
register with your Amazon EKS cluster. After the nodes join the cluster, you can deploy Kubernetes
applications to them. You can also launch self-managed Amazon Linux 2 nodes with eksctl
or the
Amazon Web Services Management Console. If you need to launch nodes on Amazon Outposts, see Launching self-managed Amazon Linux nodes on an
Outpost.
Prerequisites
-
An existing Amazon EKS cluster. To deploy one, see Creating an Amazon EKS cluster. If you have subnets in the Amazon Web Services Region where you have Amazon Outposts, Amazon Wavelength, or Amazon Local Zones enabled, those subnets must not have been passed in when you created your cluster.
-
(Optional, but recommended) The Amazon VPC CNI plugin for Kubernetes add-on configured with its own IAM role that has the necessary IAM policy attached to it. For more information, see Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts.
-
Familiarity with the considerations listed in Choosing an Amazon EC2 instance type. Depending on the instance type you choose, there may be additional prerequisites for your cluster and VPC.
- eksctl
-
Prerequisite
Version
0.159.0
or later of theeksctl
command line tool installed on your device or Amazon CloudShell. To install or updateeksctl
, see Installing or updating eksctl.To launch self-managed Linux nodes using
eksctl
-
(Optional) If the AmazonEKS_CNI_Policy managed IAM policy is attached to your Amazon EKS node IAM role, we recommend assigning it to an IAM role that you associate to the Kubernetes
aws-node
service account instead. For more information, see Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts. -
The following command creates a node group in an existing cluster. Replace
with a name for your node group. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters. Replaceal-nodes
with the name of your cluster. The name can contain only alphanumeric characters (case-sensitive) and hyphens. It must start with an alphabetic character and can't be longer than 100 characters. Replace the remainingmy-cluster
with your own values. The nodes are created with the same Kubernetes version as the control plane, by default.example value
Before choosing a value for
--node-type
, review Choosing an Amazon EC2 instance type.Replace
with the name of your Amazon EC2 key pair or public key. This key is used to SSH into your nodes after they launch. If you don't already have an Amazon EC2 key pair, you can create one in the Amazon Web Services Management Console. For more information, see Amazon EC2 key pairs in the Amazon EC2 User Guide for Linux Instances.my-key
Create your node group with the following command.
Important
If you want to deploy a node group to Amazon Outposts, Wavelength, or Local Zone subnets, there are additional considerations:
-
The subnets must not have been passed in when you created the cluster.
-
You must create the node group with a config file that specifies the subnets and
volumeType
. For more information, see Create a nodegroup from a config file: gp2 and Config file schema in the eksctl
documentation.
eksctl create nodegroup \ --cluster
my-cluster
\ --nameal-nodes
\ --node-typet3.medium
\ --nodes3
\ --nodes-min1
\ --nodes-max4
\ --ssh-access \ --managed=false \ --ssh-public-keymy-key
To deploy a node group that:
-
can assign a significantly higher number of IP addresses to Pods than the default configuration, see Increase the amount of available IP addresses for your Amazon EC2 nodes.
-
can assign
IPv4
addresses to Pods from a different CIDR block than that of the instance, see Custom networking for pods. -
can assign
IPv6
addresses to Pods and services, see Tutorial: Assigning IPv6 addresses to Pods and services. -
use the
containerd
runtime, you must deploy the node group using aconfig
file. For more information, see Enable the containerd runtime bootstrap flag. -
don't have outbound internet access, see Private cluster requirements.
For a complete list of all available options and defaults, enter the following command.
eksctl create nodegroup --help
If nodes fail to join the cluster, then see Nodes fail to join cluster in the Troubleshooting guide.
An example output is as follows. Several lines are output while the nodes are created. One of the last lines of output is the following example line.
[✔] created 1 nodegroup(s) in cluster "
my-cluster
" -
-
(Optional) Deploy a sample application to test your cluster and Linux nodes.
We recommend blocking Pod access to IMDS if the following conditions are true:
You plan to assign IAM roles to all of your Kubernetes service accounts so that Pods only have the minimum permissions that they need.
No Pods in the cluster require access to the Amazon EC2 instance metadata service (IMDS) for other reasons, such as retrieving the current Amazon Web Services Region.
For more information, see Restrict access to the instance profile assigned to the worker node
.
-
- Amazon Web Services Management Console
-
Step 1: To launch self-managed Linux nodes using the Amazon Web Services Management Console
-
Download the latest version of the Amazon CloudFormation template.
curl -O https://s3.cn-north-1.amazonaws.com.cn/amazon-eks/cloudformation/2022-12-23/amazon-eks-nodegroup.yaml
-
Wait for your cluster status to show as
ACTIVE
. If you launch your nodes before the cluster is active, the nodes fail to register with the cluster and you will have to relaunch them. -
Open the Amazon CloudFormation console at https://console.amazonaws.cn/cloudformation
. -
Choose Create stack and then select With new resources (standard).
-
For Specify template, select Upload a template file and then select Choose file.
-
Select the
file that you downloaded.amazon-eks-nodegroup.yaml
-
Select Next.
-
On the Specify stack details page, enter the following parameters accordingly, and then choose Next:
-
Stack name: Choose a stack name for your Amazon CloudFormation stack. For example, you can call it
. The name can contain only alphanumeric characters (case-sensitive) and hyphens. It must start with an alphabetic character and can't be longer than 100 characters.my-cluster-nodes
-
ClusterName: Enter the name that you used when you created your Amazon EKS cluster. This name must equal the cluster name or your nodes can't join the cluster.
-
ClusterControlPlaneSecurityGroup: Choose the SecurityGroups value from the Amazon CloudFormation output that you generated when you created your VPC.
The following steps show one operation to retrieve the applicable group.
Open the Amazon EKS console at https://console.amazonaws.cn/eks/home#/clusters
. -
Choose the name of the cluster.
-
Choose the Networking tab.
-
Use the Additional security groups value as a reference when selecting from the ClusterControlPlaneSecurityGroup dropdown list.
-
NodeGroupName: Enter a name for your node group. This name can be used later to identify the Auto Scaling node group that's created for your nodes. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters.
-
NodeAutoScalingGroupMinSize: Enter the minimum number of nodes that your node Auto Scaling group can scale in to.
-
NodeAutoScalingGroupDesiredCapacity: Enter the desired number of nodes to scale to when your stack is created.
-
NodeAutoScalingGroupMaxSize: Enter the maximum number of nodes that your node Auto Scaling group can scale out to.
-
NodeInstanceType: Choose an instance type for your nodes. For more information, see Choosing an Amazon EC2 instance type.
-
NodeImageIdSSMParam: Pre-populated with the Amazon EC2 Systems Manager parameter of a recent Amazon EKS optimized AMI for a variable Kubernetes version. To use a different Kubernetes minor version supported with Amazon EKS, replace
with a different supported version. We recommend specifying the same Kubernetes version as your cluster.1.XX
To use the Amazon EKS optimized accelerated AMI, replace
withamazon-linux-2
amazon-linux-2-gpu
. To use the Amazon EKS optimized Arm AMI, replace
withamazon-linux-2
amazon-linux-2-arm64
.Note
The Amazon EKS node AMI is based on Amazon Linux 2. You can track security or privacy events for Amazon Linux 2 at the Amazon Linux Security Center
or subscribe to the associated RSS feed . Security and privacy events include an overview of the issue, what packages are affected, and how to update your instances to correct the issue. -
NodeImageId: (Optional) If you're using your own custom AMI (instead of the Amazon EKS optimized AMI), enter a node AMI ID for your Amazon Web Services Region. If you specify a value here, it overrides any values in the NodeImageIdSSMParam field.
-
NodeVolumeSize: Specify a root volume size for your nodes, in GiB.
-
NodeVolumeType: Specify a root volume type for your nodes.
-
KeyName: Enter the name of an Amazon EC2 SSH key pair that you can use to connect using SSH into your nodes with after they launch. If you don't already have an Amazon EC2 key pair, you can create one in the Amazon Web Services Management Console. For more information, see Amazon EC2 key pairs in the Amazon EC2 User Guide for Linux Instances.
Note
If you don't provide a key pair here, the Amazon CloudFormation stack creation fails.
-
BootstrapArguments: Specify any optional arguments to pass to the node bootstrap script, such as extra
kubelet
arguments. For more information, view the bootstrap script usage informationon GitHub. To deploy a node group that:
-
can assign a significantly higher number of IP addresses to Pods than the default configuration, see Increase the amount of available IP addresses for your Amazon EC2 nodes.
-
can assign
IPv4
addresses to Pods from a different CIDR block than that of the instance, see Custom networking for pods. -
can assign
IPv6
addresses to Pods and services, see Tutorial: Assigning IPv6 addresses to Pods and services. -
use the
containerd
runtime, you must deploy the node group using aconfig
file. For more information, see Enable the containerd runtime bootstrap flag. -
don't have outbound internet access, see Private cluster requirements.
-
-
DisableIMDSv1: By default, each node supports the Instance Metadata Service Version 1 (IMDSv1) and IMDSv2. You can disable IMDSv1. To prevent future nodes and Pods in the node group from using MDSv1, set DisableIMDSv1 to true. For more information about IMDS, see Configuring the instance metadata service. For more information about restricting access to it on your nodes, see Restrict access to the instance profile assigned to the worker node
. -
VpcId: Enter the ID for the VPC that you created.
-
Subnets: Choose the subnets that you created for your VPC. If you created your VPC using the steps that are described in Creating a VPC for your Amazon EKS cluster, specify only the private subnets within the VPC for your nodes to launch into. You can see which subnets are private by opening each subnet link from the Networking tab of your cluster.
Important
-
If any of the subnets are public subnets, then they must have the automatic public IP address assignment setting enabled. If the setting isn't enabled for the public subnet, then any nodes that you deploy to that public subnet won't be assigned a public IP address and won't be able to communicate with the cluster or other Amazon services. If the subnet was deployed before March 26, 2020 using either of the Amazon EKS Amazon CloudFormation VPC templates, or by using
eksctl
, then automatic public IP address assignment is disabled for public subnets. For information about how to enable public IP address assignment for a subnet, see Modifying the publicIPv4
addressing attribute for your subnet. If the node is deployed to a private subnet, then it's able to communicate with the cluster and other Amazon services through a NAT gateway. -
If the subnets don't have internet access, make sure that you're aware of the considerations and extra steps in Private cluster requirements.
-
If you select Amazon Outposts, Wavelength, or Local Zone subnets, the subnets must not have been passed in when you created the cluster.
-
-
-
Select your desired choices on the Configure stack options page, and then choose Next.
-
Select the check box to the left of I acknowledge that Amazon CloudFormation might create IAM resources., and then choose Create stack.
-
When your stack has finished creating, select it in the console and choose Outputs.
-
Record the NodeInstanceRole for the node group that was created. You need this when you configure your Amazon EKS nodes.
Step 2: To enable nodes to join your cluster
Note
If you launched nodes inside a private VPC without outbound internet access, make sure to enable nodes to join your cluster from within the VPC.
-
Check to see if you already have an
aws-auth
ConfigMap
.kubectl describe configmap -n kube-system aws-auth
-
If you are shown an
aws-auth
ConfigMap
, then update it as needed.-
Open the
ConfigMap
for editing.kubectl edit -n kube-system configmap/aws-auth
-
Add a new
mapRoles
entry as needed. Set therolearn
value to the NodeInstanceRole value that you recorded in the previous procedure.[...] data: mapRoles: |
- rolearn:
[...]<ARN of instance role (not instance profile)>
username: system:node:{{EC2PrivateDNSName}} groups: - system:bootstrappers - system:nodes -
Save the file and exit your text editor.
-
-
If you received an error stating "
Error from server (NotFound): configmaps "aws-auth" not found
, then apply the stockConfigMap
.-
Download the configuration map.
curl -O https://s3.cn-north-1.amazonaws.com.cn/amazon-eks/cloudformation/2020-10-29/aws-auth-cm.yaml
-
In the
file, set theaws-auth-cm.yaml
rolearn
value to the NodeInstanceRole value that you recorded in the previous procedure. You can do this with a text editor, or by replacing
and running the following command:my-node-instance-role
sed -i.bak -e 's|<ARN of instance role (not instance profile)>|
my-node-instance-role
|' aws-auth-cm.yaml -
Apply the configuration. This command may take a few minutes to finish.
kubectl apply -f aws-auth-cm.yaml
-
-
Watch the status of your nodes and wait for them to reach the
Ready
status.kubectl get nodes --watch
Enter
Ctrl
+C
to return to a shell prompt.Note
If you receive any authorization or resource type errors, see Unauthorized or access denied (kubectl) in the troubleshooting topic.
If nodes fail to join the cluster, then see Nodes fail to join cluster in the Troubleshooting guide.
-
(GPU nodes only) If you chose a GPU instance type and the Amazon EKS optimized accelerated AMI, you must apply the NVIDIA device plugin for Kubernetes
as a DaemonSet on your cluster. Replace
with your desired NVIDIA/k8s-device-pluginvX.X.X
version before running the following command. kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/
vX.X.X
/nvidia-device-plugin.yml
Step 3: Additional actions
-
(Optional) Deploy a sample application to test your cluster and Linux nodes.
-
(Optional) If the AmazonEKS_CNI_Policy managed IAM policy (if you have an
IPv4
cluster) or the
(that you created yourself if you have anAmazonEKS_CNI_IPv6_Policy
IPv6
cluster) is attached to your Amazon EKS node IAM role, we recommend assigning it to an IAM role that you associate to the Kubernetesaws-node
service account instead. For more information, see Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts. We recommend blocking Pod access to IMDS if the following conditions are true:
You plan to assign IAM roles to all of your Kubernetes service accounts so that Pods only have the minimum permissions that they need.
No Pods in the cluster require access to the Amazon EC2 instance metadata service (IMDS) for other reasons, such as retrieving the current Amazon Web Services Region.
For more information, see Restrict access to the instance profile assigned to the worker node
.
-