Creating a managed node group

To create a managed node group with eksctl

This procedure requires eksctl version 0.175.0 or later. You can check your version with the following command:

eksctl version

For instructions on how to install or upgrade eksctl, see Installation in the eksctl documentation.

1. (Optional) If the AmazonEKS_CNI_Policy managed IAM policy is attached to your Amazon EKS node IAM role, we recommend assigning it to an IAM role that you associate to the Kubernetes aws-node service account instead. For more information, see Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts (IRSA).

2. Create a managed node group with or without using a custom launch template. Manually specifying a launch template allows for greater customization of a node group. For example, it can allow deploying a custom AMI or providing arguments to the boostrap.sh script in an Amazon EKS optimized AMI. For a complete list of every available option and default, enter the following command.

   eksctl create nodegroup --help
   

   In the following command, replace my-cluster with the name of your cluster and replace my-mng with the name of your node group. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters.

   Important

   If you don't use a custom launch template when first creating a managed node group, don't use one at a later time for the node group. If you didn't specify a custom launch template, the system auto-generates a launch template that we don't recommend that you modify manually. Manually modifying this auto-generated launch template might cause errors.

   Without a launch template

   eksctl creates a default Amazon EC2 launch template in your account and deploys the node group using a launch template that it creates based on options that you specify. Before specifying a value for --node-type, see Choosing an Amazon EC2 instance type.

   Replace ami-family with an allowed keyword. For more information, see Setting the node AMI Family in the eksctl documentation. Replace my-key with the name of your Amazon EC2 key pair or public key. This key is used to SSH into your nodes after they launch.

   Note

   For Windows, this command doesn't enable SSH. Instead, it associates your Amazon EC2 key pair with the instance and allows you to RDP into the instance.

   If you don't already have an Amazon EC2 key pair, you can create one in the Amazon Web Services Management Console. For Linux information, see Amazon EC2 key pairs and Linux instances in the Amazon EC2 User Guide for Linux Instances. For Windows information, see Amazon EC2 key pairs and Windows instances in the Amazon EC2 User Guide for Windows Instances.

   We recommend blocking Pod access to IMDS if the following conditions are true:

   • You plan to assign IAM roles to all of your Kubernetes service accounts so that Pods only have the minimum permissions that they need.

   • No Pods in the cluster require access to the Amazon EC2 instance metadata service (IMDS) for other reasons, such as retrieving the current Amazon Web Services Region.

   For more information, see Restrict access to the instance profile assigned to the worker node.

   If you want to block Pod access to IMDS, then add the --disable-pod-imds option to the following command.

   eksctl create nodegroup \
     --cluster my-cluster \
     --region region-code \
     --name my-mng \
     --node-ami-family ami-family \
     --node-type m5.large \
     --nodes 3 \
     --nodes-min 2 \
     --nodes-max 4 \
     --ssh-access \
     --ssh-public-key my-key
   

   Your instances can optionally assign a significantly higher number of IP addresses to Pods, assign IP addresses to Pods from a different CIDR block than the instance's, and be deployed to a cluster without internet access. For more information, see Increase the amount of available IP addresses for your Amazon EC2 nodes, Custom networking for pods, and Private cluster requirements for additional options to add to the previous command.

   Managed node groups calculates and applies a single value for the maximum number of Pods that can run on each node of your node group, based on instance type. If you create a node group with different instance types, the smallest value calculated across all instance types is applied as the maximum number of Pods that can run on every instance type in the node group. Managed node groups calculates the value using the script referenced in Amazon EKS recommended maximum Pods for each Amazon EC2 instance type.

   With a launch template

   The launch template must already exist and must meet the requirements specified in Launch template configuration basics.

   We recommend blocking Pod access to IMDS if the following conditions are true:

   • You plan to assign IAM roles to all of your Kubernetes service accounts so that Pods only have the minimum permissions that they need.

   • No Pods in the cluster require access to the Amazon EC2 instance metadata service (IMDS) for other reasons, such as retrieving the current Amazon Web Services Region.

   For more information, see Restrict access to the instance profile assigned to the worker node.

   If you want to block Pod access to IMDS, then specify the necessary settings in the launch template.

   1. Copy the following contents to your device. Replace the example values and then run the modified command to create the eks-nodegroup.yaml file. Several settings that you specify when deploying without a launch template are moved into the launch template. If you don't specify a version, the template's default version is used.

      cat >eks-nodegroup.yaml <<EOF
      apiVersion: eksctl.io/v1alpha5
      kind: ClusterConfig
      metadata:
        name: my-cluster
        region: region-code
      managedNodeGroups:
      - name: my-mng
        launchTemplate:
          id: lt-id
          version: "1"
      EOF
      

      For a complete list of eksctl config file settings, see Config file schema in the eksctl documentation. Your instances can optionally assign a significantly higher number of IP addresses to Pods, assign IP addresses to Pods from a different CIDR block than the instance's, use the containerd runtime, and be deployed to a cluster without outbound internet access. For more information, see Increase the amount of available IP addresses for your Amazon EC2 nodes, Custom networking for pods, Test migration from Docker to containerd, and Private cluster requirements for additional options to add to the config file.

      If you didn't specify an AMI ID in your launch template, managed node groups calculates and applies a single value for the maximum number of Pods that can run on each node of your node group, based on instance type. If you create a node group with different instance types, the smallest value calculated across all instance types is applied as the maximum number of Pods that can run on every instance type in the node group. Managed node groups calculates the value using the script referenced in Amazon EKS recommended maximum Pods for each Amazon EC2 instance type.

      If you specified an AMI ID in your launch template, specify the maximum number of Pods that can run on each node of your node group if you're using custom networking or want to increase the number of IP addresses assigned to your instance. For more information, see Amazon EKS recommended maximum Pods for each Amazon EC2 instance type.

   2. Deploy the nodegroup with the following command.

      eksctl create nodegroup --config-file eks-nodegroup.yaml
      

To create a managed node group using the Amazon Web Services Management Console

1. Wait for your cluster status to show as ACTIVE. You can't create a managed node group for a cluster that isn't already ACTIVE.

2. Open the Amazon EKS console at https://console.amazonaws.cn/eks/home#/clusters.

3. Choose the name of the cluster that you want to create a managed node group in.

4. Select the Compute tab.

5. Choose Add node group.

6. On the Configure node group page, fill out the parameters accordingly, and then choose Next.

   • Name – Enter a unique name for your managed node group. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters.

   • Node IAM role – Choose the node instance role to use with your node group. For more information, see Amazon EKS node IAM role.

     Important

     • You can't use the same role that is used to create any clusters.

     • We recommend using a role that's not currently in use by any self-managed node group. Otherwise, you plan to use with a new self-managed node group. For more information, see Deleting a managed node group.

   • Use launch template – (Optional) Choose if you want to use an existing launch template. Select a Launch Template Name. Then, select a Launch template version. If you don't select a version, then Amazon EKS uses the template's default version. Launch templates allow for more customization of your node group, such as allowing you to deploy a custom AMI, assign a significantly higher number of IP addresses to Pods, assign IP addresses to Pods from a different CIDR block than the instance's, enable the containerd runtime for your instances, and deploying nodes to a cluster without outbound internet access. For more information, see Increase the amount of available IP addresses for your Amazon EC2 nodes, Custom networking for pods, Test migration from Docker to containerd, and Private cluster requirements.

     The launch template must meet the requirements in Customizing managed nodes with launch templates. If you don't use your own launch template, the Amazon EKS API creates a default Amazon EC2 launch template in your account and deploys the node group using the default launch template.

     If you implement IAM roles for service accounts, assign necessary permissions directly to every Pod that requires access to Amazon services, and no Pods in your cluster require access to IMDS for other reasons, such as retrieving the current Amazon Web Services Region, then you can also disable access to IMDS for Pods that don't use host networking in a launch template. For more information, see Restrict access to the instance profile assigned to the worker node.

   • Kubernetes labels – (Optional) You can choose to apply Kubernetes labels to the nodes in your managed node group.

   • Kubernetes taints – (Optional) You can choose to apply Kubernetes taints to the nodes in your managed node group. The available options in the Effect menu are NoSchedule, NoExecute, and PreferNoSchedule. For more information, see Node taints on managed node groups.

   • Tags – (Optional) You can choose to tag your Amazon EKS managed node group. These tags don't propagate to other resources in the node group, such as Auto Scaling groups or instances. For more information, see Tagging your Amazon EKS resources.

7. On the Set compute and scaling configuration page, fill out the parameters accordingly, and then choose Next.

   • AMI type – Select an AMI type. If you are deploying Arm instances, be sure to review the considerations in Amazon EKS optimized Arm Amazon Linux AMIs before deploying.

     If you specified a launch template on the previous page, and specified an AMI in the launch template, then you can't select a value. The value from the template is displayed. The AMI specified in the template must meet the requirements in Specifying an AMI.

   • Capacity type – Select a capacity type. For more information about choosing a capacity type, see Managed node group capacity types. You can't mix different capacity types within the same node group. If you want to use both capacity types, create separate node groups, each with their own capacity and instance types.

   • Instance types – By default, one or more instance type is specified. To remove a default instance type, select the X on the right side of the instance type. Choose the instance types to use in your managed node group. For more information, see Choosing an Amazon EC2 instance type.

     The console displays a set of commonly used instance types. If you need to create a managed node group with an instance type that's not displayed, then use eksctl, the Amazon CLI, Amazon CloudFormation, or an SDK to create the node group. If you specified a launch template on the previous page, then you can't select a value because the instance type must be specified in the launch template. The value from the launch template is displayed. If you selected Spot for Capacity type, then we recommend specifying multiple instance types to enhance availability.

   • Disk size – Enter the disk size (in GiB) to use for your node's root volume.

     If you specified a launch template on the previous page, then you can't select a value because it must be specified in the launch template.

   • Desired size – Specify the current number of nodes that the managed node group should maintain at launch.

     Note

     Amazon EKS doesn't automatically scale your node group in or out. However, you can configure the Kubernetes Cluster Autoscaler to do this for you.

   • Minimum size – Specify the minimum number of nodes that the managed node group can scale in to.

   • Maximum size – Specify the maximum number of nodes that the managed node group can scale out to.

   • Node group update configuration – (Optional) You can select the number or percentage of nodes to be updated in parallel. These nodes will be unavailable during the update. For Maximum unavailable, select one of the following options and specify a Value:

     • Number – Select and specify the number of nodes in your node group that can be updated in parallel.

     • Percentage – Select and specify the percentage of nodes in your node group that can be updated in parallel. This is useful if you have a large number of nodes in your node group.

8. On the Specify networking page, fill out the parameters accordingly, and then choose Next.

   • Subnets – Choose the subnets to launch your managed nodes into.

     Important

     If you are running a stateful application across multiple Availability Zones that is backed by Amazon EBS volumes and using the Kubernetes Autoscaling, you should configure multiple node groups, each scoped to a single Availability Zone. In addition, you should enable the --balance-similar-node-groups feature.

     Important

     • If you choose a public subnet, and your cluster has only the public API server endpoint enabled, then the subnet must have MapPublicIPOnLaunch set to true for the instances to successfully join a cluster. If the subnet was created using eksctl or the Amazon EKS vended Amazon CloudFormation templates on or after March 26, 2020, then this setting is already set to true. If the subnets were created with eksctl or the Amazon CloudFormation templates before March 26, 2020, then you need to change the setting manually. For more information, see Modifying the public IPv4 addressing attribute for your subnet.

     • If you use a launch template and specify multiple network interfaces, Amazon EC2 won't auto-assign a public IPv4 address, even if MapPublicIpOnLaunch is set to true. For nodes to join the cluster in this scenario, you must either enable the cluster's private API server endpoint, or launch nodes in a private subnet with outbound internet access provided through an alternative method, such as a NAT Gateway. For more information, see Amazon EC2 instance IP addressing in the Amazon EC2 User Guide for Linux Instances.

   • Configure SSH access to nodes (Optional). Enabling SSH allows you to connect to your instances and gather diagnostic information if there are issues. We highly recommend enabling remote access when you create a node group. You can't enable remote access after the node group is created.

     If you chose to use a launch template, then this option isn't shown. To enable remote access to your nodes, specify a key pair in the launch template and ensure that the proper port is open to the nodes in the security groups that you specify in the launch template. For more information, see Using custom security groups.

     Note

     For Windows, this command doesn't enable SSH. Instead, it associates your Amazon EC2 key pair with the instance and allows you to RDP into the instance.

   • For SSH key pair (Optional), choose an Amazon EC2 SSH key to use. For Linux information, see Amazon EC2 key pairs and Linux instances in the Amazon EC2 User Guide for Linux Instances. For Windows information, see Amazon EC2 key pairs and Windows instances in the Amazon EC2 User Guide for Windows Instances. If you chose to use a launch template, then you can't select one. When an Amazon EC2 SSH key is provided for node groups using Bottlerocket AMIs, the administrative container is also enabled. For more information, see Admin container on GitHub.

   • For Allow SSH remote access from, if you want to limit access to specific instances, then select the security groups that are associated to those instances. If you don't select specific security groups, then SSH access is allowed from anywhere on the internet (0.0.0.0/0).

9. On the Review and create page, review your managed node group configuration and choose Create.

   If nodes fail to join the cluster, then see Nodes fail to join cluster in the Troubleshooting guide.

10. Watch the status of your nodes and wait for them to reach the Ready status.

    kubectl get nodes --watch
    

11. (GPU nodes only) If you chose a GPU instance type and the Amazon EKS optimized accelerated AMI, then you must apply the NVIDIA device plugin for Kubernetes as a DaemonSet on your cluster. Replace vX.X.X with your desired NVIDIA/k8s-device-plugin version before running the following command.

    kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/vX.X.X/nvidia-device-plugin.yml