Tutorial: Assigning IPv6 addresses to Pods and services

To deploy an IPv6 cluster with eksctl

1. Create the ipv6-cluster.yaml file. Copy the command that follows to your device. Make the following modifications to the command as needed and then run the modified command:

   • Replace my-cluster with a name for your cluster. The name can contain only alphanumeric characters (case-sensitive) and hyphens. It must start with an alphabetic character and can't be longer than 100 characters.

   • Replace region-code with any Amazon Web Services Region that is supported by Amazon EKS. For a list of Amazon Web Services Regions, see Amazon EKS endpoints and quotas in the Amazon General Reference guide.

   • The value for version with the version of your cluster. For more information, see supported Amazon EKS Kubernetes version.

   • Replace my-nodegroup with a name for your node group. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters.

   • Replace t3.medium with any Amazon Nitro System instance type.

   cat >ipv6-cluster.yaml <<EOF
   ---
   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig
   
   metadata:
     name: my-cluster
     region: region-code
     version: "X.XX"
   
   kubernetesNetworkConfig:
     ipFamily: IPv6
   
   addons:
     - name: vpc-cni
       version: latest
     - name: coredns
       version: latest
     - name: kube-proxy
       version: latest
   
   iam:
     withOIDC: true
   
   managedNodeGroups:
     - name: my-nodegroup
       instanceType: t3.medium
   EOF

2. Create your cluster.

   eksctl create cluster -f ipv6-cluster.yaml

   Cluster creation takes several minutes. Don't proceed until you see the last line of output, which looks similar to the following output.

   [...]
   [✓]  EKS cluster "my-cluster" in "region-code" region is ready

3. Confirm that default Pods are assigned IPv6 addresses.

   kubectl get pods -n kube-system -o wide

   An example output is as follows.

   NAME                       READY   STATUS    RESTARTS   AGE     IP                                       NODE                                            NOMINATED NODE   READINESS GATES
   aws-node-rslts             1/1     Running   1          5m36s   2600:1f13:b66:8200:11a5:ade0:c590:6ac8   ip-192-168-34-75.region-code.compute.internal   <none>           <none>
   aws-node-t74jh             1/1     Running   0          5m32s   2600:1f13:b66:8203:4516:2080:8ced:1ca9   ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   coredns-85d5b4454c-cw7w2   1/1     Running   0          56m     2600:1f13:b66:8203:34e5::                ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   coredns-85d5b4454c-tx6n8   1/1     Running   0          56m     2600:1f13:b66:8203:34e5::1               ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   kube-proxy-btpbk           1/1     Running   0          5m36s   2600:1f13:b66:8200:11a5:ade0:c590:6ac8   ip-192-168-34-75.region-code.compute.internal   <none>           <none>
   kube-proxy-jjk2g           1/1     Running   0          5m33s   2600:1f13:b66:8203:4516:2080:8ced:1ca9   ip-192-168-253-70.region-code.compute.internal  <none>           <none>

4. Confirm that default services are assigned IPv6 addresses.

   kubectl get services -n kube-system -o wide
   

   An example output is as follows.

   NAME       TYPE        CLUSTER-IP          EXTERNAL-IP   PORT(S)         AGE   SELECTOR
   kube-dns   ClusterIP   fd30:3087:b6c2::a   <none>        53/UDP,53/TCP   57m   k8s-app=kube-dns

5. (Optional) Deploy a sample application or deploy the Amazon Load Balancer Controller and a sample application to load balance application or network traffic to IPv6 Pods.

6. After you've finished with the cluster and nodes that you created for this tutorial, you should clean up the resources that you created with the following command.

   eksctl delete cluster my-cluster

To create your cluster with the Amazon CLI

Replace all example values in the steps of this procedure with your own values.

1. Run the following commands to set some variables used in later steps. Replace region-code with the Amazon Web Services Region that you want to deploy your resources in. The value can be any Amazon Web Services Region that is supported by Amazon EKS. For a list of Amazon Web Services Regions, see Amazon EKS endpoints and quotas in the Amazon General Reference guide. Replace my-cluster with a name for your cluster. The name can contain only alphanumeric characters (case-sensitive) and hyphens. It must start with an alphabetic character and can't be longer than 100 characters. Replace my-nodegroup with a name for your node group. The node group name can't be longer than 63 characters. It must start with letter or digit, but can also include hyphens and underscores for the remaining characters. Replace 111122223333 with your account ID.

   export region_code=region-code
   export cluster_name=my-cluster
   export nodegroup_name=my-nodegroup
   export account_id=111122223333

2. Create an Amazon VPC with public and private subnets that meets Amazon EKS and IPv6 requirements.

   1. Run the following command to set a variable for your Amazon CloudFormation stack name. You can replace my-eks-ipv6-vpc with any name you choose.

      export vpc_stack_name=my-eks-ipv6-vpc

   2. Create an IPv6 VPC using an Amazon CloudFormation template.

      aws cloudformation create-stack --region $region_code --stack-name $vpc_stack_name \
        --template-url https://s3.cn-north-1.amazonaws.com.cn/amazon-eks/cloudformation/2020-10-29/amazon-eks-ipv6-vpc-public-private-subnets.yaml
      

      The stack takes a few minutes to create. Run the following command. Don't continue to the next step until the output of the command is CREATE_COMPLETE.

      aws cloudformation describe-stacks --region $region_code --stack-name $vpc_stack_name --query Stacks[].StackStatus --output text
      

   3. Retrieve the IDs of the public subnets that were created.

      aws cloudformation describe-stacks --region $region_code --stack-name $vpc_stack_name \
          --query='Stacks[].Outputs[?OutputKey==`SubnetsPublic`].OutputValue' --output text
      

      An example output is as follows.

      subnet-0a1a56c486EXAMPLE,subnet-099e6ca77aEXAMPLE

   4. Enable the auto-assign IPv6 address option for the public subnets that were created.

      aws ec2 modify-subnet-attribute --region $region_code --subnet-id subnet-0a1a56c486EXAMPLE --assign-ipv6-address-on-creation
      aws ec2 modify-subnet-attribute --region $region_code --subnet-id subnet-099e6ca77aEXAMPLE --assign-ipv6-address-on-creation

   5. Retrieve the names of the subnets and security groups created by the template from the deployed Amazon CloudFormation stack and store them in variables for use in a later step.

      security_groups=$(aws cloudformation describe-stacks --region $region_code --stack-name $vpc_stack_name \
          --query='Stacks[].Outputs[?OutputKey==`SecurityGroups`].OutputValue' --output text)
      
      public_subnets=$(aws cloudformation describe-stacks --region $region_code --stack-name $vpc_stack_name \
          --query='Stacks[].Outputs[?OutputKey==`SubnetsPublic`].OutputValue' --output text)
      
      private_subnets=$(aws cloudformation describe-stacks --region $region_code --stack-name $vpc_stack_name \
          --query='Stacks[].Outputs[?OutputKey==`SubnetsPrivate`].OutputValue' --output text)
      
      subnets=${public_subnets},${private_subnets}
      

3. Create a cluster IAM role and attach the required Amazon EKS IAM managed policy to it. Kubernetes clusters managed by Amazon EKS make calls to other Amazon services on your behalf to manage the resources that you use with the service.

   1. Run the following command to create the eks-cluster-role-trust-policy.json file.

      cat >eks-cluster-role-trust-policy.json <<EOF
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": "eks.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }
      EOF
      

   2. Run the following command to set a variable for your role name. You can replace myAmazonEKSClusterRole with any name you choose.

      export cluster_role_name=myAmazonEKSClusterRole

   3. Create the role.

      aws iam create-role --role-name $cluster_role_name --assume-role-policy-document file://"eks-cluster-role-trust-policy.json"

   4. Retrieve the ARN of the IAM role and store it in a variable for a later step.

      cluster_iam_role=$(aws iam get-role --role-name $cluster_role_name --query="Role.Arn" --output text)
      

   5. Attach the required Amazon EKS managed IAM policy to the role.

      aws iam attach-role-policy --policy-arn arn:aws-cn:iam::aws:policy/AmazonEKSClusterPolicy --role-name $cluster_role_name
      

4. Create your cluster.

   aws eks create-cluster --region $region_code --name $cluster_name --kubernetes-version 1.XX \
      --role-arn $cluster_iam_role --resources-vpc-config subnetIds=$subnets,securityGroupIds=$security_groups \
      --kubernetes-network-config ipFamily=ipv6
   

   1. Note

      You might receive an error that one of the Availability Zones in your request doesn't have sufficient capacity to create an Amazon EKS cluster. If this happens, the error output contains the Availability Zones that can support a new cluster. Retry creating your cluster with at least two subnets that are located in the supported Availability Zones for your account. For more information, see Insufficient capacity.

      The cluster takes several minutes to create. Run the following command. Don't continue to the next step until the output from the command is ACTIVE.

      aws eks describe-cluster --region $region_code --name $cluster_name --query cluster.status
      

5. Create or update a kubeconfig file for your cluster so that you can communicate with your cluster.

   aws eks update-kubeconfig --region $region_code --name $cluster_name

   By default, the config file is created in ~/.kube or the new cluster's configuration is added to an existing config file in ~/.kube.

6. Create a node IAM role.

   1. Run the following command to create the vpc-cni-ipv6-policy.json file.

      cat >vpc-cni-ipv6-policy <<EOF
      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "ec2:AssignIpv6Addresses",
                      "ec2:DescribeInstances",
                      "ec2:DescribeTags",
                      "ec2:DescribeNetworkInterfaces",
                      "ec2:DescribeInstanceTypes"
                  ],
                  "Resource": "*"
              },
              {
                  "Effect": "Allow",
                  "Action": [
                      "ec2:CreateTags"
                  ],
                  "Resource": [
                      "arn:aws:ec2:*:*:network-interface/*"
                  ]
              }
          ]
      }
      EOF
      

   2. Create the IAM policy.

      aws iam create-policy --policy-name AmazonEKS_CNI_IPv6_Policy --policy-document file://vpc-cni-ipv6-policy.json

   3. Run the following command to create the node-role-trust-relationship.json file.

      cat >node-role-trust-relationship.json <<EOF
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
          }
        ]
      }
      EOF
      

   4. Run the following command to set a variable for your role name. You can replace AmazonEKSNodeRole with any name you choose.

      export node_role_name=AmazonEKSNodeRole

   5. Create the IAM role.

      aws iam create-role --role-name $node_role_name --assume-role-policy-document file://"node-role-trust-relationship.json"

   6. Attach the IAM policy to the IAM role.

      aws iam attach-role-policy --policy-arn arn:aws-cn:iam::$account_id:policy/AmazonEKS_CNI_IPv6_Policy \
          --role-name $node_role_name

      Important

      For simplicity in this tutorial, the policy is attached to this IAM role. In a production cluster however, we recommend attaching the policy to a separate IAM role. For more information, see Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts.

   7. Attach two required IAM managed policies to the IAM role.

      aws iam attach-role-policy --policy-arn arn:aws-cn:iam::aws:policy/AmazonEKSWorkerNodePolicy \
        --role-name $node_role_name
      aws iam attach-role-policy --policy-arn arn:aws-cn:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly \
        --role-name $node_role_name
      

   8. Retrieve the ARN of the IAM role and store it in a variable for a later step.

      node_iam_role=$(aws iam get-role --role-name $node_role_name --query="Role.Arn" --output text)
      

7. Create a managed node group.

   1. View the IDs of the subnets that you created in a previous step.

      echo $subnets

      An example output is as follows.

      subnet-0a1a56c486EXAMPLE,subnet-099e6ca77aEXAMPLE,subnet-0377963d69EXAMPLE,subnet-0c05f819d5EXAMPLE

   2. Create the node group. Replace 0a1a56c486EXAMPLE, 099e6ca77aEXAMPLE, 0377963d69EXAMPLE, and 0c05f819d5EXAMPLE with the values returned in the output of the previous step. Be sure to remove the commas between subnet IDs from the previous output in the following command. You can replace t3.medium with any Amazon Nitro System instance type.

      aws eks create-nodegroup --region $region_code --cluster-name $cluster_name --nodegroup-name $nodegroup_name \
          --subnets subnet-0a1a56c486EXAMPLE subnet-099e6ca77aEXAMPLE subnet-0377963d69EXAMPLE subnet-0c05f819d5EXAMPLE \
          --instance-types t3.medium --node-role $node_iam_role

      The node group takes a few minutes to create. Run the following command. Don't proceed to the next step until the output returned is ACTIVE.

      aws eks describe-nodegroup --region $region_code --cluster-name $cluster_name --nodegroup-name $nodegroup_name \
          --query nodegroup.status --output text
      

8. Confirm that the default Pods are assigned IPv6 addresses in the IP column.

   kubectl get pods -n kube-system -o wide
   

   An example output is as follows.

   NAME                       READY   STATUS    RESTARTS   AGE     IP                                       NODE                                            NOMINATED NODE   READINESS GATES
   aws-node-rslts             1/1     Running   1          5m36s   2600:1f13:b66:8200:11a5:ade0:c590:6ac8   ip-192-168-34-75.region-code.compute.internal   <none>           <none>
   aws-node-t74jh             1/1     Running   0          5m32s   2600:1f13:b66:8203:4516:2080:8ced:1ca9   ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   coredns-85d5b4454c-cw7w2   1/1     Running   0          56m     2600:1f13:b66:8203:34e5::                ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   coredns-85d5b4454c-tx6n8   1/1     Running   0          56m     2600:1f13:b66:8203:34e5::1               ip-192-168-253-70.region-code.compute.internal  <none>           <none>
   kube-proxy-btpbk           1/1     Running   0          5m36s   2600:1f13:b66:8200:11a5:ade0:c590:6ac8   ip-192-168-34-75.region-code.compute.internal   <none>           <none>
   kube-proxy-jjk2g           1/1     Running   0          5m33s   2600:1f13:b66:8203:4516:2080:8ced:1ca9   ip-192-168-253-70.region-code.compute.internal  <none>           <none>

9. Confirm that the default services are assigned IPv6 addresses in the IP column.

   kubectl get services -n kube-system -o wide
   

   An example output is as follows.

   NAME       TYPE        CLUSTER-IP          EXTERNAL-IP   PORT(S)         AGE   SELECTOR
   kube-dns   ClusterIP   fd30:3087:b6c2::a   <none>        53/UDP,53/TCP   57m   k8s-app=kube-dns

10. (Optional) Deploy a sample application or deploy the Amazon Load Balancer Controller and a sample application to load balance application or network traffic to IPv6 Pods.

11. After you've finished with the cluster and nodes that you created for this tutorial, you should clean up the resources that you created with the following commands. Make sure that you're not using any of the resources outside of this tutorial before deleting them.

    1. If you're completing this step in a different shell than you completed the previous steps in, set the values of all the variables used in previous steps, replacing the example values with the values you specified when you completed the previous steps. If you're completing this step in the same shell that you completed the previous steps in, skip to the next step.

       export region_code=region-code
       export vpc_stack_name=my-eks-ipv6-vpc
       export cluster_name=my-cluster
       export nodegroup_name=my-nodegroup
       export account_id=111122223333
       export node_role_name=AmazonEKSNodeRole
       export cluster_role_name=myAmazonEKSClusterRole

    2. Delete your node group.

       aws eks delete-nodegroup --region $region_code --cluster-name $cluster_name --nodegroup-name $nodegroup_name
       

       Deletion takes a few minutes. Run the following command. Don't proceed to the next step if any output is returned.

       aws eks list-nodegroups --region $region_code --cluster-name $cluster_name --query nodegroups --output text
       

    3. Delete the cluster.

       aws eks delete-cluster --region $region_code --name $cluster_name
       

       The cluster takes a few minutes to delete. Before continuing make sure that the cluster is deleted with the following command.

       aws eks describe-cluster --region $region_code --name $cluster_name
       

       Don't proceed to the next step until your output is similar to the following output.

       An error occurred (ResourceNotFoundException) when calling the DescribeCluster operation: No cluster found for name: my-cluster.

    4. Delete the IAM resources that you created. Replace AmazonEKS_CNI_IPv6_Policy with the name you chose, if you chose a different name than the one used in previous steps.

       aws iam detach-role-policy --role-name $cluster_role_name --policy-arn arn:aws-cn:iam::aws:policy/AmazonEKSClusterPolicy
       aws iam detach-role-policy --role-name $node_role_name --policy-arn arn:aws-cn:iam::aws:policy/AmazonEKSWorkerNodePolicy
       aws iam detach-role-policy --role-name $node_role_name --policy-arn arn:aws-cn:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
       aws iam detach-role-policy --role-name $node_role_name --policy-arn arn:aws-cn:iam::$account_id:policy/AmazonEKS_CNI_IPv6_Policy
       aws iam delete-policy --policy-arn arn:aws-cn:iam::$account_id:policy/AmazonEKS_CNI_IPv6_Policy
       aws iam delete-role --role-name $cluster_role_name
       aws iam delete-role --role-name $node_role_name

    5. Delete the Amazon CloudFormation stack that created the VPC.

       aws cloudformation delete-stack --region $region_code --stack-name $vpc_stack_name