Pod networking (CNI) - Amazon EKS
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Pod networking (CNI)

Amazon EKS supports native VPC networking with the Amazon VPC Container Network Interface (CNI) plugin for Kubernetes. This plugin assigns an IP address from your VPC to each pod. The plugin is an open-source project that is maintained on GitHub. For more information, see amazon-vpc-cni-k8s and Proposal: CNI plugin for Kubernetes networking over Amazon VPC on GitHub. The Amazon VPC CNI plugin is fully supported for use on Amazon EKS and self-managed Kubernetes clusters on Amazon.

Note

Kubernetes can use the Container Networking Interface (CNI) for configurable networking setups. The Amazon VPC CNI plugin might not meet requirements for all use cases. Amazon EKS maintains a network of partners that offer alternative CNI solutions with commercial support options. For more information, see Alternate compatible CNI plugins.

When you create an Amazon EKS node, it has one network interface. All Amazon EC2 instance types support more than one network interface. The network interface attached to the instance when the instance is created is called the primary network interface. Any additional network interface attached to the instance is called a secondary network interface. Each network interface can be assigned multiple private IP addresses. One of the private IP addresses is the primary IP address, whereas all other addresses assigned to the network interface are secondary IP addresses. For more information about network interfaces, see Elastic network interfaces in the Amazon EC2 User Guide for Linux Instances. For more information about how many network interfaces and private IP addresses are supported for each network interface, see IP addresses per network interface per instance type in the Amazon EC2 User Guide for Linux Instances. For example, an m5.large instance type supports three network interfaces and ten private IP addresses for each network interface.

The Amazon VPC Container Network Interface (CNI) plugin for Kubernetes is deployed with each of your Amazon EC2 nodes in a Daemonset with the name aws-node. The plugin consists of two primary components:

  • L-IPAM daemon – Responsible for creating network interfaces and attaching the network interfaces to Amazon EC2 instances, assigning secondary IP addresses to network interfaces, and maintaining a warm pool of IP addresses on each node for assignment to Kubernetes pods when they are scheduled. When the number of pods running on the node exceeds the number of addresses that can be assigned to a single network interface, the plugin starts allocating a new network interface, as long as the maximum number of network interfaces for the instance aren't already attached. There are configuration variables that allow you to change the default value for when the plugin creates new network interfaces. For more information, see WARM_ENI_TARGET, WARM_IP_TARGET and MINIMUM_IP_TARGET on GitHub.

    Each pod that you deploy is assigned one secondary private IP address from one of the network interfaces attached to the instance. Previously, it was mentioned that an m5.large instance supports three network interfaces and ten private IP addresses for each network interface. Even though an m5.large instance supports 30 private IP addresses, you can't deploy 30 pods to that node. To determine how many pods you can deploy to a node, use the following formula: 

    (Number of network interfaces for the instance type × (the number of IP addressess per network interface - 1)) + 2

    Using this formula, an m5.large instance type can support a maximum of 29 pods. For a list of the maximum number of pods supported by each instance type, see eni-max-pods.txt on GitHub. System pods count towards the maximum pods. For example, the CNI plugin and kube-proxy pods run on every node in a cluster, so you're only able to deploy 27 additional pods to an m5.large instance, not 29. Further, CoreDNS runs on some of the nodes in the cluster, which decrements the maximum pods by another one for the nodes it runs on.

    By default, all pods deployed to a node are assigned the same security groups and are assigned private IP addresses from a CIDR block that is assigned to the subnet that one of the instance's network interfaces is connected to. You can assign IP addresses from a different CIDR block than the subnet that the primary network interface is connected to by configuring CNI custom networking. You can also use CNI custom networking to assign all pods on a node the same security groups. The security groups assigned to all pods can be different than the security groups assigned to the primary network interface. You can assign unique security groups to pods deployed to many Amazon EC2 instance types using security groups for pods. For more information, see Security groups for pods.

  • CNI plugin – Responsible for wiring the host network (for example, configuring the network interfaces and virtual Ethernet pairs) and adding the correct network interface to the pod namespace.

Important

If you are using version 1.7.0 or later of the CNI plugin and you assign a custom pod security policy to the aws-node Kubernetes service account used for the aws-node pods deployed by the Daemonset, then the policy must have NET_ADMIN in its allowedCapabilities section along with hostNetwork: true and privileged: true in the policy's spec. For more information, see Pod security policy.