Configure your cluster for Kubernetes network policies

1. Mount the BPF filesystem

   Note

   If your cluster is version 1.27 or later, you can skip this step as all Amazon EKS optimized Amazon Linux and Bottlerocket AMIs for 1.27 or later have this feature already.

   For all other cluster versions, if you upgrade the Amazon EKS optimized Amazon Linux to version v20230703 or later or you upgrade the Bottlerocket AMI to version v1.0.2 or later, you can skip this step.

   1. Mount the Berkeley Packet Filter (BPF) file system on each of your nodes.

      sudo mount -t bpf bpffs /sys/fs/bpf
      

   2. Then, add the same command to your user data in your launch template for your Amazon EC2 Auto Scaling Groups.

2. Enable network policy in the VPC CNI

   1. See which type of the add-on is installed on your cluster. Depending on the tool that you created your cluster with, you might not currently have the Amazon EKS add-on type installed on your cluster. Replace my-cluster with the name of your cluster.

      aws eks describe-addon --cluster-name my-cluster --addon-name vpc-cni --query addon.addonVersion --output text

      If a version number is returned, you have the Amazon EKS type of the add-on installed on your cluster and don't need to complete the remaining steps in this procedure. If an error is returned, you don't have the Amazon EKS type of the add-on installed on your cluster.

   2. • Amazon EKS add-on

        Amazon Web Services Management Console

        1. Open the Amazon EKS console at https://console.amazonaws.cn/eks/home#/clusters.

        2. In the left navigation pane, select Clusters, and then select the name of the cluster that you want to configure the Amazon VPC CNI add-on for.

        3. Choose the Add-ons tab.

        4. Select the box in the top right of the add-on box and then choose Edit.

        5. On the Configure name of addon page:

           1. Select a v1.14.0-eksbuild.3 or later version in the Version dropdown list.

           2. Expand the Optional configuration settings.

           3. Enter the JSON key "enableNetworkPolicy": and value "true" in Configuration values. The resulting text must be a valid JSON object. If this key and value are the only data in the text box, surround the key and value with curly braces {}. The following example shows both network policy and the network policy logs are enabled:

              {
                                            "enableNetworkPolicy": "true",
                                            "nodeAgent": {
                                            "enableCloudWatchLogs": "true"
                                            }
                                            }

              The following screenshot shows an example of this scenario.

           

        Amazon CLI

        • Run the following Amazon CLI command. Replace my-cluster with the name of your cluster and the IAM role ARN with the role that you are using.

          aws eks update-addon --cluster-name my-cluster --addon-name vpc-cni --addon-version v1.14.0-eksbuild.3 \
              --service-account-role-arn arn:aws:iam::123456789012:role/AmazonEKSVPCCNIRole \
              --resolve-conflicts PRESERVE --configuration-values '{"enableNetworkPolicy": "true"}'
                                  

      • Self-managed add-on

        Helm

        If you have installed the Amazon VPC CNI plugin for Kubernetes through helm, you can update the configuration to enable network policy.

        • Run the following command to enable network policy.

          helm upgrade --set enableNetworkPolicy=true aws-vpc-cni --namespace kube-system eks/aws-vpc-cni

        kubectl

        1. Open the amazon-vpc-cni ConfigMap in your editor.

           kubectl edit configmap -n kube-system amazon-vpc-cni -o yaml
           

        2. Add the following line to the data in the ConfigMap.

           enable-network-policy-controller: "true"

           Once you've added the line, your ConfigMap should look like the following example.

           apiVersion: v1
            kind: ConfigMap
            metadata:
             name: amazon-vpc-cni
             namespace: kube-system
            data:
             enable-network-policy-controller: "true"

        3. Open the aws-node DaemonSet in your editor.

           kubectl edit daemonset -n kube-system aws-node
           

        4. Replace the false with true in the command argument --enable-network-policy=false in the args: in the aws-network-policy-agent container in the VPC CNI aws-node daemonset manifest.

                - args:
                   - --enable-network-policy=true

3. Confirm that the aws-node pods are running on your cluster.

   kubectl get pods -n kube-system | grep 'aws-node\|amazon'
   

   An example output is as follows.

   aws-node-gmqp7                                          2/2     Running   1 (24h ago)   24h
   aws-node-prnsh                                          2/2     Running   1 (24h ago)   24h

   If network policy is enabled, there are 2 containers in the aws-node pods. In previous versions and if network policy is disabled, there is only a single container in the aws-node pods.

   You can now deploy Kubernetes network policies to your cluster. For more information, see Kubernetes network policies.

To run the Stars policy demo

1. Apply the front-end, back-end, client, and management user interface services:

   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/namespace.yaml
   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/management-ui.yaml
   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/backend.yaml
   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/frontend.yaml
   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/client.yaml

2. View all Pods on the cluster.

   kubectl get pods -A

   An example output is as follows.

   In your output, you should see pods in the namespaces shown in the following output. The NAMES of your pods and the number of pods in the READY column are different than those in the following output. Don't continue until you see pods with similar names and they all have Running in the STATUS column.

   NAMESPACE         NAME                                       READY   STATUS    RESTARTS   AGE
   [...]
   client            client-xlffc                               1/1     Running   0          5m19s
   [...]
   management-ui     management-ui-qrb2g                        1/1     Running   0          5m24s
   stars             backend-sz87q                              1/1     Running   0          5m23s
   stars             frontend-cscnf                             1/1     Running   0          5m21s
   [...]

3. To connect to the management user interface, connect to the EXTERNAL-IP of the service running on your cluster:

   kubectl get service/management-ui -n management-ui

4. Open the a browser to the location from the previous step. You should see the management user interface. The C node is the client service, the F node is the front-end service, and the B node is the back-end service. Each node has full communication access to all other nodes, as indicated by the bold, colored lines.

   

5. Apply the following network policy in both the stars and client namespaces to isolate the services from each other:

   kind: NetworkPolicy
   apiVersion: networking.k8s.io/v1
   metadata:
     name: default-deny
   spec:
     podSelector:
       matchLabels: {}

   You can use the following commands to apply the policy to both namespaces:

   kubectl apply -n stars -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/default-deny.yaml
   kubectl apply -n client -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/default-deny.yaml

6. Refresh your browser. You see that the management user interface can no longer reach any of the nodes, so they don't show up in the user interface.

7. Apply the following different network policies to allow the management user interface to access the services. Apply this policy to allow the UI:

   kind: NetworkPolicy
   apiVersion: networking.k8s.io/v1
   metadata:
     namespace: stars
     name: allow-ui 
   spec:
     podSelector:
       matchLabels: {}
     ingress:
       - from:
           - namespaceSelector:
               matchLabels:
                 role: management-ui 

   Apply this policy to allow the client:

   kind: NetworkPolicy
   apiVersion: networking.k8s.io/v1
   metadata:
     namespace: client 
     name: allow-ui 
   spec:
     podSelector:
       matchLabels: {}
     ingress:
       - from:
           - namespaceSelector:
               matchLabels:
                 role: management-ui 

   You can use the following commands to apply both policies:

   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/allow-ui.yaml
   kubectl apply -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/apply_network_policies.files/allow-ui-client.yaml

8. Refresh your browser. You see that the management user interface can reach the nodes again, but the nodes cannot communicate with each other.

   

9. Apply the following network policy to allow traffic from the front-end service to the back-end service:

   kind: NetworkPolicy
   apiVersion: networking.k8s.io/v1
   metadata:
     namespace: stars
     name: backend-policy
   spec:
     podSelector:
       matchLabels:
         role: backend
     ingress:
       - from:
           - podSelector:
               matchLabels:
                 role: frontend
         ports:
           - protocol: TCP
             port: 6379
   

10. Refresh your browser. You see that the front-end can communicate with the back-end.

    

11. Apply the following network policy to allow traffic from the client to the front-end service:

    kind: NetworkPolicy
    apiVersion: networking.k8s.io/v1
    metadata:
      namespace: stars
      name: frontend-policy
    spec:
      podSelector:
        matchLabels:
          role: frontend 
      ingress:
        - from:
            - namespaceSelector:
                matchLabels:
                  role: client
          ports:
            - protocol: TCP
              port: 80
    

12. Refresh your browser. You see that the client can communicate to the front-end service. The front-end service can still communicate to the back-end service.

    

13. (Optional) When you are done with the demo, you can delete its resources.

    kubectl delete -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/client.yaml
    kubectl delete -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/frontend.yaml
    kubectl delete -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/backend.yaml
    kubectl delete -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/management-ui.yaml
    kubectl delete -f https://eksworkshop.com/beginner/120_network-policies/calico/stars_policy_demo/create_resources.files/namespace.yaml

    Even after deleting the resources, there can still be network policy endpoints on the nodes that might interfere in unexpected ways with networking in your cluster. The only sure way to remove these rules is to reboot the nodes or terminate all of the nodes and recycle them. To terminate all nodes, either set the Auto Scaling Group desired count to 0, then back up to the desired number, or just terminate the nodes.