Tutorial: Getting started with Amazon Batch on Amazon EKS

Amazon Batch on Amazon EKS is a managed service for scheduling and scaling batch workloads into existing Amazon EKS clusters. Amazon Batch doesn't create, administer, or perform lifecycle operations of your Amazon EKS clusters on your behalf. Amazon Batch orchestration scales up and down nodes managed by Amazon Batch and run pods on those nodes.

Amazon Batch doesn't touch nodes, auto scaling node groups or pods lifecycles that aren't associated with Amazon Batch compute environments within your Amazon EKS cluster. For Amazon Batch to operate effectively, its service-linked role needs Kubernetes role-based access control (RBAC) permissions in your existing Amazon EKS cluster. For more information, see Using RBAC Authorization in the Kubernetes documentation.

Amazon Batch requires a Kubernetes namespace where it can scope pods as Amazon Batch jobs into. We recommend a dedicated namespace to isolate the Amazon Batch pods from your other cluster workloads.

After Amazon Batch has been given RBAC access and a namespace has been established, you can associate that Amazon EKS cluster to an Amazon Batch compute environment using the CreateComputeEnvironment API operation. A job queue can be associated with this new Amazon EKS compute environment. Amazon Batch jobs are submitted to the job queue based on an Amazon EKS job definition using the SubmitJob API operation. Amazon Batch then launches Amazon Batch managed nodes and place jobs from job queue as Kubernetes pods into the EKS cluster associated with an Amazon Batch compute environment.

The following sections cover how to get set up for Amazon Batch on Amazon EKS.

Prerequisites

Before starting this tutorial, you must install and configure the following tools and resources that you need to create and manage both Amazon Batch and Amazon EKS resources.

Prepare your Amazon EKS cluster for Amazon Batch

All steps are required.

Create an Amazon EKS compute environment

Amazon Batch compute environments define compute resource parameters to meet your batch workload needs. In a managed compute environment, Amazon Batch helps you to manage the capacity and instance types of the compute resources (Kubernetes nodes) within your Amazon EKS cluster. This is based on the compute resource specification that you define when you create the compute environment. You can use EC2 On-Demand Instances or EC2 Spot Instances.

Now that the AWSServiceRoleForBatch service-linked role has access to your Amazon EKS cluster, you can create Amazon Batch resources. First, create a compute environment that points to your Amazon EKS cluster.

$ cat <<EOF > ./batch-eks-compute-environment.json
{
  "computeEnvironmentName": "My-Eks-CE1",
  "type": "MANAGED",
  "state": "ENABLED",
  "eksConfiguration": {
    "eksClusterArn": "arn:aws-cn:eks:<region>:123456789012:cluster/<cluster-name>",
    "kubernetesNamespace": "my-aws-batch-namespace"
  },
  "computeResources": {
    "type": "EC2",
    "allocationStrategy": "BEST_FIT_PROGRESSIVE",
    "minvCpus": 0,
    "maxvCpus": 128,
    "instanceTypes": [
        "m5"
    ],
    "subnets": [
        "<eks-cluster-subnets-with-access-to-internet-for-image-pull>"
    ],
    "securityGroupIds": [
        "<eks-cluster-sg>"
    ],
    "instanceRole": "<eks-instance-profile>"
  }
}
EOF
$ aws batch create-compute-environment --cli-input-json file://./batch-eks-compute-environment.json

$ aws batch describe-compute-environments --compute-environments My-Eks-CE1

Create a job queue and attach the compute environment

$ aws batch describe-compute-environments --compute-environments My-Eks-CE1

Jobs submitted to this new job queue are run as pods on Amazon Batch managed nodes that joined the Amazon EKS cluster that's associated with your compute environment.

$ cat <<EOF > ./batch-eks-job-queue.json
 {
    "jobQueueName": "My-Eks-JQ1",
    "priority": 10,
    "computeEnvironmentOrder": [
      {
        "order": 1,
        "computeEnvironment": "My-Eks-CE1"
      }
    ]
  }
EOF
$ aws batch create-job-queue --cli-input-json file://./batch-eks-job-queue.json

Create a job definition

$ cat <<EOF > ./batch-eks-job-definition.json
{
  "jobDefinitionName": "MyJobOnEks_Sleep",
  "type": "container",
  "eksProperties": {
    "podProperties": {
      "hostNetwork": true,
      "containers": [
        {
          "image": "public.ecr.aws/amazonlinux/amazonlinux:2",
          "command": [
            "sleep",
            "60"
          ],
          "resources": {
            "limits": {
              "cpu": "1",
              "memory": "1024Mi"
            }
          }
        }
      ],
      "metadata": {
        "labels": {
          "environment": "test"
        }
      }
    }
  }
}
EOF
$ aws batch register-job-definition --cli-input-json file://./batch-eks-job-definition.json

Submit a job

$ aws batch submit-job --job-queue My-Eks-JQ1 \
    --job-definition MyJobOnEks_Sleep --job-name My-Eks-Job1
$ aws batch describe-jobs --job <jobId-from-submit-response>

(Optional) Submit a job with overrides

This job overrides the command passed to the container.

$ cat <<EOF > ./submit-job-override.json
{
  "jobName": "EksWithOverrides",
  "jobQueue": "My-Eks-JQ1",
  "jobDefinition": "MyJobOnEks_Sleep",
  "eksPropertiesOverride": {
    "podProperties": {
      "containers": [
        {
          "command": [
            "/bin/sh"
          ],
          "args": [
            "-c",
            "echo hello world"
          ]
        }
      ]
    }
  }
}
EOF
$ aws batch submit-job --cli-input-json file://./submit-job-override.json