Amazon EKS jobs - Amazon Batch
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon EKS jobs

A job is the smallest unit of work in Amazon Batch. An Amazon Batch job on Amazon EKS has a one-to-one mapping to a Kubernetes pod. An Amazon Batch job definition is a template for an Amazon Batch job. When you submit an Amazon Batch job, you reference a job definition, target a job queue, and provide a name for a job. In the job definition of an Amazon Batch job on Amazon EKS, the eksProperties parameter defines the set of parameters that an Amazon Batch on Amazon EKS job supports. In a SubmitJob request, the eksPropertiesOverride parameter allows for overrides to some common parameters. This way, you can use templates of job definitions for multiple jobs. When a job is dispatched to your Amazon EKS cluster, Amazon Batch transforms the job into a podspec (Kind: Pod). The podspec uses some additional Amazon Batch parameters to ensure that jobs are scaled and scheduled correctly. Amazon Batch combines labels and taints to ensure jobs run only on Amazon Batch managed nodes and that other pods don't run on those nodes.

Important
  • If the hostNetwork parameter isn't explicitly set in an Amazon EKS job definition, the pod networking mode in Amazon Batch defaults to host mode. More specifically, the following settings are applied: hostNetwork=true and dnsPolicy=ClusterFirstWithHostNet.

  • Amazon Batch cleans up job pods soon after a pod completes its job. To see pod application logs, configure a logging service for your cluster. For more information, see Use CloudWatch Logs to monitor Amazon Batch on Amazon EKS jobs.

Map a running job to a pod and a node

The podProperties of a running job have podName and nodeName parameters set for the current job attempt. Use the DescribeJobs API operation to view these parameters.

The following is example output.

$ aws batch describe-jobs --job 2d044787-c663-4ce6-a6fe-f2baf7e51b04 { "jobs": [ { "status": "RUNNING", "jobArn": "arn:aws-cn:batch:us-east-1:123456789012:job/2d044787-c663-4ce6-a6fe-f2baf7e51b04", "jobDefinition": "arn:aws-cn:batch:us-east-1:123456789012:job-definition/MyJobOnEks_SleepWithRequestsOnly:1", "jobQueue": "arn:aws-cn:batch:us-east-1:123456789012:job-queue/My-Eks-JQ1", "jobId": "2d044787-c663-4ce6-a6fe-f2baf7e51b04", "eksProperties": { "podProperties": { "nodeName": "ip-192-168-55-175.ec2.internal", "containers": [ { "image": "public.ecr.aws/amazonlinux/amazonlinux:2", "resources": { "requests": { "cpu": "1", "memory": "1024Mi" } } } ], "podName": "aws-batch.b0aca953-ba8f-3791-83e2-ed13af39428c" } } } ] }

For a job with retries enabled, the podName and nodeName of every completed attempt is in the eksAttempts list parameter of the DescribeJobs API operation. The podName and nodeName of the current running attempt are in the podProperties object.

How to map a running pod back to its job

A pod has labels that indicate the jobId and uuid of the compute environment that it belongs to. Amazon Batch injects environment variables so the job’s runtime can reference job information. For more information, see Amazon Batch job environment variables. You can view this information by running the following command. The output is as follows.

$ kubectl describe pod aws-batch.14638eb9-d218-372d-ba5c-1c9ab9c7f2a1 -n my-aws-batch-namespace Name: aws-batch.14638eb9-d218-372d-ba5c-1c9ab9c7f2a1 Namespace: my-aws-batch-namespace Priority: 0 Node: ip-192-168-45-88.ec2.internal/192.168.45.88 Start Time: Wed, 26 Oct 2022 00:30:48 +0000 Labels: batch.amazonaws.com/compute-environment-uuid=5c19160b-d450-31c9-8454-86cf5b30548f batch.amazonaws.com/job-id=f980f2cf-6309-4c77-a2b2-d83fbba0e9f0 batch.amazonaws.com/node-uid=a4be5c1d-9881-4524-b967-587789094647 ... Status: Running IP: 192.168.45.88 IPs: IP: 192.168.45.88 Containers: default: Image: public.ecr.aws/amazonlinux/amazonlinux:2 ... Environment: AWS_BATCH_JOB_KUBERNETES_NODE_UID: a4be5c1d-9881-4524-b967-587789094647 AWS_BATCH_JOB_ID: f980f2cf-6309-4c77-a2b2-d83fbba0e9f0 AWS_BATCH_JQ_NAME: My-Eks-JQ1 AWS_BATCH_JOB_ATTEMPT: 1 AWS_BATCH_CE_NAME: My-Eks-CE1 ...
Features that Amazon Batch Amazon EKS jobs support

These are the Amazon Batch specific features that are also common to Kubernetes jobs that run on Amazon EKS:

KubernetesSecrets and ServiceAccounts

Amazon Batch supports referencing Kubernetes Secrets and ServiceAccounts. You can configure pods to use Amazon EKS IAM roles for service accounts. For more information, see Configuring pods to use a Kubernetes service account in the Amazon EKS User Guide.