Run GPU jobs - Amazon Batch
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Run GPU jobs

GPU jobs help you to run jobs that use an instance's GPUs.

The following Amazon EC2 GPU-based instance types are supported. For more information, see Amazon EC2 G3 Instances, Amazon EC2 G4 Instances, Amazon EC2 G5 Instances,Amazon EC2 G6 Instances , Amazon EC2 P2 Instances, Amazon EC2 P3 Instances, Amazon EC2 P4d Instances, Amazon EC2 P5 Instances, Amazon EC2 P6 Instances, Amazon EC2 Trn1 Instances, Amazon EC2 Trn2 Instances, Amazon EC2 Inf1 Instances, Amazon EC2 Inf2 Instances, Amazon EC2 Dl1 Instances, and Amazon EC2 Dl2 Instances.

Instance type

GPUs

GPU memory

vCPUs

Memory

Network bandwidth

g3s.xlarge

1

8 GiB

4

30.5 GiB

10 Gbps

g3.4xlarge

1

8 GiB

16

122 GiB

Up to 10 Gbps

g3.8xlarge

2

16 GiB

32

244 GiB

10 Gbps

g3.16xlarge

4

32 GiB

64

488 GiB

25 Gbps

g4dn.xlarge

1

16 GiB

4

16 GiB

Up to 25 Gbps

g4dn.2xlarge

1

16 GiB

8

32 GiB

Up to 25 Gbps

g4dn.4xlarge

1

16 GiB

16

64 GiB

Up to 25 Gbps

g4dn.8xlarge

1

16 GiB

32

128 GiB

50 Gbps

g4dn.12xlarge

4

64 GiB

48

192 GiB

50 Gbps

g4dn.16xlarge

1

16 GiB

64

256 GiB

50 Gbps

g5.xlarge

1

24 GiB

4

16 GiB

Up to 10 Gbps

g5.2xlarge

1

24 GiB

8

32 GiB

Up to 10 Gbps

g5.4xlarge

1

24 GiB

16

64 GiB

Up to 25 Gbps

g5.8xlarge

1

24 GiB

32

128 GiB

25 Gbps

g5.16xlarge

1

24 GiB

64

256 GiB

25 Gbps

g5.12xlarge

4

96 GiB

48

192 GiB

40 Gbps

g5.24xlarge

4

96 GiB

96

384 GiB

50 Gbps

g5.48xlarge

8

192 GiB

192

768 GiB

100 Gbps

g5g.xlarge

1

16 GB

4

8 GB

Up to 10 Gbps

g5g.2xlarge

1

16 GB

8

16 GB

Up to 10 Gbps

g5g.4xlarge

1

16 GB

16

32 GB

Up to 10 Gbps

g5g.8xlarge

1

16 GB

32

64 GB

12 Gbps

g5g.16xlarge

2

32 GB

64

128 GB

25 Gbps

g5g.metal

2

32 GB

64

128 GB

25 Gbps

g6.xlarge

1

24 GiB

4

16 GiB

Up to 10 Gbps

g6.2xlarge

1

24 GiB

8

32 GiB

Up to 10 Gbps

g6.4xlarge

1

24 GiB

16

64 GiB

Up to 25 Gbps

g6.8xlarge

1

24 GiB

32

128 GiB

25 Gbps

g6.16xlarge

1

24 GiB

64

256 GiB

25 Gbps

g6.12xlarge

4

96 GiB

48

192 GiB

40 Gbps

g6.24xlarge

4

96 GiB

96

384 GiB

50 Gbps

g6.48xlarge

8

192 GiB

192

768 GiB

100 Gbps

g6e.xlarge

1

48 GB

4

32 GB

Up to 20 Gbps

g6e.2xlarge

1

48 GB

8

64 GB

Up to 20 Gbps

g6e.4xlarge

1

48 GB

16

128 GB

20 Gbps

g6e.8xlarge

1

48 GB

32

256 GB

25 Gbps

g6e.16xlarge

1

48 GB

64

512 GB

35 Gbps

g6e.12xlarge

4

192 GB

48

384 GB

100 Gbps

g6e.24xlarge

4

192 GB

96

768 GB

200 Gbps

g6e.48xlarge

8

384 GB

192

1536 GB

400 Gbps

gr6.4xlarge

1

24 GB

16

128 GB

Up to 25 Gbps

gr6.8xlarge

1

24 GB

32

256 GB

25 Gbps

p2.xlarge

1

12 GiB

4

61 GiB

High

p2.8xlarge

8

96 GiB

32

488 GiB

10 Gbps

p2.16xlarge

16

192 GiB

64

732 GiB

20 Gbps

p3.2xlarge

1

16 GiB

8

61 GiB

Up to 10 Gbps

p3.8xlarge

4

64 GiB

32

244 GiB

10 Gbps

p3.16xlarge

8

128 GiB

64

488 GiB

25 Gbps

p3dn.24xlarge

8

256 GiB

96

768 GiB

100 Gbps

p4d.24xlarge

8

320 GB

96

1152 GB

400 Gbps

p4de.24xlarge

8

640 GB

96

1152 GB

400 Gbps

p5.48xlarge

8

640 GiB

192

2 TiB

3200 Gbps

p5e.48xlarge

8

1128 GB

192

2 TB

3200 Gbps

p5en.48xlarge

8

1128 GB

192

2 TB

3200 Gbps

p6-b200.48xlarge

8

1440 GB

192

2 TB

100 Gbps

trn1.2xlarge

1

32 GB

8

32 GB

Up to 12.5 Gbps

trn1.32xlarge

16

512 GB

128

512 GB

800 Gbps

trn1n.32xlarge

16

512 GB

128

512 GB

1600 Gbps

trn2.48xlarge

16

1.5 TB

192

2 TB

3.2 Tbps

inf1.xlarge

1

8 GB

4

8 GB

Up to 25 Gbps

inf1.2xlarge

1

8 GB

8

16 GB

Up to 25 Gbps

inf1.6xlarge

4

32 GB

24

48 GB

25 Gbps

inf1.24xlarge

16

128 GB

96

192 GB

100 Gbps

inf2.xlarge

1

32 GB

4

16 GB

Up to 15 Gbps

inf2.8xlarge

1

32 GB

32

128 GB

Up to 25 Gbps

inf2.24xlarge

6

192 GB

96

384 GB

50 Gbps

inf2.48xlarge

12

384 GB

192

768 GB

100 Gbps

dl1.24xlarge

8

256 GB

96

768 GB

400 Gbps

dl2q.24xlarge

8

128 GB

96

768 GB

100 Gbps
Note

For GPU jobs Amazon Batch only supports instance types that have NVIDIA GPUs. For example, the G4ad family is not supported for GPU scheduling. You can still use G4ad on Amazon Batch by defining only the vcpu and memory requirements in the job definition, then accessing the host GPUs directly through customization in an Amazon EC2 launch template user data with an Amazon ECS or Amazon EKS compute optimized AMI, or a customized AMI for using AMD GPUs.

Instance types that use an ARM64 architecture are supported for GPU jobs on custom AMIs provided to Amazon Batch or Amazon EC2 user data to access the GPUs by customized code and configurations. For example, the G5g instance family.

The resourceRequirements parameter for the job definition specifies the number of GPUs to be pinned to the container. This number of GPUs isn't available to any other job that runs on that instance for the duration of that job. All instance types in a compute environment that run GPU jobs must be from the p3, p4, p5, p6, g3, g3s, g4, g5, or g6 instance families. If this isn't done a GPU job might get stuck in the RUNNABLE status.

Jobs that don't use the GPUs can be run on GPU instances. However, they might cost more to run on the GPU instances than on similar non-GPU instances. Depending on the specific vCPU, memory, and time needed, these non-GPU jobs might block GPU jobs from running.

Topics