Get started with P5 instances - Amazon Elastic Compute Cloud
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Get started with P5 instances

P5 instances provide 8 NVIDIA H100 GPUs with 640 GB of high-bandwidth GPU memory. They feature 3rd generation AMD EPYC processors and provide 2 TB of system memory, 30 TB of local NVMe instance storage, 3,200 Gbps aggregated network bandwidth, and GPUDirect RDMA support. P5 instances also support Amazon EC2 UltraCluster technology, which provides lower latency and improved network performance using EFA.

The following table provides a summary of the p5.48xlarge specifications.

vCPUs System memory GPUs GPU memory Network bandwidth GPUDirect RDMA GPU peer to peer Instance storage
192 2 TiB 8 NVIDIA H100 GPUs 640 GB HBM3 3200 Gbps with EFAv2 Supported 900 GB/s NVSwitch 8 x 3,800 GB NVMe SSD volumes
Software configuration

The easiest way to get started with P5 instances is to launch an instance using an Amazon Deep Learning AMI that is preconfigured with all of the required software. For the latest Amazon Deep Learning AMI for use with P5 instances, see the Amazon Deep Learning Base GPU AMI (Ubuntu 20.04).

If you need to build a custom AMI for use with P5 instances, we recommend installing the following minimum software versions:

  • NVIDIA driver 535.54.03 or later

  • CUDA 12.1 or later

  • Nvidia GDRCopy 2.3 or later

  • EFA installer 1.24.1 or later

  • NCCL 2.18.3 or later

  • aws-ofi-nccl plugin 1.7.2-aws or later

We also recommend that you configure the instance to not use deeper C-states. For more information, see High performance and low latency by limiting deeper C-states. The latest Amazon Deep Learning Base GPU AMI is preconfigured to not use deeper C-states.

Networking and EFA configuration

P5 instances deliver 3200 Gbps of networking bandwidth by using multiple EFA interfaces. P5 instances support 32 network cards. We recommend that you define a single EFA network interface per network card. To configure these interfaces at launch we recommend the following settings:

  • For network interface 0, specify device index 0

  • For network interfaces 1 through 31, specify device index 1