SageMaker model parallelism library v2

The Amazon SageMaker model parallelism library is a capability of SageMaker that enables high performance and optimized large scale training on SageMaker accelerate compute instances. The Core features of the SageMaker model parallelism library v2 include techniques and optimizations to accelerate and simplify large model training, such as hybrid sharded data parallelism, tensor parallelism, activation checkpointing, and activation offloading. You can use the SMP library to accelerate the training and fine-tuning of large language models (LLMs), large vision models (LVMs), and foundation models (FMs) with hundreds of billions of parameters.

The SageMaker model parallelism library v2 (SMP v2) aligns the library’s APIs and methods with open source PyTorch Fully Sharded Data Parallelism (FSDP), which gives you the benefit of SMP performance optimizations with minimal code changes. With SMP v2, you can improve the computational performance of training a state-of-the-art large model on SageMaker by bringing your PyTorch FSDP training scripts to SageMaker.

You can use SMP v2 for the general SageMaker Training jobs and distributed training workloads on SageMaker HyperPod clusters.