Blogs and case studies Example notebooks Additional resources

Blogs, example notebooks, and additional resources

The following sections contain examples and additional resources for you to learn more about Amazon SageMaker.

Blogs and case studies

See the following table for lists of blogs and case studies for various features within SageMaker Inference. You can use the blogs to help put together solutions that work for your use case.

Feature	Resources
Real-Time Inference	Getting started with deploying real-time models on Amazon SageMaker Deploy BLOOM-176B and OPT-30B on Amazon SageMaker with large model inference Deep Learning Containers and DeepSpeed Creating a machine learning-powered REST API with Amazon API Gateway mapping templates and Amazon SageMaker
Autoscaling	Configuring autoscaling inference endpoints in Amazon SageMaker
Serverless Inference	Amazon SageMaker Serverless Inference – Machine Learning Inference without Worrying about Servers Host Hugging Face transformer models using Amazon SageMaker Serverless Inference Introducing the Amazon SageMaker Serverless Inference Benchmarking Toolkit
Asynchronous Inference	Run computer vision inference on large videos with Amazon SageMaker asynchronous endpoints Build a predictive maintenance solution with Amazon Kinesis, Amazon Glue, and Amazon SageMaker Improve high-value research with Hugging Face and Amazon SageMaker Asynchronous Inference endpoints
Batch Transform	Associating prediction results with input data using Amazon SageMaker Batch Transform
Multi-Model Endpoints	Save on inference costs by using Amazon SageMaker multi-model endpoints Run multiple deep learning models on GPU with Amazon SageMaker multi-model endpoints How to scale machine learning inference for multi-tenant SaaS use cases Run and optimize multi-model inference with Amazon SageMaker multi-model endpoints
Serial Inference Pipelines	Design patterns for serial inference on Amazon SageMaker
Multi-Container Endpoints	Cost efficient ML inference with multi-framework models on Amazon SageMaker
Running Model Ensembles	Run ensemble ML models on Amazon SageMaker
Inference Recommender	SageMaker Inference Recommender example notebook SageMaker Inference Recommender for HuggingFace BERT Sentiment Analysis example notebook Achieve hyperscale performance for model serving using NVIDIA Triton Inference Server on Amazon SageMaker
Advanced model hosting blog series	Part 1: Common design patterns for building ML application on Amazon SageMaker Part 2: Getting started with deploying real time models on SageMaker Part 3: Run and optimize multi-model inference with Amazon SageMaker multi-model endpoints Part 4: Design patterns for serial inference on Amazon SageMaker Part 5: Cost efficient ML inference with multi-framework models on Amazon SageMaker Part 6: Best practices in testing and updating models on SageMaker Part 7: Run ensemble ML models on Amazon SageMaker

Example notebooks

See the following table for example notebooks that can help you learn more about SageMaker Inference.

Feature	Example notebooks
Inference Recommender	SageMaker Inference Recommender example notebook SageMaker Inference Recommender for HuggingFace BERT Sentiment Analysis example notebook
Optimize large language models (LLMs) for SageMaker	Generative AI LLMs workshop

Additional resources

For more information about each SageMaker Inference option in detail, you can watch the following video.

Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Resources

Troubleshooting and reference