Amazon SageMaker Autopilot example notebooks

By default, Autopilot allows deploying generated models to real-time inference endpoints. In this repository, the notebook illustrates how to deploy Autopilot models trained with ENSEMBLING and HYPERPARAMETER OPTIMIZATION (HPO) modes to serverless endpoints. Serverless endpoints automatically launch compute resources and scale them in and out depending on traffic, eliminating the need to choose instance types or manage scaling policies.

Custom feature selection

Autopilot inspects your data set, and runs a number of candidates to figure out the optimal combination of data preprocessing steps, machine learning algorithms, and hyperparameters. You can easily deploy either on a real-time endpoint or for batch processing.

In some cases, you might want to have the flexibility to bring custom data processing code to Autopilot. For example, your datasets might contain a large number of independent variables, and you may wish to incorporate a custom feature selection step to remove irrelevant variables first. The resulting smaller dataset can then be used to launch an Autopilot job. Ultimately, you would also want to include both the custom processing code and models from Autopilot for real-time or batch processing.

Pipeline example

While Autopilot streamlines the process of building ML models, MLOps engineers are still responsible for creating, automating, and managing end-to-end ML workflows in production. SageMaker Pipelines can assist in automating various steps of the ML lifecycle, such as data preprocessing, model training, hyperparameter tuning, model evaluation, and deployment. This notebook serves as a demonstration of how to incorporate Autopilot into a SageMaker Pipelines end-to-end AutoML training workflow. To launch an Autopilot experiment within Pipelines, you must create a model-building workflow by writing custom integration code using Pipelines Lambda or Processing steps. For more information, refer to Move Amazon SageMaker Autopilot ML models from experimentation to production using Amazon SageMaker Pipelines.

Alternatively, when using Autopilot in Ensembling mode, you can refer to the notebook example that demonstrates how to use native AutoML step in SageMaker Pipeline's native AutoML step. With Autopilot supported as a native step within Pipelines, you can now add an automated training step (AutoMLStep) to your Pipelines and invoke an Autopilot experiment in Ensembling mode.

More notebooks

You can find more notebooks illustrating other use cases such as batch transform, time-series forecasting and more in the root directory.