# Automated ML, no-code, or low-code
Amazon SageMaker AI offers the following features to automate key machine learning tasks and use no-code or low-code solutions.
+ **Amazon SageMaker Canvas**: For a UI-based, no-code AutoML experience, new users should use the [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html) application in [Amazon SageMaker Studio](studio-updated.md).
Amazon SageMaker Canvas provides analysts and citizen data scientists no-code capabilities for tasks such as data preparation, feature engineering, algorithm selection, training and tuning, inference, and more. Users can leverage built-in visualizations and what-if analysis to explore their data and different scenarios, with automated predictions enabling them to easily productionize their models. SageMaker Canvas supports a variety of use cases, including computer vision, demand forecasting, intelligent search, and generative AI.
+ **Amazon SageMaker Autopilot**: [Amazon SageMaker Autopilot](autopilot-automate-model-development.md) is an automated machine learning (AutoML) feature-set that automates the end-to-end process of building, training, tuning, and deploying machine learning models. Amazon SageMaker Autopilot analyzes your data, selects algorithms suitable for your problem type, preprocesses the data to prepare it for training, handles automatic model training, and performs hyperparameter optimization to find the best performing model for your dataset.
+ As of November 30, 2023, the user interface (UI) for Autopilot is integrated into the [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html) application in Studio.
+ Users of [Amazon SageMaker Studio Classic](studio.md), the previous experience of Studio, can continue using the Autopilot UI in Studio Classic. Users with coding experience can continue using the [AutoML API references](autopilot-automate-model-development.md) in any supported SDK for technical implementation.
**Note**
If you have been using Autopilot in Studio Classic until now and want to migrate to SageMaker Canvas, you might have to grant additional permissions to your user profile or IAM role so that you can create and use the SageMaker Canvas application. For more information, see [(Optional) Migrate from Autopilot in Studio Classic to SageMaker Canvas](studio-updated-migrate-ui.md#studio-updated-migrate-autopilot).
+ **Amazon SageMaker JumpStart**: SageMaker JumpStart provides pretrained, open-source models for a wide range of problem types to help you get started with machine learning. You can incrementally train and tune these models before deployment. JumpStart also provides solution templates that set up infrastructure for common use cases, and executable example notebooks for machine learning with SageMaker AI.
**Topics**
+ [
# SageMaker Autopilot
](autopilot-automate-model-development.md)
+ [
# SageMaker JumpStart pretrained models
](studio-jumpstart.md)
# SageMaker Autopilot
**Important**
As of November 30, 2023, Autopilot's UI is migrating to [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html) as part of the updated [Amazon SageMaker Studio](studio-updated.md) experience. SageMaker Canvas provides analysts and citizen data scientists no-code capabilities for tasks such as data preparation, feature engineering, algorithm selection, training and tuning, inference, and more. Users can leverage built-in visualizations and what-if analysis to explore their data and different scenarios, with automated predictions enabling them to easily productionize their models. Canvas supports a variety of use cases, including computer vision, demand forecasting, intelligent search, and generative AI.
Users of [Amazon SageMaker Studio Classic](studio.md), the previous experience of [Studio](studio-updated.md), can continue using the Autopilot UI in Studio Classic. Users with coding experience can continue using all [API references](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html) in any supported SDK for technical implementation.
If you have been using Autopilot in Studio Classic until now and want to migrate to SageMaker Canvas, you might have to grant additional permissions to your user profile or IAM role so that you can create and use the SageMaker Canvas application. For more information, see [(Optional) Migrate from Autopilot in Studio Classic to SageMaker Canvas](studio-updated-migrate-ui.md#studio-updated-migrate-autopilot).
All UI-related instructions in this guide pertain to Autopilot's standalone features before migrating to [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html). Users following these instructions should use [Studio Classic](studio.md).
Amazon SageMaker Autopilot is a feature set that simplifies and accelerates various stages of the machine learning workflow by automating the process of building and deploying machine learning models (AutoML). The following page explains key information about Amazon SageMaker Autopilot.
Autopilot performs the following key tasks that you can use on autopilot or with various degrees of human guidance:
+ **Data analysis and preprocessing:** Autopilot identifies your specific problem type, handles missing values, normalizes your data, selects features, and overall prepares the data for model training.
+ **Model selection:** Autopilot explores a variety of algorithms and uses a cross-validation resampling technique to generate metrics that evaluate the predictive quality of the algorithms based on predefined objective metrics.
+ **Hyperparameter optimization:** Autopilot automates the search for optimal hyperparameter configurations.
+ **Model training and evaluation:** Autopilot automates the process of training and evaluating various model candidates. It splits the data into training and validation sets, trains the selected model candidates using the training data, and evaluates their performance on the unseen data of the validation set. Lastly, it ranks the optimized model candidates based on their performance and identifies the best performing model.
+ **Model deployment:** Once Autopilot has identified the best performing model, it provides the option to deploy the model automatically by generating the model artifacts and the endpoint exposing an API. External applications can send data to the endpoint and receive the corresponding predictions or inferences.
Autopilot supports building machine learning models on large datasets up to hundreds of GBs.
The following diagram outlines the tasks of this AutoML process managed by Autopilot.
![\[Overview of Amazon SageMaker Autopilot AutoML process.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/Autopilot-process-graphic-1.png)
Depending on your comfort level with the machine learning process and coding experience, you can use Autopilot in different ways:
+ **Using the Studio Classic UI**, users can choose between a no-code experience or have some level of human input.
**Note**
Only experiments created from tabular data for problem types such as regression or classification are available via the Studio Classic UI.
+ **Using the AutoML API**, users with coding experience can use available SDKs to create AutoML jobs. This approach provides greater flexibility and customization options and is available for all problem types.
Autopilot currently supports the following problem types:
**Note**
For regression or classification problems involving tabular data, users can choose between two options: using the Studio Classic user interface or the [API Reference](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html).
Tasks such as text and image classification, time-series forecasting, and fine-tuning of large language models are exclusively available through the version 2 of the [AutoML REST API](autopilot-reference.md). If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_auto_ml_job_v2.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
Users who prefer the convenience of a user interface can use [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas-getting-started.html) to access pre-trained models and generative AI foundation models, or create custom models tailored for specific text, image classification, forecasting needs, or generative AI.
+ **Regression, binary, and multiclass classification** with tabular data formatted as CSV or Parquet files in which each column contains a feature with a specific data type and each row contains an observation. The column data types accepted include numerical, categorical, text, and time series that consists of strings of comma-separated numbers.
+ To create an Autopilot job as a pilot experiment using the SageMaker API reference, see [Create Regression or Classification Jobs for Tabular Data Using the AutoML API](autopilot-automate-model-development-create-experiment.md).
+ To create an Autopilot job as a pilot experiment using the Studio Classic UI, see [Create a Regression or Classification Autopilot experiment for tabular data using the Studio Classic UI](autopilot-automate-model-development-create-experiment-ui.md).
+ **Text classification** with data formatted as CSV or Parquet files in which a column provides the sentences to be classified, while another column should provide the corresponding class label. See [Create an AutoML job for text classification using the API](autopilot-create-experiment-text-classification.md).
+ **Image classification** with image formats such as PNG, JPEG, or a combination of both.See [Create an Image Classification Job using the AutoML API](autopilot-create-experiment-image-classification.md).
+ **Time-series forecasting** with time-series data formatted as CSV or Parquet files.See [Create an AutoML job for time-series forecasting using the API](autopilot-create-experiment-timeseries-forecasting.md).
+ Fine-tuning of large language models (LLMs) for **text generation** with data formatted as CSV or Parquet files.See [Create an AutoML job to fine-tune text generation models using the API](autopilot-create-experiment-finetune-llms.md).
Additionally, Autopilot helps users understand how models make predictions by automatically generating reports that show the importance of each individual feature. This provides transparency and insights into the factors influencing the predictions, which can be used by risk and compliance teams and external regulators. Autopilot also provides a model performance report, which encompasses a summary of evaluation metrics, a confusion matrix, various visualizations such as receiver operating characteristic curves and precision-recall curves, and more. The specific content of each report vary depending on the problem type of the Autopilot experiment.
The explainability and performance reports for the best model candidate in an Autopilot experiment are available for text, image, and tabular data classification problem types.
For tabular data use cases such as regression or classification, Autopilot offers additional visibility into how the data was wrangled and how the model candidates were selected, trained, and tuned by generating notebooks that contain the code used to explore the data and find the best performing model. These notebooks provide an interactive and exploratory environment to help you learn about the impact of various inputs or the trade-offs made in the experiments. You can experiment further with the higher performing model candidate by making your own modifications to the data exploration and candidate definition notebooks provided by Autopilot.
With Amazon SageMaker AI, you pay only for what you use. You pay for the underlying compute and storage resources within SageMaker AI or other Amazon services, based on your usage. For more information about the cost of using SageMaker AI, see [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing).
**Topics**
+ [
# Create Regression or Classification Jobs for Tabular Data Using the AutoML API
](autopilot-automate-model-development-create-experiment.md)
+ [
# Create an Image Classification Job using the AutoML API
](autopilot-create-experiment-image-classification.md)
+ [
# Create an AutoML job for text classification using the API
](autopilot-create-experiment-text-classification.md)
+ [
# Create an AutoML job for time-series forecasting using the API
](autopilot-create-experiment-timeseries-forecasting.md)
+ [
# Create an AutoML job to fine-tune text generation models using the API
](autopilot-create-experiment-finetune-llms.md)
+ [
# Create a Regression or Classification Autopilot experiment for tabular data using the Studio Classic UI
](autopilot-automate-model-development-create-experiment-ui.md)
+ [
# Amazon SageMaker Autopilot example notebooks
](autopilot-example-notebooks.md)
+ [
# Videos: Use Autopilot to automate and explore the machine learning process
](autopilot-videos.md)
+ [
# Autopilot quotas
](autopilot-quotas.md)
+ [
# API Reference guide for Autopilot
](autopilot-reference.md)
# Create Regression or Classification Jobs for Tabular Data Using the AutoML API
You can create an Autopilot regression or classification job for tabular data programmatically by calling the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) API action in any language supported by Autopilot or the Amazon CLI. The following is a collection of mandatory and optional input request parameters for the `CreateAutoMLJobV2` API action. You can find the alternative information for the previous version of this action, `CreateAutoMLJob`. However, we recommend using `CreateAutoMLJobV2`.
For information on how this API action translates into a function in the language of your choice, see the [ See Also](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_SeeAlso) section of `CreateAutoMLJobV2` and choose an SDK. As an example, for Python users, see the full request syntax of `[create\$1auto\$1ml\$1job\$1v2](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_auto_ml_job_v2)` in Amazon SDK for Python (Boto3).
**Note**
[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) and [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) are new versions of [CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html) and [DescribeAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) which offer backward compatibility.
We recommend using the `CreateAutoMLJobV2`. `CreateAutoMLJobV2` can manage tabular problem types identical to those of its previous version `CreateAutoMLJob`, as well as non-tabular problem types such as image or text classification, or time-series forecasting.
At a minimum, all experiments on tabular data require the specification of the experiment name, providing locations for the input and output data, and specifying which target data to predict. Optionally, you can also specify the type of problem that you want to solve (regression, classification, multiclass classification), choose your modeling strategy (*stacked ensembles* or *hyperparameters optimization*), select the list of algorithms used by the Autopilot job to train the data, and more.
After the experiment runs, you can compare trials and delve into the details of the pre-processing steps, algorithms, and hyperparameter ranges of each model. You also have the option to download their [explainability](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-explainability.html) and [performance](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-insights.html) reports. Use the provided [ notebooks](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-notebook-output.html ) to see the results of the automated data exploration or the candidate model definitions.
Find guidelines on how to migrate a `CreateAutoMLJob` to `CreateAutoMLJobV2` in [Migrate a CreateAutoMLJob to CreateAutoMLJobV2](#autopilot-create-experiment-api-migrate-v1-v2).
## Required parameters
------
#### [ CreateAutoMLJobV2 ]
When calling `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)` to create an Autopilot experiment for tabular data, you must provide the following values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax)` to specify the name of your job.
+ At least one `[AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)` in `[AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)` to specify your data source.
+ Both an `[AutoMLJobObjective](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobObjective)` metric and your chosen type of supervised learning problem (binary classification, multiclass classification, regression) in `AutoMLProblemTypeConfig`, or none at all. For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`. You set the supervised learning problem in the `ProblemType` attribute of `TabularJobConfig`.
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data.
------
#### [ CreateAutoMLJob ]
When calling `[CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html)` to create an AutoML experiment, you must provide the following four values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-AutoMLJobName)` to specify the name of your job.
+ At least one `[AutoMLChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html)` in `[InputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-InputDataConfig)` to specify your data source.
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data.
------
All other parameters are optional.
## Optional parameters
The following sections provide details of some optional parameters that you can pass to your `CreateAutoMLJobV2` API action when using tabular data. You can find the alternative information for the previous version of this action, `CreateAutoMLJob`. However, we recommend using `CreateAutoMLJobV2`.
### How to set the training mode of an AutoML job
For tabular data, the set of algorithms run on your data to train your model candidates is dependent on your modeling strategy (`ENSEMBLING` or `HYPERPARAMETER_TUNING`). The following details how to set this training mode.
If you keep blank (or `null`), the `Mode` is inferred based on the size of your dataset.
For information on Autopilot's *stacked ensembles* and *hyperparameters optimization* training methods, see [Training modes and algorithm support](autopilot-model-support-validation.md)
------
#### [ CreateAutoMLJobV2 ]
For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
You can set the [training method](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html) of an AutoML job V2 with the `[TabularJobConfig.Mode](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` parameter.
------
#### [ CreateAutoMLJob ]
You can set the [training method](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html) of an AutoML job with the `[AutoMLJobConfig.Mode](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html#sagemaker-Type-AutoMLJobConfig-Mode)` parameter.
------
### How to select features and algorithms for training an AutoML job
#### Features selection
Autopilot provides automatic data-preprocessing steps including feature selection and feature extraction. However, you can manually provide the features to be used in training with the `FeatureSpecificatioS3Uri` attribute.
Selected features should be contained within a JSON file in the following format:
```
{ "FeatureAttributeNames":["col1", "col2", ...] }
```
The values listed in `["col1", "col2", ...]` are case sensitive. They should be a list of strings containing unique values that are subsets of the column names in the input data.
**Note**
The list of columns provided as features cannot include the target column.
------
#### [ CreateAutoMLJobV2 ]
For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
You can set the URL to your selected features with the `[TabularJobConfig.FeatureSpecificationS3Uri](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` parameter.
------
#### [ CreateAutoMLJob ]
You can set the `FeatureSpecificatioS3Uri` attribute of [AutoMLCandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html) within the [CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html) API with the following format:
```
{
"[AutoMLJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-AutoMLJobConfig)": {
"[CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html#sagemaker-Type-AutoMLJobConfig-CandidateGenerationConfig)": {
"[FeatureSpecificationS3Uri](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html#sagemaker-Type-AutoMLCandidateGenerationConfig-FeatureSpecificationS3Uri)":"string"
},
}
}
```
------
#### Algorithms selection
By default, your Autopilot job runs a pre-defined list of algorithms on your dataset to train model candidates. The list of algorithms depends on the training mode (`ENSEMBLING` or `HYPERPARAMETER_TUNING`) used by the job.
You can provide a subset of the default selection of algorithms.
------
#### [ CreateAutoMLJobV2 ]
For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
You can specify an array of selected `AutoMLAlgorithms` in the `AlgorithmsConfig` attribute of [CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html).
The following is an example of an `AlgorithmsConfig` attribute listing exactly three algorithms ("xgboost", "fastai", "catboost") in its `AutoMLAlgorithms` field for the ensembling training mode.
```
{
"[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)": {
"[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)": {
"[Mode](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)": "ENSEMBLING",
"[CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html)": {
"[AlgorithmsConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html#sagemaker-Type-CandidateGenerationConfig-AlgorithmsConfig)":[
{"[AutoMLAlgorithms](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html)":["xgboost", "fastai", "catboost"]}
]
},
},
},
}
```
------
#### [ CreateAutoMLJob ]
You can specify an array of selected `AutoMLAlgorithms` in the `AlgorithmsConfig` attribute of [AutoMLCandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html).
The following is an example of an `AlgorithmsConfig` attribute listing exactly three algorithms ("xgboost", "fastai", "catboost") in its `AutoMLAlgorithms` field for the ensembling training mode.
```
{
"[AutoMLJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-AutoMLJobConfig)": {
"[CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html#sagemaker-Type-AutoMLJobConfig-CandidateGenerationConfig)": {
"[AlgorithmsConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html#sagemaker-Type-AutoMLCandidateGenerationConfig-AlgorithmsConfig)":[
{"[AutoMLAlgorithms](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html#sagemaker-Type-AutoMLAlgorithmConfig-AutoMLAlgorithms)":["xgboost", "fastai", "catboost"]}
]
},
"Mode": "ENSEMBLING"
}
```
------
For the list of available algorithms per training `Mode`, see [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html#sagemaker-Type-AutoMLAlgorithmConfig-AutoMLAlgorithms](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html#sagemaker-Type-AutoMLAlgorithmConfig-AutoMLAlgorithms). For details on each algorithm, see [Training modes and algorithm support](autopilot-model-support-validation.md).
### How to specify the training and validation datasets of an AutoML job
You can provide your own validation dataset and custom data split ratio, or let Autopilot split the dataset automatically.
------
#### [ CreateAutoMLJobV2 ]
Each [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html) object (see the required parameter [AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker-api/src/AWSSageMakerAPIDoc/build/server-root/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)) has a `ChannelType`, which can be set to either `training` or `validation` values that specify how the data is to be used when building a machine learning model. At least one data source must be provided and a maximum of two data sources is allowed: one for training data and one for validation data.
How you split the data into training and validation datasets depends on whether you have one or two data sources.
+ If you only have **one data source**, the `ChannelType` is set to `training` by default and must have this value.
+ If the `ValidationFraction` value in [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html) is not set, 0.2 (20%) of the data from this source is used for validation by default.
+ If the `ValidationFraction` is set to a value between 0 and 1, the dataset is split based on the value specified, where the value specifies the fraction of the dataset used for validation.
+ If you have **two data sources**, the `ChannelType` of one of the `AutoMLJobChannel` objects must be set to `training`, the default value. The `ChannelType` of the other data source must be set to `validation`. The two data sources must have the same format, either CSV or Parquet, and the same schema. You must not set the value for the `ValidationFraction` in this case because all of the data from each source is used for either training or validation. Setting this value causes an error.
------
#### [ CreateAutoMLJob ]
Each [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html) object (see the required parameter [InputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-InputDataConfig)) has a `ChannelType`, which can be set to either `training` or `validation` values that specify how the data is to be used when building a machine learning model. At least one data source must be provided and a maximum of two data sources is allowed: one for training data and one for validation data.
How you split the data into training and validation datasets depends on whether you have one or two data sources.
+ If you only have **one data source**, the `ChannelType` is set to `training` by default and must have this value.
+ If the `ValidationFraction` value in [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html) is not set, 0.2 (20%) of the data from this source is used for validation by default.
+ If the `ValidationFraction` is set to a value between 0 and 1, the dataset is split based on the value specified, where the value specifies the fraction of the dataset used for validation.
+ If you have **two data sources**, the `ChannelType` of one of the `AutoMLChannel` objects must be set to `training`, the default value. The `ChannelType` of the other data source must be set to `validation`. The two data sources must have the same format, either CSV or Parquet, and the same schema. You must not set the value for the `ValidationFraction` in this case because all of the data from each source is used for either training or validation. Setting this value causes an error.
------
For information on split and cross-validation in Autopilot see [Cross-validation in Autopilot](autopilot-metrics-validation.md#autopilot-cross-validation).
### How to set the problem type of an AutoML job
------
#### [ CreateAutoMLJobV2 ]
For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
You can further specify the type of supervised learning problem (binary classification, multiclass classification, regression) available for the model candidates of your AutoML job V2 with the `[TabularJobConfig.ProblemType](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` parameter.
------
#### [ CreateAutoMLJob ]
You can set the [type of problem](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-datasets-problem-types.html#autopilot-problem-types) on an AutoML job with the `[CreateAutoPilot.ProblemType](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-ProblemType)` parameter. This limits the kind of preprocessing and algorithms that Autopilot tries. After the job is finished, if you had set the `[CreateAutoPilot.ProblemType](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-ProblemType)`, then the `[ResolvedAttribute.ProblemType](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ResolvedAttributes.html)` matches the `ProblemType` you set. If you keep it blank (or `null`), the `ProblemType` is inferred on your behalf.
------
**Note**
In some cases, Autopilot is unable to infer the `ProblemType` with high enough confidence, in which case you must provide the value for the job to succeed.
### How to add sample weights to an AutoML job
You can add a sample weights column to your tabular dataset and then pass it to your AutoML job to request dataset rows to be weighted during training and evaluation.
Support for sample weights is available in [ensembling mode](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html#autopilot-training-mode) only. Your weights should be numeric and non-negative. Data points with invalid or no weight value are excluded. For more information on the available objective metrics, see [Autopilot weighted metrics](autopilot-metrics-validation.md#autopilot-weighted-metrics).
------
#### [ CreateAutoMLJobV2 ]
For tabular data, you must choose `[TabularJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
To set sample weights when creating an experiment (see [CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)), you can pass the name of your sample weights column in the `SampleWeightAttributeName` attribute of the `TabularJobConfig` object. This ensures that your objective metric uses the weights for the training, evaluation, and selection of model candidates.
------
#### [ CreateAutoMLJob ]
To set sample weights when creating an experiment (see [CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html)), you can pass the name of your sample weights column in the `SampleWeightAttributeName` attribute of the [AutoMLChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html) object. This ensures that your objective metric uses the weights for the training, evaluation, and selection of model candidates.
------
### How to configure AutoML to initiate a remote job on EMR Serverless for large datasets
You can configure your AutoML job V2 to automatically initiate a remote job on Amazon EMR Serverless when additional compute resources are needed to process large datasets. By seamlessly transitioning to EMR Serverless when required, the AutoML job can handle datasets that would otherwise exceed the initially provisioned resources, without any manual intervention from you. EMR Serverless is available for the tabular and time series problem types. We recommend setting up this option for tabular datasets larger than 5 GB.
To allow your AutoML job V2 to automatically transition to EMR Serverless for large dataset, you need to provide an `EmrServerlessComputeConfig` object, which includes an `ExecutionRoleARN` field, to the `AutoMLComputeConfig` of the AutoML job V2 input request.
The `ExecutionRoleARN` is the ARN of the IAM role granting the AutoML job V2 the necessary permissions to run EMR Serverless jobs.
This role should have the following trust relationship:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "emr-serverless.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
And grant the permissions to:
+ Create, list, and update EMR Serverless applications.
+ Start, list, get, or cancel job runs on an EMR Serverless application.
+ Tag EMR Serverless resources.
+ Pass an IAM role to the EMR Serverless service for execution.
By granting the `iam:PassRole` permission, the AutoML job V2 can temporarily assume the `EMRServerlessRuntimeRole-*` role and pass it to the EMR Serverless service. These are the IAM roles used by the EMR Serverless job execution environments to access other Amazon services and resources needed during runtime, such as Amazon S3 for data access, CloudWatch for logging, access to the Amazon Glue Data Catalog or other services based on your workload requirements.
See [Job runtime roles for Amazon EMR Serverless](https://docs.amazonaws.cn/emr/latest/EMR-Serverless-UserGuide/security-iam-runtime-role.html) for details on this role permissions.
The IAM policy defined in the provided JSON document grants those permissions:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [{
"Sid": "EMRServerlessCreateApplicationOperation",
"Effect": "Allow",
"Action": "emr-serverless:CreateApplication",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessListApplicationOperation",
"Effect": "Allow",
"Action": "emr-serverless:ListApplications",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessApplicationOperations",
"Effect": "Allow",
"Action": [
"emr-serverless:UpdateApplication",
"emr-serverless:GetApplication"
],
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessStartJobRunOperation",
"Effect": "Allow",
"Action": "emr-serverless:StartJobRun",
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessListJobRunOperation",
"Effect": "Allow",
"Action": "emr-serverless:ListJobRuns",
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessJobRunOperations",
"Effect": "Allow",
"Action": [
"emr-serverless:GetJobRun",
"emr-serverless:CancelJobRun"
],
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*/jobruns/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessTagResourceOperation",
"Effect": "Allow",
"Action": "emr-serverless:TagResource",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "IAMPassOperationForEMRServerless",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws-cn:iam::*:role/EMRServerlessRuntimeRole-*",
"Condition": {
"StringEquals": {
"iam:PassedToService": "emr-serverless.amazonaws.com",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
}
]
}
```
------
## Migrate a CreateAutoMLJob to CreateAutoMLJobV2
We recommend users of `CreateAutoMLJob` to migrate to `CreateAutoMLJobV2`.
This section explains the differences in the input parameters between [CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker-api/src/AWSSageMakerAPIDoc/build/server-root/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#API_CreateAutoMLJob_RequestSyntax) and [CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax) by highlighting the changes in the position, name, or structure of the objects and attributes of the input request between the two versions.
+ **Request attributes that did not change between versions.**
```
{
"AutoMLJobName": "string",
"AutoMLJobObjective": {
"MetricName": "string"
},
"ModelDeployConfig": {
"AutoGenerateEndpointName": boolean,
"EndpointName": "string"
},
"OutputDataConfig": {
"KmsKeyId": "string",
"S3OutputPath": "string"
},
"RoleArn": "string",
"Tags": [
{
"Key": "string",
"Value": "string"
}
]
}
```
+ **Request attributes that changed position and structure between versions.**
The following attributes changed position: `DataSplitConfig`, `Security Config`, `CompletionCriteria`, `Mode`, `FeatureSpecificationS3Uri`, `SampleWeightAttributeName`, `TargetAttributeName`.
------
#### [ CreateAutoMLJob ]
```
{
"AutoMLJobConfig": {
"Mode": "string",
"CompletionCriteria": {
"MaxAutoMLJobRuntimeInSeconds": number,
"MaxCandidates": number,
"MaxRuntimePerTrainingJobInSeconds": number
},
"DataSplitConfig": {
"ValidationFraction": number
},
"SecurityConfig": {
"EnableInterContainerTrafficEncryption": boolean,
"VolumeKmsKeyId": "string",
"VpcConfig": {
"SecurityGroupIds": [ "string" ],
"Subnets": [ "string" ]
}
},
"CandidateGenerationConfig": {
"FeatureSpecificationS3Uri": "string"
}
},
"GenerateCandidateDefinitionsOnly": boolean,
"ProblemType": "string"
}
```
------
#### [ CreateAutoMLJobV2 ]
```
{
"AutoMLProblemTypeConfig": {
"TabularJobConfig": {
"Mode": "string",
"ProblemType": "string",
"GenerateCandidateDefinitionsOnly": boolean,
"CompletionCriteria": {
"MaxAutoMLJobRuntimeInSeconds": number,
"MaxCandidates": number,
"MaxRuntimePerTrainingJobInSeconds": number
},
"FeatureSpecificationS3Uri": "string",
"SampleWeightAttributeName": "string",
"TargetAttributeName": "string"
}
},
"DataSplitConfig": {
"ValidationFraction": number
},
"SecurityConfig": {
"EnableInterContainerTrafficEncryption": boolean,
"VolumeKmsKeyId": "string",
"VpcConfig": {
"SecurityGroupIds": [ "string" ],
"Subnets": [ "string" ]
}
}
}
```
------
+ **The following attributes changed position and structure between versions.**
The following JSON illustrates how [AutoMLJobConfig.CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html#sagemaker-Type-AutoMLJobConfig-CandidateGenerationConfig) of type [AutoMLCandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html) moved to [AutoMLProblemTypeConfig.TabularJobConfig.CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax) of type [CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html) in V2.
------
#### [ CreateAutoMLJob ]
```
{
"AutoMLJobConfig": {
"CandidateGenerationConfig": {
"AlgorithmsConfig": [
{
"AutoMLAlgorithms": [ "string" ]
}
],
"FeatureSpecificationS3Uri": "string"
}
}
```
------
#### [ CreateAutoMLJobV2 ]
```
{
"AutoMLProblemTypeConfig": {
"TabularJobConfig": {
"CandidateGenerationConfig": {
"AlgorithmsConfig": [
{
"AutoMLAlgorithms": [ "string" ]
}
],
},
}
},
}
```
------
+ **Request attributes that changed name and structure.**
The following JSON illustrates how [InputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-InputDataConfig) (An array of [AutoMLChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html)) changed to [AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig) (An array of [AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)) in V2. Note that the attributes `SampleWeightAttributeName` and `TargetAttributeName` move out of `InputDataConfig` and into `AutoMLProblemTypeConfig`.
------
#### [ CreateAutoMLJob ]
```
{
"InputDataConfig": [
{
"ChannelType": "string",
"CompressionType": "string",
"ContentType": "string",
"DataSource": {
"S3DataSource": {
"S3DataType": "string",
"S3Uri": "string"
}
},
"SampleWeightAttributeName": "string",
"TargetAttributeName": "string"
}
]
}
```
------
#### [ CreateAutoMLJobV2 ]
```
{
"AutoMLJobInputDataConfig": [
{
"ChannelType": "string",
"CompressionType": "string",
"ContentType": "string",
"DataSource": {
"S3DataSource": {
"S3DataType": "string",
"S3Uri": "string"
}
}
}
]
}
```
------
# Autopilot datasets and problem types
For tabular data (that is data in which each column contains a feature with a specific data type and each row contains an observation), Autopilot gives you the option of specifying the type of supervised learning problem available for the model candidates of the AutoML job, such as binary classification or regression, or of detecting it on your behalf based on the data you provide. Autopilot also supports multiple data formats and data types.
**Topics**
+ [
## Autopilot datasets, data types, and formats
](#autopilot-datasets)
+ [
## Autopilot problem types
](#autopilot-problem-types)
## Autopilot datasets, data types, and formats
Autopilot supports tabular data formatted as CSV files or as Parquet files: each column contains a feature with a specific data type and each row contains an observation. The properties of these two file formats differ considerably.
+ **CSV** (comma-separated-values) is a row-based file format that stores data in human readable plaintext which a popular choice for data exchange as they are supported by a wide range of applications.
+ **Parquet** is a column-based file format where the data is stored and processed more efficiently than row-based file formats. This makes them a better option for big data problems.
The **data types** accepted for columns include numerical, categorical, text, and time series that consists of strings of comma-separated numbers. If Autopilot detects it is dealing with** time series** sequences, it processes them through specialized feature transformers provided by the [tsfresh](https://tsfresh.readthedocs.io/en/latest/text/list_of_features.html) library. This library takes the time series as an input and outputs a feature such as the highest absolute value of the time series or descriptive statistics on autocorrelation. These outputted features are then used as inputs to one of the three problem types.
Autopilot supports building machine learning models on large datasets up to hundreds of GBs. For details on the default resource limits for input datasets and how to increase them, see [Autopilot quotas](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-quotas.html).
## Autopilot problem types
For the tabular data, you further specify the type of supervised learning problems available for the model candidates as follows:
### Regression
Regression estimates the values of a dependent target variable based on one or more other variables or attributes that are correlated with it. An example is the prediction of house prices using features like the number of bathrooms and bedrooms, square footage of the house and garden. Regression analysis can create a model that takes one or more of these features as an input and predicts the price of a house.
### Binary classification
Binary classification is a type of supervised learning that assigns an individual to one of two predefined and mutually exclusive classes based on their attributes. It is supervised because the models are trained using examples where the attributes are provided with correctly labeled objects. A medical diagnosis for whether an individual has a disease or not based on the results of diagnostic tests is an example of binary classification.
### Multiclass classification
Multiclass classification is a type of supervised learning that assigns an individual to one of several classes based on their attributes. It is supervised because the models are trained using examples where the attributes are provided with correctly labelled objects. An example is the prediction of the topic most relevant to a text document. A document may be classified as being about, say, religion or politics or finance, or about one of several other predefined topic classes.
# Training modes and algorithm support
Autopilot supports different training modes and algorithms to address machine learning problems, report on quality and objective metrics, and to use cross-validation automatically, when needed.
## Training modes
SageMaker Autopilot can automatically select the training method based on the dataset size, or you can select it manually. The choices are as follows:
+ **Ensembling** – Autopilot uses the [AutoGluon](https://auto.gluon.ai/scoredebugweight/tutorials/tabular_prediction/index.html) library to train several base models. To find the best combination for your dataset, ensemble mode runs 10 trials with different model and meta parameter settings. Then Autopilot combines these models using a stacking ensemble method to create an optimal predictive model. For a list of algorithms that Autopilot supports in ensembling mode for tabular data, see the following **Algorithms support** section.
+ **Hyperparameter optimization (HPO)** – Autopilot finds the best version of a model by tuning hyperparameters using Bayesian optimization or multi-fidelity optimization while running training jobs on your dataset. HPO mode selects the algorithms that are most relevant to your dataset and selects the best range of hyperparameters to tune your models. To tune your models, HPO mode runs up to 100 trials (default) to find the optimal hyperparameters settings within the selected range. If your dataset size is less than 100 MB, Autopilot uses Bayesian optimization. Autopilot chooses multi-fidelity optimization if your dataset is larger than 100 MB.
In multi-fidelity optimization, metrics are continuously emitted from the training containers. A trial that is performing poorly against a selected objective metric is stopped early. A trial that is performing well is allocated more resources.
For a list of algorithms that Autopilot supports in HPO mode, see the following **Algorithm support** section.
+ **Auto** – Autopilot automatically chooses either ensembling mode or HPO mode based on your dataset size. If your dataset is larger than 100 MB, Autopilot chooses HPO. Otherwise, it chooses ensembling mode. Autopilot can fail to read the size of your dataset in the following cases.
+ If you enable Virtual Private Cloud (VPC) mode, for an AutoML job but the S3 bucket containing the dataset only allows access from the VPC.
+ The input [S3DataType](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLS3DataSource.html#sagemaker-Type-AutoMLS3DataSource-S3DataType) of your dataset is a `ManifestFile`.
+ The input [S3Uri](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLS3DataSource.html#sagemaker-Type-AutoMLS3DataSource-S3Uri) contains more than 1000 items.
If Autopilot is unable to read your dataset size, it defaults to choosing HPO mode.
**Note**
For optimal runtime and performance, use ensemble training mode for datasets that are smaller than 100 MB.
## Algorithms support
In **HPO mode**, Autopilot supports the following types of machine learning algorithms:
+ [Linear learner](https://docs.amazonaws.cn/sagemaker/latest/dg/linear-learner.html) – A supervised learning algorithm that can solve either classification or regression problems.
+ [XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html) – A supervised learning algorithm that attempts to accurately predict a target variable by combining an ensemble of estimates from a set of simpler and weaker models.
+ Deep learning algorithm – A multilayer perceptron (MLP) and feedforward artificial neural network. This algorithm can handle data that is not linearly separable.
**Note**
You don't need to specify an algorithm to use for your machine learning problem. Autopilot automatically selects the appropriate algorithm to train.
In **ensembling mode**, Autopilot supports the following types of machine learning algorithms:
+ [LightGBM](https://docs.amazonaws.cn/sagemaker/latest/dg/lightgbm.html) – An optimized framework that uses tree-based algorithms with gradient boosting. This algorithm uses trees that grow in breadth, rather than depth, and is highly optimized for speed.
+ [CatBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/catboost.html) – A framework that uses tree-based algorithms with gradient boosting. Optimized for handling categorical variables.
+ [XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html) – A framework that uses tree-based algorithms with gradient boosting that grows in depth, rather than breadth.
+ [Random Forest](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) – A tree-based algorithm that uses several decision trees on random sub-samples of the data with replacement. The trees are split into optimal nodes at each level. The decisions of each tree are averaged together to prevent overfitting and improve predictions.
+ [Extra Trees](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html#sklearn.ensemble.ExtraTreesClassifier) – A tree-based algorithm that uses several decision trees on the entire dataset. The trees are split randomly at each level. The decisions of each tree are averaged to prevent overfitting and to improve predictions. Extra trees add a degree of randomization in comparison to the random forest algorithm.
+ [Linear Models](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model) – A framework that uses a linear equation to model the relationship between two variables in observed data.
+ Neural network torch – A neural network model that's implemented using [Pytorch](https://pytorch.org/).
+ Neural network fast.ai – A neural network model that's implemented using [fast.ai](https://www.fast.ai/).
# Metrics and validation
This guide shows metrics and validation techniques that you can use to measure machine learning model performance. Amazon SageMaker Autopilot produces metrics that measure the predictive quality of machine learning model candidates. The metrics calculated for candidates are specified using an array of [MetricDatum](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_MetricDatum.html) types.
## Autopilot metrics
The following list contains the names of the metrics that are currently available to measure model performance within Autopilot.
**Note**
Autopilot supports sample weights. To learn more about sample weights and the available objective metrics, see [Autopilot weighted metrics](#autopilot-weighted-metrics).
The following are the available metrics.
**`Accuracy`**
The ratio of the number of correctly classified items to the total number of (correctly and incorrectly) classified items. It is used for both binary and multiclass classification. Accuracy measures how close the predicted class values are to the actual values. Values for accuracy metrics vary between zero (0) and one (1). A value of 1 indicates perfect accuracy, and 0 indicates perfect inaccuracy.
**`AUC`**
The area under the curve (AUC) metric is used to compare and evaluate binary classification by algorithms that return probabilities, such as logistic regression. To map the probabilities into classifications, these are compared against a threshold value.
The relevant curve is the receiver operating characteristic curve. The curve plots the true positive rate (TPR) of predictions (or recall) against the false positive rate (FPR) as a function of the threshold value, above which a prediction is considered positive. Increasing the threshold results in fewer false positives, but more false negatives.
AUC is the area under this receiver operating characteristic curve. Therefore, AUC provides an aggregated measure of the model performance across all possible classification thresholds. AUC scores vary between 0 and 1. A score of 1 indicates perfect accuracy, and a score of one half (0.5) indicates that the prediction is not better than a random classifier.
**`BalancedAccuracy`**
`BalancedAccuracy` is a metric that measures the ratio of accurate predictions to all predictions. This ratio is calculated after normalizing true positives (TP) and true negatives (TN) by the total number of positive (P) and negative (N) values. It is used in both binary and multiclass classification and is defined as follows: 0.5\$1((TP/P)\$1(TN/N)), with values ranging from 0 to 1. `BalancedAccuracy` gives a better measure of accuracy when the number of positives or negatives differ greatly from each other in an imbalanced dataset, such as when only 1% of email is spam.
**`F1`**
The `F1` score is the harmonic mean of the precision and recall, defined as follows: F1 = 2 \$1 (precision \$1 recall) / (precision \$1 recall). It is used for binary classification into classes traditionally referred to as positive and negative. Predictions are said to be true when they match their actual (correct) class, and false when they do not.
Precision is the ratio of the true positive predictions to all positive predictions, and it includes the false positives in a dataset. Precision measures the quality of the prediction when it predicts the positive class.
Recall (or sensitivity) is the ratio of the true positive predictions to all actual positive instances. Recall measures how completely a model predicts the actual class members in a dataset.
F1 scores vary between 0 and 1. A score of 1 indicates the best possible performance, and 0 indicates the worst.
**`F1macro`**
The `F1macro` score applies F1 scoring to multiclass classification problems. It does this by calculating the precision and recall, and then taking their harmonic mean to calculate the F1 score for each class. Lastly, the `F1macro` averages the individual scores to obtain the `F1macro` score. `F1macro` scores vary between 0 and 1. A score of 1 indicates the best possible performance, and 0 indicates the worst.
**`InferenceLatency`**
Inference latency is the approximate amount of time between making a request for a model prediction to receiving it from a real time endpoint to which the model is deployed. This metric is measured in seconds and only available in ensembling mode.
**`LogLoss`**
Log loss, also known as cross-entropy loss, is a metric used to evaluate the quality of the probability outputs, rather than the outputs themselves. It is used in both binary and multiclass classification and in neural nets. It is also the cost function for logistic regression. Log loss is an important metric to indicate when a model makes incorrect predictions with high probabilities. Values range from 0 to infinity. A value of 0 represents a model that perfectly predicts the data.
**`MAE`**
The mean absolute error (MAE) is a measure of how different the predicted and actual values are, when they're averaged over all values. MAE is commonly used in regression analysis to understand model prediction error. If there is linear regression, MAE represents the average distance from a predicted line to the actual value. MAE is defined as the sum of absolute errors divided by the number of observations. Values range from 0 to infinity, with smaller numbers indicating a better model fit to the data.
**`MSE`**
The mean squared error (MSE) is the average of the squared differences between the predicted and actual values. It is used for regression. MSE values are always positive. The better a model is at predicting the actual values, the smaller the MSE value is.
**`Precision`**
Precision measures how well an algorithm predicts the true positives (TP) out of all of the positives that it identifies. It is defined as follows: Precision = TP/(TP\$1FP), with values ranging from zero (0) to one (1), and is used in binary classification. Precision is an important metric when the cost of a false positive is high. For example, the cost of a false positive is very high if an airplane safety system is falsely deemed safe to fly. A false positive (FP) reflects a positive prediction that is actually negative in the data.
**`PrecisionMacro`**
The precision macro computes precision for multiclass classification problems. It does this by calculating precision for each class and averaging scores to obtain precision for several classes. `PrecisionMacro` scores range from zero (0) to one (1). Higher scores reflect the model's ability to predict true positives (TP) out of all of the positives that it identifies, averaged across multiple classes.
**`R2`**
R2, also known as the coefficient of determination, is used in regression to quantify how much a model can explain the variance of a dependent variable. Values range from one (1) to negative one (-1). Higher numbers indicate a higher fraction of explained variability. `R2` values close to zero (0) indicate that very little of the dependent variable can be explained by the model. Negative values indicate a poor fit and that the model is outperformed by a constant function. For linear regression, this is a horizontal line.
**`Recall`**
Recall measures how well an algorithm correctly predicts all of the true positives (TP) in a dataset. A true positive is a positive prediction that is also an actual positive value in the data. Recall is defined as follows: Recall = TP/(TP\$1FN), with values ranging from 0 to 1. Higher scores reflect a better ability of the model to predict true positives (TP) in the data. It is used in binary classification.
Recall is important when testing for cancer because it's used to find all of the true positives. A false negative (FN) reflects a negative prediction that is actually positive in the data. It is often insufficient to measure only recall, because predicting every output as a true positive yields a perfect recall score.
**`RecallMacro`**
The `RecallMacro` computes recall for multiclass classification problems by calculating recall for each class and averaging scores to obtain recall for several classes. `RecallMacro` scores range from 0 to 1. Higher scores reflect the model's ability to predict true positives (TP) in a dataset, whereas a true positive reflects a positive prediction that is also an actual positive value in the data. It is often insufficient to measure only recall, because predicting every output as a true positive will yield a perfect recall score.
**`RMSE`**
Root mean squared error (RMSE) measures the square root of the squared difference between predicted and actual values, and is averaged over all values. It is used in regression analysis to understand model prediction error. It's an important metric to indicate the presence of large model errors and outliers. Values range from zero (0) to infinity, with smaller numbers indicating a better model fit to the data. RMSE is dependent on scale, and should not be used to compare datasets of different sizes.
Metrics that are automatically calculated for a model candidate are determined by the type of problem being addressed.
Refer to the [ Amazon SageMaker API reference documentation](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobObjective.html) for the list of available metrics supported by Autopilot.
## Autopilot weighted metrics
**Note**
Autopilot supports sample weights in ensembling mode only for all [available metrics](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-metrics-validation.html#autopilot-metrics) with the exception of `Balanced Accuracy` and `InferenceLatency`. `BalanceAccuracy` comes with its own weighting scheme for imbalanced datasets that does not require sample weights. `InferenceLatency` does not support sample weights. Both objective `Balanced Accuracy` and `InferenceLatency` metrics ignore any existing sample weights when training and evaluating a model.
Users can add a sample weights column to their data to ensure that each observation used to train a machine learning model is given a weight corresponding to its perceived importance to the model. This is especially useful in scenarios in which the observations in the dataset have varying degrees of importance, or in which a dataset contains a disproportionate number of samples from one class compared to others. Assigning a weight to each observation based on its importance or greater importance to a minority class can help a model’s overall performance, or ensure that a model is not biased toward the majority class.
For information about how to pass sample weights when creating an experiment in the Studio Classic UI, see *Step 7* in [Create an Autopilot experiment using Studio Classic](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-create-experiment.html).
For information about how to pass sample weights programmatically when creating an Autopilot experiment using the API, see *How to add sample weights to an AutoML job* in [Create an Autopilot experiment programmatically](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-create-experiment.html).
## Cross-validation in Autopilot
Cross-validation is used in to reduce overfitting and bias in model selection. It is also used to assess how well a model can predict the values of an unseen validation dataset, if the validation dataset is drawn from the same population. This method is especially important when training on datasets that have a limited number of training instances.
Autopilot uses cross-validation to build models in hyperparameter optimization (HPO) and ensemble training mode. The first step in the Autopilot cross-validation process is to split the data into k-folds.
### K-fold splitting
K-fold splitting is a method that separates an input training dataset into multiple training and validation datasets. The dataset is split into `k` equally-sized sub-samples called folds. Models are then trained on `k-1` folds and tested against the remaining kth fold, which is the validation dataset. The process is repeated `k` times using a different data set for validation.
The following image depicts k-fold splitting with k = 4 folds. Each fold is represented as a row. The dark-toned boxes represent the parts of the data used in training. The remaining light-toned boxes indicate the validation datasets.
![\[K-fold splitting with 4-folds depicted as boxes: dark for data used; light for validation datasets.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-metrics-kfold-splits.png)
Autopilot uses k-fold cross-validation for both hyperparameter optimization (HPO) mode and ensembling mode.
You can deploy Autopilot models that are built using cross-validation like you would with any other Autopilot or SageMaker AI model.
### HPO mode
K-fold cross-validation uses the k-fold splitting method for cross-validation. In HPO mode, Autopilot automatically implements k-fold cross-validation for small datasets with 50,000 or fewer training instances. Performing cross-validation is especially important when training on small datasets because it protects against overfitting and selection bias.
HPO mode uses a *k* value of 5 on each of the candidate algorithms that are used to model the dataset. Multiple models are trained on different splits, and the models are stored separately. When training is complete, validation metrics for each of the models are averaged to produce a single estimation metric. Lastly, Autopilot combines the models from the trial with the best validation metric into an ensemble model. Autopilot uses this ensemble model to make predictions.
The validation metric for the models trained by Autopilot is presented as the objective metric in the model leaderboard. Autopilot uses the default validation metric for each problem type that it handles, unless you specify otherwise. For the list of all metrics that Autopilot uses, see [Autopilot metrics](#autopilot-metrics).
For example, the [Boston Housing dataset](http://lib.stat.cmu.edu/datasets/boston) contains only 861 samples. If you build a model to predict house sale prices using this dataset without cross-validation, you risk training on a dataset that is not representative of the Boston housing stock. If you split the data only once into training and validation subsets, the training fold may only contain data mainly from the suburbs. As a result, you would train on data that isn't representative of the rest of the city. In this example, your model would likely overfit on this biased selection. K-fold cross-validation can reduce the risk of this kind of error by making full and randomized use of the available data for both training and validation.
Cross-validation can increase training times by an average of 20%. Training times may also increase significantly for complex datasets.
**Note**
In HPO mode, you can see the training and validation metrics from each fold in your `/aws/sagemaker/TrainingJobs` CloudWatch Logs. For more information about CloudWatch Logs, see [CloudWatch Logs for Amazon SageMaker AI](logging-cloudwatch.md).
### Ensembling mode
**Note**
Autopilot supports sample weights in ensembling mode. For the list of available metrics supporting sample weights, see [Autopilot metrics](#autopilot-metrics).
In ensembling mode, cross-validation is performed regardless of dataset size. Customers can either provide their own validation dataset and custom data split ratio, or let Autopilot split the dataset automatically into an 80-20% split ratio. The training data is then split into `k`-folds for cross-validation, where the value of `k` is determined by the AutoGluon engine. An ensemble consists of multiple machine learning models, where each model is known as the base model. A single base model is trained on (`k`-1) folds and makes out-of-fold predictions on the remaining fold. This process is repeated for all `k` folds, and the out-of-fold (OOF) predictions are concatenated to form a single set of predictions. All base models in the ensemble follow this same process of generating OOF predictions.
The following image depicts k-fold validation with `k` = 4 folds. Each fold is represented as a row. The dark-toned boxes represent the parts of the data used in training. The remaining light-toned boxes indicate the validation datasets.
In the upper part of the image, in each fold, the first base model makes predictions on the validation dataset after training on the training datasets. At each subsequent fold, the datasets change roles. A dataset that was previously used for training is now used for validation, and this also applies in reverse. At the end of `k` folds, all of the predictions are concatenated to form a single set of predictions called an out-of-fold (OOF) prediction. This process is repeated for each `n` base models.
![\[k-fold validation: Four rows of boxes depict 4-folds that generate a row of OOF predictions.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-metrics-kfold.PNG)
The OOF predictions for each base model are then used as features to train a stacking model. The stacking model learns the importance weights for each base model. These weights are used to combine the OOF predictions to form the final prediction. Performance on the validation dataset determines which base or stacking model is the best, and this model is returned as the final model.
In ensemble mode, you can either provide your own validation dataset or let Autopilot split the input dataset automatically into 80% train and 20% validation datasets. The training data is then split into `k`-folds for cross-validation and produces an OOF prediction and a base model for each fold.
These OOF predictions are used as features to train a stacking model, which simultaneously learns weights for each base model. These weights are used to combine the OOF predictions to form the final prediction. The validation datasets for each fold are used for hyperparameter tuning of all base models and the stacking model. Performance on the validation datasets determines which base or stacking model is the best model, and this model is returned as the final model.
# Autopilot model deployment and prediction
This Amazon SageMaker Autopilot guide includes steps for model deployment, setting up real-time inference, and running inference with batch jobs.
After you train your Autopilot models, you can deploy them to get predictions in one of two ways:
1. Use [Deploy models for real-time inference](autopilot-deploy-models-realtime.md) to set up an endpoint and obtain predictions interactively. Real-time inference is ideal for inference workloads where you have real-time, interactive, low latency requirements.
1. Use [Run batch inference jobs](autopilot-deploy-models-batch.md) to make predictions in parallel on batches of observations on an entire dataset. Batch inference is a good option for large datasets or if you don't need an immediate response to a model prediction request.
**Note**
To avoid incurring unnecessary charges: After the endpoints and resources that were created from model deployment are no longer needed, you can delete them. For information about pricing of instances by Region, see [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing/).
# Deploy models for real-time inference
Real-time inference is ideal for inference workloads where you have real-time, interactive, low latency requirements. This section shows how you can use real-time inferencing to obtain predictions interactively from your model.
To deploy the model that produced the best validation metric in an Autopilot experiment, you have several options. For example, when using Autopilot in SageMaker Studio Classic, you can deploy the model automatically or manually. You can also use SageMaker APIs to manually deploy an Autopilot model.
The following tabs show three options for deploying your model. These instructions assume that you have already created a model in Autopilot. If you don't have a model, see [Create Regression or Classification Jobs for Tabular Data Using the AutoML API](autopilot-automate-model-development-create-experiment.md). To see examples for each option, open each tab.
## Deploy using the Autopilot User Interface (UI)
The Autopilot UI contains helpful dropdown menus, toggles, tooltips, and more to help you navigate through model deployment. You can deploy using either one of the following procedures: Automatic or Manual.
+ **Automatic Deployment**: To automatically deploy the best model from an Autopilot experiment to an endpoint
1. [Create an experiment](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-create-experiment.html) in SageMaker Studio Classic.
1. Toggle the **Auto deploy** value to **Yes**.
**Note**
**Automatic deployment will fail if either the default resource quota or your customer quota for endpoint instances in a Region is too limited.** In hyperparameter optimization (HPO) mode, you are required to have at least two ml.m5.2xlarge instances. In ensembling mode, you are required to have at least one ml.m5.12xlarge instance. If you encounter a failure related to quotas, you can [request a service limit increase](https://docs.amazonaws.cn/servicequotas/latest/userguide/request-quota-increase.html) for SageMaker AI endpoint instances.
+ **Manual Deployment**: To manually deploy the best model from an Autopilot experiment to an endpoint
1. [Create an experiment](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-create-experiment.html) in SageMaker Studio Classic.
1. Toggle the **Auto deploy** value to **No**.
1. Select the model that you want to deploy under **Model name**.
1. Select the orange **Deployment and advanced settings** button located on the right of the leaderboard. This opens a new tab.
1. Configure the endpoint name, instance type, and other optional information.
1. Select the orange **Deploy model** to deploy to an endpoint.
1. Check the progress of the endpoint creation process in the [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/) by navigating to the Endpoints section. That section is located in the **Inference** dropdown menu in the navigation panel.
1. After the endpoint status changes from **Creating** to **InService**, as shown below, return to Studio Classic and invoke the endpoint.
![\[SageMaker AI console: Endpoints page to create an endpoint or check endpoint status.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-check-progress.PNG)
## Deploy using SageMaker APIs
You can also obtain real-time inference by deploying your model using **API calls**. This section shows the five steps of this process using Amazon Command Line Interface (Amazon CLI) code snippets.
For complete code examples for both Amazon CLI commands and Amazon SDK for Python (boto3), open the tabs directly following these steps.
1. **Obtain candidate definitions**
Obtain the candidate container definitions from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers). These candidate definitions are used to create a SageMaker AI model.
The following example uses the [DescribeAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) API to obtain candidate definitions for the best model candidate. See the following Amazon CLI command as an example.
```
aws sagemaker describe-auto-ml-job --auto-ml-job-name --region
```
1. **List candidates**
The following example uses the [ListCandidatesForAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html) API to list all candidates. See the following Amazon CLI command as an example.
```
aws sagemaker list-candidates-for-auto-ml-job --auto-ml-job-name --region
```
1. **Create a SageMaker AI model**
Use the container definitions from the previous steps to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker create-model --model-name '' \
--containers [', , ]' \
--execution-role-arn '' --region '
```
1. **Create an endpoint configuration**
The following example uses the [CreateEndpointConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpointConfig.html) API to create an endpoint configuration. See the following Amazon CLI command as an example.
```
aws sagemaker create-endpoint-config --endpoint-config-name '' \
--production-variants '' \
--region ''
```
1. **Create the endpoint**
The following Amazon CLI example uses the [CreateEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpoint.html) API to create the endpoint.
```
aws sagemaker create-endpoint --endpoint-name '' \
--endpoint-config-name '' \
--region ''
```
Check the progress of your endpoint deployment by using the [DescribeEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeEndpoint.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-endpoint —endpoint-name '' —region
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready to use for real-time inference.
1. **Invoke the endpoint**
The following command structure invokes the endpoint for real-time inferencing.
```
aws sagemaker invoke-endpoint --endpoint-name '' \
--region '' --body '' [--content-type] ''
```
The following tabs contain complete code examples for deploying a model with Amazon SDK for Python (boto3) or the Amazon CLI.
------
#### [ Amazon SDK for Python (boto3) ]
1. **Obtain the candidate definitions** by using the following code example.
```
import sagemaker
import boto3
session = sagemaker.session.Session()
sagemaker_client = boto3.client('sagemaker', region_name='us-west-2')
job_name = 'test-auto-ml-job'
describe_response = sm_client.describe_auto_ml_job(AutoMLJobName=job_name)
# extract the best candidate definition from DescribeAutoMLJob response
best_candidate = describe_response['BestCandidate']
# extract the InferenceContainers definition from the caandidate definition
inference_containers = best_candidate['InferenceContainers']
```
1. **Create the model** by using the following the code example.
```
# Create Model
model_name = 'test-model'
sagemaker_role = 'arn:aws:iam:444455556666:role/sagemaker-execution-role'
create_model_response = sagemaker_client.create_model(
ModelName = model_name,
ExecutionRoleArn = sagemaker_role,
Containers = inference_containers
)
```
1. **Create the endpoint configuration** by using the following the code example.
```
endpoint_config_name = 'test-endpoint-config'
instance_type = 'ml.m5.2xlarge'
# for all supported instance types, see
# https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ProductionVariant.html#sagemaker-Type-ProductionVariant-InstanceType # Create endpoint config
endpoint_config_response = sagemaker_client.create_endpoint_config(
EndpointConfigName=endpoint_config_name,
ProductionVariants=[
{
"VariantName": "variant1",
"ModelName": model_name,
"InstanceType": instance_type,
"InitialInstanceCount": 1
}
]
)
print(f"Created EndpointConfig: {endpoint_config_response['EndpointConfigArn']}")
```
1. **Create the endpoint** and deploy the model with the following code example.
```
# create endpoint and deploy the model
endpoint_name = 'test-endpoint'
create_endpoint_response = sagemaker_client.create_endpoint(
EndpointName=endpoint_name,
EndpointConfigName=endpoint_config_name)
print(create_endpoint_response)
```
**Check the status of creating the endpoint** by using the following the code example.
```
# describe endpoint creation status
status = sagemaker_client.describe_endpoint(EndpointName=endpoint_name)["EndpointStatus"]
```
1. **Invoke the endpoint** for real-time inferencing by using the following command structure.
```
# once endpoint status is InService, you can invoke the endpoint for inferencing
if status == "InService":
sm_runtime = boto3.Session().client('sagemaker-runtime')
inference_result = sm_runtime.invoke_endpoint(EndpointName='test-endpoint', ContentType='text/csv', Body='1,2,3,4,class')
```
------
#### [ Amazon Command Line Interface (Amazon CLI) ]
1. **Obtain the candidate definitions** by using the following code example.
```
aws sagemaker describe-auto-ml-job --auto-ml-job-name 'test-automl-job' --region us-west-2
```
1. **Create the model** by using the following code example.
```
aws sagemaker create-model --model-name 'test-sagemaker-model'
--containers '[{
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3", amzn-s3-demo-bucket1
"ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz",
"Environment": {
"AUTOML_SPARSE_ENCODE_RECORDIO_PROTOBUF": "1",
"AUTOML_TRANSFORM_MODE": "feature-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "application/x-recordio-protobuf",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.3-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz",
"Environment": {
"MAX_CONTENT_LENGTH": "20971520",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,probabilities"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3", aws-region
"ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz",
"Environment": {
"AUTOML_TRANSFORM_MODE": "inverse-label-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_INPUT": "predicted_label",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,labels,probabilities",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}]' \
--execution-role-arn 'arn:aws:iam::1234567890:role/sagemaker-execution-role' \
--region 'us-west-2'
```
For additional details, see [creating a model](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/create-model.html).
The `create model` command will return a response in the following format.
```
{
"ModelArn": "arn:aws:sagemaker:us-west-2:1234567890:model/test-sagemaker-model"
}
```
1. **Create an endpoint configuration** by using the following code example.
```
aws sagemaker create-endpoint-config --endpoint-config-name 'test-endpoint-config' \
--production-variants '[{"VariantName": "variant1",
"ModelName": "test-sagemaker-model",
"InitialInstanceCount": 1,
"InstanceType": "ml.m5.2xlarge"
}]' \
--region us-west-2
```
The `create endpoint` configuration command will return a response in the following format.
```
{
"EndpointConfigArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint-config/test-endpoint-config"
}
```
1. **Create an endpoint** by using the following code example.
```
aws sagemaker create-endpoint --endpoint-name 'test-endpoint' \
--endpoint-config-name 'test-endpoint-config' \
--region us-west-2
```
The `create endpoint` command will return a response in the following format.
```
{
"EndpointArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint/test-endpoint"
}
```
Check the progress of the endpoint deployment by using the following [describe-endpoint](https://docs.amazonaws.cn/cli/latest/reference/sagemaker/describe-endpoint.html) CLI code example.
```
aws sagemaker describe-endpoint --endpoint-name 'test-endpoint' --region us-west-2
```
The previous progress check will return a response in the following format.
```
{
"EndpointName": "test-endpoint",
"EndpointArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint/test-endpoint",
"EndpointConfigName": "test-endpoint-config",
"EndpointStatus": "Creating",
"CreationTime": 1660251167.595,
"LastModifiedTime": 1660251167.595
}
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready for use in real-time inference.
1. **Invoke the endpoint** for real-time inferencing by using the following command structure.
```
aws sagemaker-runtime invoke-endpoint --endpoint-name 'test-endpoint' \
--region 'us-west-2' \
--body '1,51,3.5,1.4,0.2' \
--content-type 'text/csv' \
'/tmp/inference_output'
```
For more options, see [invoking an endpoint](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker-runtime/invoke-endpoint.html).
------
## Deploy models from different accounts
You can deploy an Autopilot model from a different account than the original account that a model was generated in. To implement cross-account model deployment, this section shows how to do the following: Grant permission to assume the role to the account you want to deploy from (the generating account). Make a call to `DescribeAutoMLJob` from the deploying account to obtain model information. Grant access rights to the model artifacts from the generating account.
1. **Grant permission to the deploying account**
To assume the role in the generating account, you must grant permission to the deploying account. This allows the deploying account to describe Autopilot jobs in the generating account.
The following example uses a generating account with a trusted `sagemaker-role` entity. The example shows how to give a deploying account with the ID 111122223333 permission to assume the role of the generating account.
```
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"sagemaker.amazonaws.com"
],
"AWS": [ "111122223333"]
},
"Action": "sts:AssumeRole"
}
```
The new account with the ID 111122223333 can now assume the role for the generating account.
Next, call the `DescribeAutoMLJob` API from the deploying account to obtain a description of the job created by the generating account.
The following code example describes the model from the deploying account.
```
import sagemaker
import boto3
session = sagemaker.session.Session()
sts_client = boto3.client('sts')
sts_client.assume_role
role = 'arn:aws:iam::111122223333:role/sagemaker-role'
role_session_name = "role-session-name"
_assumed_role = sts_client.assume_role(RoleArn=role, RoleSessionName=role_session_name)
credentials = _assumed_role["Credentials"]
access_key = credentials["AccessKeyId"]
secret_key = credentials["SecretAccessKey"]
session_token = credentials["SessionToken"]
session = boto3.session.Session()
sm_client = session.client('sagemaker', region_name='us-west-2',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=session_token)
# now you can call describe automl job created in account A
job_name = "test-job"
response= sm_client.describe_auto_ml_job(AutoMLJobName=job_name)
```
1. **Grant access to the deploying account** to the model artifacts in the generating account.
The deploying account only needs access to the model artifacts in the generating account to deploy it. These are located in the [S3OutputPath](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html#sagemaker-Type-AutoMLOutputDataConfig-S3OutputPath) that was specified in the original `CreateAutoMLJob` API call during model generation.
To give the deploying account access to the model artifacts, choose one of the following options:
1. [Give access](https://www.amazonaws.cn/premiumsupport/knowledge-center/cross-account-access-s3/) to the `ModelDataUrl` from the generating account to the deploying account.
Next, you need to give the deploying account permission to assume the role. follow the [real-time inferencing steps](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-deploy-models.html#autopilot-deploy-models-realtime) to deploy.
1. [Copy model artifacts](https://www.amazonaws.cn/premiumsupport/knowledge-center/copy-s3-objects-account/) from the generating account's original [S3OutputPath](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html#sagemaker-Type-AutoMLOutputDataConfig-S3OutputPath) to the generating account.
To grant access to the model artifacts, you must define a `best_candidate` model and reassign model containers to the new account.
The following example shows how to define a `best_candidate` model and reassign the `ModelDataUrl`.
```
best_candidate = automl.describe_auto_ml_job()['BestCandidate']
# reassigning ModelDataUrl for best_candidate containers below
new_model_locations = ['new-container-1-ModelDataUrl', 'new-container-2-ModelDataUrl', 'new-container-3-ModelDataUrl']
new_model_locations_index = 0
for container in best_candidate['InferenceContainers']:
container['ModelDataUrl'] = new_model_locations[new_model_locations_index++]
```
After this assignment of containers, follow the steps in [Deploy using SageMaker APIs](#autopilot-deploy-models-api) to deploy.
To build a payload in real-time inferencing, see the notebook example to [ define a test payload](https://www.amazonaws.cn/getting-started/hands-on/machine-learning-tutorial-automatically-create-models). To create the payload from a CSV file and invoke an endpoint, see the **Predict with your model** section in [Create a machine learning model automatically](https://www.amazonaws.cn/getting-started/hands-on/create-machine-learning-model-automatically-sagemaker-autopilot/#autopilot-cr-room).
# Run batch inference jobs
Batch inferencing, also known as offline inferencing, generates model predictions on a batch of observations. Batch inference is a good option for large datasets or if you don't need an immediate response to a model prediction request. By contrast, online inference ([real-time inferencing](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-deploy-models.html#autopilot-deploy-models-realtime)) generates predictions in real time. You can make batch inferences from an Autopilot model using the [SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/), the Autopilot user interface (UI), the [Amazon SDK for Python (boto3)](https://www.amazonaws.cn/sdk-for-python/), or the Amazon Command Line Interface ([Amazon CLI](https://docs.amazonaws.cn/cli/)).
The following tabs show three options for deploying your model: Using APIs, Autopilot UI, or using APIs to deploy from different accounts. These instructions assume that you have already created a model in Autopilot. If you don't have a model, see [Create Regression or Classification Jobs for Tabular Data Using the AutoML API](autopilot-automate-model-development-create-experiment.md). To see examples for each option, open each tab.
## Deploy a model using Autopilot UI
The Autopilot UI contains helpful dropdown menus, toggles, tooltips, and more to help you navigate through model deployment.
The following steps show how to deploy a model from an Autopilot experiment for batch predictions.
1. Sign in at [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/) and select **Studio** from the navigation pane.
1. On the left navigation pane, choose **Studio**.
1. Under **Get started**, select the Domain that you want to launch the Studio application in. If your user profile only belongs to one Domain, you do not see the option for selecting a Domain.
1. Select the user profile that you want to launch the Studio Classic application for. If there is no user profile in the domain, choose **Create user profile**. For more information, see [Add user profiles](https://docs.amazonaws.cn/sagemaker/latest/dg/domain-user-profile-add.html).
1. Choose **Launch Studio**. If the user profile belongs to a shared space, choose **Open Spaces**.
1. When the SageMaker Studio Classic console opens, choose the **Launch SageMaker Studio** button.
1. Select **AutoML** from the left navigation pane.
1. Under **Name**, select the Autopilot experiment corresponding to the model that you want to deploy. This opens a new **AUTOPILOT JOB** tab.
1. In the **Model name** section, select the model that you want to deploy.
1. Choose **Deploy model**. This opens a new tab.
1. Choose **Make batch predictions** at the top of the page.
1. For **Batch transform job configuration**, input the **Instance type**, **Instance count** and other optional information.
1. In the **Input data configuration** section, open the dropdown menu.
1. For **S3 data type**, choose **ManifestFile** or **S3Prefix**.
1. For **Split type**, choose **Line**, **RecordIO**, **TFRecord** or **None**.
1. For **Compression**, choose **Gzip** or **None**.
1. For **S3 location**, enter the Amazon S3 bucket location of the input data and other optional information.
1. Under **Output data configuration**, enter the S3 bucket for the output data, and choose how to [assemble the output](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TransformOutput.html#sagemaker-Type-TransformOutput-AssembleWith) of your job.
1. For **Additional configuration (optional)**, you can enter a MIME type and an **S3 Encryption key**.
1. For **Input/output filtering and data joins (optional)**, you enter a JSONpath expression to filter your input data, join the input source data with your output data, and enter a JSONpath expression to filter your output data.
1. For examples for each type of filter, see the [DataProcessing API](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DataProcessing.html#sagemaker-Type-DataProcessing-InputFilter).
1. To perform batch predictions on your input dataset, select **Create batch transform job**. A new **Batch Transform Jobs** tab appears.
1. In the **Batch Transform Jobs** tab: Locate the name of your job in **Status** section. Then check the progress of the job.
## Deploy using SageMaker APIs
To use the SageMaker APIs for batch inferencing, there are three steps:
1. **Obtain candidate definitions**
Candidate definitions from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers) are used to create a SageMaker AI model.
The following example shows how to use the [DescribeAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) API to obtain candidate definitions for the best model candidate. See the following Amazon CLI command as an example.
```
aws sagemaker describe-auto-ml-job --auto-ml-job-name --region
```
Use the [ListCandidatesForAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html) API to list all candidates. See the following Amazon CLI command as an example.
```
aws sagemaker list-candidates-for-auto-ml-job --auto-ml-job-name --region
```
1. **Create a SageMaker AI model**
To create a SageMaker AI model using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API, use the container definitions from the previous steps. See the following Amazon CLI command as an example.
```
aws sagemaker create-model --model-name '' \
--containers [', , ]' \
--execution-role-arn '' --region '
```
1. **Create a SageMaker AI transform job**
The following example creates a SageMaker AI transform job with the [CreateTransformJob](https://docs.amazonaws.cn/cli/latest/reference/sagemaker/create-transform-job.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker create-transform-job --transform-job-name '' --model-name ''\
--transform-input '{
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": ""
}
},
"ContentType": "text/csv",
"SplitType": "Line"
}'\
--transform-output '{
"S3OutputPath": "",
"AssembleWith": "Line"
}'\
--transform-resources '{
"InstanceType": "",
"InstanceCount": 1
}' --region ''
```
Check the progress of your transform job using the [DescribeTransformJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeTransformJob.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-transform-job --transform-job-name '' --region
```
After the job is finished, the predicted result will be available in ``.
The output file name has the following format: `.out`. As an example, if your input file is `text_x.csv`, the output name will be `text_x.csv.out`.
The following tabs show code examples for SageMaker Python SDK, Amazon SDK for Python (boto3), and the Amazon CLI.
------
#### [ SageMaker Python SDK ]
The following example uses the **[SageMaker Python SDK ](https://sagemaker.readthedocs.io/en/stable/overview.html)** to make predictions in batches.
```
from sagemaker import AutoML
sagemaker_session= sagemaker.session.Session()
job_name = 'test-auto-ml-job' # your autopilot job name
automl = AutoML.attach(auto_ml_job_name=job_name)
output_path = 's3://test-auto-ml-job/output'
input_data = 's3://test-auto-ml-job/test_X.csv'
# call DescribeAutoMLJob API to get the best candidate definition
best_candidate = automl.describe_auto_ml_job()['BestCandidate']
best_candidate_name = best_candidate['CandidateName']
# create model
model = automl.create_model(name=best_candidate_name,
candidate=best_candidate)
# create transformer
transformer = model.transformer(instance_count=1,
instance_type='ml.m5.2xlarge',
assemble_with='Line',
output_path=output_path)
# do batch transform
transformer.transform(data=input_data,
split_type='Line',
content_type='text/csv',
wait=True)
```
------
#### [ Amazon SDK for Python (boto3) ]
The following example uses **Amazon SDK for Python (boto3)** to make predictions in batches.
```
import sagemaker
import boto3
session = sagemaker.session.Session()
sm_client = boto3.client('sagemaker', region_name='us-west-2')
role = 'arn:aws:iam::1234567890:role/sagemaker-execution-role'
output_path = 's3://test-auto-ml-job/output'
input_data = 's3://test-auto-ml-job/test_X.csv'
best_candidate = sm_client.describe_auto_ml_job(AutoMLJobName=job_name)['BestCandidate']
best_candidate_containers = best_candidate['InferenceContainers']
best_candidate_name = best_candidate['CandidateName']
# create model
reponse = sm_client.create_model(
ModelName = best_candidate_name,
ExecutionRoleArn = role,
Containers = best_candidate_containers
)
# Lauch Transform Job
response = sm_client.create_transform_job(
TransformJobName=f'{best_candidate_name}-transform-job',
ModelName=model_name,
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': input_data
}
},
'ContentType': "text/csv",
'SplitType': 'Line'
},
TransformOutput={
'S3OutputPath': output_path,
'AssembleWith': 'Line',
},
TransformResources={
'InstanceType': 'ml.m5.2xlarge',
'InstanceCount': 1,
},
)
```
The batch inference job returns a response in the following format.
```
{'TransformJobArn': 'arn:aws:sagemaker:us-west-2:1234567890:transform-job/test-transform-job',
'ResponseMetadata': {'RequestId': '659f97fc-28c4-440b-b957-a49733f7c2f2',
'HTTPStatusCode': 200,
'HTTPHeaders': {'x-amzn-requestid': '659f97fc-28c4-440b-b957-a49733f7c2f2',
'content-type': 'application/x-amz-json-1.1',
'content-length': '96',
'date': 'Thu, 11 Aug 2022 22:23:49 GMT'},
'RetryAttempts': 0}}
```
------
#### [ Amazon Command Line Interface (Amazon CLI) ]
1. **Obtain the candidate definitions** by using the following the code example.
```
aws sagemaker describe-auto-ml-job --auto-ml-job-name 'test-automl-job' --region us-west-2
```
1. **Create the model** by using the following the code example.
```
aws sagemaker create-model --model-name 'test-sagemaker-model'
--containers '[{
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/data-processor-models/test-job1-dpp0-1-e569ff7ad77f4e55a7e549a/output/model.tar.gz",
"Environment": {
"AUTOML_SPARSE_ENCODE_RECORDIO_PROTOBUF": "1",
"AUTOML_TRANSFORM_MODE": "feature-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "application/x-recordio-protobuf",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.3-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/tuning/flicdf10v2-dpp0-xgb/test-job1E9-244-7490a1c0/output/model.tar.gz",
"Environment": {
"MAX_CONTENT_LENGTH": "20971520",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,probabilities"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/data-processor-models/test-job1-dpp0-1-e569ff7ad77f4e55a7e549a/output/model.tar.gz",
"Environment": {
"AUTOML_TRANSFORM_MODE": "inverse-label-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_INPUT": "predicted_label",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,labels,probabilities",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}]' \
--execution-role-arn 'arn:aws:iam::1234567890:role/sagemaker-execution-role' \
--region 'us-west-2'
```
1. **Create the transform job** by using the following the code example.
```
aws sagemaker create-transform-job --transform-job-name 'test-tranform-job'\
--model-name 'test-sagemaker-model'\
--transform-input '{
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3://amzn-s3-demo-bucket/data.csv"
}
},
"ContentType": "text/csv",
"SplitType": "Line"
}'\
--transform-output '{
"S3OutputPath": "s3://amzn-s3-demo-bucket/output/",
"AssembleWith": "Line"
}'\
--transform-resources '{
"InstanceType": "ml.m5.2xlarge",
"InstanceCount": 1
}'\
--region 'us-west-2'
```
1. **Check the progress of the transform job** by using the following the code example.
```
aws sagemaker describe-transform-job --transform-job-name 'test-tranform-job' --region us-west-2
```
The following is the response from the transform job.
```
{
"TransformJobName": "test-tranform-job",
"TransformJobArn": "arn:aws:sagemaker:us-west-2:1234567890:transform-job/test-tranform-job",
"TransformJobStatus": "InProgress",
"ModelName": "test-model",
"TransformInput": {
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3://amzn-s3-demo-bucket/data.csv"
}
},
"ContentType": "text/csv",
"CompressionType": "None",
"SplitType": "Line"
},
"TransformOutput": {
"S3OutputPath": "s3://amzn-s3-demo-bucket/output/",
"AssembleWith": "Line",
"KmsKeyId": ""
},
"TransformResources": {
"InstanceType": "ml.m5.2xlarge",
"InstanceCount": 1
},
"CreationTime": 1662495635.679,
"TransformStartTime": 1662495847.496,
"DataProcessing": {
"InputFilter": "$",
"OutputFilter": "$",
"JoinSource": "None"
}
}
```
After the `TransformJobStatus` changes to `Completed`, you can check the inference result in the `S3OutputPath`.
------
## Deploy models from different accounts
To create a batch inferencing job in a different account than the one that the model was generated in, follow the instructions in [Deploy models from different accounts](autopilot-deploy-models-realtime.md#autopilot-deploy-models-realtime-across-accounts). Then you can create models and transform jobs by following the [Deploy using SageMaker APIs](#autopilot-deploy-models-batch-steps).
# View model details
Autopilot generates details about the candidate models that you can obtain. These details include the following:
+ A plot of the aggregated SHAP values that indicate the importance of each feature. This helps explain your models predictions.
+ The summary statistics for various training and validation metrics, including the objective metric.
+ A list of the hyperparameters used to train and tune the model.
To view model details after running an Autopilot job, follow these steps:
1. Choose the **Home** icon (![\[Black square icon representing a placeholder or empty image.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/studio/icons/house.png)) from the left navigation pane to view the top-level **Amazon SageMaker Studio Classic** navigation menu.
1. Select the **AutoML** card from the main working area. This opens a new **Autopilot** tab.
1. In the **Name** section, select the Autopilot job that has the details that you want to examine. This opens a new **Autopilot job** tab.
1. The **Autopilot job** panel lists the metric values including the **Objective** metric for each model under **Model name**. The **Best model** is listed at the top of the list under **Model name** and is also highlighted in the **Models** tab.
1. To review model details, select the model that you are interested in and select **View model details**. This opens a new **Model Details** tab.
1. The **Model Details** tab is divided into four subsections.
1. The top of the **Explainability** tab contains a plot of aggregated SHAP values that indicate the importance of each feature. Following that are the metrics and hyperparameter values for this model.
1. The **Performance** tab contains metrics statistics a confusion matrix.
1. The **Artifacts** tab contains information about model inputs, outputs, and intermediate results.
1. The **Network** tab summarizes your network isolation and encryption choices.
**Note**
Feature importance and information in the **Performance** tab is only generated for the **Best model**.
For more information about how the SHAP values help explain predictions based on feature importance, see the whitepaper [Understanding the model explainability](https://pages.awscloud.com/rs/112-TZM-766/images/Amazon.AI.Fairness.and.Explainability.Whitepaper.pdf). Additional information is also available in the [Model Explainability](clarify-model-explainability.md) topic in the SageMaker AI Developer Guide.
# View an Autopilot model performance report
An Amazon SageMaker AI model quality report (also referred to as performance report) provides insights and quality information for the best model candidate generated by an AutoML job. This includes information about the job details, model problem type, objective function, and other information related to the problem type. This guide shows how to view Amazon SageMaker Autopilot performance metrics graphically, or view metrics as raw data in a JSON file.
For example, in classification problems, the model quality report includes the following:
+ Confusion matrix
+ Area under the receiver operating characteristic curve (AUC)
+ Information to understand false positives and false negatives
+ Tradeoffs between true positives and false positives
+ Tradeoffs between precision and recall
Autopilot also provides performance metrics for all of your candidate models. These metrics are calculated using all of the training data and are used to estimate model performance. The main working area includes these metrics by default. The type of metric is determined by the type of problem being addressed.
Refer to the [ Amazon SageMaker API reference documentation](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobObjective.html) for the list of available metrics supported by Autopilot.
You can sort your model candidates with the relevant metric to help you select and deploy the model that addresses your business needs. For definitions of these metrics, see the [Autopilot candidate metrics](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-metrics-validation.html#autopilot-metrics) topic.
To view a performance report from an Autopilot job, follow these steps:
1. Choose the **Home** icon (![\[Black square icon representing a placeholder or empty image.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/studio/icons/house.png)) from the left navigation pane to view the top-level **Amazon SageMaker Studio Classic** navigation menu.
1. Select the **AutoML** card from the main working area. This opens a new **Autopilot** tab.
1. In the **Name** section, select the Autopilot job that has the details that you want to examine. This opens a new **Autopilot job** tab.
1. The **Autopilot job** panel lists the metric values including the **Objective** metric for each model under **Model name**. The **Best model** is listed at the top of the list under **Model name** and it is highlighted in the **Models** tab.
1. To review model details, select the model that you are interested in and select **View in model details**. This opens a new **Model Details** tab.
1. Choose the **Performance** tab between the **Explainability** and **Artifacts** tab.
1. On the top right section of the tab, select the down arrow on the **Download Performance Reports** button.
1. The down arrow provides two options to view Autopilot performance metrics:
1. You can download a PDF of the performance report to view the metrics graphically.
1. You can view metrics as raw data and download it as a JSON file.
For instructions on how to create and run an AutoML job in SageMaker Studio Classic, see [Create Regression or Classification Jobs for Tabular Data Using the AutoML API](autopilot-automate-model-development-create-experiment.md).
The performance report contains two sections. The first contains details about the Autopilot job that produced the model. The second section contains a model quality report.
## Autopilot Job details
This first section of the report gives some general information about the Autopilot job that produced the model. These job details include the following information:
+ Autopilot candidate name
+ Autopilot job name
+ Problem type
+ Objective metric
+ Optimization direction
## Model quality report
Model quality information is generated by Autopilot model insights. The report's content that is generated depends on the problem type it addressed: regression, binary classification, or multiclass classification. The report specifies the number of rows that were included in the evaluation dataset and the time at which the evaluation occurred.
### Metrics tables
The first part of the model quality report contains metrics tables. These are appropriate for the type of problem that the model addressed.
The following image is an example of a metrics table that Autopilot generates for a regression problem. It shows the metric name, value, and standard deviation.
![\[Amazon SageMaker Autopilot model insights regression metrics report example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-regression-metrics.png)
The following image is an example of a metrics table generated by Autopilot for a multiclass classification problem. It shows the metric name, value, and standard deviation.
![\[Amazon SageMaker Autopilot model insights multiclass classification metrics report example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-multiclass-metrics-report.png)
### Graphical model performance information
The second part of the model quality report contains graphical information to help you evaluate model performance. The contents of this section depend on the problem type used in modeling.
#### The area under the receiver operating characteristic curve
The area under the receiver operating characteristic curve represents the trade-off between true positive and false positive rates. It is an industry-standard accuracy metric used for binary classification models. AUC (area under the curve) measures the ability the model to predict a higher score for positive examples, as compared to negative examples. The AUC metric provides an aggregated measure of the model performance across all possible classification thresholds.
The AUC metric returns a decimal value from 0 to 1. AUC values near 1 indicate that the machine learning model is highly accurate. Values near 0.5 indicate that the model is performing no better than guessing at random. AUC values close to 0 indicate that the model has learned the correct patterns, but is making predictions that are as inaccurate as possible. Values near zero can indicate a problem with the data. For more information about the AUC metric, see the [Receiver operating characteristic](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) article on Wikipedia.
The following is an example of an area under the receiver operating characteristic curve graph to evaluate predictions made by a binary classification model. The dashed thin line represents the area under the receiver operating characteristic curve that a model which classifies no-better-than-random guessing would score, with an AUC score of 0.5. The curves of more accurate classification models lie above this random baseline, where the rate of true positives exceeds the rate of false positives. The area under the receiver operating characteristic curve representing the performance of the binary classification model is the thicker solid line.
![\[Amazon SageMaker Autopilot area under the receiver operating characteristic curve example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-receiver-operating-characteristic-curve.png)
A summary of the graph's components of **false positive rate **(FPR) and **true positive rate **(TPR) are defined as follows.
+ Correct predictions
+ **True positive** (TP): The predicted value is 1, and the true value is 1.
+ **True negative** (TN): The predicted value is 0, and the true value is 0.
+ Erroneous predictions
+ **False positive** (FP): The predicted value is 1, but the true value is 0.
+ **False negative** (FN): The predicted value is 0, but the true value is 1.
The **false positive rate **(FPR) measures the fraction of true negatives (TN) that were falsely predicted as positives (FP), over the sum of FP and TN. The range is 0 to 1. A smaller value indicates better predictive accuracy.
+ FPR = FP/(FP\$1TN)
The **true positive rate **(TPR) measures the fraction true positives that were correctly predicted as positives (TP) over the sum of TP and false negatives (FN). The range is 0 to 1. A larger value indicates better predictive accuracy.
+ TPR = TP/(TP\$1FN)
#### Confusion matrix
A confusion matrix provides a way to visualize the accuracy of the predictions made by a model for binary and multiclass classification for different problems. The confusion matrix in the model quality report contains the following.
+ The number and percentage of correct and incorrect predictions for the actual labels
+ The number and percentage of accurate predictions on the diagonal from the upper-left to the lower-right corner
+ The number and percentage of inaccurate predictions on the diagonal from the upper-right to the lower-left corner
The incorrect predictions on a confusion matrix are the confusion values.
The following diagram is an example of a confusion matrix for a binary classification problem. It contains the following information:
+ The vertical axis is divided into two rows containing true and false actual labels.
+ The horizontal axis is divided into two columns containing true and false labels that were predicted by the model.
+ The color bar assigns a darker tone to a larger number of samples to visually indicate the number of values that were classified in each category.
In this example, the model predicted actual 2817 false values correctly, and 353 actual true values correctly. The model incorrectly predicted 130 actual true values to be false and 33 actual false values to be true. The difference in tone indicates that the dataset is not balanced. The imbalance is because there are many more actual false labels than actual true labels.
![\[Amazon SageMaker Autopilot binary confusion matrix example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-confusion-matrix-binary.png)
The following diagram is an example of a confusion matrix for a multi-class classification problem. The confusion matrix in the model quality report contains the following.
+ The vertical axis is divided into three rows containing three different actual labels.
+ The horizontal axis is divided into three columns containing labels that were predicted by the model.
+ The color bar assigns a darker tone to a larger number of samples to visually indicate the number of values that were classified in each category.
In the example below, the model correctly predicted actual 354 values for label **f**, 1094 values for label **i** and 852 values for label **m**. The difference in tone indicates that the dataset is not balanced because there are many more labels for the value **i** than for **f** or **m**.
![\[Amazon SageMaker Autopilot multiclass confusion matrix example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-confusion-matrix-multiclass.png)
The confusion matrix in the model quality report provided can accommodate a maximum of 15 labels for multiclass classification problem types. If a row corresponding to a label shows a `Nan` value, it means that the validation dataset used to check model predictions does not contain data with that label.
#### Gain curve
In binary classification, a gain curve predicts the cumulative benefit of using a percentage of the dataset to find a positive label. The gain value is calculated during training by dividing the cumulative number of positive observations by the total number of positive observations in the data, at each decile. If the classification model created during training is representative of the unseen data, you can use the gain curve to predict the percentage of data that you must target to obtain a percentage of positive labels. The greater the percentage of the dataset used, the higher the percentage of positive labels found.
In the following example graph, the gain curve is the line with changing slope. The straight line is the percentage of positive labels found by selecting a percentage of data from the dataset at random. Upon targeting 20% of the dataset, you would expect to find larger than 40% of the positive labels. As an example, you might consider using a gain curve to determine your efforts in a marketing campaign. Using our gain curve example, for 83% of people in a neighborhood to purchase cookies, you'd send an advertisement to about 60% of the neighborhood.
![\[Amazon SageMaker Autopilot gain curve example with percentage and gain value.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-gain-curve.png)
#### Lift curve
In binary classification, the lift curve illustrates the uplift of using a trained model to predict the likelihood of finding a positive label compared to a random guess. The lift value is calculated during training using the ratio of percentage gain to the ratio of positive labels at each decile. If the model created during training is representative of the unseen data, use the lift curve to predict the benefit of using the model over randomly guessing.
In the following example graph, the lift curve is the line with changing slope. The straight line is the lift curve associated with selecting the corresponding percentage randomly from the dataset. Upon targeting 40% of the dataset with your model's classification labels, you would expect to find about 1.7 times the number of the positive labels that you would have found by randomly selecting 40% of the unseen data.
![\[Amazon SageMaker Autopilot lift curve example with percentage and lift value.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-lift-curve.png)
#### Precision-recall curve
The precision-recall curve represents the tradeoff between precision and recall for binary classification problems.
**Precision** measures the fraction of actual positives that are predicted as positive (TP) out of all positive predictions (TP and false positive). The range is 0 to 1. A larger value indicates better accuracy in the predicted values.
+ Precision = TP/(TP\$1FP)
**Recall** measures the fraction of actual positives that are predicted as positive (TP) out of all actual positive predictions (TP and false negative). This is also known as the sensitivity or as the true positive rate. The range is 0 to 1. A larger value indicates better detection of positive values from the sample.
+ Recall = TP/(TP\$1FN)
The objective of a classification problem is to correctly label as many elements as possible. A system with high recall but low precision returns a high percentage of false positives.
The following graphic depicts a spam filter that marks every email as spam. It has high recall, but low precision, because recall doesn't measure false positives.
Give more weight to recall over precision if your problem has a low penalty for false positive values, but a high penalty for missing a true positive result. For example, detecting an impending collision in a self-driving vehicle.
![\[Autopilot example of high recall and low precision system, modelling all samples as positives.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-high-recall-low-precision.PNG)
By contrast, a system with high precision, but low recall, returns a high percentage of false negatives. A spam filter that marks every email as desirable (not spam) has high precision but low recall because precision doesn't measure false negatives.
If your problem has a low penalty for false negative values, but a high penalty for missing a true negative results, give more weight to precision over recall. For example, flagging a suspicious filter for a tax audit.
The following graphic depicts a spam filter that has high precision but low recall, because precision doesn't measure false negatives.
![\[Autopilot example of high-precision and low-recall system, modeling all samples as negatives.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-high-precision-low-recall.PNG)
A model that makes predictions with both high precision and high recall produces a high number of correctly labeled results. For more information, see [Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) article in Wikipedia.
#### Area under precision-recall curve (AUPRC)
For binary classification problems, Amazon SageMaker Autopilot includes a graph of the area under the precision-recall curve (AUPRC). The AUPRC metric provides an aggregated measure of the model performance across all possible classification thresholds and uses both precision and recall. AUPRC does not take the number of true negatives into account. Therefore, it can be useful to evaluate model performance in cases where there's a large number of true negatives in the data. For example, to model a gene containing a rare mutation.
The following graphic is an example of an AUPRC graph. Precision at its highest value is 1, and recall is at 0. In the lower right corner of the graph, recall is its highest value (1) and precision is 0. In between these two points , the AUPRC curve illustrates the tradeoff between precision and recall at different thresholds.
![\[Precision-recall curve depicts tradeoff between precision and recall at different thresholds.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-binary-precision-recall.png)
#### Actual against predicted plot
The actual against predicted plot shows the difference between actual and predicted model values. In the following example graph, the solid line is a linear line of best fit. If the model were 100% accurate, each predicted point would equal its corresponding actual point and lie on this line of best fit. The distance away from the line of best fit is a visual indication of model error. The larger the distance away from the line of best fit, the higher the model error.
![\[Example with linear line of best fit, differing actual and predicted plot, and model error.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-actual-vs-predicted-plot.png)
#### Standardized residual plot
A standardized residual plot incorporates the following statistical terms:
**`residual`**
A (raw) residual shows the difference between actual and values predicted by your model. The larger the difference, the larger the residual value.
**`standard deviation`**
The standard deviation is a measure of how values vary from an average value. A high standard deviation indicates that many values are very different from their average value. A low standard deviation indicates that many values are close to their average value.
**`standardized residual`**
A standardized residual divides the raw residuals by their standard deviation. Standardized residuals have units of standard deviation and are useful in identifying outliers in data regardless of the difference in scale of the raw residuals. If a standardized residual is much smaller or larger than the other standardized residuals, it indicates that the model is not fitting these observations well.
The standardized residual plot measures the strength of the difference between observed and expected values. The actual predicted value is displayed on the x axis. A point with a value larger than an absolute value of 3 is commonly regarded as an outlier.
The following example graph shows that a large number of standardized residuals are clustered around 0 on the horizontal axis. The values close to zero indicate that the model is fitting these points well. The points towards the top and bottom of the plot are not predicted well by the model.
![\[Amazon SageMaker Autopilot standardized residual plot example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-standardized-residual.png)
#### Residual histogram
A residual histogram incorporates the following statistical terms:
**`residual`**
A (raw) residual shows the difference between actual and values predicted by your model. The larger the difference, the larger the residual value.
**`standard deviation`**
The standard deviation is a measure of how much values vary from an average value. A high standard deviation indicates that many values are very different from their average value. A low standard deviation indicates that many values are close to their average value.
**`standardized residual`**
A standardized residual divides the raw residuals by their standard deviation. Standardized residuals have units of standard deviation. These are useful in identifying outliers in data regardless of the difference in scale of the raw residuals. If a standardized residual is much smaller or larger than the other standardized residuals, it would indicate that the model is not fitting these observations well.
**`histogram`**
A histogram is a graph that shows how often a value occurred.
The residual histogram shows the distribution of standardized residual values. A histogram distributed in a bell shape and centered at zero indicates that the model does not systematically overpredict or underpredict any particular range of target values.
In the following graphic, the standardized residual values indicate that the model is fitting the data well. If the graph showed values far away from the center value, it would indicate that those values don't fit the model well.
![\[Standardized residual value close to zero, indicating that the model fits the data well.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-residual-histogram.png)
# Autopilot notebooks generated to manage AutoML tasks
Amazon SageMaker Autopilot manages the key tasks in an automatic machine learning (AutoML) process using an AutoML job. The AutoML job creates three notebook-based reports that describe the plan that Autopilot follows to generate candidate models.
A candidate model consists of a (pipeline, algorithm) pair. First, there’s a **data exploration** notebook that describes what Autopilot learned about the data that you provided. Second, there’s a **candidate definition** notebook, which uses the information about the data to generate candidates. Third, a **model insights** report that can help detail the performance characteristics of the best model in the leaderboard of an Autopilot experiment.
**Topics**
+ [
# Autopilot data exploration report
](autopilot-data-exploration-report.md)
+ [
# Find and run the candidate definition notebook
](autopilot-candidate-generation-notebook.md)
You can run these notebooks in Amazon SageMaker AI, or locally, if you have installed the [Amazon SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable). You can share the notebooks just like any other SageMaker Studio Classic notebook. The notebooks are created for you to conduct experiments. For example, you could edit the following items in the notebooks:
+ Preprocessors used on the data
+ Amount of hyperparameter optimization (HPO) runs and their parallelism
+ Algorithms to try
+ Instance types used for the HPO jobs
+ Hyperparameter ranges
Modifications to the candidate definition notebook are encouraged as a learning tool. With this capability, you learn how decisions made during the machine learning process impact your results.
**Note**
When you run the notebooks in your default instance, you incur baseline costs. However, when you run HPO jobs from the candidate notebook, these jobs use additional compute resources that incur additional costs.
# Autopilot data exploration report
Amazon SageMaker Autopilot cleans and pre-processes your dataset automatically. High-quality data improves machine learning efficiency and produces models that make more accurate predictions.
There are issues with customer-provided datasets that cannot be fixed automatically without the benefit of some domain knowledge. Large outlier values in the target column for regression problems, for example, may cause suboptimal predictions for the non-outlier values. Outliers may need to be removed depending on the modeling objective. If a target column is included by accident as one of the input features, the final model will validate well, but be of little value for future predictions.
To help customers discover these sorts of issues, Autopilot provides a data exploration report that contains insights into potential issues with their data. The report also suggests how to handle the issues.
A data exploration notebook containing the report is generated for every Autopilot job. The report is stored in an Amazon S3 bucket and can be accessed from your output path. The path of the data exploration report usually adheres to the following pattern.
```
[s3 output path]/[name of the automl job]/sagemaker-automl-candidates/[name of processing job used for data analysis]/notebooks/SageMaker AIAutopilotDataExplorationNotebook.ipynb
```
The location of the data exploration notebook can be obtained from the Autopilot API using the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) operation response, which is stored in [DataExplorationNotebookLocation](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobArtifacts.html#sagemaker-Type-AutoMLJobArtifacts-DataExplorationNotebookLocation).
When running Autopilot from SageMaker Studio Classic, you can open the data exploration report using the following steps:
1. Choose the **Home** icon ![\[Black square icon representing a placeholder or empty image.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/studio/icons/house.png) from the *left navigation pane* to view the top-level **Amazon SageMaker Studio Classic** navigation menu.
1. Select the **AutoML** card from the main working area. This opens a new **Autopilot** tab.
1. In the **Name** section, select the Autopilot job that has the data exploration notebook that you want to examine. This opens a new **Autopilot job** tab.
1. Select **Open data exploration notebook** from the top right section of the **Autopilot job** tab.
The data exploration report is generated from your data before the training process begins. This allows you to stop Autopilot jobs that might lead to meaningless results. Likewise, you can address any issues or improvements with your dataset before rerunning Autopilot. This way, you can use your domain expertise to improve the data quality manually, before you train a model on a better-curated dataset.
The data report contains only static markdown and can be opened in any Jupyter environment. The notebook that contains the report can be converted to other formats, such as PDF or HTML. For more information about conversions, see [Using the nbconvert script to convert Jupyter notebooks to other formats.](https://nbconvert.readthedocs.io/en/latest/usage.html ).
**Topics**
+ [
## Dataset Summary
](#autopilot-data-exploration-report-dataset-summary)
+ [
## Target Analysis
](#autopilot-data-exploration-report-target-analysis)
+ [
## Data Sample
](#autopilot-data-exploration-report-data-sample)
+ [
## Duplicate rows
](#autopilot-data-exploration-report-duplicate-rows)
+ [
## Cross column correlations
](#autopilot-data-exploration-report-cross-column-correlations)
+ [
## Anomalous Rows
](#autopilot-data-exploration-report-cross-anomolous-rows)
+ [
## Missing values, cardinality, and descriptive statistics
](#autopilot-data-exploration-report-description-statistics-and-values)
## Dataset Summary
This **Dataset Summary** provides key statistics characterizing your dataset including the number of rows, columns, percent duplicate rows and missing target values. It is intended to provide you with a quick alert when there are issue with your dataset that Amazon SageMaker Autopilot has detected and that are likely to require your intervention. The insights are surfaced as warnings that are classified as being of either “high” or “low” severity. The classification depends on the level of confidence that the issue will adversely impact the performance of the model.
The high and low severity insights appear in the summary as pop-ups. For most of the insights, recommendations are offered for how to confirm that there is an issue with the dataset that requires your attention. Proposals are also provided for how to resolve the issues.
Autopilot provides additional statistics about missing or not valid target values in our dataset to help you detect other issues that may not be captured by high severity insights. An unexpected number of columns of a particular type might indicate that some columns that you want to use may be missing from the dataset. It could also indicate that there was an issue with how the data was prepared or stored. Fixing these data problems brought to your attention by Autopilot can improve the performance of the machine learning models trained on your data.
High severity insights are shown in the summary section and in other relevant sections in the report. Examples of high and low-severity insights are usually given depending on the section of the data report.
## Target Analysis
Various high and low-severity insights are shown in this section related to the distribution of values in the target column. Check that target column contains the correct values. Incorrect values in target column will likely result in a machine learning model that doesn't serve the intended business purpose. Several data insights of high and low severity are present in this section. Here are several examples.
+ **Outlier target values **- Skewed or unusual target distribution for regression, such as heavy tailed targets.
+ **High or low target cardinality **- Infrequent number of class labels or a large number of unique classes for classification.
For both regression and classification problem types, not valid values such as numeric infinity, `NaN` or empty space in target column are surfaced. Depending on the problem type, different dataset statistics are presented. A distribution of target column values for a regression problem allows you to verify if the distribution is what you expected.
The following screenshot shows an Autopilot data report, which includes statistics such as the mean, median, minimum, maximum, percentage of outliers in your dataset. The screenshot also includes a histogram showing the distribution of labels in the target column. The histogram shows **Target Column Values** on the horizontal axis and **Count **on the vertical axis. A box highlights the **Outliers Percentage** section of the screenshot to indicate where this statistic appears.
![\[Autopilot data report on the distribution of target column values.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-target-analysis.png)
Multiple statistics are shown regarding target values and their distribution. If any of the outliers, not valid values, or missing percentages are greater than zero, these values are surfaced so you can investigate why your data contains unusable target values. Some unusable target values are highlighted as a low severity insight warning.
In the following screenshot, a ` symbol was added accidentally to the target column, which prevented the numeric value of the target from being parsed. A **Low severity insight: "Invalid target values"** warning appears. The warning in this example states "0.14% of the labels in the target column could not be converted to numeric values. The most common non-numeric values are: ["-3.8e-05","-9-05","-4.7e-05","-1.4999999999999999e-05","-4.3e-05"]. That usually indicates that there are problems with data collection or processing. Amazon SageMaker Autopilot ignores all observations with invalid target label."
![\[Autopilot data report low severity warning about invalid target values.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-target-analysis-invalid-target-values.png)
Autopilot also provides a histogram showing the distribution of labels for classification.
The following screenshot shows an example of statistics given for your target column including the number of classes, missing or not valid values. A histogram with **Target Label** on the horizontal axis and **Frequency** on the vertical axis shows the distribution of each label category.
![\[Autopilot data report high cardinality for classification.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-target-analysis-invalid-classification.png)
**Note**
You can find definitions of all the terms presented in this and other sections in **Definitions** section at the bottom of the report notebook.
## Data Sample
Autopilot presents an actual sample of your data to help you spot issues with your dataset. The sample table scrolls horizontally. Inspect the sample data to verify that all the necessary columns are present in the dataset.
Autopilot also calculates a measure of prediction power, that can be used to identify a linear or nonlinear relationship between a feature and the target variable. A value of `0` indicates that the feature has no predictive value in predicting the target variable. A value of `1` indicates the highest predictive power for the target variable. For more information on predictive power, see the **Definitions** section.
**Note**
It is not recommended that you use prediction power as a substitute for feature importance. Only use it if you're certain that prediction power is an appropriate measure for your use case.
The following screenshot shows example data sample. The top row contains the prediction power of each column in your dataset. The second row contains the column data type. Subsequent rows contain the labels. The columns contain the target column followed by each feature column. Each feature column has an associated prediction power, highlighted in this screenshot, with a box. In this example, the column containing the feature `x51` has a predictive power of `0.68` for the target variable `y`. The feature `x55` is slightly less predictive with a prediction power of `0.59`.
![\[Autopilot data report data sample prediction power.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-data-sample-prediction.png)
## Duplicate rows
If duplicate rows are present in the dataset, Amazon SageMaker Autopilot displays a sample of them.
**Note**
It is not recommended to balance a dataset by up-sampling before providing it to Autopilot. This may result in inaccurate validation scores for the models trained by Autopilot, and the models that are produced may be unusable.
## Cross column correlations
Autopilot uses the Pearson's correlation coefficient, a measure of linear correlation between two features, to populate a correlation matrix. In the correlation matrix, numeric features are plotted on both the horizontal and vertical axes, with the Pearson's correlation coefficient plotted at their intersections. The higher the correlation between two features, the higher the coefficient, with a maximum value of `|1|`.
+ A value of `-1` indicates that the features are perfectly negatively correlated.
+ A value of `1`, which occurs when a feature is correlated with itself, indicates perfect positive correlation.
You can use the information in the correlation matrix to remove highly correlated features. A smaller number of features reduces chances of overfitting a model and can reduce the costs of production in two ways. It lessens the Autopilot runtime needed and, for some applications, can make data collection procedures cheaper.
The following screenshot shows an example of a correlation matrix between `7` features. Each feature is displayed in a matrix on both the horizontal and vertical axes. The Pearson's correlation coefficient is displayed at the intersection between two features. Each feature intersection has a color tone associated with it. The higher the correlation, the darker the tone. The darkest tones occupy the diagonal of the matrix, where each feature is correlated with itself, representing perfect correlation.
![\[Autopilot data report data cross-correlation matrix.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-data-cross-column-statistics.png)
## Anomalous Rows
Amazon SageMaker Autopilot detects which rows in your dataset might be anomalous. It then assigns an anomaly score to each row. Rows with negative anomaly scores are considered anomalous.
The following screenshot shows the output from an Autopilot analysis for rows containing anomalies. A column containing an anomalous score appears next to the dataset columns for each row.
![\[Autopilot dataset with anomalous rows, showing negative anomaly scores.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-data-anomalous-rows.png)
## Missing values, cardinality, and descriptive statistics
Amazon SageMaker Autopilot examines and reports on properties of the individual columns of your dataset. In each section of the data report that presents this analysis, the content is arranged in order. This is so you can check the most “suspicious” values first. Using these statistics you can improve contents of individual columns, and improve the quality of the model produced by Autopilot.
Autopilot calculates several statistics on the categorical values in columns that contain them. These include the number of unique entries and, for text, the number of unique words.
Autopilot calculates several standard statistics on the numerical values in columns that contain them. The following image depicts these statistics, including the mean, median, minimum and maximum values, and the percentages of numerical types and of outlier values.
![\[Autopilot data report statistics on columns with numerical values.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-data-report-data-descriptive-statistics.png)
# Find and run the candidate definition notebook
The candidate definition notebook contains each suggested preprocessing step, algorithm, and hyperparameter ranges.
You can choose which candidate to train and tune in two ways. The first, by running sections of the notebook. The second, by running the entire notebook to optimize all candidates to identify a best candidate. If you run the entire notebook, only the best candidate is displayed after job completion.
To run Autopilot from SageMaker Studio Classic, open the candidate definition notebook by following these steps:
1. Choose the **Home** icon ![\[Black square icon representing a placeholder or empty image.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/studio/icons/house.png) from the left navigation pane to view the top-level **Amazon SageMaker Studio Classic** navigation menu.
1. Select the **AutoML** card from the main working area. This opens a new **Autopilot** tab.
1. In the **Name** section, select the Autopilot job that has the candidate definition notebook that you want to examine. This opens a new **Autopilot job** tab.
1. Choose **Open candidate generation notebook** from the top right section of the **Autopilot job** tab. This opens a new read-only preview of the **Amazon SageMaker Autopilot Candidate Definition Notebook**.
To run the candidate definition notebook, follow these steps:
1. Choose **Import notebook** at the top right of the **Amazon SageMaker Autopilot Candidate Definition Notebook** tab. This opens a tab to set up a new notebook environment to run the notebook.
1. Select an existing SageMaker **Image** or use a **Custom Image**.
1. Select a **Kernel**, an **Instance type**, and an optional **Start-up script**.
You can now run the notebook in this new environment.
# Configure inference output in generated containers
Autopilot generates an ordered [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ContainerDefinition.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ContainerDefinition.html) list. This can be used to build a model to deploy in a machine learning pipeline. This model can be used for online hosting and inference.
Customers can list inference container definitions with the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidateForAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidateForAutoMLJob.html) API. The list of inference container definitions that represent the best candidate is also available in the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) response.
## Inference container definitions for regression and classification problem types
Autopilot generates inference containers specific to the [training mode](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html#autopilot-training-mode) and the [problem type](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-datasets-problem-types.html#autopilot-problem-types) of the job.
### Container definitions for hyperparameter optimization (HPO) mode
+ **Regression**: HPO generates two containers:
1. A feature engineering container that transforms the original features into features that the regression algorithms can train on.
1. An algorithm container that transforms features and generates a regression score for the dataset.
+ **Classification**: HPO generates three containers:
1. A feature engineering container that transforms the original features into features that the classification algorithms can train on.
1. An algorithm container that generates the `predicted_label` with the highest probability. This container can also produce the various probabilities associated with the classification outcomes in the inference response.
1. A feature engineering container that performs post-processing of the algorithm prediction. For example, it can perform an inverse transform on the predicted label and change it to the original label.
### Container definitions for ensembling mode
In ensembling mode, both regression and classification problem types have only one inference container. This inference container transforms the features and generates the predictions based on problem type.
## Inference responses per problem type
### Inference responses for classification models
For classification inference containers, you can select the content of the inference response by using four predefined keys:
+ `predicted_label`: The label with the highest probability of predicting the correct label, as determined by Autopilot.
+ `probability`:
+ **HPO models:** The probability of the `True` class for binary classification. The probability of the `predicted_label` for multiclass classification.
+ **Ensemble models:** The probability of the `predicted_label` for binary and multiclass classification.
+ `probabilities`: The list of probabilities for all corresponding classes.
+ `labels`: The list of all labels.
For example, for a binary classification problem, if you pass the inference response keys `['predicted_label', 'probability', 'probabilities', 'labels']` and the output response appears as `[1, 0.1, "[0.9, 0.1]", "['1', '0']"]`, you should interpret it as follows:
1. `predicted_label` equals `1` because label "1" has a higher probability (`0.9` in this case).
1. For HPO models, `probability` equals `0.1` which is the probability of the `positive_class` (`0` in this case) selected by Autopilot.
For Ensemble models, `probability` equals `0.9` which is the probability of the `predicted_label`.
1. `probabilities` lists the `probability` of each label in `labels`.
1. `labels` are the unique labels in the dataset, where the second label ("0" in this case) is the `positive_class` selected by Autopilot.
By default, inference containers are configured to generate only the `predicted_label`. To select additional inference content, you can update the `inference_response_keys` parameter to include up to these three environment variables:
+ `SAGEMAKER_INFERENCE_SUPPORTED`: This is set to provide hints to you about what content each container supports.
+ `SAGEMAKER_INFERENCE_INPUT`: This should be set to the keys that the container expects in input payload.
+ `SAGEMAKER_INFERENCE_OUTPUT`: This should be populated with the set of keys that the container outputs.
### Inference responses for classification models in HPO mode
This section shows how to configure the inference response from classification models using hyperparameter optimization (HPO) mode.
To choose the inference response content in HPO mode: Add the `SAGEMAKER_INFERENCE_INPUT` and `SAGEMAKER_INFERENCE_OUTPUT` variables to the second and third containers that are generated in HPO mode for classification problems.
The keys supported by the second container (algorithm) are predicted\$1label, probability, and probabilities. Note that `labels` is deliberately not added to `SAGEMAKER_INFERENCE_SUPPORTED`.
The keys supported by the third classification model container are `predicted_label`, `labels`, `probability`, and `probabilities`. Therefore, the `SAGEMAKER_INFERENCE_SUPPORTED` environment includes the names of these keys.
To update the definition of the inference containers to receive `predicted_label` and `probability`, use the following code example.
```
containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label, probability'})
containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
```
The following code example updates the definition of the inference containers to receive `predicted_label`, `probabilities`, and `labels`. Do not pass the `labels` to the second container (the algorithm container), because it is generated by the third container independently.
```
containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label,probabilities'})
containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label,probabilities'})
containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probabilities,labels'})
```
The following collapsible sections provide code examples for Amazon SDK for Python (Boto3) and for SageMaker SDK for Python. Each section shows how to select the content of the inference responses in HPO mode for the respective code example.
#### Amazon SDK for Python (Boto3)
```
import boto3
sm_client = boto3.client('sagemaker', region_name='')
role = ''
input_data = ''
output_path = ''
best_candidate = sm_client.describe_auto_ml_job(AutoMLJobName='')['BestCandidate']
best_candidate_containers = best_candidate['InferenceContainers']
best_candidate_name = best_candidate['CandidateName']
best_candidate_containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
best_candidate_containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label, probability'})
best_candidate_containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
# create model
reponse = sm_client.create_model(
ModelName = '',
ExecutionRoleArn = role,
Containers = best_candidate_containers
)
# Lauch Transform Job
response = sm_client.create_transform_job(
TransformJobName='',
ModelName='',
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': input_data
}
},
'ContentType': "text/CSV",
'SplitType': 'Line'
},
TransformOutput={
'S3OutputPath': output_path,
'AssembleWith': 'Line',
},
TransformResources={
'InstanceType': 'ml.m4.xlarge',
'InstanceCount': 1,
},
)
```
#### SageMaker SDK for Python
```
from sagemaker import AutoML
aml = AutoML.attach(auto_ml_job_name='')
aml_best_model = aml.create_model(name='',
candidate=None,
inference_response_keys**=['probabilities', 'labels'])
aml_transformer = aml_best_model.transformer(accept='text/csv',
assemble_with='Line',
instance_type='ml.m5.xlarge',
instance_count=1,)
aml_transformer.transform('',
content_type='text/csv',
split_type='Line',
job_name='',
wait=True)
```
### Inference responses for classification models in ensembling mode
This section shows how to configure the inference response from classification models using ensembling mode.
In **ensembling mode**, to choose the content of the inference response, update the `SAGEMAKER_INFERENCE_OUTPUT` environment variable.
The keys supported by the classification model container are `predicted_label`, `labels`, `probability`, and `probabilities`. These keys are included in the `SAGEMAKER_INFERENCE_SUPPORTED` environment.
To update the inference container definition to receive `predicted_label` and `probability`, refer to the following code example.
```
containers[0]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
```
The following collapsible section provides a code example for selecting the content of the inference responses in ensembling mode. The example uses Amazon SDK for Python (Boto3).
#### Amazon SDK for Python (Boto3)
```
import boto3
sm_client = boto3.client('sagemaker', region_name='')
role = ''
input_data = ''
output_path = ''
best_candidate = sm_client.describe_auto_ml_job(AutoMLJobName='')['BestCandidate']
best_candidate_containers = best_candidate['InferenceContainers']
best_candidate_name = best_candidate['CandidateName']
*best_candidate_containers[0]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
*
# create model
reponse = sm_client.create_model(
ModelName = '',
ExecutionRoleArn = role,
Containers = best_candidate_containers
)
# Lauch Transform Job
response = sm_client.create_transform_job(
TransformJobName='',
ModelName='',
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': input_data
}
},
'ContentType': "text/CSV",
'SplitType': 'Line'
},
TransformOutput={
'S3OutputPath': output_path,
'AssembleWith': 'Line',
},
TransformResources={
'InstanceType': 'ml.m4.xlarge',
'InstanceCount': 1,
},
)
```
The following collapsible section provides a code example that is identical to the SageMaker SDK for Python example for HPO. It is included for your convenience.
#### SageMaker SDK for Python
The following HPO code example uses SageMaker SDK for Python.
```
from sagemaker import AutoML
aml = AutoML.attach(auto_ml_job_name='')
aml_best_model = aml.create_model(name='',
candidate=None,
*inference_response_keys**=['probabilities', 'labels'])*
aml_transformer = aml_best_model.transformer(accept='text/csv',
assemble_with='Line',
instance_type='ml.m5.xlarge',
instance_count=1,)
aml_transformer.transform('',
content_type='text/csv',
split_type='Line',
job_name='',
wait=True)
```
# Create an Image Classification Job using the AutoML API
The following instructions show how to create an Amazon SageMaker Autopilot job as a pilot experiment for image classification problem types using SageMaker [API Reference](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html).
**Note**
Tasks such as text and image classification, time-series forecasting, and fine-tuning of large language models are exclusively available through the version 2 of the [AutoML REST API](autopilot-reference.md). If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_auto_ml_job_v2.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
Users who prefer the convenience of a user interface can use [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas-getting-started.html) to access pre-trained models and generative AI foundation models, or create custom models tailored for specific text, image classification, forecasting needs, or generative AI.
You can create an Autopilot image classification experiment programmatically by calling the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) API action in any language supported by Amazon SageMaker Autopilot or the Amazon CLI.
For information on how this API action translates into a function in the language of your choice, see the [ See Also](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_SeeAlso) section of `CreateAutoMLJobV2` and choose an SDK. As an example, for Python users, see the full request syntax of `[create\$1auto\$1ml\$1job\$1v2](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_auto_ml_job_v2)` in Amazon SDK for Python (Boto3).
The following is a collection of mandatory and optional input request parameters for the `CreateAutoMLJobV2` API action used in image classification.
## Required parameters
When calling `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)` to create an Autopilot experiment for image classification, you must provide the following values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax)` to specify the name of your job.
+ At least one `[AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)` in `[AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)` to specify your data source.
+ An `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)` of type `[ImageClassificationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ImageClassificationJobConfig.html)`.
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data.
All other parameters are optional.
## Optional parameters
The following sections provide details of some optional parameters that you can pass to your image classification AutoML job.
### How to specify the training and validation datasets of an AutoML job
You can provide your own validation dataset and custom data split ratio, or let Autopilot split the dataset automatically.
Each [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html) object (see the required parameter [AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker-api/src/AWSSageMakerAPIDoc/build/server-root/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)) has a `ChannelType`, which can be set to either `training` or `validation` values that specify how the data is to be used when building a machine learning model.
At least one data source must be provided and a maximum of two data sources is allowed: one for training data and one for validation data. How you split the data into training and validation datasets depends on whether you have one or two data sources.
How you split the data into training and validation datasets depends on whether you have one or two data sources.
+ If you only have **one data source**, the `ChannelType` is set to `training` by default and must have this value.
+ If the `ValidationFraction` value in [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html) is not set, 0.2 (20%) of the data from this source is used for validation by default.
+ If the `ValidationFraction` is set to a value between 0 and 1, the dataset is split based on the value specified, where the value specifies the fraction of the dataset used for validation.
+ If you have **two data sources**, the `ChannelType` of one of the `AutoMLJobChannel` objects must be set to `training`, the default value. The `ChannelType` of the other data source must be set to `validation`. The two data sources must have the same format, either CSV or Parquet, and the same schema. You must not set the value for the `ValidationFraction` in this case because all of the data from each source is used for either training or validation. Setting this value causes an error.
### How to specify the automatic model deployment configuration for an AutoML job
To enable automatic deployment for the best model candidate of an AutoML job, include a `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the AutoML job request. This will allow the deployment of the best model to a SageMaker AI endpoint. Below are the available configurations for customization.
+ To let Autopilot generate the endpoint name, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)` to `True`.
+ To provide your own name for the endpoint, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents) to False and provide a name of your choice in [EndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)`.
# Datasets format and objective metric for image classification
In this section we learn about the available formats for datasets used in image classification as well as the objective metric used to evaluate the predictive quality of machine learning model candidates. The metrics calculated for candidates are specified using an array of [MetricDatum](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_MetricDatum.html) types.
## Datasets formats
Autopilot supports .png, .jpg, and .jpeg image formats. If your dataset contains all .png images use `image/png`, if it contains all .jpg or .jpeg images use `image/jpeg`, and if your dataset contains a mix of image formats use `image/*`.
## Objective metric
The following list contains the names of the metrics that are currently available to measure the performance of models for image classification.
**`Accuracy`**
The ratio of the number of correctly classified items to the total number of (correctly and incorrectly) classified items. Accuracy measures how close the predicted class values are to the actual values. Values for accuracy metrics vary between zero (0) and one (1). A value of 1 indicates perfect accuracy, and 0 indicates perfect inaccuracy.
# Deploy Autopilot models for real-time inference
After you train your Amazon SageMaker Autopilot models, you can set up an endpoint and obtain predictions interactively. The following section describes the steps for deploying your model to a SageMaker AI real-time inference endpoint to get predictions from your model.
## Real-time inferencing
Real-time inference is ideal for inference workloads where you have real-time, interactive, low latency requirements. This section shows how you can use real-time inferencing to obtain predictions interactively from your model.
You can use SageMaker APIs to manually deploy the model that produced the best validation metric in an Autopilot experiment as follows.
Alternatively, you can chose the automatic deployment option when creating your Autopilot experiment. For information on setting up the automatic deployment of models, see `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the request parameters of `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestParameters)`. This creates an endpoint automatically.
**Note**
To avoid incurring unnecessary charges, you can delete unneeded endpoint and resources created from model deployment. For information about pricing of instances by Region, see [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing/).
1. **Obtain the candidate container definitions**
Obtain the candidate container definitions from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers). A container definition for inference refers to the containerized environment designed for deploying and running your trained SageMaker AI model to make predictions.
The following Amazon CLI command example uses the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) API to obtain candidate definitions for the best model candidate.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name job-name --region region
```
1. **List candidates**
The following Amazon CLI command example uses the [ListCandidatesForAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html) API to list all model candidates.
```
aws sagemaker list-candidates-for-auto-ml-job --auto-ml-job-name --region
```
1. **Create a SageMaker AI model**
Use the container definitions from the previous steps and a candidate of your choice to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker create-model --model-name '' \
--containers [', , ]' \
--execution-role-arn '' --region '
```
1. **Create an endpoint configuration**
The following Amazon CLI command example uses the [CreateEndpointConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpointConfig.html) API to create an endpoint configuration.
```
aws sagemaker create-endpoint-config --endpoint-config-name '' \
--production-variants '' \
--region ''
```
1. **Create the endpoint**
The following Amazon CLI example uses the [CreateEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpoint.html) API to create the endpoint.
```
aws sagemaker create-endpoint --endpoint-name '' \
--endpoint-config-name '' \
--region ''
```
Check the progress of your endpoint deployment by using the [DescribeEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeEndpoint.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-endpoint —endpoint-name '' —region
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready to use for real-time inference.
1. **Invoke the endpoint**
The following command structure invokes the endpoint for real-time inferencing.
```
aws sagemaker invoke-endpoint --endpoint-name '' \
--region '' --body '' [--content-type] ''
```
# Explainability report
Amazon SageMaker Autopilot provides an explainability report to help explain how a best model candidate makes predictions for image classification problems. This report can assist ML engineers, product managers, and other internal stakeholders in understanding the characteristics of the model. Both consumers and regulators rely on transparency in machine learning to trust and interpret decisions made on model predictions. You can use these explanations for auditing and meeting regulatory requirements, establishing trust in the model, supporting human decision-making, and debugging and improving model performance.
The Autopilot explanatory functionality for image classification uses a visual class activation map (CAM) approach that produces a heatmap where the distribution and intensity of each color highlights the areas of an image that contribute the most to a specific prediction. This approach relies on principal components derived from an implementation of [Eigen-CAM](https://arxiv.org/ftp/arxiv/papers/2008/2008.00299.pdf).
Autopilot generates the explainability report as a JSON file. The report includes analysis details that are based on the validation dataset. Each image used to generate the report contains the following information:
+ `input_image_uri`: The Amazon S3 URI to the input image taken as input for the heatmap.
+ `heatmap_image_uri`: The Amazon S3 URI to the heatmap image generated by Autopilot.
+ `predicted_label`: The label class predicted by best model trained by Autopilot.
+ `probability`: The confidence with which the `predicted_label` is predicted.
You can find the Amazon S3 prefix to the explainability artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.Explainability](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-Explainability)`.
The following examples illustrates what the heatmaps look like on few samples from [Oxford-IIIT Pet Dataset](https://www.robots.ox.ac.uk/~vgg/data/pets/). The heatmap image displays color gradients that indicate the relative importance of different features within the image. The red color represents regions with greater importance in predicting the "predicted\$1label" of the input image compared to the features represented by the blue color.
****
| Input Image | Heatmap Image |
| --- | --- |
| ![\[The original image of a dog.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-image-classification-explainability-img1-input.png) | ![\[A dog with a heatmap highlighting the regions with the greater contribution to the predicted label.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-image-classification-explainability-img1-output.png) |
| ![\[The original image of a cat.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-image-classification-explainability-img2-input.png) | ![\[A cat with a heatmap highlighting the regions with the greater contribution to the predicted label.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-image-classification-explainability-img2-output.png) |
# Model performance report
An Amazon SageMaker AI model quality report (also referred to as performance report) provides insights and quality information for the best model candidate generated by an AutoML job. This includes information about the job details, model problem type, objective function, and various metrics. This section details the content of a performance report for image classification problems and explains how to access the metrics as raw data in a JSON file.
You can find the Amazon S3 prefix to the model quality report artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.ModelInsights](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-ModelInsights)`.
The performance report contains two sections:
+ The first section contains details about the Autopilot job that produced the model.
+ The second section contains a model quality report with various performance metrics.
## Autopilot job details
This first section of the report gives some general information about the Autopilot job that produced the model. These details include the following information:
+ Autopilot candidate name: The name of the best model candidate.
+ Autopilot job name: The name of the job.
+ Problem type: The problem type. In our case, *image classification*.
+ Objective metric: The objective metric used to optimize the performance of the model. In our case, *Accuracy*.
+ Optimization direction: Indicates whether to minimize or maximize the objective metric.
## Model quality report
Model quality information is generated by Autopilot model insights. The report's content that is generated depends on the problem type it addressed. The report specifies the number of rows that were included in the evaluation dataset and the time at which the evaluation occurred.
### Metrics tables
The first part of the model quality report contains metrics tables. These are appropriate for the type of problem that the model addressed.
The following image is an example of a metrics table generates by Autopilot for an image or text classification problem. It shows the metric name, value, and standard deviation.
![\[Amazon SageMaker Autopilot model insights image or text classification metrics report example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-multiclass-metrics-report.png)
### Graphical model performance information
The second part of the model quality report contains graphical information to help you evaluate model performance. The contents of this section depend on the selected problem type.
#### Confusion matrix
A confusion matrix provides a way to visualize the accuracy of the predictions made by a model for binary and multiclass classification for different problems.
A summary of the graph's components of **false positive rate **(FPR) and **true positive rate **(TPR) are defined as follows.
+ Correct predictions
+ **True positive** (TP): The predicted the value is 1, and the true value is 1.
+ **True negative** (TN): The predicted the value is 0, and the true value is 0.
+ Erroneous predictions
+ **False positive** (FP): The predicted the value is 1, but the true value is 0.
+ **False negative** (FN): The predicted the value is 0, but the true value is 1.
The confusion matrix in the model quality report contains the following.
+ The number and percentage of correct and incorrect predictions for the actual labels
+ The number and percentage of accurate predictions on the diagonal from the upper-left to the lower-right corner
+ The number and percentage of inaccurate predictions on the diagonal from the upper-right to the lower-left corner
The incorrect predictions on a confusion matrix are the confusion values.
The following diagram is an example of a confusion matrix for a multi-class classification problem. The confusion matrix in the model quality report contains the following.
+ The vertical axis is divided into three rows containing three different actual labels.
+ The horizontal axis is divided into three columns containing labels that were predicted by the model.
+ The color bar assigns a darker tone to a larger number of samples to visually indicate the number of values that were classified in each category.
In the example below, the model correctly predicted actual 354 values for label **f**, 1094 values for label **i** and 852 values for label **m**. The difference in tone indicates that the dataset is not balanced because there are many more labels for the value **i** than for **f** or **m**.
![\[Amazon SageMaker Autopilot multiclass confusion matrix example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-confusion-matrix-multiclass.png)
The confusion matrix in the model quality report provided can accommodate a maximum of 15 labels for multiclass classification problem types. If a row corresponding to a label shows a `Nan` value, it means that the validation dataset used to check model predictions does not contain data with that label.
# Create an AutoML job for text classification using the API
The following instructions show how to create an Amazon SageMaker Autopilot job as a pilot experiment for text classification problem types using SageMaker [API Reference](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html).
**Note**
Tasks such as text and image classification, time-series forecasting, and fine-tuning of large language models are exclusively available through the version 2 of the [AutoML REST API](autopilot-reference.md). If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_auto_ml_job_v2.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
Users who prefer the convenience of a user interface can use [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas-getting-started.html) to access pre-trained models and generative AI foundation models, or create custom models tailored for specific text, image classification, forecasting needs, or generative AI.
You can create an Autopilot text classification experiment programmatically by calling the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) API action in any language supported by Amazon SageMaker Autopilot or the Amazon CLI.
For information on how this API action translates into a function in the language of your choice, see the [ See Also](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_SeeAlso) section of `CreateAutoMLJobV2` and choose an SDK. As an example, for Python users, see the full request syntax of `[create\$1auto\$1ml\$1job\$1v2](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_auto_ml_job_v2)` in Amazon SDK for Python (Boto3).
The following is a collection of mandatory and optional input request parameters for the `CreateAutoMLJobV2` API action used in text classification.
## Required parameters
When calling `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)` to create an Autopilot experiment for text classification, you must provide the following values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax)` to specify the name of your job.
+ At least one `[AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)` in `[AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)` to specify your data source.
+ An `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)` of type `[TextClassificationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextClassificationJobConfig.html)`.
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data.
All other parameters are optional.
## Optional parameters
The following sections provide details of some optional parameters that you can pass to your text classification AutoML job.
### How to specify the training and validation datasets of an AutoML job
You can provide your own validation dataset and custom data split ratio, or let Autopilot split the dataset automatically.
Each [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html) object (see the required parameter [AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker-api/src/AWSSageMakerAPIDoc/build/server-root/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)) has a `ChannelType`, which can be set to either `training` or `validation` values that specify how the data is to be used when building a machine learning model.
At least one data source must be provided and a maximum of two data sources is allowed: one for training data and one for validation data. How you split the data into training and validation datasets depends on whether you have one or two data sources.
How you split the data into training and validation datasets depends on whether you have one or two data sources.
+ If you only have **one data source**, the `ChannelType` is set to `training` by default and must have this value.
+ If the `ValidationFraction` value in [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html) is not set, 0.2 (20%) of the data from this source is used for validation by default.
+ If the `ValidationFraction` is set to a value between 0 and 1, the dataset is split based on the value specified, where the value specifies the fraction of the dataset used for validation.
+ If you have **two data sources**, the `ChannelType` of one of the `AutoMLJobChannel` objects must be set to `training`, the default value. The `ChannelType` of the other data source must be set to `validation`. The two data sources must have the same format, either CSV or Parquet, and the same schema. You must not set the value for the `ValidationFraction` in this case because all of the data from each source is used for either training or validation. Setting this value causes an error.
### How to specify the automatic model deployment configuration for an AutoML job
To enable automatic deployment for the best model candidate of an AutoML job, include a `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the AutoML job request. This will allow the deployment of the best model to a SageMaker AI endpoint. Below are the available configurations for customization.
+ To let Autopilotgenerate the endpoint name, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)` to `True`.
+ To provide your own name for the endpoint, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents) to False and provide a name of your choice in [EndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)`.
# Datasets format and objective metric for text classification
In this section we learn about the available formats for datasets used in text classification as well as the metric used to evaluate the predictive quality of machine learning model candidates. The metrics calculated for candidates are specified using an array of [MetricDatum](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_MetricDatum.html) types.
## Datasets formats
Autopilot supports tabular data formatted as CSV files or as Parquet files. For tabular data, each column contains a feature with a specific data type and each row contains an observation. The properties of these two file formats differ considerably.
+ **CSV** (comma-separated-values) is a row-based file format that stores data in human readable plaintext which a popular choice for data exchange as they are supported by a wide range of applications.
+ **Parquet** is a column-based file format where the data is stored and processed more efficiently than row-based file formats. This makes them a better option for big data problems.
The **data types** accepted for columns include numerical, categorical, text.
Autopilot supports building machine learning models on large datasets up to hundreds of GBs. For details on the default resource limits for input datasets and how to increase them, see [Amazon SageMaker Autopilot quotas](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-quotas.html).
## Objective metric
The following list contains the names of the metrics that are currently available to measure the performance of models for text classification.
**`Accuracy`**
The ratio of the number of correctly classified items to the total number of (correctly and incorrectly) classified items. Accuracy measures how close the predicted class values are to the actual values. Values for accuracy metrics vary between zero (0) and one (1). A value of 1 indicates perfect accuracy, and 0 indicates perfect inaccuracy.
# Deploy Autopilot models for real-time inference
After you train your Amazon SageMaker Autopilot models, you can set up an endpoint and obtain predictions interactively. The following section describes the steps for deploying your model to a SageMaker AI real-time inference endpoint to get predictions from your model.
## Real-time inferencing
Real-time inference is ideal for inference workloads where you have real-time, interactive, low latency requirements. This section shows how you can use real-time inferencing to obtain predictions interactively from your model.
You can use SageMaker APIs to manually deploy the model that produced the best validation metric in an Autopilot experiment as follows.
Alternatively, you can chose the automatic deployment option when creating your Autopilot experiment. For information on setting up the automatic deployment of models, see `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the request parameters of `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestParameters)`. This creates an endpoint automatically.
**Note**
To avoid incurring unnecessary charges, you can delete unneeded endpoint and resources created from model deployment. For information about pricing of instances by Region, see [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing/).
1. **Obtain the candidate container definitions**
Obtain the candidate container definitions from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers). A container definition for inference refers to the containerized environment designed for deploying and running your trained SageMaker AI model to make predictions.
The following Amazon CLI command example uses the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) API to obtain candidate definitions for the best model candidate.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name job-name --region region
```
1. **List candidates**
The following Amazon CLI command example uses the [ListCandidatesForAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html) API to list all model candidates.
```
aws sagemaker list-candidates-for-auto-ml-job --auto-ml-job-name --region
```
1. **Create a SageMaker AI model**
Use the container definitions from the previous steps and a candidate of your choice to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker create-model --model-name '' \
--containers [', , ]' \
--execution-role-arn '' --region '
```
1. **Create an endpoint configuration**
The following Amazon CLI command example uses the [CreateEndpointConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpointConfig.html) API to create an endpoint configuration.
```
aws sagemaker create-endpoint-config --endpoint-config-name '' \
--production-variants '' \
--region ''
```
1. **Create the endpoint**
The following Amazon CLI example uses the [CreateEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpoint.html) API to create the endpoint.
```
aws sagemaker create-endpoint --endpoint-name '' \
--endpoint-config-name '' \
--region ''
```
Check the progress of your endpoint deployment by using the [DescribeEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeEndpoint.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-endpoint —endpoint-name '' —region
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready to use for real-time inference.
1. **Invoke the endpoint**
The following command structure invokes the endpoint for real-time inferencing.
```
aws sagemaker invoke-endpoint --endpoint-name '' \
--region '' --body '' [--content-type] ''
```
# Explainability report
Amazon SageMaker Autopilot provides an explainability report to help explain how a best model candidate makes predictions for text classification problems. This report can assist ML engineers, product managers, and other internal stakeholders in understanding the characteristics of the model. Both consumers and regulators rely on transparency in machine learning to trust and interpret decisions made on model predictions. You can use these explanations for auditing and meeting regulatory requirements, establishing trust in the model, supporting human decision-making, and debugging and improving model performance.
The Autopilot explanatory functionality for text classification uses the axiomatic attribution method *Integrated Gradients*. This approach relies on an implementation of [Axiomatic Attribution for Deep Network](https://arxiv.org/pdf/1703.01365.pdf).
Autopilot generates the explainability report as a JSON file. The report includes analysis details that are based on the validation dataset. Each sample used to generate the report contains the following information:
+ `text`: The input text content explained.
+ `token_scores`: The list of scores for every token in the text.
+
+ `attribution`: The score depicting the importance of the token.
+ `description.partial_text`: The partial substring that represents the token.
+ `predicted_label`: The label class predicted by the best model candidate.
+ `probability`: The confidence with which the `predicted_label` was predicted.
You can find the Amazon S3 prefix to the explainability artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.Explainability](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-Explainability)`.
The following is an example of analysis content that you could find in the explainability artifacts.
```
{
"text": "It was a fantastic movie!",
"predicted_label": 2,
"probability": 0.9984835,
"token_scores": [
{
"attribution": 0,
"description": {
"partial_text": "It"
}
},
{
"attribution": -0.022447118861679088,
"description": {
"partial_text": "was"
}
},
{
"attribution": -0.2164326456817965,
"description": {
"partial_text": "a"
}
},
{
"attribution": 0.675,
"description": {
"partial_text": "fantastic"
}
},
{
"attribution": 0.416,
"description": {
"partial_text": "movie!"
}
}
]
}
```
In this sample of the JSON report, the explanatory functionality evaluates the text `It was a fantastic movie!` and scores the contribution of each of its token to the overall predicted label. The predicted label is `2`, which is a strong positive sentiment, with a probability of 99.85%. The JSON sample then details the contribution of each individual token to this prediction. For example, the token `fantastic` has a stronger attribution than the token `was`. It is the token that contributed the most to the final prediction.
# Model performance report
An Amazon SageMaker AI model quality report (also referred to as performance report) provides insights and quality information for the best model candidate generated by an AutoML job. This includes information about the job details, model problem type, objective function, and various metrics. This section details the content of a performance report for text classification problems and explains how to access the metrics as raw data in a JSON file.
You can find the Amazon S3 prefix to the model quality report artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.ModelInsights](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-ModelInsights)`.
The performance report contains two sections:
+ The first section contains details about the Autopilot job that produced the model.
+ The second section contains a model quality report with various performance metrics.
## Autopilot job details
This first section of the report gives some general information about the Autopilot job that produced the model. These details include the following information:
+ Autopilot candidate name: The name of the best model candidate.
+ Autopilot job name: The name of the job.
+ Problem type: The problem type. In our case, *text classification*.
+ Objective metric: The objective metric used to optimize the performance of the model. In our case, *Accuracy*.
+ Optimization direction: Indicates whether to minimize or maximize the objective metric.
## Model quality report
Model quality information is generated by Autopilot model insights. The report's content that is generated depends on the problem type it addressed. The report specifies the number of rows that were included in the evaluation dataset and the time at which the evaluation occurred.
### Metrics tables
The first part of the model quality report contains metrics tables. These are appropriate for the type of problem that the model addressed.
The following image is an example of a metrics table generated by Autopilot for an image or text classification problem. It shows the metric name, value, and standard deviation.
![\[Amazon SageMaker Autopilot model insights image or text classification metrics report example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-multiclass-metrics-report.png)
### Graphical model performance information
The second part of the model quality report contains graphical information to help you evaluate model performance. The contents of this section depend on the selected problem type.
#### Confusion matrix
A confusion matrix provides a way to visualize the accuracy of the predictions made by a model for binary and multiclass classification for different problems.
A summary of the graph's components of **false positive rate **(FPR) and **true positive rate **(TPR) are defined as follows.
+ Correct predictions
+ **True positive** (TP): The predicted the value is 1, and the true value is 1.
+ **True negative** (TN): The predicted the value is 0, and the true value is 0.
+ Erroneous predictions
+ **False positive** (FP): The predicted the value is 1, but the true value is 0.
+ **False negative** (FN): The predicted the value is 0, but the true value is 1.
The confusion matrix in the model quality report contains the following.
+ The number and percentage of correct and incorrect predictions for the actual labels
+ The number and percentage of accurate predictions on the diagonal from the upper-left to the lower-right corner
+ The number and percentage of inaccurate predictions on the diagonal from the upper-right to the lower-left corner
The incorrect predictions on a confusion matrix are the confusion values.
The following diagram is an example of a confusion matrix for a multi-class classification problem. The confusion matrix in the model quality report contains the following.
+ The vertical axis is divided into three rows containing three different actual labels.
+ The horizontal axis is divided into three columns containing labels that were predicted by the model.
+ The color bar assigns a darker tone to a larger number of samples to visually indicate the number of values that were classified in each category.
In the example below, the model correctly predicted actual 354 values for label **f**, 1094 values for label **i** and 852 values for label **m**. The difference in tone indicates that the dataset is not balanced because there are many more labels for the value **i** than for **f** or **m**.
![\[Amazon SageMaker Autopilot multiclass confusion matrix example.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-model-insights-confusion-matrix-multiclass.png)
The confusion matrix in the model quality report provided can accommodate a maximum of 15 labels for multiclass classification problem types. If a row corresponding to a label shows a `Nan` value, it means that the validation dataset used to check model predictions does not contain data with that label.
# Create an AutoML job for time-series forecasting using the API
Forecasting in machine learning refers to the process of predicting future outcomes or trends based on historical data and patterns. By analyzing past time-series data and identifying underlying patterns, machine learning algorithms can make predictions and provide valuable insights into future behavior. In forecasting, the goal is to develop models that can accurately capture the relationship between input variables and the target variable over time. This involves examining various factors such as trends, seasonality, and other relevant patterns within the data. The collected information is then used to train a machine learning model. The trained model is capable of generating predictions by taking new input data and applying the learned patterns and relationships. It can provide forecasts for a wide range of use cases, such as sales projections, stock market trends, weather forecasts, demand forecasting, and many more.
The following instructions show how to create an Amazon SageMaker Autopilot job as a pilot experiment for time-series forecasting problem types using SageMaker [API Reference](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html).
**Note**
Tasks such as text and image classification, time-series forecasting, and fine-tuning of large language models are exclusively available through the version 2 of the [AutoML REST API](autopilot-reference.md). If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_auto_ml_job_v2.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
Users who prefer the convenience of a user interface can use [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas-getting-started.html) to access pre-trained models and generative AI foundation models, or create custom models tailored for specific text, image classification, forecasting needs, or generative AI.
You can create an Autopilot time-series forecasting experiment programmatically by calling the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) API in any language supported by Amazon SageMaker Autopilot or the Amazon CLI.
For information on how this API action translates into a function in the language of your choice, see the [ See Also](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_SeeAlso) section of `CreateAutoMLJobV2` and choose an SDK. As an example, for Python users, see the full request syntax of `[create\$1auto\$1ml\$1job\$1v2](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_auto_ml_job_v2)` in Amazon SDK for Python (Boto3).
Autopilot trains several model candidates with your target time-series, then selects an optimal forecasting model for a given objective metric. When your model candidates have been trained, you can find the best candidate metrics in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateProperties.html#sagemaker-Type-CandidateProperties-CandidateMetrics)`.
The following sections define the mandatory and optional input request parameters for the `CreateAutoMLJobV2` API used in time-series forecasting.
**Note**
Refer to the notebook [Time-Series Forecasting with Amazon SageMaker Autopilot](https://github.com/aws/amazon-sagemaker-examples/blob/main/autopilot/autopilot_time_series.ipynb) for a practical, hands-on time-series forecasting example. In this notebook, you use Amazon SageMaker Autopilot to train a time-series model and produce predictions using the trained model. The notebook provides instructions for retrieving a ready-made dataset of tabular historical data on Amazon S3.
## Prerequisites
Before using Autopilot to create a time-series forecasting experiment in SageMaker AI, make sure to:
+ Prepare your time-series dataset. Dataset preparation involves collecting relevant data from various sources, cleaning and filtering it to remove noise and inconsistencies, and organizing it into a structured format. See [Time-series datasets format and missing values filling methods](timeseries-forecasting-data-format.md) to learn more about time-series formats requirements in Autopilot. Optionally, you can supplement your dataset with the public holiday calendar of the country of your choice to capture associated patterns. For more information on holiday calendars, see [National holiday calendars](autopilot-timeseries-forecasting-holiday-calendars.md).
**Note**
We recommend providing at least 3-5 historical data points for each 1 future data point you want to predict. For example, to forecast 7 days ahead (horizon of 1 week) based on daily data, train your model on a minimum of 21-35 days of historical data. Make sure to provide enough data to capture seasonal and recurrent patterns.
+ Place your time-series data in an Amazon S3 bucket.
+ Grant full access to the Amazon S3 bucket containing your input data for the SageMaker AI execution role used to run your experiment. Once this is done, you can use the ARN of this execution role in Autopilot API requests.
+ For information on retrieving your SageMaker AI execution role, see [Get your execution role](sagemaker-roles.md#sagemaker-roles-get-execution-role).
+ For information on granting your SageMaker AI execution role permissions to access one or more specific buckets in Amazon S3, see * Add Additional Amazon S3 Permissions to a SageMaker AI Execution Role* in [Create execution role](sagemaker-roles.md#sagemaker-roles-create-execution-role).
## Required parameters
When calling `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)` to create an Autopilot experiment for time-series forecasting, you must provide the following values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax)` to specify the name of your job. The name should be of type `string`, and should have a minimum length of 1 character and a maximum length of 32.
+ At least one `[AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)` in `[AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)` in which you specify the name of the Amazon S3 bucket that contains your data. Optionally, you can specify the content (CSV or Parquet files) and compression (GZip) types.
+ An `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)` of type `[TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html)` to configure the settings of your time-series forecasting job. In particular, you must specify:
+ The **frequency** of predictions, which refers to the desired granularity (hourly, daily, monthly, etc) of your forecast.
Valid intervals are an integer followed by `Y` (Year), `M` (Month), `W` (Week), `D` (Day), `H` (Hour), and `min` (Minute). For example, `1D` indicates every day and `15min` indicates every 15 minutes. The value of a frequency must not overlap with the next larger frequency. For example, you must use a frequency of `1H` instead of `60min`.
The valid values for each frequency are the following:
+ Minute - 1-59
+ Hour - 1-23
+ Day - 1-6
+ Week - 1-4
+ Month - 1-11
+ Year - 1
+ The **horizon** of predictions in your forecast, which refers to the number of time-steps that the model predicts. The forecast horizon is also called the prediction length. The maximum forecast horizon is the lesser of 500 time-steps or 1/4 of the time-steps in the dataset.
+ A [TimeSeriesConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesConfig.html) in which you define the schema of your dataset to map the column headers to your forecast by specifying:
+ A `TargetAttributeName`: The column that contains historical data of the target field to forecast.
+ A `TimestampAttributeName`: The column that contains a point in time at which the target value of a given item is recorded.
+ A `ItemIdentifierAttributeName`: The column that contains the item identifiers for which you want to predict the target value.
The following is an example of those request parameters. In this example, you are setting up a daily forecast for the expected quantity or level of demand of specific items over a period of 20 days.
```
"AutoMLProblemTypeConfig": {
"ForecastFrequency": "D",
"ForecastHorizon": 20,
"TimeSeriesConfig": {
"TargetAttributeName": "demand",
"TimestampAttributeName": "timestamp",
"ItemIdentifierAttributeName": "item_id"
},
```
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data. You can use the ARN of the execution role to which you have granted access to your data.
All other parameters are optional. For example, you can set specific forecast quantiles, choose a filling method for missing values in the dataset, or define how to aggregate data that does not align with forecast frequency. To learn how to set those additional parameters, see [Optional parameters](#timeseries-forecasting-api-optional-params).
## Optional parameters
The following sections provide details of some optional parameters that you can pass to your time-series forecasting AutoML job.
### How to specify algorithms
By default, your Autopilot job trains a pre-defined list of algorithms on your dataset. However, you can provide a subset of the default selection of algorithms.
For time-series forecasting, you must choose `[TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html)` as the type of `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
Then, you can specify an array of selected `AutoMLAlgorithms` in the `AlgorithmsConfig` attribute of [CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html).
The following is an example of an `AlgorithmsConfig` attribute listing exactly three algorithms ("cnn-qr", "prophet", "arima") in its `AutoMLAlgorithms` field.
```
{
"[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)": {
"[TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html)": {
"[CandidateGenerationConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html)": {
"[AlgorithmsConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html#sagemaker-Type-CandidateGenerationConfig-AlgorithmsConfig)":[
{"[AutoMLAlgorithms](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html)":["cnn-qr", "prophet", "arima"]}
]
},
},
},
}
```
For the list of available algorithms for time-series forecasting, see [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html#sagemaker-Type-AutoMLAlgorithmConfig-AutoMLAlgorithms](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html#sagemaker-Type-AutoMLAlgorithmConfig-AutoMLAlgorithms). For details on each algorithm, see [Algorithms support for time-series forecasting](timeseries-forecasting-algorithms.md).
### How to specify custom quantiles
Autopilot trains 6 models candidates with your target time-series, then combines these models using a stacking ensemble method to create an optimal forecasting model for a given objective metric. Each Autopilot forecasting model generates a probabilistic forecast by producing forecasts at quantiles between P1 and P99. These quantiles are used to account for forecast uncertainty. By default, forecasts will be generated for the 0.1 (`p10`), 0.5 (`p50`), and 0.9 (`p90`). You can choose to specify your own quantiles.
In Autopilot, you can specify up to five forecast quantiles from 0.01 (`p1`) to 0.99 (`p99`), by increments of 0.01 or higher in the `ForecastQuantiles` attribute of [TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html).
In the following example, you are setting up a daily 10th, 25th, 50th, 75th, and 90th percentile forecast for the expected quantity or level of demand of specific items over a period of 20 days.
```
"AutoMLProblemTypeConfig": {
"ForecastFrequency": "D",
"ForecastHorizon": 20,
"ForecastQuantiles": ["p10", "p25", "p50", "p75", "p90"],
"TimeSeriesConfig": {
"TargetAttributeName": "demand",
"TimestampAttributeName": "timestamp",
"ItemIdentifierAttributeName": "item_id"
},
```
### How to aggregate data for different forecast frequencies
To create a forecast model (also referred to as the best model candidate from your experiment), you must specify a forecast frequency. The forecast frequency determines the frequency of predictions in your forecasts. For example, monthly sales forecasts. Autopilot best model can generate forecasts for data frequencies that are higher than the frequency at which your data is recorded.
During training, Autopilot aggregates any data that does not align with the forecast frequency you specify. For example, you might have some daily data but specify a weekly forecast frequency. Autopilot aligns the daily data based on the week that it belongs in. Autopilot then combines it into single record for each week.
During aggregation, the default transformation method is to sum the data. You can configure the aggregation when you create your AutoML job in the `Transformations` attribute of [TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html). The supported aggregation methods are `sum` (default), `avg`, `first`, `min`, `max`. Aggregation is only supported for the target column.
In the following example, you configure the aggregation to calculate the average of the individual promo forecasts to provide the final aggregated forecast values.
```
"Transformations": {
"Aggregation": {
"promo": "avg"
}
}
```
### How to handle missing values in your input datasets
Autopilot provides a number of filling methods to handle missing values in the target and other numeric columns of your time-series datasets. For information on the list of supported filling methods and their available filling logic, see [Handle missing values](timeseries-forecasting-data-format.md#timeseries-missing-values).
You configure your filling strategy in the `Transformations` attribute of [TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html) when creating your AutoML job.
To set a filling method, you need to provide a key-value pair:
+ The key is the name of the column for which you want to specify the filling method.
+ The value associated with the key is an object that defines the filling strategy for that column.
You can specify multiple filling methods for a single column.
To set a specific value for the filling method, you should set the fill parameter to the desired filling method value (for example `"backfill" : "value"`), and define the actual filling value in an additional parameter suffixed with "\$1value". For example, to set `backfill` to a value of `2`, you must include two parameters: `"backfill": "value"` and `"backfill_value":"2"`.
In the following example, you specify the filling strategy for the incomplete data column, "price" as follows: All missing values between the first data point of an item and the last are set to `0` after which all missing values are filled with the value `2` until the end date of the dataset.
```
"Transformations": {
"Filling": {
"price": {
"middlefill" : "zero",
"backfill" : "value",
"backfill_value": "2"
}
}
}
```
### How to specify an objective metric
Autopilot produces accuracy metrics to evaluate the model candidates and help you choose which to use to generate forecasts. When you run a time-series forecasting experiment, you can either choose AutoML to let Autopilot optimize the predictor for you, or you can manually choose an algorithm for your predictor.
By default, Autopilot uses the Average Weighted Quantile Loss. However, you can configure the objective metric when you create your AutoML job in the `MetricName` attribute of [AutoMLJobObjective](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobObjective.html).
For the list of available algorithms, see [Algorithms support for time-series forecasting](timeseries-forecasting-algorithms.md).
### How to incorporate national holiday information to your dataset
In Autopilot, you can incorporate a feature-engineered dataset of national holiday information to your time-series. Autopilot provide native support for the holiday calendars of over 250 countries. After you choose a country, Autopilot applies that country’s holiday calendar to every item in your dataset during training. This allows the model to identify patterns associated with specific holidays.
You can enable the holiday featurization when you create your AutoML job by passing an [HolidayConfigAttributes](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_HolidayConfigAttributes.html) object to the `HolidayConfig` attribute of [TimeSeriesForecastingJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html). The `HolidayConfigAttributes` object contains the two letters `CountryCode` attribute that determines the country of the public national holiday calendar used to augment your time-series dataset.
Refer to [Country Codes](autopilot-timeseries-forecasting-holiday-calendars.md#holiday-country-codes) for the list of supported calendars and their corresponding country code.
### How to enable automatic deployment
Autopilot allows you to automatically deploy your forecast model to an endpoint. To enable automatic deployment for the best model candidate of an AutoML job, include a `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the AutoML job request. This allows the deployment of the best model to a SageMaker AI endpoint. Below are the available configurations for customization.
+ To let Autopilotgenerate the endpoint name, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)` to `True`.
+ To provide your own name for the endpoint, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents) to False and provide a name of your choice in [EndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)`.
### How to configure AutoML to initiate a remote job on EMR Serverless for large datasets
You can configure your AutoML job V2 to automatically initiate a remote job on Amazon EMR Serverless when additional compute resources are needed to process large datasets. By seamlessly transitioning to EMR Serverless when required, the AutoML job can handle datasets that would otherwise exceed the initially provisioned resources, without any manual intervention from you. EMR Serverless is available for the tabular and time series problem types. We recommend setting up this option for time-series datasets larger than 30 GB.
To allow your AutoML job V2 to automatically transition to EMR Serverless for large dataset, you need to provide an `EmrServerlessComputeConfig` object, which includes an `ExecutionRoleARN` field, to the `AutoMLComputeConfig` of the AutoML job V2 input request.
The `ExecutionRoleARN` is the ARN of the IAM role granting the AutoML job V2 the necessary permissions to run EMR Serverless jobs.
This role should have the following trust relationship:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "emr-serverless.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
And grant the permissions to:
+ Create, list, and update EMR Serverless applications.
+ Start, list, get, or cancel job runs on an EMR Serverless application.
+ Tag EMR Serverless resources.
+ Pass an IAM role to the EMR Serverless service for execution.
By granting the `iam:PassRole` permission, the AutoML job V2 can temporarily assume the `EMRServerlessRuntimeRole-*` role and pass it to the EMR Serverless service. These are the IAM roles used by the EMR Serverless job execution environments to access other Amazon services and resources needed during runtime, such as Amazon S3 for data access, CloudWatch for logging, access to the Amazon Glue Data Catalog or other services based on your workload requirements.
See [Job runtime roles for Amazon EMR Serverless](https://docs.amazonaws.cn/emr/latest/EMR-Serverless-UserGuide/security-iam-runtime-role.html) for details on this role permissions.
The IAM policy defined in the provided JSON document grants those permissions:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [{
"Sid": "EMRServerlessCreateApplicationOperation",
"Effect": "Allow",
"Action": "emr-serverless:CreateApplication",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessListApplicationOperation",
"Effect": "Allow",
"Action": "emr-serverless:ListApplications",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessApplicationOperations",
"Effect": "Allow",
"Action": [
"emr-serverless:UpdateApplication",
"emr-serverless:GetApplication"
],
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessStartJobRunOperation",
"Effect": "Allow",
"Action": "emr-serverless:StartJobRun",
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessListJobRunOperation",
"Effect": "Allow",
"Action": "emr-serverless:ListJobRuns",
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessJobRunOperations",
"Effect": "Allow",
"Action": [
"emr-serverless:GetJobRun",
"emr-serverless:CancelJobRun"
],
"Resource": "arn:aws-cn:emr-serverless:*:*:/applications/*/jobruns/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "EMRServerlessTagResourceOperation",
"Effect": "Allow",
"Action": "emr-serverless:TagResource",
"Resource": "arn:aws-cn:emr-serverless:*:*:/*",
"Condition": {
"StringEquals": {
"aws:RequestTag/sagemaker:is-canvas-resource": "True",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
},
{
"Sid": "IAMPassOperationForEMRServerless",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws-cn:iam::*:role/EMRServerlessRuntimeRole-*",
"Condition": {
"StringEquals": {
"iam:PassedToService": "emr-serverless.amazonaws.com",
"aws:ResourceAccount": "${aws:PrincipalAccount}"
}
}
}
]
}
```
------
# Time-series datasets format and missing values filling methods
Time-series data refers to a collection of observations or measurements recorded over regular intervals of time. In this type of data, each observation is associated with a specific timestamp or time period, creating a sequence of data points ordered chronologically.
The specific columns you include in your time-series dataset depend on the goals of your analysis and the data available to you. At a minimum, the time-series data is composed of a 3-column table where:
+ One column contains unique identifiers assigned to individual items to refer to their value at a specific moment.
+ Another column represents the point-in-time value or **target** to log the value of a given item at a specific moment. After the model is trained on those target values, this target column contains the values that the model predicts at a specified frequency within a defined horizon.
+ And a timestamp column is included to record the date and time when the value was measured.
+ Additional columns can contain other factors that may influence the forecast performance. For example, in a time-series dataset for retail where the target is the sales or revenue, you might include features that provide information about units sold, product ID, store location, customer count, inventory levels, as well as covariate indicators such as weather data or demographic information.
**Note**
You can add a feature-engineered dataset of national holiday information to your time-series. By including holidays in your time series model, you can capture the periodic patterns that holidays create. This helps your forecasts better reflect the underlying seasonality of your data. For information on the available calendars per country, see [National holiday calendars](autopilot-timeseries-forecasting-holiday-calendars.md)
## Datasets format for time-series forecasting
Autopilot supports numeric, categorical, text, and datetime data types. The data type of the target column must be numeric.
Autopilot supports time-series data formatted as CSV (default) files or as Parquet files.
+ **CSV** (comma-separated-values) is a row-based file format that stores data in human readable plaintext which a popular choice for data exchange as they are supported by a wide range of applications.
+ **Parquet** is a column-based file format where the data is stored and processed more efficiently than row-based file formats. This makes them a better option for big data problems.
For more information about the resource limits on time-series datasets for forecasting in Autopilot, see [Time-series forecasting resource limits for Autopilot](timeseries-forecasting-limits.md).
## Handle missing values
A common issue in time-series forecasting data is the presence of missing values. Your data might contain missing values for a number of reasons, including measurement failures, formatting problems, human errors, or a lack of information to record. For instance, if you are forecasting product demand for a retail store and an item is sold out or unavailable, there would be no sales data to record while that item is out of stock. If prevalent enough, missing values can significantly impact a model's accuracy.
Autopilot provides a number of filling methods to handle missing values, with distinct approaches for the target column and other additional columns. Filling is the process of adding standardized values to missing entries in your dataset.
Refer to [How to handle missing values in your input datasets](autopilot-create-experiment-timeseries-forecasting.md#timeseries-forecasting-fill-missing-values) to learn how to set the method for filling missing values in your time-series dataset.
Autopilot supports the following filling methods:
+ **Front filling:** Fills any missing values between the earliest recorded data point among all items and the starting point of each item (each item can start at a different time). This ensures that the data for each item is complete and spans from the earliest recorded data point to its respective starting point.
+ **Middle filling:** Fills any missing values between the start and end dates of the items in the dataset.
+ **Back filling:** Fills any missing values between the last data point of each item (each item can stop at a different time) and the last recorded data point among all items.
+ **Future filling:** Fills any missing values between the last recorded data point among all items and the end of the forecast horizon.
The following image provides a visual representation of the different filling methods.
![\[The different filling methods for time series forecasting in Amazon SageMaker Autopilot.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/autopilot/autopilot-forecast-filling-methods.png)
### Choose a filling logic
When choosing a filling logic, you should consider how the logic will be interpreted by your model. For instance, in a retail scenario, recording 0 sales of an available item is different from recording 0 sales of an unavailable item, as the latter does not imply a lack of customer interest in the item. Because of this, `0` filling in the target column of the time-series might cause the predictor to be under-biased in its predictions, while `NaN` filling might ignore actual occurrences of 0 available items being sold and cause the predictor to be over-biased.
### Filling logic
You can perform filling on the target column and other numeric columns in your datasets. Target columns have different filling guidelines and restrictions than the rest of the numeric columns.
Filling Guidelines
| Column type | Filling by default? | Supported filling methods | Default filling logic | Accepted filling logic |
| --- | --- | --- | --- | --- |
| Target column | Yes | Middle and back filling | 0 | [\[See the AWS documentation website for more details\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/timeseries-forecasting-data-format.html) |
| Other numeric columns | No | Middle, back, and future filling | No default | [\[See the AWS documentation website for more details\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/timeseries-forecasting-data-format.html) |
**Note**
For both the target and other numeric columns, `mean`, `median`, `min`, and `max` are calculated based on a rolling window of the 64 most recent data entries before the missing values.
# National holiday calendars
Autopilot supports a feature-engineered dataset of national holiday information that provides access to the holiday calendars of over 250 countries. Holiday calendar features are especially useful in the retail domain, where public holidays can significantly affect demand. The following section lists the country codes that you can use to access the holiday calendars of each supported country.
Refer to [How to incorporate national holiday information to your dataset](autopilot-create-experiment-timeseries-forecasting.md#timeseries-forecasting-add-holiday-calendar) to learn how to add a calendar to your dataset.
## Country Codes
Autopilot provides native support for the public holiday calendars of the following countries. Use the **Country Code** when specifying a country with the API.
| Country | Country Code |
| --- | --- |
| Afghanistan | AF |
| Åland Islands | AX |
| Albania | AL |
| Algeria | DZ |
| American Samoa | AS |
| Andorra | AD |
| Angola | AO |
| Anguilla | AI |
| Antartica | AQ |
| Antigua and Barbuda | AG |
| Argentina | AR |
| Armenia | AM |
| Aruba | AW |
| Australia | AU |
| Austria | AT |
| Azerbaijan | AZ |
| Bahamas | BS |
| Bahrain | BH |
| Bangladesh | BD |
| Barbados | BB |
| Belarus | BY |
| Belgium | BE |
| Belize | BZ |
| Benin | BJ |
| Bermuda | BM |
| Bhutan | BT |
| Bolivia | BO |
| Bosnia and Herzegovina | BA |
| Botswana | BW |
| Bouvet Island | BV |
| Brazil | BR |
| British Indian Ocean Territory | IO |
| British Virgin Islands | VG |
| Brunei Darussalam | BN |
| Bulgaria | BG |
| Burkina Faso | BF |
| Burundi | BI |
| Cambodia | KH |
| Cameroon | CM |
| Canada | CA |
| Cape Verde | CV |
| Caribbean Netherlands | BQ |
| Cayman Islands | KY |
| Central African Republic | CF |
| Chad | TD |
| Chile | CL |
| China | CN |
| Christmas Island | CX |
| Cocos (Keeling) Islands | CC |
| Colombia | CO |
| Comoros | KM |
| Cook Islands | CK |
| Costa Rica | CR |
| Croatia | HR |
| Cuba | CU |
| Curaçao | CW |
| Cyprus | CY |
| Czechia | CZ |
| Democratic Republic of the Congo | CD |
| Denmark | DK |
| Djibouti | DJ |
| Dominica | DM |
| Dominican Republic | DO |
| Ecuador | EC |
| Egypt | EG |
| El Salvador | SV |
| Equatorial Guinea | GQ |
| Eritrea | ER |
| Estonia | EE |
| Eswatini | SZ |
| Ethiopia | ET |
| Falkland Islands | FK |
| Faroe Islands | FO |
| Fiji | FJ |
| Finland | FI |
| France | FR |
| French Guiana | GF |
| French Polynesia | PF |
| French Southern Territories | TF |
| Gabon | GA |
| Gambia | GM |
| Georgia | GE |
| Germany | DE |
| Ghana | GH |
| Gibraltar | GI |
| Greece | GR |
| Greenland | GL |
| Grenada | GD |
| Guadeloupe | GP |
| Guam | GU |
| Guatemala | GT |
| Guernsey | GG |
| Guinea | GN |
| Guinea-Bissau | GW |
| Guyana | GY |
| Haiti | HT |
| Heard Island and McDonald Islands | HM |
| Honduras | HN |
| Hong Kong | HK |
| Hungary | HU |
| Iceland | IS |
| India | IN |
| Indonesia | ID |
| Iran | IR |
| Iraq | IQ |
| Ireland | IE |
| Isle of Man | IM |
| Israel | IL |
| Italy | IT |
| Ivory Coast | CI |
| Jamaica | JM |
| Japan | JP |
| Jersey | JE |
| Jordan | JO |
| Kazakhstan | KZ |
| Kenya | KE |
| Kiribati | KI |
| Kosovo | XK |
| Kuwait | KW |
| Kyrgyzstan | KG |
| Laos | LA |
| Latvia | LV |
| Lebanon | LB |
| Lesotho | LS |
| Liberia | LR |
| Libya | LY |
| Liechtenstein | LI |
| Lithuania | LT |
| Luxembourg | LU |
| Macao | MO |
| Madagascar | MG |
| Malawi | MW |
| Malaysia | MY |
| Maldives | MV |
| Mali | ML |
| Malta | MT |
| Marshall Islands | MH |
| Martinique | MQ |
| Mauritania | MR |
| Mauritius | MU |
| Mayotte | YT |
| Mexico | MX |
| Micronesia | FM |
| Moldova | MD |
| Monaco | MC |
| Mongolia | MN |
| Montenegro | ME |
| Montserrat | MS |
| Morocco | MA |
| Mozambique | MZ |
| Myanmar | MM |
| Namibia | NA |
| Nauru | NR |
| Nepal | NP |
| Netherlands | NL |
| New Caledonia | NC |
| New Zealand | NZ |
| Nicaragua | NI |
| Niger | NE |
| Nigeria | NG |
| Niue | NU |
| Norfolk Island | NF |
| North Korea | KP |
| North Macedonia | MK |
| Northern Mariana Islands | MP |
| Norway | NO |
| Oman | OM |
| Pakistan | PK |
| Palau | PW |
| Palestine | PS |
| Panama | PA |
| Papua New Guinea | PG |
| Paraguay | PY |
| Peru | PE |
| Philippines | PH |
| Pitcairn Islands | PN |
| Poland | PL |
| Portugal | PT |
| Puerto Rico | PR |
| Qatar | QA |
| Republic of the Congo | CG |
| Réunion | RE |
| Romania | RO |
| Russian Federation | RU |
| Rwanda | RW |
| Saint Barthélemy | BL |
| "Saint Helena, Ascension and Tristan da Cunha " | SH |
| Saint Kitts and Nevis | KN |
| Saint Lucia | LC |
| Saint Martin | MF |
| Saint Pierre and Miquelon | PM |
| Saint Vincent and the Grenadines | VC |
| Samoa | WS |
| San Marino | SM |
| Sao Tome and Principe | ST |
| Saudi Arabia | SA |
| Senegal | SN |
| Serbia | RS |
| Seychelles | SC |
| Sierra Leone | SL |
| Singapore | SG |
| Sint Maarten | SX |
| Slovakia | SK |
| Slovenia | SI |
| Solomon Islands | SB |
| Somalia | SO |
| South Africa | ZA |
| South Georgia and the South Sandwich Islands | GS |
| South Korea | KR |
| South Sudan | SS |
| Spain | ES |
| Sri Lanka | LK |
| Sudan | SD |
| Suriname | SR |
| Svalbard and Jan Mayen | SJ |
| Sweden | SE |
| Switzerland | CH |
| Syrian Arab Republic | SY |
| Tajikistan | TJ |
| Tanzania | TZ |
| Thailand | TH |
| Timor-Leste | TL |
| Togo | TG |
| Tokelau | TK |
| Tonga | TO |
| Trinidad and Tobago | TT |
| Tunisia | TN |
| Turkey | TR |
| Turkmenistan | TM |
| Turks and Caicos Islands | TC |
| Tuvalu | TV |
| Uganda | UG |
| Ukraine | UA |
| United Arab Emirates | AE |
| United Kingdom | UK |
| United Nations | UN |
| United States | US |
| United States Minor Outlying Islands | UM |
| United States Virgin Islands | VI |
| Uruguay | UY |
| Uzbekistan | UZ |
| Vanuatu | VU |
| Vatican City | VA |
| Venezuela | VE |
| Vietnam | VN |
| Wallis and Futuna | WF |
| Western Sahara | EH |
| Yemen | YE |
| Zambia | ZM |
| Zimbabwe | ZW |
# Objective metrics
Autopilot produces accuracy metrics to evaluate the model candidates and help you choose which to use to generate forecasts. You can either let Autopilot optimize the predictor for you, or you can manually choose an algorithm for your predictor. By default, Autopilot uses the Average Weighted Quantile Loss.
The following list contains the names of the metrics that are currently available to measure the performance of models for time-series forecasting.
**`RMSE`**
Root mean squared error (RMSE) – Measures the square root of the squared difference between predicted and actual values, and is averaged over all values. It's an important metric to indicate the presence of large model errors and outliers. Values range from zero (0) to infinity, with smaller numbers indicating a better model fit to the data. RMSE is dependent on scale, and should not be used to compare datasets of different sizes.
**`wQL`**
Weighted Quantile Loss (wQL) – Assess the accuracy of the forecast by measuring the weighted absolute differences between predicted and actual P10, P50, and P90 quantiles with lower values indicating better performance.
**`Average wQL (default)`**
Average Weighted Quantile Loss (Average wQL) – Evaluates the forecast by averaging the accuracy at the P10, P50, and P90 quantiles. A lower value indicates a more accurate model.
**`MASE`**
Mean Absolute Scaled Error (MASE) – The mean absolute error of the forecast normalized by the mean absolute error of a simple baseline forecasting method. A lower value indicates a more accurate model, where MASE < 1 is estimated to be better than the baseline and MASE > 1 is estimated to be worse than the baseline.
**`MAPE`**
Mean Absolute Percent Error (MAPE) – The percentage error (percent difference of the mean forecasted value versus the actual value) averaged over all time points. A lower value indicates a more accurate model, where MAPE = 0 is a model with no errors.
**`WAPE`**
Weighted Absolute Percent Error (WAPE) – The sum of the absolute error normalized by the sum of the absolute target, which measure the overall deviation of forecasted values from observed values. A lower value indicates a more accurate model.
# Algorithms support for time-series forecasting
Autopilot trains the following six built-in algorithms with your target time-series. Then, using a stacking ensemble method, it combines these model candidates to create an optimal forecasting model for a given objective metric.
+ **Convolutional Neural Network - Quantile Regression (CNN-QR)** – CNN-QR is a proprietary machine learning algorithm for forecasting time-series using causal convolutional neural networks (CNNs). CNN-QR works best with large datasets containing hundreds of time-series.
+ **DeepAR\$1** – DeepAR\$1 is a proprietary machine learning algorithm for forecasting time-series using recurrent neural networks (RNNs). DeepAR\$1 works best with large datasets containing hundreds of feature time-series.
+ **Prophet** – [Prophet](https://facebook.github.io/prophet/) is a popular local Bayesian structural time series model based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality. The Autopilot Prophet algorithm uses the [Prophet class](https://facebook.github.io/prophet/docs/quick_start.html#python-ap) of the Python implementation of Prophet. It works best with time-series with strong seasonal effects and several seasons of historical data.
+ **Non-Parametric Time Series (NPTS)** – The NPTS proprietary algorithm is a scalable, probabilistic baseline forecaster. It predicts the future value distribution of a given time-series by sampling from past observations. NPTS is especially useful when working with sparse or intermittent time series.
+ **Autoregressive Integrated Moving Average (ARIMA)** – ARIMA is a commonly used statistical algorithm for time-series forecasting. The algorithm captures standard temporal structures (patterned organizations of time) in the input dataset. It is especially useful for simple datasets with under 100 time series.
+ **Exponential Smoothing (ETS)** – ETS is a commonly used statistical algorithm for time-series forecasting. The algorithm is especially useful for simple datasets with under 100 time series, and datasets with seasonality patterns. ETS computes a weighted average over all observations in the time series dataset as its prediction, with exponentially decreasing weights over time.
# Forecast a deployed Autopilot model
After training your models using the AutoML API, you can deploy them for real-time or batch-based forecasting.
The AutoML API trains several model candidates for your time-series data and selects an optimal forecasting model based on your target objective metric. Once your model candidates have been trained, you can find the best candidate in the response [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) at [BestCandidate](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-CandidateName).
To get predictions using this best performing model, you can either set up an endpoint to obtain forecasts interactively or use batch forecasting to make predictions on a batch of observations.
**Considerations**
+ When providing input data for forecasting, the schema of your data should remain the same as the one used to train your model, including the number of columns, column headers, and data types. You can forecast for existing or new item IDs within the same or different timestamp range to predict for a different time period.
+ Forecasting models predict for the forecast horizon points in the future specified in the input request at training, which is from the *target end date* to the *target end date \$1 forecast horizon*. To use the model for predicting specific dates, you should provide the data in the same format as the original input data, extending up to a specified *target end date*. In this scenario, the model will start predicting from the new target end date.
For example, if your dataset had monthly data from January to June with a Forecast horizon of 2, then the model would predict the target value for the next 2 months, which would be July and August. If in August, you want to predict for the next 2 months, this time your input data should be from January to August and the model will predict for the next 2 months (September, October).
+ When forecasting future data points, there is no set minimum for the amount of historical data to provide. Include enough data to capture seasonal and recurrent patterns in your time-series.
**Topics**
+ [
# Real-time forecasting
](timeseries-forecasting-realtime.md)
+ [
# Batch forecasting
](timeseries-forecasting-batch.md)
# Real-time forecasting
Real-time forecasting is useful when you need to generate predictions on-the-fly, such as for applications that require immediate responses or when forecasting for individual data points.
By deploying your AutoML model as a real-time endpoint, you can generate forecasts on-demand and minimize the latency between receiving new data and obtaining predictions. This makes real-time forecasting well-suited for applications that require immediate, personalized, or event-driven forecasting capabilities.
For real time forecasting, the dataset should be a subset of the input dataset. The real time endpoint has an input data size of approximately 6MB and a response timeout limitation of 60 seconds. We recommend bringing in one or few items at a time.
You can use SageMaker APIs to retrieve the best candidate of an AutoML job and then create a SageMaker AI endpoint using that candidate.
Alternatively, you can chose the automatic deployment option when creating your Autopilot experiment. For information on setting up the automatic deployment of models, see [How to enable automatic deployment](autopilot-create-experiment-timeseries-forecasting.md#timeseries-forecasting-auto-model-deployment).
**To create a SageMaker AI endpoint using your best model candidate:**
1.
**Retrieve the details of the AutoML job.**
The following Amazon CLI command example uses the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) API to obtain details of the AutoML job, including the information about the best model candidate.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name job-name --region region
```
1.
**Extract the container definition from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers) for the best model candidate.**
A container definition is the containerized environment used to host the trained SageMaker AI model for making predictions.
```
BEST_CANDIDATE=$(aws sagemaker describe-auto-ml-job-v2 \
--auto-ml-job-name job-name
--region region \
--query 'BestCandidate.InferenceContainers[0]' \
--output json
```
This command extracts the container definition for the best model candidate and stores it in the `BEST_CANDIDATE` variable.
1.
**Create a SageMaker AI model using the best candidate container definition.**
Use the container definitions from the previous steps to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API.
```
aws sagemaker create-model \
--model-name 'your-candidate-name>' \
--primary-container "$BEST_CANDIDATE"
--execution-role-arn 'execution-role-arn>' \
--region 'region>
```
The `--execution-role-arn` parameter specifies the IAM role that SageMaker AI assumes when using the model for inference. For details on the permissions required for this role, see [CreateModel API: Execution Role Permissions](https://docs.amazonaws.cn/).
1.
**Create a SageMaker AI endpoint configuration using the model.**
The following Amazon CLI command uses the [CreateEndpointConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpointConfig.html) API to create an endpoint configuration.
```
aws sagemaker create-endpoint-config \
--production-variants file://production-variants.json \
--region 'region'
```
Where the `production-variants.json` file contains the model configuration, including the model name and instance type.
**Note**
We recommend using [m5.12xlarge](https://www.amazonaws.cn/ec2/instance-types/m5/) instances for real-time forecasting.
```
[
{
"VariantName": "variant-name",
"ModelName": "model-name",
"InitialInstanceCount": 1,
"InstanceType": "m5.12xlarge"
}
]
}
```
1.
**Create the SageMaker AI endpoint using the endpoint configuration.**
The following Amazon CLI example uses the [CreateEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpoint.html) API to create the endpoint.
```
aws sagemaker create-endpoint \
--endpoint-name 'endpoint-name>' \
--endpoint-config-name 'endpoint-config-name' \
--region 'region'
```
Check the progress of your real-time inference endpoint deployment by using the [DescribeEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeEndpoint.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-endpoint \
--endpoint-name 'endpoint-name' \
--region 'region'
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready to use for real-time inference.
1.
**Invoke the SageMaker AI endpoint to make predictions.**
```
aws sagemaker invoke-endpoint \
--endpoint-name 'endpoint-name' \
--region 'region' \
--body file://input-data-in-bytes.json \
--content-type 'application/json' outfile
```
Where the `input-data-in-bytes.json` file contains the input data for the prediction.
# Batch forecasting
Batch forecasting, also known as offline inferencing, generates model predictions on a batch of observations. Batch inference is a good option for large datasets or if you don't need an immediate response to a model prediction request.
By contrast, online inference (real-time inferencing) generates predictions in real time.
You can use SageMaker APIs to retrieve the best candidate of an AutoML job and then submit a batch of input data for inference using that candidate.
1.
**Retrieve the details of the AutoML job.**
The following Amazon CLI command example uses the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) API to obtain details of the AutoML job, including the information about the best model candidate.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name job-name --region region
```
1.
**Extract the container definition from [InferenceContainers](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html#sagemaker-Type-AutoMLCandidate-InferenceContainers) for the best model candidate.**
A container definition is the containerized environment used to host the trained SageMaker AI model for making predictions.
```
BEST_CANDIDATE=$(aws sagemaker describe-auto-ml-job-v2 \
--auto-ml-job-name job-name
--region region \
--query 'BestCandidate.InferenceContainers[0]' \
--output json
```
This command extracts the container definition for the best model candidate and stores it in the `BEST_CANDIDATE` variable.
1.
**Create a SageMaker AI model using the best candidate container definition.**
Use the container definitions from the previous steps to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API.
```
aws sagemaker create-model \
--model-name 'model-name' \
--primary-container "$BEST_CANDIDATE"
--execution-role-arn 'execution-role-arn>' \
--region 'region>
```
The `--execution-role-arn` parameter specifies the IAM role that SageMaker AI assumes when using the model for inference. For details on the permissions required for this role, see [CreateModel API: Execution Role Permissions](https://docs.amazonaws.cn/).
1.
**Create a batch transform job.**
The following example creates a transform job using the [CreateTransformJob](https://docs.amazonaws.cn/cli/latest/reference/sagemaker/create-transform-job.html) API.
```
aws sagemaker create-transform-job \
--transform-job-name 'transform-job-name' \
--model-name 'model-name'\
--transform-input file://transform-input.json \
--transform-output file://transform-output.json \
--transform-resources file://transform-resources.json \
--region 'region'
```
The input, output, and resource details are defined in separate JSON files:
+ `transform-input.json`:
```
{
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3://my-input-data-bucket/path/to/input/data"
}
},
"ContentType": "text/csv",
"SplitType": "None"
}
```
+ `transform-output.json`:
```
{
"S3OutputPath": "s3://my-output-bucket/path/to/output",
"AssembleWith": "Line"
}
```
+ `transform-resources.json`:
**Note**
We recommend using [m5.12xlarge](https://www.amazonaws.cn/ec2/instance-types/m5/) instances for general-purpose workloads and `m5.24xlarge` instances for big data forecasting tasks.
```
{
"InstanceType": "instance-type",
"InstanceCount": 1
}
```
1.
**Monitor the progress of your transform job using the [DescribeTransformJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeTransformJob.html) API.**
See the following Amazon CLI command as an example.
```
aws sagemaker describe-transform-job \
--transform-job-name 'transform-job-name' \
--region region
```
1.
**Retrieve the batch transform output.**
After the job is finished, the predicted result is available in the `S3OutputPath`.
The output file name has the following format: `input_data_file_name.out`. As an example, if your input file is `text_x.csv`, the output name will be `text_x.csv.out`.
```
aws s3 ls s3://my-output-bucket/path/to/output/
```
The following code examples illustrate the use of the Amazon SDK for Python (boto3) and the Amazon CLI for batch forecasting.
------
#### [ Amazon SDK for Python (boto3) ]
The following example uses **Amazon SDK for Python (boto3)** to make predictions in batches.
```
import sagemaker
import boto3
session = sagemaker.session.Session()
sm_client = boto3.client('sagemaker', region_name='us-west-2')
role = 'arn:aws:iam::1234567890:role/sagemaker-execution-role'
output_path = 's3://test-auto-ml-job/output'
input_data = 's3://test-auto-ml-job/test_X.csv'
best_candidate = sm_client.describe_auto_ml_job_v2(AutoMLJobName=job_name)['BestCandidate']
best_candidate_containers = best_candidate['InferenceContainers']
best_candidate_name = best_candidate['CandidateName']
# create model
reponse = sm_client.create_model(
ModelName = best_candidate_name,
ExecutionRoleArn = role,
Containers = best_candidate_containers
)
# Lauch Transform Job
response = sm_client.create_transform_job(
TransformJobName=f'{best_candidate_name}-transform-job',
ModelName=model_name,
TransformInput={
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': input_data
}
},
'ContentType': "text/csv",
'SplitType': 'None'
},
TransformOutput={
'S3OutputPath': output_path,
'AssembleWith': 'Line',
},
TransformResources={
'InstanceType': 'ml.m5.2xlarge',
'InstanceCount': 1,
},
)
```
The batch inference job returns a response in the following format.
```
{'TransformJobArn': 'arn:aws:sagemaker:us-west-2:1234567890:transform-job/test-transform-job',
'ResponseMetadata': {'RequestId': '659f97fc-28c4-440b-b957-a49733f7c2f2',
'HTTPStatusCode': 200,
'HTTPHeaders': {'x-amzn-requestid': '659f97fc-28c4-440b-b957-a49733f7c2f2',
'content-type': 'application/x-amz-json-1.1',
'content-length': '96',
'date': 'Thu, 11 Aug 2022 22:23:49 GMT'},
'RetryAttempts': 0}}
```
------
#### [ Amazon Command Line Interface (Amazon CLI) ]
1. **Obtain the best candidate container definitions**.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name 'test-automl-job' --region us-west-2
```
1. **Create the model**.
```
aws sagemaker create-model --model-name 'test-sagemaker-model'
--containers '[{
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/data-processor-models/test-job1-dpp0-1-e569ff7ad77f4e55a7e549a/output/model.tar.gz",
"Environment": {
"AUTOML_SPARSE_ENCODE_RECORDIO_PROTOBUF": "1",
"AUTOML_TRANSFORM_MODE": "feature-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "application/x-recordio-protobuf",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.3-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/tuning/flicdf10v2-dpp0-xgb/test-job1E9-244-7490a1c0/output/model.tar.gz",
"Environment": {
"MAX_CONTENT_LENGTH": "20971520",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,probabilities"
}
}, {
"Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3",
"ModelDataUrl": "s3://amzn-s3-demo-bucket/out/test-job1/data-processor-models/test-job1-dpp0-1-e569ff7ad77f4e55a7e549a/output/model.tar.gz",
"Environment": {
"AUTOML_TRANSFORM_MODE": "inverse-label-transform",
"SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
"SAGEMAKER_INFERENCE_INPUT": "predicted_label",
"SAGEMAKER_INFERENCE_OUTPUT": "predicted_label",
"SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,labels,probabilities",
"SAGEMAKER_PROGRAM": "sagemaker_serve",
"SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
}
}]' \
--execution-role-arn 'arn:aws:iam::1234567890:role/sagemaker-execution-role' \
--region 'us-west-2'
```
1. **Create a transform job**.
```
aws sagemaker create-transform-job --transform-job-name 'test-tranform-job'\
--model-name 'test-sagemaker-model'\
--transform-input '{
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3://amzn-s3-demo-bucket/data.csv"
}
},
"ContentType": "text/csv",
"SplitType": "None"
}'\
--transform-output '{
"S3OutputPath": "s3://amzn-s3-demo-bucket/output/",
"AssembleWith": "Line"
}'\
--transform-resources '{
"InstanceType": "ml.m5.2xlarge",
"InstanceCount": 1
}'\
--region 'us-west-2'
```
1. **Check the progress of the transform job**.
```
aws sagemaker describe-transform-job --transform-job-name 'test-tranform-job' --region us-west-2
```
The following is the response from the transform job.
```
{
"TransformJobName": "test-tranform-job",
"TransformJobArn": "arn:aws:sagemaker:us-west-2:1234567890:transform-job/test-tranform-job",
"TransformJobStatus": "InProgress",
"ModelName": "test-model",
"TransformInput": {
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3://amzn-s3-demo-bucket/data.csv"
}
},
"ContentType": "text/csv",
"CompressionType": "None",
"SplitType": "None"
},
"TransformOutput": {
"S3OutputPath": "s3://amzn-s3-demo-bucket/output/",
"AssembleWith": "Line",
"KmsKeyId": ""
},
"TransformResources": {
"InstanceType": "ml.m5.2xlarge",
"InstanceCount": 1
},
"CreationTime": 1662495635.679,
"TransformStartTime": 1662495847.496,
"DataProcessing": {
"InputFilter": "$",
"OutputFilter": "$",
"JoinSource": "None"
}
}
```
After the `TransformJobStatus` changes to `Completed`, you can check the inference result in the `S3OutputPath`.
------
# Amazon SageMaker Autopilot data exploration notebook
Amazon SageMaker Autopilot cleans and pre-processes your dataset automatically. To help users understand their data, uncover patterns, relationships, and anomalies about the time-series, Amazon SageMaker Autopilot generates a **data exploration** static report in the form of a notebook for users to reference.
The data exploration notebook is generated for every Autopilot job. The report is stored in an Amazon S3 bucket and can be accessed from the job output path.
You can find the Amazon S3 prefix to the data exploration notebook in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[AutoMLJobArtifacts.DataExplorationNotebookLocation](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html#sagemaker-DescribeAutoMLJobV2-response-AutoMLJobArtifacts)`.
# Reports generated by Amazon SageMaker Autopilot
In addition to the data exploration notebook, Autopilot generates various reports for the best model candidate of each experiment.
+ An explainability report provides insights into how the model makes forecasts.
+ A performance report provides a quantitative assessment of the model's forecasting capabilities.
+ A backtest results report is generated after testing the model's performance on historical data.
## Explainability report
Autopilot explainability report helps you better understand how the attributes in your datasets impact forecasts for specific time-series (item and dimension combinations) and time points. Autopilot uses a metric called *Impact scores* to quantify the relative impact of each attribute and determine whether they increase or decrease forecast values.
For example, consider a forecasting scenario where the target is `sales` and there are two related attributes: `price` and `color`. Autopilot may find that the item’s color has a high impact on sales for certain items, but a negligible effect for other items. It may also find that a promotion in the summer has a high impact on sales, but a promotion in the winter has little effect.
The explainability report is generated only when:
+ The time series dataset includes additional feature columns or is associated with a holiday calendar.
+ The base models CNN-QR and DeepAR\$1 are included in the final ensemble.
### Interpret Impact scores
Impact scores measure the relative impact attributes have on forecast values. For example, if the `price` attribute has an impact score that is twice as large as the `store location` attribute, you can conclude that the price of an item has twice the impact on forecast values than the store location.
Impact scores also provide information on whether attributes increase or decrease forecast values.
The Impact scores range from -1 to 1, where the sign denotes the direction of the impact. A score of 0 indicates no impact, while scores close to 1 or -1 indicate a significant impact.
It is important to note that Impact scores measure the relative impact of attributes, not the absolute impact. Therefore, Impact scores cannot be used to determine whether particular attributes improve model accuracy. If an attribute has a low Impact score, that does not necessarily mean that it has a low impact on forecast values; it means that it has a lower impact on forecast values than other attributes used by the predictor.
### Find the explainability report
You can find the Amazon S3 prefix to the explainability artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.Explainability](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-Explainability)`.
## Model performance report
Autopilot model quality report (also referred to as performance report) provides insights and quality information for the best model candidate (best predictor) generated by an AutoML job. This includes information about the job details, objective function, and accuracy metrics (`wQL`, `MAPE`, `WAPE`, `RMSE`, `MASE`).
You can find the Amazon S3 prefix to the model quality report artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.ModelInsights](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-ModelInsights)`.
## Backtests results report
Backtests results provide insights into the performance of a time-series forecasting model by evaluating its predictive accuracy and reliability. It helps analysts and data scientists assess its performance on historical data and assists in understanding its potential performance on future, unseen data.
Autopilot uses backtesting to tune parameters and produce accuracy metrics. During backtesting, Autopilot automatically splits your time-series data into two sets, a training set and a testing set. The training set is used to train a model which is then used to generate forecasts for data points in the testing set. Autopilot uses this testing dataset to evaluate the model's accuracy by comparing forecasted values with observed values in the testing set.
You can find the Amazon S3 prefix to the model quality report artifacts generated for the best candidate in the response to `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` at `[BestCandidate.CandidateProperties.CandidateArtifactLocations.BacktestResults](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html#sagemaker-Type-CandidateArtifactLocations-BacktestResults)`.
# Time-series forecasting resource limits for Autopilot
The following table lists the resource limits for time-series forecasting jobs in Amazon SageMaker Autopilot and whether or not you can adjust each limit.
| **Resource limits** | **Default limit** | **Adjustable** |
| --- | --- | --- |
| Size of input dataset | 30 GB | Yes |
| Size of a single Parquet file | 2 GB | No |
| Maximum number of rows in a dataset | 3 billion | Yes |
| Maximum number of grouping columns | 5 | No |
| Maximum number of numerical features | 13 | No |
| Maximum number of categorical features | 10 | No |
| Maximum number of time-series (unique combinations of item and grouping columns) per dataset | 5,000,000 | Yes |
| Maximum Forecast horizon | 500 | Yes |
# Create an AutoML job to fine-tune text generation models using the API
Large language models (LLMs) excel in multiple generative tasks, including text generation, summarization, completion, question answering, and more. Their performance can be attributed to their significant size and extensive training on diverse datasets and various tasks. However, specific domains, such as healthcare and financial services, may require customized fine-tuning to adapt to unique data and use cases. By tailoring their training to their particular domain, LLMs can improve their performance and provide more accurate outputs for targeted applications.
Autopilot offers the capability to fine-tune a selection of pre-trained generative text models. In particular, Autopilot supports the **instruction-based fine tuning** of a selection of general-purpose large language models (LLMs) powered by JumpStart.
**Note**
The text generation models that support fine-tuning in Autopilot are currently accessible exclusively in Regions supported by SageMaker Canvas. See the documentation of SageMaker Canvas for the [full list of its supported Regions](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html).
Fine-tuning a pre-trained model requires a specific dataset of clear instructions that guide the model on how to generate output or behave for that task. The model learns from the dataset, adjusting its parameters to conform to the provided instructions. Instruction-based fine-tuning involves using labeled examples formatted as prompt-response pairs and phrased as instructions. For more information about fine-tuning, see [Fine-tune a foundation model](https://docs.amazonaws.cn/sagemaker/latest/dg/jumpstart-foundation-models-fine-tuning.html).
The following guidelines outline the process of creating an Amazon SageMaker Autopilot job as a pilot experiment to fine-tune text generation LLMs using the SageMaker [API Reference](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html).
**Note**
Tasks such as text and image classification, time-series forecasting, and fine-tuning of large language models are exclusively available through the version 2 of the [AutoML REST API](autopilot-reference.md). If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_auto_ml_job_v2.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
Users who prefer the convenience of a user interface can use [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas-getting-started.html) to access pre-trained models and generative AI foundation models, or create custom models tailored for specific text, image classification, forecasting needs, or generative AI.
To create an Autopilot experiment programmatically for fine-tuning an LLM, you can call the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) API in any language supported by Amazon SageMaker Autopilot or the Amazon CLI.
For information about how this API action translates into a function in the language of your choice, see the [ See Also](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_SeeAlso) section of `CreateAutoMLJobV2` and choose an SDK. As an example, for Python users, see the full request syntax of `[create\$1auto\$1ml\$1job\$1v2](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_auto_ml_job_v2)` in Amazon SDK for Python (Boto3).
**Note**
Autopilot fine-tunes large language models without requiring multiple candidates to be trained and evaluated. Instead, using your dataset, Autopilot directly fine-tunes your target model to enhance a default objective metric, the cross-entropy loss. Fine-tuning language models in Autopilot does not require setting the `AutoMLJobObjective` field.
Once your LLM is fine-tuned, you can evaluate its performance by accessing various ROUGE scores through the `[BestCandidate](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateProperties.html#sagemaker-Type-CandidateProperties-CandidateMetrics)` when making a `[DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)` API call. The model also provides information about its training and validation loss as well as perplexity. For a comprehensive list of metrics for evaluating the quality of the text generated by the fine-tuned models, see [Metrics for fine-tuning large language models in Autopilot](autopilot-llms-finetuning-metrics.md).
## Prerequisites
Before using Autopilot to create a fine-tuning experiment in SageMaker AI, make sure to take the following steps:
+ (Optional) Choose the pre-trained model you want to fine-tune.
For the list of pre-trained models available for fine-tuning in Amazon SageMaker Autopilot, see [Supported large language models for fine-tuning](autopilot-llms-finetuning-models.md). The selection of a model is not mandatory; if no model is specified, Autopilot automatically defaults to the model *Falcon7BInstruct*.
+ Create a dataset of instructions. See [Dataset file types and input data format](autopilot-llms-finetuning-data-format.md) to learn about the format requirements for your instruction-based dataset.
+ Place your dataset in an Amazon S3 bucket.
+ Grant full access to the Amazon S3 bucket containing your input data for the SageMaker AI execution role used to run your experiment.
+ For information on retrieving your SageMaker AI execution role, see [Get your execution role](sagemaker-roles.md#sagemaker-roles-get-execution-role).
+ For information on granting your SageMaker AI execution role permissions to access one or more specific buckets in Amazon S3, see * Add Additional Amazon S3 Permissions to a SageMaker AI Execution Role* in [Create execution role](sagemaker-roles.md#sagemaker-roles-create-execution-role).
+ Additionally, you should provide your execution role with the necessary permissions to access the default storage Amazon S3 bucket used by JumpStart. This access is required for storing and retrieving pre-trained model artifacts in JumpStart. To grant access to this Amazon S3 bucket, you must create a new inline custom policy on your execution role.
Here's an example policy that you can use in your JSON editor when configuring AutoML fine-tuning jobs in `us-west-2`:
*JumpStart's bucket names follow a predetermined pattern that depends on the Amazon Web Services Regions. You must adjust the name of the bucket accordingly.*
```
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::jumpstart-cache-prod-us-west-2",
"arn:aws:s3:::jumpstart-cache-prod-us-west-2/*"
]
}
```
Once this is done, you can use the ARN of this execution role in Autopilot API requests.
## Required parameters
When calling `[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)` to create an Autopilot experiment for LLM fine-tuning, you must provide the following values:
+ An `[AutoMLJobName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#API_CreateAutoMLJobV2_RequestSyntax)` to specify the name of your job. The name should be of type `string`, and should have a minimum length of 1 character and a maximum length of 32.
+ At least one `[AutoMLJobChannel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)` of the `training` type within the `[AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)`. This channel specifies the name of the Amazon S3 bucket where your fine-tuning dataset is located. You have the option to define a `validation` channel. If no validation channel is provided, and a `ValidationFraction` is configured in the [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html), this fraction is utilized to randomly divide the training dataset into training and validation sets. Additionally, you can specify the type of content (CSV or Parquet files) for the dataset.
+ An `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)` of type `[TextGenerationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html)` to configure the settings of your training job.
In particular, you can specify the name of the base model to fine-tune in the `BaseModelName` field. For the list of pre-trained models available for fine-tuning in Amazon SageMaker Autopilot, see [Supported large language models for fine-tuning](autopilot-llms-finetuning-models.md).
+ An `[OutputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)` to specify the Amazon S3 output path to store the artifacts of your AutoML job.
+ A `[RoleArn](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html#sagemaker-CreateAutoMLJob-request-RoleArn)` to specify the ARN of the role used to access your data.
The following is an example of the full request format used when making an API call to `CreateAutoMLJobV2` for fine-tuning a (`Falcon7BInstruct`) model.
```
{
"AutoMLJobName": "",
"AutoMLJobInputDataConfig": [
{
"ChannelType": "training",
"CompressionType": "None",
"ContentType": "text/csv",
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "s3:///.csv"
}
}
}
],
"OutputDataConfig": {
"S3OutputPath": "s3:///output",
"KmsKeyId": "arn:aws:kms:::key/"
},
"RoleArn":"arn:aws:iam:::role/",
"AutoMLProblemTypeConfig": {
"TextGenerationJobConfig": {
"BaseModelName": "Falcon7BInstruct"
}
}
}
```
All other parameters are optional.
## Optional parameters
The following sections provide details of some optional parameters that you can pass to your fine-tuning AutoML job.
### How to specify the training and validation datasets of an AutoML job
You can provide your own validation dataset and custom data split ratio, or let Autopilot split the dataset automatically.
Each [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html) object (see the required parameter [AutoMLJobInputDataConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLJobInputDataConfig)) has a `ChannelType`, which can be set to either `training` or `validation` values that specify how the data is to be used when building a machine learning model.
At least one data source must be provided and a maximum of two data sources is allowed: one for training data and one for validation data. How you split the data into training and validation datasets depends on whether you have one or two data sources.
+ If you only have **one data source**, the `ChannelType` is set to `training` by default and must have this value.
+ If the `ValidationFraction` value in [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html) is not set, 0.2 (20%) of the data from this source is used for validation by default.
+ If the `ValidationFraction` is set to a value between 0 and 1, the dataset is split based on the value specified, where the value specifies the fraction of the dataset used for validation.
+ If you have **two data sources**, the `ChannelType` of one of the `AutoMLJobChannel` objects must be set to `training`, the default value. The `ChannelType` of the other data source must be set to `validation`. The two data sources must have the same format, either CSV or Parquet, and the same schema. You must not set the value for the `ValidationFraction` in this case because all of the data from each source is used for either training or validation. Setting this value causes an error.
### How to enable automatic deployment
With Autopilot, you can automatically deploy your fine-tuned model to an endpoint. To enable automatic deployment for your fine-tuned model, include a `[ModelDeployConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-ModelDeployConfig)` in the AutoML job request. This allows the deployment of your fine-tuned model to a SageMaker AI endpoint. Below are the available configurations for customization.
+ To let Autopilot generate the endpoint name, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)` to `True`.
+ To provide your own name for the endpoint, set `[AutoGenerateEndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents) to False and provide a name of your choice in [EndpointName](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html#API_ModelDeployConfig_Contents)`.
### How to set the EULA acceptance when fine-tuning a model using the AutoML API
For models requiring the acceptance of an end-user license agreement before fine-tuning, you can accept the EULA by setting the `AcceptEula` attribute of the `[ModelAccessConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelAccessConfig.html)` to `True` in `[TextGenerationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html)` when configuring your `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
### How to set hyperparameters to optimize the learning process of a model
You can optimize the learning process of your text generation model by setting hyperparameter values in the `TextGenerationHyperParameters` attribute of `[TextGenerationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html)` when configuring your `[AutoMLProblemTypeConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html#sagemaker-CreateAutoMLJobV2-request-AutoMLProblemTypeConfig)`.
Autopilot allows for the setting of four common hyperparameters across all models.
+ `epochCount`: Its value should be a string containing an integer value within the range of `1` to `10`.
+ `batchSize`: Its value should be a string containing an integer value within the range of `1` to `64`.
+ `learningRate`: Its value should be a string containing a floating-point value within the range of `0` to `1`.
+ `learningRateWarmupSteps`: Its value should be a string containing an integer value within the range of `0` to `250`.
For more details on each hyperparameter, see [Hyperparameters for optimizing the learning process of your text generation models](autopilot-llms-finetuning-hyperparameters.md).
The following JSON example shows a `TextGenerationHyperParameters` field passed to the TextGenerationJobConfig where all four hyperparameters are configured.
```
"AutoMLProblemTypeConfig": {
"TextGenerationJobConfig": {
"BaseModelName": "Falcon7B",
"TextGenerationHyperParameters": {"epochCount":"5", "learningRate":"0.000001", "batchSize": "32", "learningRateWarmupSteps": "10"}
}
}
```
# Supported large language models for fine-tuning
Using Autopilot API, users can fine-tune large language models (LLMs) that are powered by Amazon SageMaker JumpStart.
**Note**
For fine-tuning models that require the acceptance of an end-user license agreement, you must explicitly declare EULA acceptance when creating your AutoML job. Note that after fine-tuning a pretrained model, the weights of the original model are changed, so you do not need to later accept a EULA when deploying the fine-tuned model.
For information on how to accept the EULA when creating a fine-tuning job using the AutoML API, see [How to set the EULA acceptance when fine-tuning a model using the AutoML API](autopilot-create-experiment-finetune-llms.md#autopilot-llms-finetuning-set-eula).
You can find the full details of each model by searching for your **JumpStart Model ID** in the following [model table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html#built-in-algorithms-with-pre-trained-model-table), and then following the link in the **Source** column. These details might include the languages supported by the model, biases it may exhibit, the datasets employed for fine-tuning, and more.
The following table lists the supported JumpStart models that you can fine-tune with an AutoML job.
| JumpStart Model ID | `BaseModelName` in API request | Description |
| --- | --- | --- |
| huggingface-textgeneration-dolly-v2-3b-bf16 | Dolly3B | Dolly 3B is a 2.8 billion parameter instruction-following large language model based on [pythia-2.8b](https://huggingface.co/EleutherAI/pythia-2.8b#pythia-28b). It is trained on the instruction/response fine tuning dataset [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and can perform tasks including brainstorming, classification, questions and answers, text generation, information extraction, and summarization. |
| huggingface-textgeneration-dolly-v2-7b-bf16 | Dolly7B | Dolly 7B is a 6.9 billion parameter instruction-following large language model based on [pythia-6.9b](https://huggingface.co/EleutherAI/pythia-6.9b). It is trained on the instruction/response fine tuning dataset [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and can perform tasks including brainstorming, classification, questions and answers, text generation, information extraction, and summarization. |
| huggingface-textgeneration-dolly-v2-12b-bf16 | Dolly12B | Dolly 12B is a 12 billion parameter instruction-following large language model based on [pythia-12b](https://huggingface.co/EleutherAI/pythia-12b). It is trained on the instruction/response fine tuning dataset [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and can perform tasks including brainstorming, classification, questions and answers, text generation, information extraction, and summarization. |
| huggingface-llm-falcon-7b-bf16 | Falcon7B | Falcon 7B is a 7 billion parameter causal large language model trained on 1,500 billion tokens enhanced with curated corpora. Falcon-7B is trained on English and French data only, and does not generalize appropriately to other languages. Because the model was trained on large amounts of web data, it carries the stereotypes and biases commonly found online. |
| huggingface-llm-falcon-7b-instruct-bf16 | Falcon7BInstruct | Falcon 7B Instruct is a 7 billion parameter causal large language model built on Falcon 7B and fine-tuned on a 250 million tokens mixture of chat/instruct datasets. Falcon 7B Instruct is mostly trained on English data, and does not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it carries the stereotypes and biases commonly encountered online. |
| huggingface-llm-falcon-40b-bf16 | Falcon40B | Falcon 40B is a 40 billion parameter causal large language model trained on 1,000 billion tokens enhanced with curated corpora. It is trained mostly on English, German, Spanish, and French, with limited capabilities in Italian, Portuguese, Polish, Dutch, Romanian, Czech, and Swedish. It does not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it carries the stereotypes and biases commonly encountered online. |
| huggingface-llm-falcon-40b-instruct-bf16 | Falcon40BInstruct | Falcon 40B Instruct is a 40 billion parameter causal large language model built on Falcon40B and fine-tuned on a mixture of Baize. It is mostly trained on English and French data, and does not generalize appropriately to other languages. Furthermore, as it is trained on a large-scale corpora representative of the web, it carries the stereotypes and biases commonly encountered online. |
| huggingface-text2text-flan-t5-large | FlanT5L | The [https://huggingface.co/docs/transformers/model_doc/t5](https://huggingface.co/docs/transformers/model_doc/t5) model family is a set of large language models that are fine-tuned on multiple tasks and can be further trained. These models are well-suited for tasks such as language translation, text generation, sentence completion, word sense disambiguation, summarization, or question answering. Flan T5 L is a 780 million parameter large language model trained on numerous languages. You can find the list of the languages supported by Flan T5 L in the details of the model retrieved from your search by model ID in JumpStart's [model table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html#built-in-algorithms-with-pre-trained-model-table). |
| huggingface-text2text-flan-t5-xl | FlanT5XL | The [https://huggingface.co/docs/transformers/model_doc/t5](https://huggingface.co/docs/transformers/model_doc/t5) model family is a set of large language models that are fine-tuned on multiple tasks and can be further trained. These models are well-suited for tasks such as language translation, text generation, sentence completion, word sense disambiguation, summarization, or question answering. Flan T5 XL is a 3 billion parameter large language model trained on numerous languages. You can find the list of the languages supported by Flan T5 XL in the details of the model retrieved from your search by model ID in JumpStart's [model table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html#built-in-algorithms-with-pre-trained-model-table). |
| huggingface-text2text-flan-t5-xxll | FlanT5XXL | The [https://huggingface.co/docs/transformers/model_doc/t5](https://huggingface.co/docs/transformers/model_doc/t5) model family is a set of large language models that are fine-tuned on multiple tasks and can be further trained. These models are well-suited for tasks such as language translation, text generation, sentence completion, word sense disambiguation, summarization, or question answering. Flan T5 XXL is a 11 billion parameter model. You can find the list of the languages supported by Flan T5 XXL in the details of the model retrieved from your search by model ID in JumpStart's [model table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html#built-in-algorithms-with-pre-trained-model-table). |
| meta-textgeneration-llama-2-7b | Llama2-7B | Llama 2 is a collection of pretrained and fine-tuned generative text models, ranging in scale from 7 billion to 70 billion parameters. Llama2-7B is the 7 billion parameter model that is intended for English use and can be adapted for a variety of natural language generation tasks. |
| meta-textgeneration-llama-2-7b-f | Llama2-7BChat | Llama 2 is a collection of pretrained and fine-tuned generative text models, ranging in scale from 7 billion to 70 billion parameters. Llama2-7B is the 7 billion parameter chat model that is optimized for dialogue use cases. |
| meta-textgeneration-llama-2-13b | Llama2-13B | Llama 2 is a collection of pretrained and fine-tuned generative text models, ranging in scale from 7 billion to 70 billion parameters. Llama2-13B is the 13 billion parameter model that is intended for English use and can be adapted for a variety of natural language generation tasks. |
| meta-textgeneration-llama-2-13b-f | Llama2-13BChat | Llama 2 is a collection of pretrained and fine-tuned generative text models, ranging in scale from 7 billion to 70 billion parameters. Llama2-13B is the 13 billion parameter chat model that is optimized for dialogue use cases. |
| huggingface-llm-mistral-7b | Mistral7B | Mistral 7B is a seven billion parameters code and general purpose English text generation model. It can be used in a variety of use cases including text summarization, classification, text completion, or code completion. |
| huggingface-llm-mistral-7b-instruct | Mistral7BInstruct | Mistral 7B Instruct is the fine-tuned version of Mistral 7B for conversational use cases. It was specialized using a variety of publicly available conversation datasets in English. |
| huggingface-textgeneration1-mpt-7b-bf16 | MPT7B | MPT 7B is a decoder-style transformer large language model with 6.7 billion parameters, pre-trained from scratch on 1 trillion tokens of English text and code. It is prepared to handle long context lengths. |
| huggingface-textgeneration1-mpt-7b-instruct-bf16 | MPT7BInstruct | MPT 7B Instruct is a model for short-form instruction following tasks. It is built by fine-tuning MPT 7B on a dataset derived from [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and the [Anthropic Helpful and Harmless (HH-RLHF)](https://huggingface.co/datasets/Anthropic/hh-rlhf) datasets. |
# Dataset file types and input data format
Instruction-based fine-tuning uses labeled datasets to improve the performance of pre-trained LLMs on specific natural language processing (NLP) tasks. The labeled examples are formatted as prompt-response pairs and phrased as instructions.
To learn about the supported dataset file types, see [Supported dataset file types](#autopilot-llms-finetuning-dataset-format).
To learn about input data format, see [Input data format for instruction-based fine-tuning](#autopilot-llms-finetuning-input-format).
## Supported dataset file types
Autopilot supports instruction-based fine-tuning datasets formatted as CSV files (default) or as Parquet files.
+ **CSV** (comma separated values) is a row-based file format that stores data in human readable plaintext, which is a popular choice for data exchange as it is supported by a wide range of applications.
+ **Parquet** is a binary, column-based file format where the data is stored and processed more efficiently than in human readable file formats such as CSV. This makes it a better option for big data problems.
**Note**
The dataset may consist of multiple files, each of which must adhere to a specific template. For information on how to format your input data, see [Input data format for instruction-based fine-tuning](#autopilot-llms-finetuning-input-format).
## Input data format for instruction-based fine-tuning
Each file in the dataset must adhere to the following format:
+ The dataset must contain exactly two comma-separated and named columns, `input` and `output`. Autopilot does not allow any additional columns.
+ The `input` columns contain the prompts, and their corresponding `output` contains the expected answer. Both the `input` and `output` are in string format.
The following example illustrates the input data format for instruction-based fine-tuning in Autopilot.
```
input,output
"",""
```
**Note**
We recommend using datasets with a minimum of 1000 rows to ensure optimal learning and performance of the model.
Additionally, Autopilot sets a maximum limit on the number of rows in the dataset and the context length based on the type of model being used.
+ The limits on the number of rows in a dataset apply to the cumulative count of rows across all files within the dataset, including multiple files. If there are two [channel types](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html) defined (one for training and one for validation), the limit applies to the total number of rows across all datasets within both channels. When the number of rows exceeds the threshold, the job fails with a validation error.
+ When the length of the input or output of a row in the dataset exceeds the limit set on the context of the language model, it is automatically truncated. If more than 60% of the rows in the dataset are truncated, whether in their input or output, Autopilot fails the job with a validation error.
The following table presents those limits for each model.
| JumpStart Model ID | `BaseModelName` in API request | Row Limit | Context Length Limit |
| --- | --- | --- | --- |
| huggingface-textgeneration-dolly-v2-3b-bf16 | Dolly3B | 10,000 rows | 1024 tokens |
| huggingface-textgeneration-dolly-v2-7b-bf16 | Dolly7B | 10,000 rows | 1024 tokens |
| huggingface-textgeneration-dolly-v2-12b-bf16 | Dolly12B | 10,000 rows | 1024 tokens |
| huggingface-llm-falcon-7b-bf16 | Falcon7B | 1,000 rows | 1024 tokens |
| huggingface-llm-falcon-7b-instruct-bf16 | Falcon7BInstruct | 1,000 rows | 1024 tokens |
| huggingface-llm-falcon-40b-bf16 | Falcon40B | 10,000 rows | 1024 tokens |
| huggingface-llm-falcon-40b-instruct-bf16 | Falcon40BInstruct | 10,000 rows | 1024 tokens |
| huggingface-text2text-flan-t5-large | FlanT5L | 10,000 rows | 1024 tokens |
| huggingface-text2text-flan-t5-xl | FlanT5XL | 10,000 rows | 1024 tokens |
| huggingface-text2text-flan-t5-xxll | FlanT5XXL | 10,000 rows | 1024 tokens |
| meta-textgeneration-llama-2-7b | Llama2-7B | 10,000 rows | 2048 tokens |
| meta-textgeneration-llama-2-7b-f | Llama2-7BChat | 10,000 rows | 2048 tokens |
| meta-textgeneration-llama-2-13b | Llama2-13B | 7,000 rows | 2048 tokens |
| meta-textgeneration-llama-2-13b-f | Llama2-13BChat | 7,000 rows | 2048 tokens |
| huggingface-llm-mistral-7b | Mistral7B | 10,000 rows | 2048 tokens |
| huggingface-llm-mistral-7b-instruct | Mistral7BInstruct | 10,000 rows | 2048 tokens |
| huggingface-textgeneration1-mpt-7b-bf16 | MPT7B | 10,000 rows | 1024 tokens |
| huggingface-textgeneration1-mpt-7b-instruct-bf16 | MPT7BInstruct | 10,000 rows | 1024 tokens |
# Hyperparameters for optimizing the learning process of your text generation models
You can optimize the learning process of your base model by adjusting any combination of the following hyperparameters. These parameters are available for all models.
+ **Epoch Count**: The `epochCount` hyperparameter determines how many times the model goes through the entire training dataset. It influences the training duration and can prevent overfitting when set appropriately. Large number of epochs may increase the overall runtime of fine-tuning jobs. We recommend setting a large `MaxAutoMLJobRuntimeInSeconds` within the `CompletionCriteria` of the `[TextGenerationJobConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html)` to avoid fine-tuning jobs from stopping prematurely.
+ **Batch Size**: The `batchSize` hyperparameter defines the number of data samples used in each iteration of training. It can affect the convergence speed and memory usage. With large batch size, the risk of out of memory (OOM) errors increases, which may surface as an internal server error in Autopilot. To check for such error, check the `/aws/sagemaker/TrainingJobs` log group for the training jobs launched by your Autopilot job. You can access those logs in CloudWatch from in the Amazon management console. Choose **Logs**, and then choose the `/aws/sagemaker/TrainingJobs` **log group**. To remedy OOM errors, reduce the batch size.
We recommend starting with a batch size of 1, then incrementally increase it until an out of memory error occurs. As a reference, 10 epochs typically takes up to 72h to complete.
+ **Learning Rate**: The `learningRate` hyperparameter controls the step size at which a model's parameters are updated during training. It determines how quickly or slowly the model's parameters are updated during training. A high learning rate means that the parameters are updated by a large step size, which can lead to faster convergence but may also cause the optimization process to overshoot the optimal solution and become unstable. A low learning rate means that the parameters are updated by a small step size, which can lead to more stable convergence but at the cost of slower learning.
+ **Learning Rate Warmup Steps**: The `learningRateWarmupSteps` hyperparameter specifies the number of training steps during which the learning rate gradually increases before reaching its target or maximum value. This helps the model converge more effectively and avoid issues like divergence or slow convergence that can occur with an initially high learning rate.
To learn about how to adjust hyperparameters for your fine-tuning experiment in Autopilot and discover their possible values, see [How to set hyperparameters to optimize the learning process of a model](autopilot-create-experiment-finetune-llms.md#autopilot-llms-finetuning-set-hyperparameters).
# Metrics for fine-tuning large language models in Autopilot
The following section describes the metrics that you can use to understand your fine-tuned large language models (LLMs). Using your dataset, Autopilot directly fine-tunes a target LLM to enhance a default objective metric, the cross-entropy loss.
Cross-entropy loss is a widely used metric to assess the dissimilarity between the predicted probability distribution and the actual distribution of words in the training data. By minimizing cross-entropy loss, the model learns to make more accurate and contextually relevant predictions, particularly in tasks related to text generation.
After fine-tuning an LLM you can evaluate the quality of its generated text using a range of ROUGE scores. Additionally, you can analyze the perplexity and cross-entropy training and validation losses as part of the evaluation process.
+ Perplexity loss measures how well the model can predict the next word in a sequence of text, with lower values indicating a better understanding of the language and context.
+ Recall-Oriented Understudy for Gisting Evaluation (ROUGE) is a set of metrics used in the field of natural language processing (NLP) and machine learning to evaluate the quality of machine-generated text, such as text summarization or text generation. It primarily assesses the similarities between the generated text and the ground truth reference (human-written) text of a validation dataset. ROUGE measures are designed to assess various aspects of text similarity, including the precision and recall of n-grams (contiguous sequences of words) in the system-generated and reference texts. The goal is to assess how well a model captures the information present in the reference text.
There are several variants of ROUGE metrics, depending on the type of n-grams used and the specific aspects of text quality being evaluated.
The following list contains the name and description of the ROUGE metrics available after the fine-tuning of large language models in Autopilot.
**`ROUGE-1`, `ROUGE-2`**
ROUGE-N, the primary ROUGE metric, measures the overlap of n-grams between the system-generated and reference texts. ROUGE-N can be adjusted to different values of `n` (here `1` or `2`) to evaluate how well the system-generated text captures the n-grams from the reference text.
**`ROUGE-L`**
ROUGE-L (ROUGE-Longest Common Subsequence) calculates the longest common subsequence between the system-generated text and the reference text. This variant considers word order in addition to content overlap.
**`ROUGE-L-Sum`**
ROUGE-L-SUM (Longest Common Subsequence for Summarization) is designed for the evaluation of text summarization systems. It focuses on measuring the longest common subsequence between the machine-generated summary and the reference summary. ROUGE-L-SUM takes into account the order of words in the text, which is important in text summarization tasks.
# Autopilot model deployment and predictions
After fine-tuning a large language model (LLM), you can deploy the model for real-time text generation by setting up an endpoint to obtain interactive predictions.
**Note**
We recommend running real-time inference jobs on `ml.g5.12xlarge` for better performances. Alternatively, `ml.g5.8xlarge` instances are suitable for Falcon-7B-Instruct and MPT-7B-Instruct text generation tasks.
You can find the specifics of these instances within the [Accelerated Computing](https://www.amazonaws.cn/ec2/instance-types/) category in the selection of instance types provided by Amazon EC2.
## Real-time text generation
You can use SageMaker APIs to manually deploy your fine-tuned model to a SageMaker AI Hosting [real-time inference endpoint](https://docs.amazonaws.cn/sagemaker/latest/dg/realtime-endpoints.html), then begin making predictions by invoking the endpoint as follows.
**Note**
Alternatively, you can chose the automatic deployment option when creating your fine-tuning experiment in Autopilot. For information on setting up the automatic deployment of models, see [How to enable automatic deployment](autopilot-create-experiment-finetune-llms.md#autopilot-llms-finetuning-auto-model-deployment).
You can also use the SageMaker Python SDK and the `JumpStartModel` class to perform inferences with models fine-tuned by Autopilot. This can be done by specifying a custom location for the model's artifact in Amazon S3. For information on defining your model as a JumpStart model and deploying your model for inference, see [Low-code deployment with the JumpStartModel class](https://sagemaker.readthedocs.io/en/stable/overview.html#deploy-a-pre-trained-model-directly-to-a-sagemaker-endpoint).
1. **Obtain the candidate inference container definitions**
You can find the `InferenceContainerDefinitions` within the `BestCandidate` object retrieved from the response to the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html#API_DescribeAutoMLJobV2_ResponseSyntax) API call. A container definition for inference refers to the containerized environment designed for deploying and running your trained model to make predictions.
The following Amazon CLI command example uses the [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) API to obtain recommended container definitions for your job name.
```
aws sagemaker describe-auto-ml-job-v2 --auto-ml-job-name job-name --region region
```
1. **Create a SageMaker AI model**
Use the container definitions from the previous step to create a SageMaker AI model by using the [CreateModel](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateModel.html) API. See the following Amazon CLI command as an example. Use the `CandidateName` for your model name.
```
aws sagemaker create-model --model-name '' \
--primary-container '' --region '
```
1. **Create an endpoint configuration**
The following Amazon CLI command example uses the [CreateEndpointConfig](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpointConfig.html) API to create an endpoint configuration.
**Note**
To prevent the endpoint creation from timing out due to a lengthy model download, we recommend setting `ModelDataDownloadTimeoutInSeconds = 3600` and `ContainerStartupHealthCheckTimeoutInSeconds = 3600`.
```
aws sagemaker create-endpoint-config --endpoint-config-name '' \
--production-variants '' ModelDataDownloadTimeoutInSeconds=3600 ContainerStartupHealthCheckTimeoutInSeconds=3600 \
--region ''
```
1. **Create the endpoint**
The following Amazon CLI example uses the [CreateEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateEndpoint.html) API to create the endpoint.
```
aws sagemaker create-endpoint --endpoint-name '' \
--endpoint-config-name '' \
--region ''
```
Check the progress of your endpoint deployment by using the [DescribeEndpoint](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeEndpoint.html) API. See the following Amazon CLI command as an example.
```
aws sagemaker describe-endpoint —endpoint-name '' —region
```
After the `EndpointStatus` changes to `InService`, the endpoint is ready to use for real-time inference.
1. **Invoke the endpoint**
The following command invokes the endpoint for real-time inferencing. Your prompt needs to be encoded in bytes.
**Note**
The format of your input prompt depends on the language model. For more information on the format of text generation prompts, see [Request format for text generation models real-time inference](#autopilot-llms-finetuning-realtime-prompt-examples).
```
aws sagemaker invoke-endpoint --endpoint-name '' \
--region '' --body '' [--content-type] 'application/json'
```
## Request format for text generation models real-time inference
Different large language models (LLMs) may have specific software dependencies, runtime environments, and hardware requirements influencing Autopilot's recommended container to host the model for inference. Additionally, each model dictates the required input data format and the expected format for predictions and outputs.
Here are example inputs for some models and recommended containers.
+ For Falcon models with the recommended container `huggingface-pytorch-tgi-inference:2.0.1-tgi1.0.3-gpu-py39-cu118-ubuntu20.04`:
```
payload = {
"inputs": "Large language model fine-tuning is defined as",
"parameters": {
"do_sample": false,
"top_p": 0.9,
"temperature": 0.1,
"max_new_tokens": 128,
"stop": ["<|endoftext|>", ""]
}
}
```
+ For all other models with the recommended container `djl-inference:0.22.1-fastertransformer5.3.0-cu118`:
```
payload= {
"text_inputs": "Large language model fine-tuning is defined as"
}
```
# Create a Regression or Classification Autopilot experiment for tabular data using the Studio Classic UI
**Important**
As of November 30, 2023, Autopilot's UI is migrating to [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html) as part of the updated [Amazon SageMaker Studio](studio-updated.md) experience. SageMaker Canvas provides analysts and citizen data scientists no-code capabilities for tasks such as data preparation, feature engineering, algorithm selection, training and tuning, inference, and more. Users can leverage built-in visualizations and what-if analysis to explore their data and different scenarios, with automated predictions enabling them to easily productionize their models. Canvas supports a variety of use cases, including computer vision, demand forecasting, intelligent search, and generative AI.
Users of [Amazon SageMaker Studio Classic](studio.md), the previous experience of [Studio](studio-updated.md), can continue using the Autopilot UI in Studio Classic. Users with coding experience can continue using all [API references](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-reference.html) in any supported SDK for technical implementation.
If you have been using Autopilot in Studio Classic until now and want to migrate to SageMaker Canvas, you might have to grant additional permissions to your user profile or IAM role so that you can create and use the SageMaker Canvas application. For more information, see [(Optional) Migrate from Autopilot in Studio Classic to SageMaker Canvas](studio-updated-migrate-ui.md#studio-updated-migrate-autopilot).
All UI-related instructions in this guide pertain to Autopilot's standalone features before migrating to [Amazon SageMaker Canvas](https://docs.amazonaws.cn/sagemaker/latest/dg/canvas.html). Users following these instructions should use [Studio Classic](studio.md).
You can use the Amazon SageMaker Studio Classic UI to create Autopilot experiments for classification or regression problems on tabular data. The UI helps you specify the name of your experiment, provide locations for the input and output data, and specify which target data to predict. Optionally, you can also specify the type of problem that you want to solve (regression, classification, multiclass classification), choose your modeling strategy (*stacked ensembles* or *hyperparameters optimization*), select the list of algorithms used by the Autopilot job to train the data, and more.
The UI has descriptions, toggle switches, dropdown menus, radio buttons, and more to help you navigate creating your model candidates. After the experiment runs, you can compare trials and delve into the details of the pre-processing steps, algorithms, and hyperparameter ranges of each model. Optionally, you can download their [explainability](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-explainability.html) and [performance](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-insights.html) reports. Use the provided [ notebooks](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-notebook-output.html ) to see the results of the automated data exploration or the candidate model definitions.
Alternatively, you can use Autopilot AutoML API in [Create Regression or Classification Jobs for Tabular Data Using the AutoML API](autopilot-automate-model-development-create-experiment.md).
# Configure the default parameters of an Autopilot experiment (for administrators)
Autopilot supports setting default values to simplify the configuration of Amazon SageMaker Autopilot when you create an Autopilot experiment using the Studio Classic UI. Administrators can use Studio Classic [lifecycle configurations](studio-lcc.md) (LCC) to set infrastructure, networking, and security values in configuration files and pre-populate the [advanced settings](autopilot-automate-model-development-create-experiment-ui.md#advanced-settings) of `AutoML` jobs.
By doing so, they can fully control network connectivity and access permissions for the resources associated with Amazon SageMaker Studio Classic, including SageMaker AI instances, data sources, output data, and other related services. Specifically, administrators can configure a desired network architecture, such as Amazon VPC, subnets, and security groups, for a Studio Classic domain or individual user profiles. Data scientists can focus on data science specific parameters when creating their Autopilot experiments using the Studio Classic UI. Furthermore, administrators can manage the encryption of data on the instance in which Autopilot experiments run by setting default encryption keys.
**Note**
This feature is currently not available in the Asia Pacific (Hong Kong) and Middle East (Bahrain) opt-in Regions.
In the following sections, you can find the full list of parameters supporting the setting of defaults when creating an Autopilot experiment using the Studio Classic UI, and learn how to set those default values.
**Topics**
+ [
## List of default parameters supported
](#autopilot-list-default-parameters-create-experiment)
+ [
## Set default Autopilot experiment parameters
](#autopilot-set-default-parameters-create-experiment-howto)
## List of default parameters supported
The following parameters support setting default values with a configuration file for creating an Autopilot experiment using the Studio Classic UI. Once set, the values automatically fill in their corresponding field in the Autopilot' **Create Experiment** tab in the Studio Classic UI. See [Advanced settings (optional)](autopilot-automate-model-development-create-experiment-ui.md#advanced-settings) for a full description of each field.
+ **Security:** Amazon VPC, subnets, and security groups.
+ **Access:** Amazon IAM role ARNs.
+ **Encryption:** Amazon KMS key IDs.
+ **Tags:** Key-value pairs used to label and organize SageMaker AI resources.
## Set default Autopilot experiment parameters
Administrators can set default values in a configuration file, then manually place the file in a recommended location within the Studio Classic environment of specific users, or they can pass the file to a lifecycle configuration script (LCC) to automate the customization of the Studio Classic environment for a given domain or user profile.
+ To set up the configuration file, start by filling in its default parameters.
To configure any or all default values listed in [List of default parameters supported](#autopilot-list-default-parameters-create-experiment), administrators can create a configuration file named `config.yaml`, the structure of which should adhere to this [sample configuration file](https://sagemaker.readthedocs.io/en/stable/overview.html#configuration-file-structure). The following snippet shows a sample configuration file with all the supported `AutoML` parameters. For more information on the format of this file, refer to the [full schema](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/config/config_schema.py).
```
SchemaVersion: '1.0'
SageMaker:
AutoMLJob:
# https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateAutoMLJob.html
AutoMLJobConfig:
SecurityConfig:
EnableInterContainerTrafficEncryption: true
VolumeKmsKeyId: 'kms-key-id'
VpcConfig:
SecurityGroupIds:
- 'security-group-id-1'
- 'security-group-id-2'
Subnets:
- 'subnet-1'
- 'subnet-2'
OutputDataConfig:
KmsKeyId: 'kms-key-id'
RoleArn: 'arn:aws:iam::111222333444:role/Admin'
Tags:
- Key: 'tag_key'
Value: 'tag_value'
```
+ Then, place the configuration file in the recommended location by either [manually copying the file](#autopilot-intelligent-defaults-manual-setup) to its recommended paths or using a [lifecycle configuration](#autopilot-intelligent-defaults-lcc-setup) (LCC).
The configuration file needs to be present in at least one of the following locations in the user's Studio Classic environment. By default, SageMaker AI searches for a configuration file in two locations:
+ First, in `/etc/xdg/sagemaker/config.yaml`. We refer to this file as the *administrator configuration file*.
+ Then, in `/root/.config/sagemaker/config.yaml`. We refer to this file as the *user configuration file*.
Using the *administrator* configuration file, administrators can define a set of default values. Optionally, they can use the *user* configuration file to override values set in the *administrator* configuration file, or set additional default parameter values.
The following snippet shows a sample script which writes the default parameters configuration file to the *administrator* location in the user's Studio Classic environment. You can replace `/etc/xdg/sagemaker` with `/root/.config/sagemaker` to write the file to the *user* location.
```
## Sample script with AutoML intelligent defaults
#!/bin/bash
sudo mkdir -p /etc/xdg/sagemaker
echo "SchemaVersion: '1.0'
CustomParameters:
AnyStringKey: 'AnyStringValue'
SageMaker:
AutoMLJob:
# https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateAutoMLJob.html
AutoMLJobConfig:
SecurityConfig:
EnableInterContainerTrafficEncryption: true
VolumeKmsKeyId: 'kms-key-id'
VpcConfig:
SecurityGroupIds:
- 'security-group-id-1'
- 'security-group-id-2'
Subnets:
- 'subnet-1'
- 'subnet-2'
OutputDataConfig:
KmsKeyId: 'kms-key-id'
RoleArn: 'arn:aws:iam::111222333444:role/Admin'
Tags:
- Key: 'tag_key'
Value: 'tag_value'
" | sudo tee /etc/xdg/sagemaker/config.yaml
```
+ **Copy the files manually** – To copy the configuration files manually, run the [script](#autopilot-intelligent-defaults-manual-setup) created in the previous step from a Studio Classic terminal. In this case, the user profile that executed the script can create Autopilot experiments with the default values applicable only to them.
+ **Create a SageMaker AI lifecycle configuration** – Alternatively, you can use a [lifecycle configuration](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-lcc.html) (LCC) to automate the customization of your Studio Classic environment. LCC are shell scripts triggered by Amazon SageMaker Studio Classic lifecycle events such as starting a Studio Classic application. This customization includes installing custom packages, configuring notebook extensions, pre-loading datasets, setting up source code repositories, or, in our case, pre-populating default parameters. Administrators can attach the LCC to a Studio Classic domain to automate the configuration of default values for each user profile within that domain.
The following sections detail how to create a lifecycle configuration so users can load Autopilot default parameters automatically when launching Studio Classic. You can choose to create an LCC using the SageMaker AI Console or the Amazon CLI.
------
#### [ Create a LCC from the SageMaker AI Console ]
Use the following steps to create an LCC containing your default parameters, attach the LCC to a domain or a user profile, then launch a Studio Classic application pre-populated with the default parameters set by the LCC using the SageMaker AI Console.
+ **To create a lifecycle configuration that runs the [script](#autopilot-intelligent-defaults-script) containing your default values using the SageMaker AI Console**
+ Open the SageMaker AI console at [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/).
+ On the left side, navigate to **Admin configurations**, then **Lifecycle configurations**.
+ From the **Lifecycle configurations** page, navigate to the Studio Classic tab, then choose **Create configuration**.
+ For **Name**, type a name using alphanumeric characters and "-", but no spaces. The name can have a maximum of 63 characters.
+ Paste your [script](#autopilot-intelligent-defaults-script) in the **Scripts** section.
+ Choose **Create configuration** to create the lifecycle configuration. This creates an LCC of type `Kernel gateway app`.
+ **To attach the lifecycle configuration to a Studio Classic domain, a space, or a user profile**
Follow the steps in [Attach the lifecycle configuration to Studio Classic domain or user profile ](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-lcc-create-console.html#studio-lcc-create-console-step2) to attach your LCC to a Studio Classic domain or a specific user profile.
+ **To launch your Studio Classic application with the lifecycle configuration**
Once the LCC is attached to a domain or a user profile, impacted users can start a Studio Classic application from the landing page of Studio Classic in Studio to pick up the defaults set by the LCC automatically. This auto-populates the Studio Classic UI when creating an Autopilot experiment.
------
#### [ Create a LCC from the Amazon CLI ]
Use the following snippets to launch a Studio Classic application that runs your [script](#autopilot-intelligent-defaults-manual-setup) using the Amazon CLI. Note that `lifecycle_config.sh` is the name given to your script in this example.
Before getting started:
+ Ensure that you have updated and configured Amazon CLI by completing the prerequisites described in [Create a lifecycle configuration from the Amazon CLI](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-lcc-create-cli.html).
+ Install [OpenSSL](https://www.openssl.org/source/) documentation. The Amazon CLI command uses the open-source library *OpenSSL* to encode your script in Base64 format. This requirement prevents errors that occur from spacing and line break encoding.
You can now follow these three steps:
+ **Create a new lifecycle configuration referencing the configuration script `lifecycle_config.sh`**
```
LCC_CONTENT=`openssl base64 -A -in lifecycle_config.sh`
## Create a new lifecycle config
aws sagemaker create-studio-lifecycle-config --region region \
--studio-lifecycle-config-name lcc-name \
--studio-lifecycle-config-content $LCC_CONTENT \
--studio-lifecycle-config-app-type default
```
Note the ARN of the newly created lifecycle configuration that is returned. This ARN is required to attach the lifecycle configuration to your application.
+ **Attach the lifecycle configuration to your `JupyterServerApp`**
The following example shows how to create a new user profile with a lifecycle configuration attached. To update an existing user profile, use the Amazon CLI [update-user-profile](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/update-user-profile.html) command. To create or update a domain, see [create-domain](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/create-domain.html) and [update-domain](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/update-domain.html). Add the lifecycle configuration ARN from the previous step to the settings of the `JupyterServerAppSettings` application type. You can add multiple lifecycle configurations at the same time by using a list of lifecycle configurations.
```
# Create a new UserProfile
aws sagemaker create-user-profile --domain-id domain-id \
--user-profile-name user-profile-name \
--region region \
--user-settings '{
"JupyterServerAppSettings": {
"LifecycleConfigArns":
["lifecycle-configuration-arn"]
}
}'
```
Once the LCC is attached to a domain or a user profile, impacted users can shut down and update their existing Studio Classic application by following the steps in [Shut down and Update Amazon SageMaker Studio Classic](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-tasks-update-studio.html), or start a new Studio Classic application from the Amazon Console to pick up the defaults set by the LCC automatically. This auto-populates the Studio Classic UI when creating an Autopilot experiment. Alternatively, they can launch a new Studio Classic application using the Amazon CLI as follows.
+ **Launch your Studio Classic application with the lifecycle configuration using the Amazon CLI**
```
# Create a Jupyter Server application
aws sagemaker create-app --domain-id domain-id \
--user-profile-name user-profile-name \
--region region \
--app-type JupyterServer \
--resource-spec LifecycleConfigArn=lifecycle-configuration-arn \
--app-name default
```
For more information on creating a lifecycle configuration using the Amazon CLI, see [Create a Lifecycle Configuration from the Amazon CLI](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-lcc-create-cli.html).
------
**To create an Autopilot experiment using Studio Classic UI**
1. Sign in at [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/), choose **Studio** from the left navigation pane, select your Domain and user profile, then **Open Studio**.
1. In Studio, choose the Studio Classic icon in the top left navigation pane. This opens a Studio Classic app.
1. Run or open a Studio Classic application from the space of your choice, or **Create Studio Classic space. **. On the **Home** tab, choose the **AutoML** card. This opens a new **AutoML** tab.
1. Choose **Create an AutoML experiment**. This opens a new **Create experiment** tab.
1. In the **Experiment and data details** section, enter the following information:
1. **Experiment name** – Must be unique to your account in the current Amazon Web Services Region and contain a maximum of 63 alphanumeric characters. Can include hyphens (-) but not spaces.
1. **Input data** – Provide the Amazon Simple Storage Service (Amazon S3) bucket location of your input data. This S3 bucket must be in your current Amazon Web Services Region. The URL must be in an `s3://` format where Amazon SageMaker AI has write permissions. The file must be in CSV or Parquet format and contain at least 500 rows. Select **Browse** to scroll through available paths and **Preview** to see a sample of your input data.
1. **Is your S3 input a manifest file?** – A manifest file includes metadata with your input data. The metadata specifies the location of your data in Amazon S3. It also specifies how the data is formatted and which attributes from the dataset to use when training your model. You can use a manifest file as an alternative to preprocessing when your labeled data is being streamed in `Pipe` mode.
1. **Auto split data?** – Autopilot can split your data into an 80-20% split for training and validation data. If you prefer a custom split, you can choose the **Specify split ratio**. To use a custom dataset for validation, choose **Provide a validation set**.
1. **Output data location (S3 bucket)** – The name of the S3 bucket location where you want to store the output data. The URL for this bucket must be in an Amazon S3 format where Amazon SageMaker AI has write permissions. The S3 bucket must be in the current Amazon Web Services Region. Autopilot can also create this for you in the same location as your input data.
1. Choose **Next: Target and features**. The **Target and features** tab opens.
1. In the **Target and features** section:
+ Select a column to set as a target for model predictions.
+ Optionally, you can pass the name of a sample weights column in the **Sample weight** section to request your dataset rows to be weighted during training and evaluation. For more information on the available objective metrics, see [Autopilot weighted metrics](autopilot-metrics-validation.md#autopilot-weighted-metrics).
**Note**
Support for sample weights is available in [ensembling mode](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html#autopilot-training-mode) only.
+ You can also select features for training and change their data type. The following data types are available: `Text`, `Numerical`, `Categorical`, `Datetime`, `Sequence`, and `Auto`. All features are selected by default.
1. Choose **Next: Training method**. The **Training method** tab opens.
1. In the **Training method** section, select your training option: **Ensembling**, **Hyperparameter optimization (HPO)**, or **Auto** to let Autopilot choose the training method automatically based on the dataset size. Each training mode runs a pre-defined set of algorithms on your dataset to train model candidates. By default, Autopilot pre-selects all the available algorithms for the given training mode. You can run an Autopilot training experiment with all the algorithms or choose your own subset.
For more information on the training modes and the available algorithms, see the **Autopilot training modes** section in the [Training modes and algorithms](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html) page.
1. Choose **Next: Deployment and advanced settings** to open the **Deployment and advanced settings** tab. Settings include the auto-display endpoint name, machine learning problem type, and additional choices for running your experiment.
1. **Deployment settings** – Autopilot can automatically create an endpoint and deploy your model for you.
To auto-deploy to an automatically generated endpoint, or to provide an endpoint name for custom deployment, set the toggle to **Yes** under **Auto deploy?** If you are importing data from Amazon SageMaker Data Wrangler, you have additional options to auto-deploy the best model with or without the transforms from Data Wrangler.
**Note**
If your Data Wrangler flow contains multi-row operations such as `groupby`, `join`, or `concatenate`, you can't auto-deploy with these transforms. For more information, see [Automatically Train Models on Your Data Flow](https://docs.amazonaws.cn/sagemaker/latest/dg/data-wrangler-autopilot.html).
1. **Advanced settings (optional)** – Autopilot provides additional controls to manually set experimental parameters such as defining your problem type, time constraints on your Autopilot job and trials, security, and encryption settings.
1. **Machine learning problem type** – Autopilot can automatically infer the type of supervised learning problem from your dataset. If you prefer to choose it manually, you can use the **Select the machine learning problem type** dropdown menu. Note that it defaults to **Auto**. In some cases, SageMaker AI is unable to infer accurately. When that happens, you must provide the value for the job to succeed. In particular, you can choose from the following types:
+ **Binary classification**– Binary classification assigns input data to one of two predefined and mutually exclusive classes, based on their attributes, such as medical diagnosis based on results of diagnostic tests that determine if someone has a disease.
+ **Regression** – Regression establishes a relationship between the input variables (also known as independent variables or features) and the target variable (also known as the dependent variable). This relationship is captured through a mathematical function or model that maps the input variables to a continuous output. It is commonly used for tasks such as predicting house prices based on features like square footage and the number of bathrooms, stock market trends, or estimating sales figures.
+ **Multiclass classification** – Multiclass classification assigns input data to one of several classes based on their attributes, like the prediction of the topic most relevant to a text document, such as politics, finance, or philosophy.
1. **Runtime** – You can define a maximum time limit. Upon reaching the time limit, trials and jobs that exceed the time constraint automatically stop.
1. **Access** – You can choose the role that Amazon SageMaker Studio Classic assumes to gain temporary access to Amazon Web Services services (in particular, SageMaker AI and Amazon S3) on your behalf. If no role is explicitly defined, Studio Classic automatically uses the default SageMaker AI execution role attached to your user profile.
1. **Encryption** – To enhance the security of your data at rest and protect it against unauthorized access, you can specify encryption keys to encrypt data in your Amazon S3 buckets and in the Amazon Elastic Block Store (Amazon EBS) volume attached to your Studio Classic domain.
1. **Security** – You can choose the virtual private cloud (Amazon VPC) in which your SageMaker AI job runs. Ensure that the Amazon VPC has access to your input and output Amazon S3 buckets.
1. **Project** – Specify the name of the SageMaker AI project to associate with this Autopilot experiment and model outputs. When you specify a project, Autopilot tags the project to an experiment. This lets you know which model outputs are associated with this project.
1. **Tags** – Tags are an array of key-value pairs. Use tags to categorize your resources from Amazon Web Services services, such as their purpose, owner, or environment.
1. Choose **Next: Review and create** to get a summary of your Autopilot experiment before you create it.
1. Select **Create experiment**.The creation of the experiment starts an Autopilot job in SageMaker AI. Autopilot provides the status of the experiment, information on the data exploration process and model candidates in notebooks, a list of generated models and their reports, and the job profile used to create them.
For information on the notebooks generated by an Autopilot job, see [Autopilot notebooks generated to manage AutoML tasks](autopilot-automate-model-development-notebook-output.md). For information on the details of each model candidate and their reports, see [View model details](autopilot-models-details.md) and [View an Autopilot model performance report](autopilot-model-insights.md).
**Note**
To avoid incurring unnecessary charges: If you deploy a model that is no longer needed, delete the endpoints and resources that were created during that deployment. Information about pricing instances by Region is available at [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing/).
# Amazon SageMaker Autopilot example notebooks
The following notebooks serve as practical, hands-on examples that address various use cases of Autopilot.
You can find all of Autopilot's notebooks in the [https://github.com/aws/amazon-sagemaker-examples/tree/main/autopilot](https://github.com/aws/amazon-sagemaker-examples/tree/main/autopilot) directory of SageMaker AI GitHub examples repository.
We recommend cloning the full Git repository within Studio Classic to access and run the notebooks directly. For information on how to clone a Git repository in Studio Classic, see [Clone a Git Repository in Amazon SageMaker Studio Classic](studio-tasks-git.md).
| **Use case** | **Description** |
| --- | --- |
| [Serverless inference](https://github.com/aws/amazon-sagemaker-examples/tree/main/autopilot/autopilot-serverless-inference) | By default, Autopilot allows deploying generated models to real-time inference endpoints. In this repository, the notebook illustrates how to deploy Autopilot models trained with `ENSEMBLING` and `HYPERPARAMETER OPTIMIZATION (HPO)` modes to serverless endpoints. Serverless endpoints automatically launch compute resources and scale them in and out depending on traffic, eliminating the need to choose instance types or manage scaling policies. |
| [Custom feature selection](https://github.com/aws/amazon-sagemaker-examples/tree/main/autopilot/custom-feature-selection) | Autopilot inspects your data set, and runs a number of candidates to figure out the optimal combination of data preprocessing steps, machine learning algorithms, and hyperparameters. You can easily deploy either on a real-time endpoint or for batch processing. In some cases, you might want to have the flexibility to bring custom data processing code to Autopilot. For example, your datasets might contain a large number of independent variables, and you may wish to incorporate a custom feature selection step to remove irrelevant variables first. The resulting smaller dataset can then be used to launch an Autopilot job. Ultimately, you would also want to include both the custom processing code and models from Autopilot for real-time or batch processing. |
| [Pipeline example](https://github.com/aws/amazon-sagemaker-examples/tree/main/autopilot/sagemaker-autopilot-pipelines) | While Autopilot streamlines the process of building ML models, MLOps engineers are still responsible for creating, automating, and managing end-to-end ML workflows in production. SageMaker Pipelines can assist in automating various steps of the ML lifecycle, such as data preprocessing, model training, hyperparameter tuning, model evaluation, and deployment. This notebook serves as a demonstration of how to incorporate Autopilot into a SageMaker Pipelines end-to-end AutoML training workflow. To launch an Autopilot experiment within Pipelines, you must create a model-building workflow by writing custom integration code using Pipelines [Lambda](https://docs.amazonaws.cn/sagemaker/latest/dg/build-and-manage-steps.html#step-type-lambda) or [Processing](https://docs.amazonaws.cn/sagemaker/latest/dg/build-and-manage-steps.html#step-type-processing) steps. For more information, refer to [Move Amazon SageMaker Autopilot ML models from experimentation to production using Amazon SageMaker Pipelines](https://amazonaws-china.com/blogs/machine-learning/move-amazon-sagemaker-autopilot-ml-models-from-experimentation-to-production-using-amazon-sagemaker-pipelines/). Alternatively, when using Autopilot in [Ensembling mode](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-model-support-validation.html), you can refer to the notebook example that demonstrates how to use native AutoML step in [SageMaker Pipeline's native AutoML step](https://github.com/aws/amazon-sagemaker-examples/blob/main/autopilot/sagemaker-autopilot-pipelines/autopilot_pipelines_demo_notebook.ipynb). With Autopilot supported as a native step within Pipelines, you can now add an automated training step ([AutoMLStep](https://docs.amazonaws.cn/sagemaker/latest/dg/build-and-manage-steps.html#step-type-automl)) to your Pipelines and invoke an Autopilot experiment in Ensembling mode. |
| [ Direct marketing with Amazon SageMaker Autopilot](https://sagemaker-examples.readthedocs.io/en/latest/autopilot/sagemaker_autopilot_direct_marketing.html) | This notebook demonstrates how uses the [Bank Marketing Data Set](https://archive.ics.uci.edu/ml/datasets/bank+marketing) to predict whether a customer will enroll for a term deposit at a bank. You can use Autopilot on this dataset to get the most accurate ML pipeline by exploring options contained in various candidate pipelines. Autopilot generates each candidate in a two-step procedure. The first step performs automated feature engineering on the dataset. The second step trains and tunes an algorithm to produce a model. The notebook contains instructions on how to train the model and how to deploy the model to perform batch inference using the best candidate. |
| [Customer Churn Prediction with Amazon SageMaker Autopilot](https://sagemaker-examples.readthedocs.io/en/latest/autopilot/autopilot_customer_churn.html) | This notebook describes using machine learning for the automated identification of unhappy customers, also known as customer churn prediction. The example shows how to analyze a publicly available dataset and perform feature engineering on it. Next it shows how to tune a model by selecting the best performing pipeline along with the optimal hyperparameters for the training algorithm. Finally, it shows how to deploy the model to a hosted endpoint and how to evaluate its predictions against ground truth. However, ML models rarely give perfect predictions. That's why this notebook also shows how to incorporate the relative costs of prediction mistakes when determining the financial outcome of using ML. |
| [Top Candidates Customer Churn Prediction with Amazon SageMaker Autopilot and Batch Transform (Python SDK)](https://sagemaker-examples.readthedocs.io/en/latest/autopilot/autopilot_customer_churn_high_level_with_evaluation.html) | This notebook also describes using machine learning for the automated identification of unhappy customers, also known as customer churn prediction. This notebook demonstrates how to configure the model to obtain the inference probability, select the top N models, and make Batch Transform on a hold-out test set for evaluation. This notebook works with SageMaker Python SDK >= 1.65.1 released on 6/19/2020. |
| [Bringing your own data processing code to Amazon SageMaker Autopilot](https://sagemaker-examples.readthedocs.io/en/latest/autopilot/custom-feature-selection/Feature_selection_autopilot.html) | This notebook demonstrates how to incorporate and deploy custom data processing code when using Amazon SageMaker Autopilot. It adds a custom feature selection step to remove irrelevant variables to an Autopilot job. It then shows how to deploy both the custom processing code and models generated by Autopilot on a real-time endpoint and, alternatively, for batch processing. |
| More notebooks | You can find more notebooks illustrating other use cases such as [batch transform](https://github.com/aws/amazon-sagemaker-examples/blob/main/autopilot/ap-batch-transform.ipynb), [time-series forecasting](https://github.com/aws/amazon-sagemaker-examples/blob/main/autopilot/autopilot_time_series.ipynb) and more in the root directory. |
# Videos: Use Autopilot to automate and explore the machine learning process
Here is a video series that provides a tour of Amazon SageMaker Autopilot capabilities using Studio Classic. They show how to start an AutoML job, analyze and preprocess data, how to do feature engineering and hyperparameter optimization on candidate models, and how to visualize and compare the resulting model metrics.
**Topics**
+ [
## Start an AutoML job with Amazon SageMaker Autopilot
](#autopilot-video-start-automl-job)
+ [
## Review data exploration and feature engineering automated in Autopilot.
](#autopilot-video-generated-notebooks)
+ [
## Tune models to optimize performance
](#autopilot-video-optimizing-model-performance)
+ [
## Choose and deploy the best model
](#autopilot-video-choose-and-deploy-the-best-model)
+ [
## Amazon SageMaker Autopilot tutorial
](#autopilot-walkthrough)
## Start an AutoML job with Amazon SageMaker Autopilot
This video shows you to how to start an AutoML job with Autopilot. (Length: 8:41)
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/qMEtqJPhqpA)
## Review data exploration and feature engineering automated in Autopilot.
This video shows you how to review the data exploration and candidate definition notebooks generated by Amazon SageMaker Autopilot. (Length: 10:04)
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/WsfRAeGzgm8)
## Tune models to optimize performance
This video shows you how to optimize model performance during training using hyperparameter tuning. (Length: 4:59)
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/KZSTsWrDGXs)
## Choose and deploy the best model
This video shows you how to use job metrics to choose the best model and then how to deploy it. (Length: 5:20)
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/vRHyX3kDstI)
## Amazon SageMaker Autopilot tutorial
This video walks you through an end to end demo where we first build a binary classification model automatically with Amazon SageMaker Autopilot. We see how candidate models have been built and optimized using auto-generated notebooks. We also look at the top candidates with Amazon SageMaker Experiments. Finally, we deploy the top candidate (based on XGBoost), and configure data capture with SageMaker Model Monitor.
[](http://www.youtube.com/watch?v=https://www.youtube.com/embed/DRjOOaR2prQ)
# Autopilot quotas
There are quotas that limit the resources available to you when using Amazon SageMaker Autopilot. Some of these limits are increasable and some are not.
**Note**
The resource quotas documented in the following sections are valid for versions of Amazon SageMaker Studio Classic 3.22.2 and higher. For information on updating your version of SageMaker Studio Classic, see [Shut Down and Update Amazon SageMaker Studio Classic and Apps](studio-tasks-update.md).
**Topics**
+ [
## Quotas that you can increase
](#autopilot-quotas-limits-increasable)
+ [
## Resource quotas
](#autopilot-quotas-resource-limits)
## Quotas that you can increase
The following table contains the resource limits for quotas you can increase:
| Resource | Regions | Default limits | Can be increased up to |
| --- | --- | --- | --- |
| Size of input dataset | All | 100 GB | Hundreds of GBs |
| Size of a single Parquet file\$1 | All | 2 GB | N/A |
| Target dataset size for subsampling\$1\$1 | All | 5 GB | Hundreds of GBs |
| Number of concurrent Autopilot jobs | us-east-1, us-east-2,us-west-2, ap-northeast-1, eu-west-1, eu-central-1 | 4 | Hundreds |
| Number of concurrent Autopilot jobs | ap-northeast-2, ap-southeast-2, eu-west-2, ap-southeast-1 | 2 | Hundreds |
| Number of concurrent Autopilot jobs | All other Regions | 1 | Tens |
**Note**
\$1This 2 GB size limit is for a single compressed Parquet file. You can provide a Parquet dataset that includes multiple compressed Parquet files up to the input dataset maximum size. After the files are decompressed, they may each expand to a larger size.
\$1\$1Autopilot automatically subsamples input datasets that are larger than the target dataset size while accounting for class imbalance and preserving rare class labels.
**To request a quota increase:**
1. Open the [ Service Quotas console](https://console.amazonaws.cn/servicequotas/home/services/sagemaker/quotas).
1. Select your quota increase, then choose **Request increase at account level**.
1. In the **Increase quota value**, enter the new limit value that you are requesting.
1. Choose **Request**.
## Resource quotas
The following table contains the runtime resource limits for an Amazon SageMaker Autopilot job in an Amazon Web Services Region.
| Resource | Limit per Autopilot job |
| --- | --- |
| Maximum runtime for an Autopilot job | 30 days |
# API Reference guide for Autopilot
This section provides a subset of the HTTP service REST APIs for creating and managing Amazon SageMaker Autopilot resources (AutoML jobs) programmatically.
If your language of choice is Python, you can refer to [Amazon SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html) or the [AutoMLV2 object](https://sagemaker.readthedocs.io/en/stable/api/training/automlv2.html#sagemaker.automl.automlv2.AutoMLV2) of the Amazon SageMaker Python SDK directly.
## AutoML API Actions
This list details the operations available in the Reference API to manage AutoML jobs programmatically.
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListAutoMLJobs.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListAutoMLJobs.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListCandidatesForAutoMLJob.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_StopAutoMLJob.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_StopAutoMLJob.html)
**Note**
[CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJobV2.html) and [DescribeAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJobV2.html) are new versions of [CreateAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CreateAutoMLJob.html) and [DescribeAutoMLJob](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_DescribeAutoMLJob.html) which offer backward compatibility.
We recommend using the `CreateAutoMLJobV2`. `CreateAutoMLJobV2` can manage tabular problem types identical to those of its previous version `CreateAutoMLJob`, as well as non-tabular problem types such as image or text classification, or time-series forecasting.
Find guidelines about how to migrate a `CreateAutoMLJob` to `CreateAutoMLJobV2` in [Migrate a CreateAutoMLJob to CreateAutoMLJobV2](https://docs.amazonaws.cn/sagemaker/latest/dg/autopilot-automate-model-development-create-experiment.html#autopilot-create-experiment-api-migrate-v1-v2).
## AutoML API Data Types
This list details the API AutoML objects used by the actions above as inbound requests or outbound responses.
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLAlgorithmConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidate.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateGenerationConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateStep.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLCandidateStep.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLChannel.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLContainerDefinition.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLContainerDefinition.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSource.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSource.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLDataSplitConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLInferenceContainerDefinitions.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLInferenceContainerDefinitions.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobArtifacts.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobArtifacts.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobChannel.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobCompletionCriteria.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobCompletionCriteria.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobInputDataConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobInputDataConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobObjective.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobObjective.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobStepMetadata.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobStepMetadata.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobSummary.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobSummary.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobCompletionCriteria.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobCompletionCriteria.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobSummary.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLJobSummary.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLOutputDataConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLPartialFailureReason.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLPartialFailureReason.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeResolvedAttributes.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLProblemTypeResolvedAttributes.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLResolvedAttributes.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLResolvedAttributes.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLSecurityConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLSecurityConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLS3DataSource.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_AutoMLS3DataSource.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateArtifactLocations.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateGenerationConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateProperties.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_CandidateProperties.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_FinalAutoMLJobObjectiveMetric.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_FinalAutoMLJobObjectiveMetric.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_HolidayConfigAttributes.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_HolidayConfigAttributes.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ImageClassificationJobConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ImageClassificationJobConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_MetricDatum.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_MetricDatum.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployResult.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ModelDeployResult.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ResolvedAttributes.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ResolvedAttributes.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularJobConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularResolvedAttributes.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TabularResolvedAttributes.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationJobConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationResolvedAttribute.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TextGenerationResolvedAttribute.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesForecastingJobConfig.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesTransformations.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TimeSeriesTransformations.html)
+ [https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TuningJobCompletionCriteria.html](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_TuningJobCompletionCriteria.html)
# SageMaker JumpStart pretrained models
Amazon SageMaker JumpStart provides pretrained, open-source models for a wide range of problem types to help you get started with machine learning. You can incrementally train and tune these models before deployment. JumpStart also provides solution templates that set up infrastructure for common use cases, and executable example notebooks for machine learning with SageMaker AI.
You can deploy, fine-tune, and evaluate pretrained models from popular models hubs through the Models landing page in the updated Studio experience.
You can also access pretrained models, solution templates, and examples through the Models landing page in Amazon SageMaker Studio Classic.
The following steps show how to access JumpStart models using Amazon SageMaker Studio and Amazon SageMaker Studio Classic.
You can also access JumpStart models using the SageMaker Python SDK. For information about how to use JumpStart models programmatically, see [Use SageMaker JumpStart Algorithms with Pretrained Models](https://sagemaker.readthedocs.io/en/stable/overview.html#use-sagemaker-jumpstart-algorithms-with-pretrained-models).
## Open JumpStart in Studio
In Amazon SageMaker Studio, open the Models landing page either through the **Home** page or the **Models** item in the left-side panel. This opens the **SageMaker Models** landing page where you can explore models in the SageMakerPublicHub, models in Private Hubs or Curated Hubs, and customized models.
+ From the **Home** page, choose **Explore models** in the **Start your model customization workflow** pane.
+ From the menu in the left panel, navigate to the **Models** node.
For more information on getting started with Amazon SageMaker Studio, see [Amazon SageMaker Studio](studio-updated.md).
![\[Amazon SageMaker Studio interface with access to JumpStart.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-studio-nav.png)
## Use JumpStart in Studio
**Important**
Before downloading or using third-party content: You are responsible for reviewing and complying with any applicable license terms and making sure that they are acceptable for your use case.
On 3/13/2026, we delisted a few models from the JumpStart catalog across regions to improve discoverability and focus on high-quality, well-supported options. Existing endpoints for delisted models will remain functional. For license information on delisted open-weight models, please refer to the Hugging Face listing of the respective models.
From the **SageMaker Models** landing page in Studio, you can explore JumpStart base models from both proprietary and publicly available model providers. You can search directly for models, filter by specific model provider, or filter based on a list of provided use cases and actions.
![\[Amazon SageMaker Studio Models landing page.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-studio-landing.png)
Choose a model to see its model detail card. In the upper right-hand corner of the model detail card, choose **Fine-tune**, **Customize**, **Deploy**, or **Evaluate** to start working through the fine-tuning, deployment, or evaluation workflows, respectively. Note that not all models are available for customization, fine-tuning or evaluation. For more information on each of these options, see [Use foundation models in Studio](jumpstart-foundation-models-use-studio-updated.md).
You can also access **Private or Curated Hub** models through a dedicated tab. These work exactly like JumpStart base models, and clicking on a model card will take you to the details page, where actions are available.
Additionally, select **My models** to access your fine-tuned and registered models. Outputs from customization jobs can be found here, under the **Logged** models tab. **Deployable** models can also be found here.
## Open and use JumpStart in Studio Classic
The following sections give information on how to open, use, and manage JumpStart from the Amazon SageMaker Studio Classic UI.
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
### Open JumpStart in Studio Classic
In Amazon SageMaker Studio Classic, open the JumpStart landing page either through the **Home** page or the **Home** menu on the left-side panel.
+ From the **Home** page you can either:
+ Choose **JumpStart** in the **Prebuilt and automated solutions** pane. This opens the **SageMaker JumpStart** landing page.
+ Choose a model directly in the **SageMaker JumpStart** landing page, or choose the **Explore All** option to see available solutions or models of a specific type.
+ From the **Home** menu in the left panel you can either:
+ Navigate to the **SageMaker JumpStart** node, then choose **Models, notebooks, solutions**. This opens the **SageMaker JumpStart** landing page.
+ Navigate to the **JumpStart** node, then choose **Launched JumpStart assets**.
The **Launched JumpStart assets** page lists your currently launched solutions, deployed model endpoints, and training jobs created with JumpStart. You can access the JumpStart landing page from this tab by clicking on the **Browse JumpStart** button at the top right of the tab.
The JumpStart landing page lists available end-to-end machine learning solutions, pretrained models, and example notebooks. From any individual solution or model page, you can choose the **Browse JumpStart** button (![\[Button labeled "Browse JumpStart" with an icon indicating a browsing action.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-browse-button.png)) at the top right of the tab to return to the **SageMaker JumpStart** page.
![\[SageMaker Studio Classic interface with access to JumpStart.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-assets.png)
**Important**
Before downloading or using third-party content: You are responsible for reviewing and complying with any applicable license terms and making sure that they are acceptable for your use case.
On 3/13/2026, we delisted a few models from the JumpStart catalog across regions to improve discoverability and focus on high-quality, well-supported options. Existing endpoints for delisted models will remain functional. For license information on delisted open-weight models, please refer to the Hugging Face listing of the respective models.
### Use JumpStart in Studio Classic
From the **SageMaker JumpStart** landing page, you can browse for solutions, models, notebooks, and other resources.
![\[SageMaker Studio Classic JumpStart landing page.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-use.png)
You can find JumpStart resources by using the search bar, or by browsing each category. Use the tabs to filter the available solutions by categories:
+ **Solutions** – In one step, launch comprehensive machine learning solutions that tie SageMaker AI to other Amazon Web Services services. Select **Explore All Solutions** to view all available solutions.
+ **Resources** – Use example notebooks, blogs, and video tutorials to learn and head start your problem types.
+ **Blogs** – Read details and solutions from machine learning experts.
+ **Video tutorials** – Watch video tutorials for SageMaker AI features and machine learning use cases from machine learning experts.
+ **Example notebooks** – Run example notebooks that use SageMaker AI features like Spot Instance training and experiments over a large variety of model types and use cases.
+ **Data types** – Find a model by data type (e.g., Vision, Text, Tabular, Audio, Text Generation). Select **Explore All Models** to view all available models.
+ **ML tasks** – Find a model by problem type (e.g., Image Classification, Image Embedding, Object Detection, Text Generation). Select **Explore All Models** to view all available models.
+ **Notebooks** – Find example notebooks that use SageMaker AI features across multiple model types and use cases. Select **Explore All Notebooks** to view all available example notebooks.
+ **Frameworks** – Find a model by framework (e.g., PyTorch, TensorFlow, Hugging Face).
### Manage JumpStart in Studio Classic
From the **Home** menu in the left panel, navigate to **SageMaker JumpStart**, then choose **Launched JumpStart assets** to list your currently launched solutions, deployed model endpoints, and training jobs created with JumpStart.
**Topics**
+ [
## Open JumpStart in Studio
](#jumpstart-open-studio)
+ [
## Use JumpStart in Studio
](#jumpstart-use-studio)
+ [
## Open and use JumpStart in Studio Classic
](#jumpstart-open-use)
+ [
# Amazon SageMaker JumpStart Foundation Models
](jumpstart-foundation-models.md)
+ [
# Private curated hubs for foundation model access control in JumpStart
](jumpstart-curated-hubs.md)
+ [
# Amazon SageMaker JumpStart in Studio Classic
](jumpstart-studio-classic.md)
# Amazon SageMaker JumpStart Foundation Models
Amazon SageMaker JumpStart offers state-of-the-art foundation models for use cases such as content writing, code generation, question answering, copywriting, summarization, classification, information retrieval, and more. Use JumpStart foundation models to build your own generative AI solutions and integrate custom solutions with additional SageMaker AI features. For more information, see [Getting started with Amazon SageMaker JumpStart](https://www.amazonaws.cn/sagemaker/jumpstart/getting-started/).
A foundation model is a large pre-trained model that is adaptable to many downstream tasks and often serves as the starting point for developing more specialized models. Examples of foundation models include LLaMa-3-70b, BLOOM 176B, FLAN-T5 XL, or GPT-J 6B, which are pre-trained on massive amounts of text data and can be fine-tuned for specific language tasks.
Amazon SageMaker JumpStart onboards and maintains publicly available foundation models for you to access, customize, and integrate into your machine learning lifecycles. For more information, see [Publicly available foundation models](jumpstart-foundation-models-latest.md#jumpstart-foundation-models-latest-publicly-available). Amazon SageMaker JumpStart also includes proprietary foundation models from third-party providers. For more information, see [Proprietary foundation models](jumpstart-foundation-models-latest.md#jumpstart-foundation-models-latest-proprietary).
To get started exploring and experimenting with available models, see [JumpStart foundation model usage](jumpstart-foundation-models-use.md). All foundation models are available to use programmatically with the SageMaker Python SDK. For more information, see [Use foundation models with the SageMaker Python SDK](jumpstart-foundation-models-use-python-sdk.md).
For more information on considerations to make when choosing a model, see [Model sources and license agreements](jumpstart-foundation-models-choose.md).
For specifics about customization and fine-tuning foundation models, see [Foundation model customization](jumpstart-foundation-models-customize.md).
For more general information on foundation models, see the paper [On the Opportunities and Risks of Foundation Models](https://arxiv.org/abs/2108.07258).
**Topics**
+ [
# Available foundation models
](jumpstart-foundation-models-latest.md)
+ [
# JumpStart foundation model usage
](jumpstart-foundation-models-use.md)
+ [
# Model sources and license agreements
](jumpstart-foundation-models-choose.md)
+ [
# Foundation model customization
](jumpstart-foundation-models-customize.md)
+ [
# Evaluate a text generation foundation model in Studio
](jumpstart-foundation-models-evaluate.md)
+ [
# Example notebooks
](jumpstart-foundation-models-example-notebooks.md)
# Available foundation models
Amazon SageMaker JumpStart offers state-of-the-art, built-in publicly available and proprietary foundation models to customize and integrate into your generative AI workflows.
## Publicly available foundation models
Amazon SageMaker JumpStart onboards and maintains open source foundation models from third-party sources. To get started with one of these publicly available models, see [JumpStart foundation model usage](jumpstart-foundation-models-use.md) or explore one of the available [Example notebooks](jumpstart-foundation-models-example-notebooks.md). In a given example notebook for a publicly available model, try switching out the model ID to experiment with different models within the same model family.
For more information on model IDs and resources on deploying publicly available JumpStart foundation models with the SageMaker Python SDK, see [Use foundation models with the SageMaker Python SDK](jumpstart-foundation-models-use-python-sdk.md).
By definition, foundation models are adaptable to many downstream tasks. Foundation models are trained on massive amounts of general domain data and the same model can be implemented or customized for multiple use cases. When choosing your foundation model, start with defining a specific task, such as text generation or image generation.
### Publicly available time series forecasting models
Time series forecasting models are designed to analyze and make predictions on sequential data over time. These models can be applied to various domains such as finance, weather forecasting, or energy demand forecasting. The Chronos models are tailored for time series forecasting tasks, enabling accurate predictions based on historical data patterns.
| Model Name | Model ID | Model Source | Fine-tunable |
| --- | --- | --- | --- |
| Chronos T5 Small | autogluon-forecasting-chronos-t5-small | Amazon | No |
| Chronos T5 Base | autogluon-forecasting-chronos-t5-base | Amazon | No |
| Chronos T5 Large | autogluon-forecasting-chronos-t5-large | Amazon | No |
| Chronos-Bolt Small | autogluon-forecasting-chronos-bolt-small | Amazon | No |
| Chronos-Bolt Base | autogluon-forecasting-chronos-bolt-base | Amazon | No |
### Publicly available text generation models
Text generation foundation models can be used for a variety of downstream tasks, including text summarization, text classification, question answering, long-form content generation, short-form copywriting, information extraction, and more.
To explore the latest text generation JumpStart foundation models, use the **Text Generation** filter on the [Getting started with Amazon SageMaker JumpStart](https://www.amazonaws.cn/sagemaker/jumpstart/getting-started/?sagemaker-jumpstart-cards.sort-by=item.additionalFields.priority&sagemaker-jumpstart-cards.sort-order=asc&awsf.sagemaker-jumpstart-filter-product-type=product-type%23foundation-model&awsf.sagemaker-jumpstart-filter-text=ml-task-type%23text-generation&awsf.sagemaker-jumpstart-filter-vision=*all&awsf.sagemaker-jumpstart-filter-tabular=*all&awsf.sagemaker-jumpstart-filter-audio-tasks=*all&awsf.sagemaker-jumpstart-filter-multimodal=*all&awsf.sagemaker-jumpstart-filter-RL=*all&awsm.page-sagemaker-jumpstart-cards=1) product description page. You can also explore foundation models based on tasks directly in the Amazon SageMaker Studio UI or SageMaker Studio Classic UI. Only a subset of publicly available text generation models are available for fine-tuning in JumpStart. For more information, see [Use foundation models in Amazon SageMaker Studio Classic](jumpstart-foundation-models-use-studio.md).
### Publicly available image generation models
JumpStart provides a wide variety of Stable Diffusion image generation foundation models including base models from Stability AI as well as pre-trained models for specific text-to-image tasks from Hugging Face. If you need to fine-tune your text-to-image foundation model, you can use Stable Diffusion 2.1 base from Stability AI. If you want to explore models that are already trained on specific art styles, you can explore one of the many third-party models from Hugging Face directly in the Amazon SageMaker Studio UI or SageMaker Studio Classic UI.
To explore the latest image generation JumpStart foundation models, use the **Text to Image** filter on the [Getting started with Amazon SageMaker JumpStart](https://www.amazonaws.cn/sagemaker/jumpstart/getting-started/?sagemaker-jumpstart-cards.sort-by=item.additionalFields.priority&sagemaker-jumpstart-cards.sort-order=asc&awsf.sagemaker-jumpstart-filter-product-type=product-type%23foundation-model&awsf.sagemaker-jumpstart-filter-text=*all&awsf.sagemaker-jumpstart-filter-vision=*all&awsf.sagemaker-jumpstart-filter-tabular=*all&awsf.sagemaker-jumpstart-filter-audio-tasks=*all&awsf.sagemaker-jumpstart-filter-multimodal=ml-task-type%23txt2img&awsf.sagemaker-jumpstart-filter-RL=*all&awsm.page-sagemaker-jumpstart-cards=1) product description page. To get started with your chosen text-to-image foundation model, see [JumpStart foundation model usage](jumpstart-foundation-models-use.md).
## Proprietary foundation models
Amazon SageMaker JumpStart provides access to proprietary foundation models from third-party providers such as [AI21 Labs](https://www.ai21.com/), [Cohere](https://cohere.com/), and [LightOn](https://www.lighton.ai/).
To get started with one of these proprietary models, see [JumpStart foundation model usage](jumpstart-foundation-models-use.md). To use a proprietary foundation model, you must first subscribe to the model in Amazon Web Services Marketplace. After subscribing to the model, locate the foundation model in Studio or SageMaker Studio Classic. For more information, see [SageMaker JumpStart pretrained models](studio-jumpstart.md).
To explore the latest proprietary foundation models for a variety of use cases, see [Getting started with Amazon SageMaker JumpStart](https://www.amazonaws.cn/sagemaker/jumpstart/getting-started/?sagemaker-jumpstart-cards.sort-by=item.additionalFields.priority&sagemaker-jumpstart-cards.sort-order=asc&awsf.sagemaker-jumpstart-filter-product-type=product-type%23foundation-model&awsf.sagemaker-jumpstart-filter-text=*all&awsf.sagemaker-jumpstart-filter-vision=*all&awsf.sagemaker-jumpstart-filter-tabular=*all&awsf.sagemaker-jumpstart-filter-audio-tasks=*all&awsf.sagemaker-jumpstart-filter-multimodal=*all&awsf.sagemaker-jumpstart-filter-RL=*all&sagemaker-jumpstart-cards.q=proprietary&sagemaker-jumpstart-cards.q_operator=AND).
# JumpStart foundation model usage
Choose, train, or deploy foundation models through Amazon SageMaker Studio or Amazon SageMaker Studio Classic, use JumpStart foundation models programmatically with the SageMaker Python SDK, or discover JumpStart foundation models directly through the SageMaker AI console.
**Topics**
+ [
# Use foundation models in Studio
](jumpstart-foundation-models-use-studio-updated.md)
+ [
# Use foundation models in Amazon SageMaker Studio Classic
](jumpstart-foundation-models-use-studio.md)
+ [
# Use foundation models with the SageMaker Python SDK
](jumpstart-foundation-models-use-python-sdk.md)
+ [
# Discover foundation models in the SageMaker AI Console
](jumpstart-foundation-models-use-console.md)
# Use foundation models in Studio
Amazon SageMaker Studio allows you to fine-tune, deploy, and evaluate both publicly available and proprietary JumpStart foundation models directly through the Studio UI.
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the updated Studio experience. For information about using the Studio Classic application, see [Amazon SageMaker Studio Classic](studio.md).
To get started, navigate to the JumpStart landing page in Amazon SageMaker Studio. You can access it from the **Home** page or the left-side panel menu. On the **JumpStart** landing page, you can explore model hubs from providers of both publicly available and proprietary models, and search for models.
Within each model hub, you can sort models by **Most likes**, **Most downloads**, **Recently updated**, or filter them by task. Choose a model to view its detail card. On the model detail card, you can choose to **Fine-tune**, **Deploy**, or **Evaluate** the model, depending on the available option. Note that not all models are available for fine-tuning or evaluation.
For more information on getting started with Amazon SageMaker Studio, see [Amazon SageMaker Studio](studio-updated.md).
**Topics**
+ [
# Fine-tune a model in Studio
](jumpstart-foundation-models-use-studio-updated-fine-tune.md)
+ [
# Deploy a model in Studio
](jumpstart-foundation-models-use-studio-updated-deploy.md)
+ [
# Evaluate a model in Studio
](jumpstart-foundation-models-use-studio-updated-evaluate.md)
+ [
# Use your SageMaker JumpStart Models in Amazon Bedrock
](jumpstart-foundation-models-use-studio-updated-register-bedrock.md)
# Fine-tune a model in Studio
Fine-tuning trains a pre-trained model on a new dataset without training from scratch. This process, also known as transfer learning, can produce accurate models with smaller datasets and less training time. To fine-tune JumpStart foundation models, navigate to a model detail card in the Studio UI. For more information on how to open JumpStart in Studio, see [Open JumpStart in Studio](studio-jumpstart.md#jumpstart-open-studio). After navigating to the model detail card of your choice, choose **Train** in the upper right corner. Note that not all models have fine-tuning available.
**Important**
Some foundation models require explicit acceptance of an end-user license agreement (EULA) before fine-tuning. For more information, see [EULA acceptance in Amazon SageMaker Studio](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula-studio).
## Model settings
When using a pre-trained JumpStart foundation model in Amazon SageMaker Studio, the **Model artifact location (Amazon S3 URI)** is populated by default. To edit the default Amazon S3 URI, choose **Enter model artifact location**. Not all models support changing the model artifact location.
## Data settings
In the **Data** field, provide an Amazon S3 URI point to your training dataset location. The default Amazon S3 URI points to an example training dataset. To edit the default Amazon S3 URI, choose **Enter training dataset** and change the URI. Be sure to review the model detail card in Amazon SageMaker Studio for information on formatting training data.
## Hyperparameters
You can customize the hyperparameters of the training job that are used to fine-tune the model. The hyperparameters available for each fine-tunable model differ depending on the model.
The following hyperparameters are common among models:
+ **Epochs** – One epoch is one cycle through the entire dataset. Multiple intervals complete a batch, and multiple batches eventually complete an epoch. Multiple epochs are run until the accuracy of the model reaches an acceptable level, or when the error rate drops below an acceptable level.
+ **Learning rate** – The amount that values should be changed between epochs. As the model is refined, its internal weights are being nudged and error rates are checked to see if the model improves. A typical learning rate is 0.1 or 0.01, where 0.01 is a much smaller adjustment and could cause the training to take a long time to converge, whereas 0.1 is much larger and can cause the training to overshoot. It is one of the primary hyperparameters that you might adjust for training your model. Note that for text models, a much smaller learning rate (5e-5 for BERT) can result in a more accurate model.
+ **Batch size** – The number of records from the dataset that is to be selected for each interval to send to the GPUs for training.
Review the tool tip prompts and additional information in the model detail card in the Studio UI to learn more about hyperparameters specific to the model of your choice.
For more information on available hyperparameters, see [Commonly supported fine-tuning hyperparameters](jumpstart-foundation-models-fine-tuning.md#jumpstart-foundation-models-fine-tuning-hyperparameters).
## Deployment
Specify the training instance type and output artifact location for your training job. You can only choose from instances that are compatible with the model of your choice within the fine-tuning the Studio UI. The default output artifact location is the SageMaker AI default bucket. To change the output artifact location, choose **Enter output artifact location** and change the Amazon S3 URI.
## Security
Specify the security settings to use for your training job, including the IAM role that SageMaker AI uses to train your model, whether your training job should connect to a virtual private cloud (VPC), and any encryption keys to secure your data.
## Additional information
In the **Additional Information** field you can edit the training job name. You can also add and remove tags in the form of key-value pairs to help organize and categorize your fine-tuning training jobs.
After providing information for your fine-tuning configuration, choose **Submit**. If the pre-trained foundation model that you chose to fine-tune requires explicit agreement of an end-user license agreement (EULA) before training, the EULA is provided in a pop-up window. To accept the terms of the EULA, choose **Accept**. You are responsible for reviewing and complying with any applicable license terms and making sure they are acceptable for your use case before downloading or using a model.
# Deploy a model in Studio
To deploy JumpStart foundation models, navigate to a model detail card in the Studio UI. For more information on how to open JumpStart in Studio, see [Open JumpStart in Studio](studio-jumpstart.md#jumpstart-open-studio). After navigating to the model detail page of your choice, choose **Deploy** in the upper right corner of the Studio UI. Then, follow the steps in [Deploy models with SageMaker Studio](https://docs.amazonaws.cn/sagemaker/latest/dg/realtime-endpoints-deploy-models.html#deploy-models-studio).
Amazon SageMaker JumpStart also offers optimized deployments, which provide pre-defined deployment configurations designed for specific use cases such as content generation, summarization, or chat-style interactions. When deploying a supported model, you can select your target use case and choose a constraint optimization — Cost optimized, Throughput optimized, Latency optimized, or Balanced — and Amazon SageMaker JumpStart automatically configures the endpoint for that scenario. This gives you visibility into key performance metrics like P50 latency, time-to-first-token (TTFT), and throughput, while ensuring the deployment is tuned for your workload. To get started, open a supported model's detail page in Studio, choose **Deploy**, and use the **Performance** panel to configure your optimized deployment.
**Important**
Some foundation models require explicit acceptance of an end-user license agreement (EULA) before deployment. For more information, see [EULA acceptance in Amazon SageMaker Studio](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula-studio).
# Evaluate a model in Studio
Amazon SageMaker JumpStart has integrations with SageMaker Clarify foundation model evaluations (FME) in Studio. If a JumpStart model has built-in evaluation capabilities available, you can choose **Evaluate** in the upper right corner of the model detail page in the JumpStart Studio UI. For more information, see [Evaluate a foundation model](https://docs.amazonaws.cn/sagemaker/latest/dg/jumpstart-foundation-models-evaluate.html).
# Use your SageMaker JumpStart Models in Amazon Bedrock
You can register the models that you've deployed from Amazon SageMaker JumpStart to Amazon Bedrock. With Amazon Bedrock, you can host your model behind multiple endpoints. You can also use Amazon Bedrock features, such as Agents and Knowledge Bases. For more information about using Amazon Bedrock's models, see [https://docs.amazonaws.cn/bedrock/latest/userguide/amazon-bedrock-marketplace.html](https://docs.amazonaws.cn/bedrock/latest/userguide/amazon-bedrock-marketplace.html).
**Important**
To migrate your models to Amazon Bedrock, we recommend attaching [AmazonBedrockFullAccess](https://docs.amazonaws.cn/aws-managed-policy/latest/reference/AmazonBedrockFullAccess.html) policy to your IAM role. If you can't attach the managed policy, make sure your IAM role has the following permissions:
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "BedrockAll",
"Effect": "Allow",
"Action": [
"bedrock:*"
],
"Resource": "*"
},
{
"Sid": "DescribeKey",
"Effect": "Allow",
"Action": [
"kms:DescribeKey"
],
"Resource": "arn:*:kms:*:::*"
},
{
"Sid": "APIsWithAllResourceAccess",
"Effect": "Allow",
"Action": [
"iam:ListRoles",
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups"
],
"Resource": "*"
},
{
"Sid": "MarketplaceModelEndpointMutatingAPIs",
"Effect": "Allow",
"Action": [
"sagemaker:CreateEndpoint",
"sagemaker:CreateEndpointConfig",
"sagemaker:CreateModel",
"sagemaker:CreateInferenceComponent",
"sagemaker:DeleteInferenceComponent",
"sagemaker:DeleteEndpoint",
"sagemaker:UpdateEndpoint"
],
"Resource": [
"arn:aws-cn:sagemaker:*:*:endpoint/*",
"arn:aws-cn:sagemaker:*:*:endpoint-config/*",
"arn:aws-cn:sagemaker:*:*:model/*"
],
"Condition": {
"StringEquals": {
"aws:CalledViaLast": "bedrock.amazonaws.com"
}
}
},
{
"Sid": "BedrockEndpointTaggingOperations",
"Effect": "Allow",
"Action": [
"sagemaker:AddTags",
"sagemaker:DeleteTags"
],
"Resource": [
"arn:aws-cn:sagemaker:*:*:endpoint/*",
"arn:aws-cn:sagemaker:*:*:endpoint-config/*",
"arn:aws-cn:sagemaker:*:*:model/*"
]
},
{
"Sid": "MarketplaceModelEndpointNonMutatingAPIs",
"Effect": "Allow",
"Action": [
"sagemaker:DescribeEndpoint",
"sagemaker:DescribeEndpointConfig",
"sagemaker:DescribeModel",
"sagemaker:DescribeInferenceComponent",
"sagemaker:ListEndpoints",
"sagemaker:ListTags"
],
"Resource": [
"arn:aws-cn:sagemaker:*:*:endpoint/*",
"arn:aws-cn:sagemaker:*:*:endpoint-config/*",
"arn:aws-cn:sagemaker:*:*:model/*"
],
"Condition": {
"StringEquals": {
"aws:CalledViaLast": "bedrock.amazonaws.com"
}
}
},
{
"Sid": "BedrockEndpointInvokingOperations",
"Effect": "Allow",
"Action": [
"sagemaker:InvokeEndpoint",
"sagemaker:InvokeEndpointWithResponseStream"
],
"Resource": [
"arn:aws-cn:sagemaker:*:*:endpoint/*"
],
"Condition": {
"StringEquals": {
"aws:CalledViaLast": "bedrock.amazonaws.com"
}
}
},
{
"Sid": "DiscoveringMarketplaceModel",
"Effect": "Allow",
"Action": [
"sagemaker:DescribeHubContent"
],
"Resource": [
"arn:aws-cn:sagemaker:*:aws:hub-content/SageMakerPublicHub/Model/*",
"arn:aws-cn:sagemaker:*:aws:hub/SageMakerPublicHub"
]
},
{
"Sid": "AllowMarketplaceModelsListing",
"Effect": "Allow",
"Action": [
"sagemaker:ListHubContents"
],
"Resource": "arn:aws-cn:sagemaker:*:aws:hub/SageMakerPublicHub"
},
{
"Sid": "RetrieveSubscribedMarketplaceLicenses",
"Effect": "Allow",
"Action": [
"license-manager:ListReceivedLicenses"
],
"Resource": [
"*"
]
},
{
"Sid": "PassRoleToSageMaker",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": [
"arn:aws-cn:iam::*:role/*Sagemaker*ForBedrock*"
],
"Condition": {
"StringEquals": {
"iam:PassedToService": [
"sagemaker.amazonaws.com",
"bedrock.amazonaws.com"
]
}
}
},
{
"Sid": "PassRoleToBedrock",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "arn:aws-cn:iam::*:role/*AmazonBedrock*",
"Condition": {
"StringEquals": {
"iam:PassedToService": [
"bedrock.amazonaws.com"
]
}
}
}
]
}
```
The Amazon Bedrock Full Access policy only provides permissions to the Amazon Bedrock API. To use Amazon Bedrock in the Amazon Web Services Management Console, your IAM role must also have the following permissions:
```
{
"Sid": "AllowConsoleS3AccessForBedrockMarketplace",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:GetBucketCORS",
"s3:ListBucket",
"s3:ListBucketVersions",
"s3:GetBucketLocation"
],
"Resource": "*"
}
```
If you’re writing your own policy, you must include the policy statement that allows the Amazon Bedrock Marketplace action for the resource. For example, the following policy allows Amazon Bedrock to use the `InvokeModel` operation for a model that you’ve deployed to an endpoint.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "BedrockAll",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel"
],
"Resource": [
"arn:aws-cn:bedrock:us-east-1:111122223333:marketplace/model-endpoint/all-access"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": ["sagemaker:InvokeEndpoint"],
"Resource": "arn:aws-cn:sagemaker:us-east-1:111122223333:endpoint/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/project": "example-project-id",
"aws:CalledViaLast": "bedrock.amazonaws.com"
}
}
}
]
}
```
After you've deployed a model, you might be able to use it in Amazon Bedrock. To see if you can use it in Amazon Bedrock, navigate to the model detail card in the Studio UI. If the model card says that it's **Bedrock Ready**, you can register the model with Amazon Bedrock.
**Important**
By default Amazon SageMaker JumpStart disables network access for the models that you deploy. If you've enabled network access, you won't be able to use the model with Amazon Bedrock. If you want to use the model with Amazon Bedrock, you must redeploy it with network access disabled.
To use it with Amazon Bedrock, navigate to the **Endpoint details** page and choose **Use with Bedrock** in the upper right corner of the Studio UI. After you see the pop-up, choose **Register to Bedrock**.
# Use foundation models in Amazon SageMaker Studio Classic
You can fine-tune and deploy both publicly available and proprietary JumpStart foundation models through the Studio Classic UI.
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
To get started with Studio Classic, see [Launch Amazon SageMaker Studio Classic](studio-launch.md).
![\[JumpStart foundation models available in Amazon SageMaker Studio Classic.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-fm-studio.png)
After opening Amazon SageMaker Studio Classic, choose **Models, notebooks, solutions** in the SageMaker JumpStart section of the navigation pane. Then, scroll down to find either the **Foundation Models: Text Generation** or **Foundation Models: Image Generation** section depending on your use case.
You can choose **View model** on a suggested foundation model card, or choose **Explore All Models** to see all available foundation models for either text generation or image generation. If you choose to see all available models, you can further filter available models by task, data type, content type, or framework. You can also search for a model name directly in the **Search** bar. If you need guidance on selecting a model, see [Available foundation models](jumpstart-foundation-models-latest.md).
**Important**
Some foundation models require explicit acceptance of an end-user license agreement (EULA). For more information, see [EULA acceptance in Amazon SageMaker Studio](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula-studio).
After you choose **View model** for the foundation model of your choice in Studio Classic, you can deploy the model. For more information, see [Deploy a Model](jumpstart-deploy.md).
You can also choose **Open notebook** in the **Run in notebook** section to run an example notebook for the foundation model directly in Studio Classic.
**Note**
To deploy a proprietary foundation model in Studio Classic, you must first subscribe to the model in Amazon Web Services Marketplace. The Amazon Web Services Marketplace link is provided in the associated example notebook within Studio Classic.
If the model is fine-tunable, you can also fine-tune the model. For more information, see [Fine-Tune a Model](jumpstart-fine-tune.md). For a list of which JumpStart foundation models are fine-tunable, see [Foundation models and hyperparameters for fine-tuning](jumpstart-foundation-models-fine-tuning.md).
# Use foundation models with the SageMaker Python SDK
All JumpStart foundation models are available for programmatic deployment using the SageMaker Python SDK.
To deploy publicly available foundation models, you can use their model ID. You can find the model IDs for all publicly available foundation models in the [Built-in Algorithms with pre-trained Model Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html). Search for the name of a foundation model in the **Search** bar. Use the **Show entries** dropdown or the pagination controls to navigate the available models.
Proprietary models must be deployed using the model package information after subscribing to the model in Amazon Web Services Marketplace.
You can find the list of JumpStart available models in [Available foundation models](jumpstart-foundation-models-latest.md).
**Important**
Some foundation models require explicit acceptance of an end-user license agreement (EULA). For more information, see [EULA acceptance with the SageMaker Python SDK](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula-python-sdk).
The following sections show how to fine-tune publicly available foundation models using the `JumpStartEstimator` class, deploy publicly available foundation models using the `JumpStartModel` class, and deploy proprietary foundation models using the`ModelPackage` class.
**Topics**
+ [
# Fine-tune publicly available foundation models with the `JumpStartEstimator` class
](jumpstart-foundation-models-use-python-sdk-estimator-class.md)
+ [
# Deploy publicly available foundation models with the `JumpStartModel` class
](jumpstart-foundation-models-use-python-sdk-model-class.md)
+ [
# Deploy proprietary foundation models with the `ModelPackage` class
](jumpstart-foundation-models-use-python-sdk-proprietary.md)
# Fine-tune publicly available foundation models with the `JumpStartEstimator` class
**Note**
For instructions on fine-tuning foundation models in a private curated hub, see [Fine-tune curated hub models](jumpstart-curated-hubs-fine-tune.md).
You can fine-tune a built-in algorithm or pre-trained model in just a few lines of code using the SageMaker Python SDK.
1. First, find the model ID for the model of your choice in the [Built-in Algorithms with pre-trained Model Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html).
1. Using the model ID, define your training job as a JumpStart estimator.
```
from sagemaker.jumpstart.estimator import JumpStartEstimator
model_id = "huggingface-textgeneration1-gpt-j-6b"
estimator = JumpStartEstimator(model_id=model_id)
```
1. Run `estimator.fit()` on your model, pointing to the training data to use for fine-tuning.
```
estimator.fit(
{"train": training_dataset_s3_path, "validation": validation_dataset_s3_path}
)
```
1. Then, use the `deploy` method to automatically deploy your model for inference. In this example, we use the GPT-J 6B model from Hugging Face.
```
predictor = estimator.deploy()
```
1. You can then run inference with the deployed model using the `predict` method.
```
question = "What is Southern California often abbreviated as?"
response = predictor.predict(question)
print(response)
```
**Note**
This example uses the foundation model GPT-J 6B, which is suitable for a wide range of text generation use cases including question answering, named entity recognition, summarization, and more. For more information about model use cases, see [Available foundation models](jumpstart-foundation-models-latest.md).
You can optionally specify model versions or instance types when creating your `JumpStartEstimator`. For more information about the `JumpStartEstimator `class and its parameters, see [JumpStartEstimator](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.jumpstart.estimator.JumpStartEstimator).
## Check default instance types
You can optionally include specific model versions or instance types when fine-tuning a pre-trained model using the `JumpStartEstimator` class. All JumpStart models have a default instance type. Retrieve the default training instance type using the following code:
```
from sagemaker import instance_types
instance_type = instance_types.retrieve_default(
model_id=model_id,
model_version=model_version,
scope="training")
print(instance_type)
```
You can see all supported instance types for a given JumpStart model with the `instance_types.retrieve()` method.
## Check default hyperparameters
To check the default hyperparameters used for training, you can use the `retrieve_default()` method from the `hyperparameters` class.
```
from sagemaker import hyperparameters
my_hyperparameters = hyperparameters.retrieve_default(model_id=model_id, model_version=model_version)
print(my_hyperparameters)
# Optionally override default hyperparameters for fine-tuning
my_hyperparameters["epoch"] = "3"
my_hyperparameters["per_device_train_batch_size"] = "4"
# Optionally validate hyperparameters for the model
hyperparameters.validate(model_id=model_id, model_version=model_version, hyperparameters=my_hyperparameters)
```
For more information on available hyperparameters, see [Commonly supported fine-tuning hyperparameters](jumpstart-foundation-models-fine-tuning.md#jumpstart-foundation-models-fine-tuning-hyperparameters).
## Check default metric definitions
You can also check the default metric definitions:
```
print(metric_definitions.retrieve_default(model_id=model_id, model_version=model_version))
```
# Deploy publicly available foundation models with the `JumpStartModel` class
You can deploy a built-in algorithm or pre-trained model to a SageMaker AI endpoint in just a few lines of code using the SageMaker Python SDK.
1. First, find the model ID for the model of your choice in the [Built-in Algorithms with pre-trained Model Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html).
1. Using the model ID, define your model as a JumpStart model.
```
from sagemaker.jumpstart.model import JumpStartModel
model_id = "huggingface-text2text-flan-t5-xl"
my_model = JumpStartModel(model_id=model_id)
```
1. Use the `deploy` method to automatically deploy your model for inference. In this example, we use the FLAN-T5 XL model from Hugging Face.
```
predictor = my_model.deploy()
```
1. You can then run inference with the deployed model using the `predict` method.
```
question = "What is Southern California often abbreviated as?"
response = predictor.predict(question)
print(response)
```
**Note**
This example uses the foundation model FLAN-T5 XL, which is suitable for a wide range of text generation use cases including question answering, summarization, chatbot creation, and more. For more information about model use cases, see [Available foundation models](jumpstart-foundation-models-latest.md).
For more information about the `JumpStartModel `class and its parameters, see [JumpStartModel](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.jumpstart.model.JumpStartModel).
## Check default instance types
You can optionally include specific model versions or instance types when deploying a pre-trained model using the `JumpStartModel` class. All JumpStart models have a default instance type. Retrieve the default deployment instance type using the following code:
```
from sagemaker import instance_types
instance_type = instance_types.retrieve_default(
model_id=model_id,
model_version=model_version,
scope="inference")
print(instance_type)
```
See all supported instance types for a given JumpStart model with the `instance_types.retrieve()` method.
## Use inference components to deploy multiple models to a shared endpoint
An inference component is a SageMaker AI hosting object that you can use to deploy one or more models to an endpoint for increased flexibility and scalability. You must change the `endpoint_type` for your JumpStart model to be inference-component-based rather than the default model-based endpoint.
```
predictor = my_model.deploy(
endpoint_name = 'jumpstart-model-id-123456789012',
endpoint_type = EndpointType.INFERENCE_COMPONENT_BASED
)
```
For more information on creating endpoints with inference components and deploying SageMaker AI models, see [Shared resource utilization with multiple models](realtime-endpoints-deploy-models.md#deployed-shared-utilization).
## Check valid input and output inference formats
To check valid data input and output formats for inference, you can use the `retrieve_options()` method from the `Serializers` and `Deserializers` classes.
```
print(sagemaker.serializers.retrieve_options(model_id=model_id, model_version=model_version))
print(sagemaker.deserializers.retrieve_options(model_id=model_id, model_version=model_version))
```
## Check supported content and accept types
Similarly, you can use the `retrieve_options()` method to check the supported content and accept types for a model.
```
print(sagemaker.content_types.retrieve_options(model_id=model_id, model_version=model_version))
print(sagemaker.accept_types.retrieve_options(model_id=model_id, model_version=model_version))
```
For more information about utilities, see [Utility APIs](https://sagemaker.readthedocs.io/en/stable/api/utility/index.html).
# Deploy proprietary foundation models with the `ModelPackage` class
Proprietary models must be deployed using the model package information after subscribing to the model in Amazon Web Services Marketplace. For more information about SageMaker AI and Amazon Web Services Marketplace, see [Buy and Sell Amazon SageMaker AI Algorithms and Models in Amazon Web Services Marketplace](https://docs.amazonaws.cn/sagemaker/latest/dg/sagemaker-marketplace.html). To find Amazon Web Services Marketplace links for the latest proprietary models, see [Getting started with Amazon SageMaker JumpStart](https://www.amazonaws.cn/sagemaker/jumpstart/getting-started/?sagemaker-jumpstart-cards.sort-by=item.additionalFields.priority&sagemaker-jumpstart-cards.sort-order=asc&awsf.sagemaker-jumpstart-filter-product-type=product-type%23foundation-model&awsf.sagemaker-jumpstart-filter-text=*all&awsf.sagemaker-jumpstart-filter-vision=*all&awsf.sagemaker-jumpstart-filter-tabular=*all&awsf.sagemaker-jumpstart-filter-audio-tasks=*all&awsf.sagemaker-jumpstart-filter-multimodal=*all&awsf.sagemaker-jumpstart-filter-RL=*all&sagemaker-jumpstart-cards.q=proprietary&sagemaker-jumpstart-cards.q_operator=AND).
After subscribing to the model of your choice in Amazon Web Services Marketplace, you can deploy the foundation model using the SageMaker Python SDK and the SDK associated with the model provider. For example, AI21 Labs, Cohere, and LightOn use the `"ai21[SM]"`, `cohere-sagemaker`, and `lightonsage` packages, respectively.
For example, to define a JumpStart model using Jurassic-2 Jumbo Instruct from AI21 Labs, use the following code:
```
import sagemaker
import ai21
role = get_execution_role()
sagemaker_session = sagemaker.Session()
model_package_arn = "arn:aws:sagemaker:us-east-1:865070037744:model-package/j2-jumbo-instruct-v1-1-43-4e47c49e61743066b9d95efed6882f35"
my_model = ModelPackage(
role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session
)
```
For step-by-step examples, find and run the notebook associated with the proprietary foundation model of your choice in SageMaker Studio Classic. See [Use foundation models in Amazon SageMaker Studio Classic](jumpstart-foundation-models-use-studio.md) for more information. For more information on the SageMaker Python SDK, see [https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.ModelPackage](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.ModelPackage).
# Discover foundation models in the SageMaker AI Console
You can explore JumpStart foundation models directly through the Amazon SageMaker AI Console.
1. Open the Amazon SageMaker AI console at [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/).
1. Find **JumpStart** on the left navigation panel and choose **Foundation models**.
1. Browse models or search for a specific model. If you need guidance for model selection, see [Available foundation models](jumpstart-foundation-models-latest.md). Choose **View model** to view the model detail page for the foundation model of your choice.
1. If the model is a proprietary model, choose **Subscribe** in the upper right corner of the model detail page to subscribe to the model in Amazon Web Services Marketplace. You should receive an email confirming your subscription to the model of your choice. For more information about SageMaker AI and Amazon Web Services Marketplace, see [Buy and Sell Amazon SageMaker AI Algorithms and Models in Amazon Web Services Marketplace](https://docs.amazonaws.cn/sagemaker/latest/dg/sagemaker-marketplace.html). Publicly available foundation models do not require a subscription.
1. To view an example notebook in GitHub, choose **View code** in the upper right corner of the model detail page.
1. To view and run an example notebook directly in Amazon SageMaker Studio Classic, choose **Open notebook in Studio** in the upper right corner of the model detail page.
# Model sources and license agreements
Amazon SageMaker JumpStart provides access to hundreds of publicly available and proprietary foundation models from third-party sources and partners. You can explore the JumpStart foundation model selection directly in the SageMaker AI console, Studio, or Studio Classic.
## Licenses and model sources
Amazon SageMaker JumpStart provides access to both publicly available and proprietary foundation models. Foundation models are onboarded and maintained from third-party open source and proprietary providers. As such, they are released under different licenses as designated by the model source. Be sure to review the license for any foundation model that you use. You are responsible for reviewing and complying with any applicable license terms and making sure they are acceptable for your use case before downloading or using the content. Some examples of common foundation model licenses include:
+ Alexa Teacher Model
+ Apache 2.0
+ BigScience Responsible AI License v1.0
+ CreativeML Open RAIL\$1\$1-M license
Similarly, for any proprietary foundation models, be sure to review and comply with any terms of use and usage guidelines from the model provider. If you have questions about license information for a specific proprietary model, reach out to model provider directly. You can find model provider contact information in the **Support** tab of each model page in Amazon Web Services Marketplace.
## End-user license agreements
Some JumpStart foundation models require explicit acceptance of an end-user license agreement (EULA) before use.
### EULA acceptance in Amazon SageMaker Studio
You may be prompted to accept an end-user license agreement before fine-tuning, deploying, or evaluating a JumpStart foundation model in Studio. To get started with JumpStart foundation models in Studio, see [Use foundation models in Studio](jumpstart-foundation-models-use-studio-updated.md).
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the updated Studio experience. For information about using the Studio Classic application, see [Amazon SageMaker Studio Classic](studio.md).
Some JumpStart foundation models require acceptance of an end-user license agreement before deployment. If this applies to the foundation model that you choose to use, Studio prompts you with a window containing the EULA content. You are responsible for reviewing and complying with any applicable license terms and making sure they are acceptable for your use case before downloading or using a model.
#### EULA acceptance in Amazon SageMaker Studio Classic
You may be prompted to accept an end-user license agreement before deploying a JumpStart foundation model or opening a JumpStart foundation model notebook in Studio Classic. To get started with JumpStart foundation models in Studio Classic, see [Use foundation models in Amazon SageMaker Studio Classic](jumpstart-foundation-models-use-studio.md).
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
Some JumpStart foundation models require acceptance of an end-user license agreement before deployment. If this applies to the foundation model that you choose to use, Studio Classic prompts you with a window titled **Review the End User License Agreement (EULA) and Acceptable Use Policy (AUP) below** after you choose either **Deploy** or **Open notebook**. You are responsible for reviewing and complying with any applicable license terms and making sure they are acceptable for your use case before downloading or using a model.
### EULA acceptance with the SageMaker Python SDK
The following sections show you how to explicitly declare EULA acceptance when deploying or fine-tuning a JumpStart model with the SageMaker Python SDK. For more information on getting started with JumpStart foundation models using the SageMaker Python SDK, see [Use foundation models with the SageMaker Python SDK](jumpstart-foundation-models-use-python-sdk.md).
Before you begin, make sure that you do the following:
+ Upgrade to the latest version of the model that you use.
+ Install the latest version of the SageMaker Python SDK.
**Important**
To use the following workflow you must have [v2.198.0](https://github.com/aws/sagemaker-python-sdk/releases/tag/v2.198.0) or later of the SageMaker Python SDK installed.
#### EULA acceptance when deploying a JumpStart model
For models that require the acceptance of an end-user license agreement, you must explicitly declare EULA acceptance when deploying your JumpStart model.
```
from sagemaker.jumpstart.model import JumpStartModel
model_id = "meta-textgeneration-llama-2-13b"
my_model = JumpStartModel(model_id=model_id)
# Declare EULA acceptance when deploying your JumpStart model
predictor = my_model.deploy(accept_eula=True)
```
The `accept_eula` value is `None` by default and must be explicitly redefined as `True` in order to accept the end-user license agreement. For more information, see [JumpStartModel](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.jumpstart.model.JumpStartModel).
#### EULA acceptance when fine-tuning a JumpStart model
For fine-tuning models that require the acceptance of an end-user license agreement, you must explicitly declare EULA acceptance when running the `fit()` method for your JumpStart estimator. After fine-tuning a pre-trained model, the weights of the original model are changed. Therefore, when you deploy the fine-tuned model later, you do not need to accept a EULA.
**Note**
The following example sets `accept_eula=False`. You should manually change the value to `True` in order to accept the EULA.
```
from sagemaker.jumpstart.estimator import JumpStartEstimator
model_id = "meta-textgeneration-llama-2-13b"
# Declare EULA acceptance when defining your JumpStart estimator
estimator = JumpStartEstimator(model_id=model_id)
estimator.fit(accept_eula=False,
{"train": training_dataset_s3_path, "validation": validation_dataset_s3_path}
)
```
The `accept_eula` value is `None` by default and must be explicitly redefined as `"true"` within the `fit()` method in order to accept the end-user license agreement. For more information, see [JumpStartEstimator](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.jumpstart.estimator.JumpStartEstimator).
#### EULA acceptance SageMaker Python SDK versions earlier than 2.198.0
**Important**
When using versions earlier than [2.198.0](https://github.com/aws/sagemaker-python-sdk/releases/tag/v2.198.0) of the SageMaker Python SDK, you must use the SageMaker `Predictor` class to accept a model EULA.
After deploying a JumpStart foundation model programmatically using the SageMaker Python SDK, you can run inference against your deployed endpoint with the SageMaker `[Predictor](https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html)` class. For models that require the acceptance of an end-user license agreement, you must explicitly declare EULA acceptance in your call to the `Predictor` class:
```
predictor.predict(payload, custom_attributes="accept_eula=true")
```
The `accept_eula` value is `false` by default and must be explicitly redefined as `true` in order to accept the end-user license agreement. The predictor returns an error if you try to run inference while `accept_eula` is set to `false`. For more information on getting started with JumpStart foundation models using the SageMaker Python SDK, see [Use foundation models with the SageMaker Python SDK](jumpstart-foundation-models-use-python-sdk.md).
**Important**
The `custom_attributes` parameter accepts key-value pairs in the format `"key1=value1;key2=value2"`. If you use the same key multiple times, the inference server uses the last value associated with the key. For example, if you pass `"accept_eula=false;accept_eula=true"` to the `custom_attributes` parameter, then the inference server associates the value `true` with the `accept_eula` key.
# Foundation model customization
Foundation models are extremely powerful models able to solve a wide array of tasks. To solve most tasks effectively, these models require some form of customization.
The recommended way to first customize a foundation model to a specific use case is through prompt engineering. Providing your foundation model with well-engineered, context-rich prompts can help achieve desired results without any fine-tuning or changing of model weights. For more information, see [Prompt engineering for foundation models](jumpstart-foundation-models-customize-prompt-engineering.md).
If prompt engineering alone is not enough to customize your foundation model to a specific task, you can fine-tune a foundation model on additional domain-specific data. For more information, see [Foundation models and hyperparameters for fine-tuning](jumpstart-foundation-models-fine-tuning.md). The fine-tuning process involves changing model weights.
If you want to customize your model with information from a knowledge library without any retraining, see [Retrieval Augmented Generation](jumpstart-foundation-models-customize-rag.md).
# Prompt engineering for foundation models
Prompt engineering is the process of designing and refining the prompts or input stimuli for a language model to generate specific types of output. Prompt engineering involves selecting appropriate keywords, providing context, and shaping the input in a way that encourages the model to produce the desired response and is a vital technique to actively shape the behavior and output of foundation models.
Effective prompt engineering is crucial for directing model behavior and achieving desired responses. Through prompt engineering, you can control a model’s tone, style, and domain expertise without more involved customization measures like fine-tuning. We recommend dedicating time to prompt engineering before you consider fine-tuning a model on additional data. The goal is to provide sufficient context and guidance to the model so that it can generalize and perform well on unseen or limited data scenarios.
## Zero-shot learning
Zero-shot learning involves training a model to generalize and make predictions on unseen classes or tasks. To perform prompt engineering in zero-shot learning environments, we recommend constructing prompts that explicitly provide information about the target task and the desired output format. For example, if you want to use a foundation model for zero-shot text classification on a set of classes that the model did not see during training, a well-engineered prompt could be: `"Classify the following text as either sports, politics, or entertainment: [input text]."` By explicitly specifying the target classes and the expected output format, you can guide the model to make accurate predictions even on unseen classes.
## Few-shot learning
Few-shot learning involves training a model with a limited amount of data for new classes or tasks. Prompt engineering in few-shot learning environments focuses on designing prompts that effectively use the limited available training data. For example, if you use a foundation model for an image classification task and only have a few examples of a new image class, you can engineer a prompt that includes the available labeled examples with a placeholder for the target class. For example, the prompt could be: `"[image 1], [image 2], and [image 3] are examples of [target class]. Classify the following image as [target class]"`. By incorporating the limited labeled examples and explicitly specifying the target class, you can guide the model to generalize and make accurate predictions even with minimal training data.
## Supported inference parameters
Changing inference parameters might also affect the responses to your prompts. While you can try to add as much specificity and context as possible to your prompts, you can also experiment with supported inference parameters. The following are examples of some commonly supported inference parameters:
| Inference Parameter | Description |
| --- | --- |
| `max_new_tokens` | The maximum output length of a foundation model response. Valid values: integer, range: Positive integer. |
| `temperature` | Controls the randomness in the output. Higher temperature results in an output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If `temperature=0`, the response is made up of only the highest probability words (greedy decoding). Valid values: float, range: Positive float. |
| `top_p` | In each step of text generation, the model samples from the smallest possible set of words with a cumulative probability of `top_p`. Valid values: float, range: 0.0, 1.0. |
| `return_full_text` | If `True`, then the input text is part of the generated output text. Valid values: boolean, default: False. |
For more information on foundation model inference, see [Deploy publicly available foundation models with the `JumpStartModel` class](jumpstart-foundation-models-use-python-sdk-model-class.md).
If prompt engineering is not sufficient to adapt your foundation model to specific business needs, domain-specific language, target tasks, or other requirements, you can consider fine-tuning your model on additional data or using Retrieval Augmented Generation (RAG) to augment your model architecture with enhanced context from archived knowledge sources. For more information, see [Foundation models and hyperparameters for fine-tuning](jumpstart-foundation-models-fine-tuning.md) or [Retrieval Augmented Generation](jumpstart-foundation-models-customize-rag.md).
# Foundation models and hyperparameters for fine-tuning
Foundation models are computationally expensive and trained on a large, unlabeled corpus. Fine-tuning a pre-trained foundation model is an affordable way to take advantage of their broad capabilities while customizing a model on your own small, corpus. Fine-tuning is a customization method that involved further training and does change the weights of your model.
Fine-tuning might be useful to you if you need:
+ to customize your model to specific business needs
+ your model to successfully work with domain-specific language, such as industry jargon, technical terms, or other specialized vocabulary
+ enhanced performance for specific tasks
+ accurate, relative, and context-aware responses in applications
+ responses that are more factual, less toxic, and better-aligned to specific requirements
There are two main approaches that you can take for fine-tuning depending on your use case and chosen foundation model.
1. If you're interested in fine-tuning your model on domain-specific data, see [Fine-tune a large language model (LLM) using domain adaptation](jumpstart-foundation-models-fine-tuning-domain-adaptation.md).
1. If you're interested in instruction-based fine-tuning using prompt and response examples, see [Fine-tune a large language model (LLM) using prompt instructions](jumpstart-foundation-models-fine-tuning-instruction-based.md).
## Foundation models available for fine-tuning
You can fine-tune any of the following JumpStart foundation models:
+ Bloom 3B
+ Bloom 7B1
+ BloomZ 3B FP16
+ BloomZ 7B1 FP16
+ Code Llama 13B
+ Code Llama 13B Python
+ Code Llama 34B
+ Code Llama 34B Python
+ Code Llama 70B
+ Code Llama 70B Python
+ Code Llama 7B
+ Code Llama 7B Python
+ CyberAgentLM2-7B-Chat (CALM2-7B-Chat)
+ Falcon 40B BF16
+ Falcon 40B Instruct BF16
+ Falcon 7B BF16
+ Falcon 7B Instruct BF16
+ Flan-T5 Base
+ Flan-T5 Large
+ Flan-T5 Small
+ Flan-T5 XL
+ Flan-T5 XXL
+ Gemma 2B
+ Gemma 2B Instruct
+ Gemma 7B
+ Gemma 7B Instruct
+ GPT-2 XL
+ GPT-J 6B
+ GPT-Neo 1.3B
+ GPT-Neo 125M
+ GPT-NEO 2.7B
+ LightGPT Instruct 6B
+ Llama 2 13B
+ Llama 2 13B Chat
+ Llama 2 13B Neuron
+ Llama 2 70B
+ Llama 2 70B Chat
+ Llama 2 7B
+ Llama 2 7B Chat
+ Llama 2 7B Neuron
+ Mistral 7B
+ Mixtral 8x7B
+ Mixtral 8x7B Instruct
+ RedPajama INCITE Base 3B V1
+ RedPajama INCITE Base 7B V1
+ RedPajama INCITE Chat 3B V1
+ RedPajama INCITE Chat 7B V1
+ RedPajama INCITE Instruct 3B V1
+ RedPajama INCITE Instruct 7B V1
+ Stable Diffusion 2.1
## Commonly supported fine-tuning hyperparameters
Different foundation models support different hyperparameters when fine-tuning. The following are commonly-supported hyperparameters that can further customize your model during training:
| Inference Parameter | Description |
| --- | --- |
| `epoch` | The number of passes that the model takes through the fine-tuning dataset during training. Must be an integer greater than 1. |
| `learning_rate` | The rate at which the model weights are updated after working through each batch of fine-tuning training examples. Must be a positive float greater than 0. |
| `instruction_tuned` | Whether to instruction-train the model or not. Must be `'True'` or `'False'`. |
| `per_device_train_batch_size` | The batch size per GPU core or CPU for training. Must be a positive integer. |
| `per_device_eval_batch_size` | The batch size per GPU core or CPU for evaluation. Must be a positive integer. |
| `max_train_samples` | For debugging purposes or quicker training, truncate the number of training examples to this value. Value -1 means that the model uses all of the training samples. Must be a positive integer or -1. |
| `max_val_samples` | For debugging purposes or quicker training, truncate the number of validation examples to this value. Value -1 means that the model uses all of the validation samples. Must be a positive integer or -1. |
| `max_input_length` | Maximum total input sequence length after tokenization. Sequences longer than this will be truncated. If -1, `max_input_length` is set to the minimum of 1024 and the `model_max_length` defined by the tokenizer. If set to a positive value, `max_input_length` is set to the minimum of the provided value and the `model_max_length` defined by the tokenizer. Must be a positive integer or -1. |
| `validation_split_ratio` | If there is no validation channel, ratio of train-validation split from the training data. Must be between 0 and 1. |
| `train_data_split_seed` | If validation data is not present, this fixes the random splitting of the input training data to training and validation data used by the model. Must be an integer. |
| `preprocessing_num_workers` | The number of processes to use for the pre-processing. If `None`, main process is used for pre-processing. |
| `lora_r` | Low-rank adaptation (LoRA) r value, which acts as the scaling factor for weight updates. Must be a positive integer. |
| `lora_alpha` | Low-rank adaptation (LoRA) alpha value, which acts as the scaling factor for weight updates. Generally 2 to 4 times the size of `lora_r`. Must be a positive integer. |
| `lora_dropout` | Dropout value for low-rank adaptation (LoRA) layers Must be a positive float between 0 and 1. |
| `int8_quantization` | If `True`, model is loaded with 8 bit precision for training. |
| `enable_fsdp` | If `True`, training uses Fully Sharded Data Parallelism. |
You can specify hyperparameter values when you fine-tune your model in Studio. For more information, see [Fine-tune a model in Studio](jumpstart-foundation-models-use-studio-updated-fine-tune.md).
You can also override default hyperparameter values when fine-tuning your model using the SageMaker Python SDK. For more information, see [Fine-tune publicly available foundation models with the `JumpStartEstimator` class](jumpstart-foundation-models-use-python-sdk-estimator-class.md).
# Fine-tune a large language model (LLM) using domain adaptation
Domain adaptation fine-tuning allows you to leverage pre-trained foundation models and adapt them to specific tasks using limited domain-specific data. If prompt engineering efforts do not provide enough customization, you can use domain adaption fine-tuning to get your model working with domain-specific language, such as industry jargon, technical terms, or other specialized data. This fine-tuning process modifies the weights of the model.
To fine-tune your model on a domain-specific dataset:
1. Prepare your training data. For instructions, see [Prepare and upload training data for domain adaptation fine-tuning](#jumpstart-foundation-models-fine-tuning-domain-adaptation-prepare-data).
1. Create your fine-tuning training job. For instructions, see [Create a training job for instruction-based fine-tuning](#jumpstart-foundation-models-fine-tuning-domain-adaptation-train).
You can find end-to-end examples in [Example notebooks](#jumpstart-foundation-models-fine-tuning-domain-adaptation-examples).
Domain adaptation fine-tuning is available with the following foundation models:
**Note**
Some JumpStart foundation models, such as Llama 2 7B, require acceptance of an end-user license agreement before fine-tuning and performing inference. For more information, see [End-user license agreements](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula).
+ Bloom 3B
+ Bloom 7B1
+ BloomZ 3B FP16
+ BloomZ 7B1 FP16
+ GPT-2 XL
+ GPT-J 6B
+ GPT-Neo 1.3B
+ GPT-Neo 125M
+ GPT-NEO 2.7B
+ Llama 2 13B
+ Llama 2 13B Chat
+ Llama 2 13B Neuron
+ Llama 2 70B
+ Llama 2 70B Chat
+ Llama 2 7B
+ Llama 2 7B Chat
+ Llama 2 7B Neuron
## Prepare and upload training data for domain adaptation fine-tuning
Training data for domain adaptation fine-tuning can be provided in CSV, JSON, or TXT file format. All training data must be in a single file within a single folder.
The training data is taken from the **Text** column for CSV or JSON training data files. If no column is labeled **Text**, then the training data is taken from the first column for CSV or JSON training data files.
The following is an example body of a TXT file to be used for fine-tuning:
```
This report includes estimates, projections, statements relating to our
business plans, objectives, and expected operating results that are “forward-
looking statements” within the meaning of the Private Securities Litigation
Reform Act of 1995, Section 27A of the Securities Act of 1933, and Section 21E
of ....
```
### Split data for training and testing
You can optionally provide another folder containing validation data. This folder should also include one CSV, JSON, or TXT file. If no validation dataset is provided, then a set amount of the training data is set aside for validation purposes. You can adjust the percentage of training data used for validation when you choose the hyperparameters for fine-tuning your model.
### Upload fine-tuning data to Amazon S3
Upload your prepared data to Amazon Simple Storage Service (Amazon S3) to use when fine-tuning a JumpStart foundation model. You can use the following commands to upload your data:
```
from sagemaker.s3 import S3Uploader
import sagemaker
import random
output_bucket = sagemaker.Session().default_bucket()
local_data_file = "train.txt"
train_data_location = f"s3://{output_bucket}/training_folder"
S3Uploader.upload(local_data_file, train_data_location)
S3Uploader.upload("template.json", train_data_location)
print(f"Training data: {train_data_location}")
```
## Create a training job for instruction-based fine-tuning
After your data is uploaded to Amazon S3, you can fine-tune and deploy your JumpStart foundation model. To fine-tune your model in Studio, see [Fine-tune a model in Studio](jumpstart-foundation-models-use-studio-updated-fine-tune.md). To fine-tune your model using the SageMaker Python SDK, see [Fine-tune publicly available foundation models with the `JumpStartEstimator` class](jumpstart-foundation-models-use-python-sdk-estimator-class.md).
## Example notebooks
For more information on domain adaptation fine-tuning, see the following example notebooks:
+ [SageMaker JumpStart Foundation Models - Fine-tuning text generation GPT-J 6B model on domain specific dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/domain-adaption-finetuning-gpt-j-6b.html)
+ [Fine-tune LLaMA 2 models on JumpStart](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/llama-2-finetuning.html)
# Fine-tune a large language model (LLM) using prompt instructions
Instruction-based fine-tuning uses labeled examples to improve the performance of a pre-trained foundation model on a specific task. The labeled examples are formatted as prompt, response pairs and phrased as instructions. This fine-tuning process modifies the weights of the model. For more information on instruction-based fine-tuning, see the papers [Introducing FLAN: More generalizable Language Models with Instruction Fine-Tuning](https://ai.googleblog.com/2021/10/introducing-flan-more-generalizable.html) and [Scaling Instruction-Finetuned Language Models](https://arxiv.org/abs/2210.11416).
Fine-tuned LAnguage Net (FLAN) models use instruction tuning to make models more amenable to solving general downstream NLP tasks. Amazon SageMaker JumpStart provides a number of foundation models in the FLAN model family. For example, FLAN-T5 models are instruction fine-tuned on a wide range of tasks to increase zero-shot performance for a variety of common use cases. With additional data and fine-tuning, instruction-based models can be further adapted to more specific tasks that weren’t considered during pre-training.
To fine-tune a LLM on a specific task using prompt-response pairs task instructions:
1. Prepare your instructions in JSON files. For more information about the required format for the prompt-response pair files and the structure of the data folder, see [Prepare and upload training data for instruction-based fine-tuning](#jumpstart-foundation-models-fine-tuning-instruction-based-prepare-data).
1. Create your fine-tuning training job. For instructions, see [Create a training job for instruction-based fine-tuning](#jumpstart-foundation-models-fine-tuning-instruction-based-train).
You can find end-to-end examples in [Example notebooks](#jumpstart-foundation-models-fine-tuning-instruction-based-examples).
Only a subset of JumpStart foundation models are compatible with instruction-based fine-tuning. Instruction-based fine-tuning is available with the following foundation models:
**Note**
Some JumpStart foundation models, such as Llama 2 7B, require acceptance of an end-user license agreement before fine-tuning and performing inference. For more information, see [End-user license agreements](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula).
+ Flan-T5 Base
+ Flan-T5 Large
+ Flan-T5 Small
+ Flan-T5 XL
+ Flan-T5 XXL
+ Llama 2 13B
+ Llama 2 13B Chat
+ Llama 2 13B Neuron
+ Llama 2 70B
+ Llama 2 70B Chat
+ Llama 2 7B
+ Llama 2 7B Chat
+ Llama 2 7B Neuron
+ Mistral 7B
+ RedPajama INCITE Base 3B V1
+ RedPajama INCITE Base 7B V1
+ RedPajama INCITE Chat 3B V1
+ RedPajama INCITE Chat 7B V1
+ RedPajama INCITE Instruct 3B V1
+ RedPajama INCITE Instruct 7B V1
## Prepare and upload training data for instruction-based fine-tuning
Training data for instruction-based fine-tuning must be provided in JSON Lines text file format, where each line is a dictionary. All training data must be in a single folder. The folder can include multiple .jsonl files.
The training folder can also include a template JSON file (`template.json`) that describes the input and output formats of your data. If no template file is provided, the following template file is used:
```
{
"prompt": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{context}",
"completion": "{response}"
}
```
According to the `template.json` file, each .jsonl entry of the training data must include `{instruction}`, `{context}`, and `{response}` fields.
If you provide a custom template JSON file, use the `"prompt"` and `"completion"` keys to define your own required fields. According to the following custom template JSON file, each .jsonl entry of the training data must include `{question}`, `{context}`, and `{answer}` fields:
```
{
"prompt": "question: {question} context: {context}",
"completion": "{answer}"
}
```
### Split data for training and testing
You can optionally provide another folder containing validation data. This folder should also include one or more .jsonl files. If no validation dataset is provided, then a set amount of the training data is set aside for validation purposes. You can adjust the percentage of training data used for validation when you choose the hyperparameters for fine-tuning your model.
### Upload fine-tuning data to Amazon S3
Upload your prepared data to Amazon Simple Storage Service (Amazon S3) to use when fine-tuning a JumpStart foundation model. You can use the following commands to upload your data:
```
from sagemaker.s3 import S3Uploader
import sagemaker
import random
output_bucket = sagemaker.Session().default_bucket()
local_data_file = "train.jsonl"
train_data_location = f"s3://{output_bucket}/dolly_dataset"
S3Uploader.upload(local_data_file, train_data_location)
S3Uploader.upload("template.json", train_data_location)
print(f"Training data: {train_data_location}")
```
## Create a training job for instruction-based fine-tuning
After your data is uploaded to Amazon S3, you can fine-tune and deploy your JumpStart foundation model. To fine-tune your model in Studio, see [Fine-tune a model in Studio](jumpstart-foundation-models-use-studio-updated-fine-tune.md). To fine-tune your model using the SageMaker Python SDK, see [Fine-tune publicly available foundation models with the `JumpStartEstimator` class](jumpstart-foundation-models-use-python-sdk-estimator-class.md).
## Example notebooks
For more information on instruction-based fine-tuning, see the following example notebooks:
+ [Fine-tune LLaMA 2 models on JumpStart](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/llama-2-finetuning.html)
+ [Introduction to SageMaker JumpStart - Text Generation with Mistral models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/mistral-7b-instruction-domain-adaptation-finetuning.html)
+ [Introduction to SageMaker JumpStart - Text Generation with Falcon models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/falcon-7b-instruction-domain-adaptation-finetuning.html)
+ [SageMaker JumpStart Foundation Models - HuggingFace Text2Text Instruction Fine-Tuning](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/instruction-fine-tuning-flan-t5.html)
# Retrieval Augmented Generation
Foundation models are usually trained offline, making the model agnostic to any data that is created after the model was trained. Additionally, foundation models are trained on very general domain corpora, making them less effective for domain-specific tasks. You can use Retrieval Augmented Generation (RAG) to retrieve data from outside a foundation model and augment your prompts by adding the relevant retrieved data in context. For more information about RAG model architectures, see [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401).
With RAG, the external data used to augment your prompts can come from multiple data sources, such as a document repositories, databases, or APIs. The first step is to convert your documents and any user queries into a compatible format to perform relevancy search. To make the formats compatible, a document collection, or knowledge library, and user-submitted queries are converted to numerical representations using embedding language models. *Embedding* is the process by which text is given numerical representation in a vector space. RAG model architectures compare the embeddings of user queries within the vector of the knowledge library. The original user prompt is then appended with relevant context from similar documents within the knowledge library. This augmented prompt is then sent to the foundation model. You can update knowledge libraries and their relevant embeddings asynchronously.
![\[A model architecture diagram of Retrieval Augmented Generation (RAG).\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-fm-rag.jpg)
The retrieved document should be large enough to contain useful context to help augment the prompt, but small enough to fit into the maximum sequence length of the prompt. You can use task-specific JumpStart models, such as the General Text Embeddings (GTE) model from Hugging Face, to provide the embeddings for your prompts and knowledge library documents. After comparing the prompt and document embeddings to find the most relevant documents, construct a new prompt with the supplemental context. Then, pass the augmented prompt to a text generation model of your choosing.
## Example notebooks
For more information on RAG foundation model solutions, see the following example notebooks:
+ [Retrieval-Augmented Generation: Question Answering using LangChain and Cohere’s Generate and Embedding Models from SageMaker JumpStart](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_Cohere+langchain_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering using LLama-2, Pinecone and Custom Dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_pinecone_llama-2_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering based on Custom Dataset with Open-sourced LangChain Library](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_langchain_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering based on Custom Dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_jumpstart_knn.html)
+ [Retrieval-Augmented Generation: Question Answering using Llama-2 and Text Embedding Models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_text_embedding_llama-2_jumpstart.html)
+ [Amazon SageMaker JumpStart - Text Embedding and Sentence Similarity](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/text-embedding-sentence-similarity.html)
You can clone the [Amazon SageMaker AI examples repository](https://github.com/aws/amazon-sagemaker-examples/tree/main/introduction_to_amazon_algorithms/jumpstart-foundation-models) to run the available JumpStart foundation model examples in the Jupyter environment of your choice within Studio. For more information on applications that you can use to create and access Jupyter in SageMaker AI, see [Applications supported in Amazon SageMaker Studio](studio-updated-apps.md).
# Evaluate a text generation foundation model in Studio
**Note**
Foundation Model Evaluations (FMEval) is in preview release for Amazon SageMaker Clarify and is subject to change.
**Important**
In order to use SageMaker Clarify Foundation Model Evaluations, you must upgrade to the new Studio experience. As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The foundation evaluation feature can only be used in the updated experience. For information about how to update Studio, see [Migration from Amazon SageMaker Studio Classic](studio-updated-migrate.md). For information about using the Studio Classic application, see [Amazon SageMaker Studio Classic](studio.md).
Amazon SageMaker JumpStart has integrations with SageMaker Clarify Foundation Model Evaluations (FMEval) in Studio. If a JumpStart model has built-in evaluation capabilities available, you can choose **Evaluate** in the upper right corner of the model detail page in the JumpStart Studio UI. For more information on navigating the JumpStart Studio UI, see [Open JumpStart in Studio](studio-jumpstart.md#jumpstart-open-studio),
Use Amazon SageMaker JumpStart to evaluate text-based foundation models with FMEval. You can use these model evaluations to compare model quality and responsibility metrics for one model, between two models, or between different versions of the same model, to help you quantify model risks. FMEval can evaluate text-based models that perform the following tasks:
+ **Open-ended generation** – The production of natural human responses to text that does not have a pre-defined structure.
+ **Text summarization** – The generation of a concise and condensed summary while retaining the meaning and key information contained in larger text.
+ **Question Answering** – The generation of an answer in natural language to a question.
+ **Classification ** – The assignment of a class, such as `positive` versus `negative` to a text passage based on its content.
You can use FMEval to automatically evaluate model responses based on specific benchmarks. You can also evaluate model responses against your own criteria by bringing your own prompt datasets. FMEval provides a user interface (UI) that guides you through the setup and configuration of an evaluation job. You can also use the FMEval library inside your own code.
Every evaluation requires quota for two instances:
+ Hosting instance – An instance that hosts and deploys an LLM.
+ Evaluation instance – An instance that is used to prompt and perform an evaluation of an LLM on the hosting instance.
If your LLM is already deployed, provide the endpoint, and SageMaker AI will use your **hosting instance** to host and deploy the LLM.
If you are evaluating a JumpStart model that is not yet deployed to your account, FMEval creates a temporary **hosting instance** for you in your account, and keeps it deployed only for the length of your evaluation. FMEval uses the default instance that JumpStart recommends for the chosen LLM as your hosting instance. You must have sufficient quota for this recommended instance.
Every evaluation also uses an evaluation instance to provide prompts to and score the responses from the LLM. You must also have sufficient quota and memory to run the evaluation algorithms. The quota and memory requirements of the evaluation instance are generally smaller than those required for a hosting instance. We recommend selecting the `ml.m5.2xlarge` instance. For more information about quota and memory, see [Resolve errors when creating a model evaluation job in Amazon SageMaker AI](clarify-foundation-model-evaluate-troubleshooting.md).
Automatic evaluations can be used to score LLMs across the following dimensions:
+ Accuracy – For text summarization, question answering, and text classification
+ Semantic robustness – For open-ended generation, text summarization and text classification tasks
+ Factual knowledge – For open-ended generation
+ Prompt stereotyping – For open-ended generation
+ Toxicity – For open-ended generation, text summarization, and question answering
You can also use human evaluations to manually evaluate model responses. The FMEval UI guides you through a workflow of selecting one or more models, provisioning resources, and writing instructions for and contacting your human workforce. After the human evaluation is complete, the results are displayed in FMEval.
You can access model evaluation through the JumpStart landing page in Studio by selecting a model to evaluate and then choosing **Evaluate**. Note that not all JumpStart models have evaluation capabilities available. For more information about how to configure, provision and run FMEval, see [What are Foundation Model Evaluations?](https://docs.amazonaws.cn/sagemaker/latest/dg/clarify-foundation-model-evaluate.html)
# Example notebooks
For step-by-step examples on how to use publicly available JumpStart foundation models with the SageMaker Python SDK, refer to the following notebooks on text generation, image generation, and model customization.
**Note**
Proprietary and publicly available JumpStart foundation models have different SageMaker AI Python SDK deployment workflows. Discover proprietary foundation model example notebooks through Amazon SageMaker Studio Classic or the SageMaker AI console. For more information, see [JumpStart foundation model usage](jumpstart-foundation-models-use.md).
You can clone the [Amazon SageMaker AI examples repository](https://github.com/aws/amazon-sagemaker-examples/tree/main/introduction_to_amazon_algorithms/jumpstart-foundation-models) to run the available JumpStart foundation model examples in the Jupyter environment of your choice within Studio. For more information on applications that you can use to create and access Jupyter in SageMaker AI, see [Applications supported in Amazon SageMaker Studio](studio-updated-apps.md).
## Time series forecasting
You can use the Chronos models to forecast time series data. They're based on the language model architecture. Use the [Introduction to SageMaker JumpStart - Time Series Forecasting with Chronos](https://github.com/aws/amazon-sagemaker-examples/blob/default/%20%20%20%20generative_ai/sm-jumpstart_time_series_forecasting.ipynb) notebook to get started.
For information about the available Chronos models, see [Available foundation models](jumpstart-foundation-models-latest.md).
## Text generation
Explore text generation example notebooks, including guidance on general text generation workflows, multilingual text classification, real-time batch inference, few-shot learning, chatbot interactions, and more.
+ [SageMaker JumpStart Foundation Models - HuggingFace Text2Text Generation with FLAN-T5 XL as an example](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/text2text-generation-flan-t5.html)
+ [SageMaker JumpStart Foundation Models - BloomZ: Multilingual Text Classification, Question and Answering, Code Generation, Paragraph rephrase, and More](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/text2text-generation-bloomz.html)
+ [SageMaker JumpStart Foundation Models - HuggingFace Text2Text Generation Batch Transform and Real-Time Batch Inference](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/text2text-generation-Batch-Transform.html)
+ [SageMaker JumpStart Foundation Models - GPT-J, GPT-Neo Few-shot learning](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/text-generation-few-shot-learning.html)
+ [SageMaker JumpStart Foundation Models - Chatbots](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/text-generation-chatbot.html)
+ [Introduction to SageMaker JumpStart - Text Generation with Mistral models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/mistral-7b-instruction-domain-adaptation-finetuning.html)
+ [Introduction to SageMaker JumpStart - Text Generation with Falcon models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/falcon-7b-instruction-domain-adaptation-finetuning.html)
## Image generation
Get started with text-to-image Stable Diffusion models, learn how to deploy an inpainting model, and experiment with a simple workflow to generate images of your dog.
+ [Introduction to JumpStart - Text to Image](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart_text_to_image/Amazon_JumpStart_Text_To_Image.html)
+ [Introduction to JumpStart Image editing - Stable Diffusion Inpainting](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart_inpainting/Amazon_JumpStart_Inpainting.html)
+ [Generate fun images of your dog](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart_text_to_image/custom_dog_image_generator.html)
## Model customization
Sometimes your use case requires greater foundation model customization for specific tasks. For more information on model customization approaches, see [Foundation model customization](jumpstart-foundation-models-customize.md) or explore one of the following example notebooks.
+ [SageMaker JumpStart Foundation Models - Fine-tuning text generation GPT-J 6B model on domain specific dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/domain-adaption-finetuning-gpt-j-6b.html)
+ [SageMaker JumpStart Foundation Models - HuggingFace Text2Text Instruction Fine-Tuning](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/instruction-fine-tuning-flan-t5.html)
+ [Retrieval-Augmented Generation: Question Answering using LangChain and Cohere’s Generate and Embedding Models from SageMaker JumpStart](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_Cohere+langchain_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering using LLama-2, Pinecone and Custom Dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_pinecone_llama-2_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering based on Custom Dataset with Open-sourced LangChain Library](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_langchain_jumpstart.html)
+ [Retrieval-Augmented Generation: Question Answering based on Custom Dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_jumpstart_knn.html)
+ [Retrieval-Augmented Generation: Question Answering using Llama-2 and Text Embedding Models](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/question_answering_text_embedding_llama-2_jumpstart.html)
+ [Amazon SageMaker JumpStart - Text Embedding and Sentence Similarity](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/jumpstart-foundation-models/question_answering_retrieval_augmented_generation/text-embedding-sentence-similarity.html)
# Private curated hubs for foundation model access control in JumpStart
Curate pretrained JumpStart foundation models for your organization with private hubs. Use the latest publicly available and proprietary foundation models while enforcing governance guardrails and ensuring that your organization can only access approved models.
Use private model hubs to share models and notebooks, centralize model artifacts, improve model discoverability, and streamline model use within your organization. Administrators can create private hubs that include subsets of models tailored to different teams, use cases, or security requirements. Administrators can create a JumpStart private model hub using the SageMaker Python SDK. Users can then browse, train, and deploy the curated set of models using Amazon SageMaker Studio or the SageMaker Python SDK.
For more information on creating a private model hub, see [Admin guide for private model hubs in Amazon SageMaker JumpStart](jumpstart-curated-hubs-admin-guide.md).
For more information on sharing private model hubs across accounts, see [Cross-account sharing for private model hubs with Amazon Resource Access Manager](jumpstart-curated-hubs-ram.md).
For more information on accessing a private model hub, see [User guide](jumpstart-curated-hubs-user-guide.md).
# Admin guide for private model hubs in Amazon SageMaker JumpStart
There are actions that administrators can take related to curated model hubs that users within your organization can access. This includes creating, adding, deleting, and managing access of private hubs. This page also includes information about the supported Amazon Regions for curated private hubs, as well as the prerequisites needed to use curated private model hubs.
## Supported Amazon Regions
Curated private hubs are currently generally available in the following Amazon commercial Regions:
+ us-east-1
+ us-east-2
+ us-west-2
+ eu-west-1
+ eu-central-1
+ ap-northeast-1
+ ap-northeast-2
+ ap-south-1
+ ap-southeast-1
+ ap-southeast-2
+ il-central-1 (SDK only)
The default maximum number of hubs allowed in a single Region is 50.
## Prerequisites
To use a curated private hub in Studio, you must have the following prerequisites:
+ An Amazon account with administrator access
+ An Amazon Identity and Access Management (IAM) role with access to Amazon SageMaker Studio
+ An Amazon SageMaker AI domain with JumpStart enabled
+ If your users try to use proprietary models, they must have subscriptions to those models in Amazon Marketplace.
+ Amazon accounts that are deploying proprietary models must have subscriptions to those models in Amazon Marketplace.
For more information on getting started with Studio, see [Amazon SageMaker Studio](studio-updated.md).
# Create a private model hub
Use the following steps to create a private hub to manage access control for pretrained JumpStart foundation models for your organization. You must intstall the SageMaker Python SDK and configure the necessary IAM permissions before creating a model hub.
**Create a private hub**
1. Install the SageMaker Python SDK and import the necessary Python packages.
```
# Install the SageMaker Python SDK
!pip3 install sagemaker --force-reinstall --quiet
# Import the necessary Python packages
import boto3
from sagemaker import Session
from sagemaker.jumpstart.hub.hub import Hub
```
1. Initialize a SageMaker AI Session.
```
sm_client = boto3.client('sagemaker')
session = Session(sagemaker_client=sm_client)
session.get_caller_identity_arn()
```
1. Configure the details of your private hub such as the internal hub name, UI display name, and UI hub description.
**Note**
If you do not specify an Amazon S3 bucket name when creating your hub, the SageMaker hub service creates a new bucket on your behalf. The new bucket has the following naming structure: `sagemaker-hubs-REGION-ACCOUNT_ID`.
```
HUB_NAME="Example-Hub"
HUB_DISPLAY_NAME="Example Hub UI Name"
HUB_DESCRIPTION="A description of the example private curated hub."
REGION="us-west-2"
```
1. Check that your **Admin** IAM role has the necessary Amazon S3 permissions to create a private hub. If your role does not have the necessary permissions, navigate to the **Roles** page in the IAM console. Choose the **Admin** role and then choose **Add permissions** in the **Permissions policies** pane to create an inline policy with the following permissions using the JSON editor:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectTagging"
],
"Resource": [
"arn:aws-cn:s3:::jumpstart-cache-prod-REGION",
"arn:aws-cn:s3:::jumpstart-cache-prod-REGION/*"
],
"Effect": "Allow"
}
]
}
```
------
1. Create a private model hub using your configurations from **Step 3** using `hub.create()`.
```
hub = Hub(hub_name=HUB_NAME, sagemaker_session=session)
try:
# Create the private hub
hub.create(
description=HUB_DESCRIPTION,
display_name=HUB_DISPLAY_NAME
)
print(f"Successfully created Hub with name {HUB_NAME} in {REGION}")
# Check that no other hubs with this internal name exist
except Exception as e:
if "ResourceInUse" in str(e):
print(f"A hub with the name {HUB_NAME} already exists in your account.")
else:
raise e
```
1. Verify the configuration of your new private hub with the following `describe` command:
```
hub.describe()
```
# Add models to a private hub
After creating a private hub, you can then add allow-listed models. For the full list of available JumpStart models, see the [Built-in Algorithms with pre-trained Model Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html) in the SageMaker Python SDK reference.
1. You can filter through the available models programmatically using the `hub.list_sagemaker_public_hub_models()` method. You can optionally filter by categories such as framework (`"framework == pytorch"`), tasks such as image classification (`"task == ic"`), and more. For more information about filters, see [https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/jumpstart/notebook_utils.py](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/jumpstart/notebook_utils.py). The filter parameter in the `hub.list_sagemaker_public_hub_models()` method is optional.
```
filter_value = "framework == meta"
response = hub.list_sagemaker_public_hub_models(filter=filter_value)
models = response["hub_content_summaries"]
while response["next_token"]:
response = hub.list_sagemaker_public_hub_models(filter=filter_value, next_token=response["next_token"])
models.extend(response["hub_content_summaries"])
print(models)
```
1. You can then add the filtered models by specifying the model ARN in the `hub.create_model_reference()` method.
```
for model in models:
print(f"Adding {model.get('hub_content_name')} to Hub")
hub.create_model_reference(model_arn=model.get("hub_content_arn"), model_name=model.get("hub_content_name"))
```
# Update resources in a private hub
You can update resources in your private hub to make changes to their metadata. The resources that you can update include model references to Amazon SageMaker JumpStart models, custom models, notebooks, datasets, and JsonDoc.
When updating model, notebook, datasets, or JsonDoc resources, you can update the content description, display name, keywords, and support status. When updating model references to JumpStart models, you can only update the field specifying the minimum model version that you'd like to use.
+ “Update model or notebook resources” to include DataSet/JsonDoc. In CLI command, DataSets/JsonDocs should added to the hub-content-type argument.
Follow the section specific to the resource that you want to update.
## Update model or notebook resources
To update a model or a notebook resource, use the [UpdateHubContent](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_UpdateHubContent.html) API.
The valid metadata fields that you can update with this API are the following:
+ `HubContentDescription` – The description of the resource.
+ `HubContentDisplayName` – The display name of the resource.
+ `HubContentMarkdown` – The description of the resource, in Markdown formatting.
+ `HubContentSearchKeywords` – The searchable keywords of the resource.
+ `SupportStatus` – The current status of the resource.
In your request, include a change for one or more of the preceding fields. If you attempt to update any other fields, such as the hub content type, you receive an error.
------
#### [ Amazon SDK for Python (Boto3) ]
The following example shows how you can use the Amazon SDK for Python (Boto3) to submit an [ UpdateHubContent](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_UpdateHubContent.html) request.
**Note**
The `HubContentVersion` you specify in the request means that the specific version's metadata is updated. To find all of the available versions of your hub content, you can use the [ ListHubContentVersions](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListHubContentVersions.html) API.
```
import boto3
sagemaker_client = boto3.Session(region_name=).client("sagemaker")
sagemaker_client.update_hub_contents(
HubName=,
HubContentName=,
HubContentType=<"Model"|"Notebook">,
HubContentVersion='1.0.0', # specify the correct version that you want to update
HubContentDescription=
)
```
------
#### [ Amazon CLI ]
The following example shows how you can use the Amazon CLI to submit an [ update-hub-content](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/update-hub-content.html) request.
```
aws sagemaker update-hub-content \
--hub-name \
--hub-content-name \
--hub-content-type <"Model"|"Notebook"> \
--hub-content-version "1.0.0" \
--hub-content-description
```
------
## Update model references
To update a model reference to a JumpStart model, use the [ UpdateHubContentReference](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_UpdateHubContentReference.html) API.
You can only update the `MinVersion` field for model references.
------
#### [ Amazon SDK for Python (Boto3) ]
The following example shows how you can use the Amazon SDK for Python (Boto3) to submit an [ UpdateHubContentReference](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_UpdateHubContentReference.html) request.
```
import boto3
sagemaker_client = boto3.Session(region_name=).client("sagemaker")
update_response = sagemaker_client.update_hub_content_reference(
HubName=,
HubContentName=,
HubContentType='ModelReference',
MinVersion='1.0.0'
)
```
------
#### [ Amazon CLI ]
The following example shows how you can use the Amazon CLI to submit an [ update-hub-content-reference](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/update-hub-content-reference.html) request.
```
aws sagemaker update-hub-content-reference \
--hub-name \
--hub-content-name \
--hub-content-type "ModelReference" \
--min-version "1.0.0"
```
------
# Cross-account sharing for private model hubs with Amazon Resource Access Manager
After creating a private model hub, you can share the hub to the necessary accounts using Amazon Resource Access Manager (Amazon RAM). For more information on creating a private hub, see [Create a private model hub](jumpstart-curated-hubs-admin-guide-create.md). The following page gives in-depth information about managed permissions related to private hubs within Amazon RAM. For information about how to create a resource share within Amazon RAM, see [Set up cross-account hub sharing](jumpstart-curated-hubs-ram-setup.md).
## Managed permissions for curated private hubs
The available access permissions are read, read and use, and full access permissions. The permission name, description, and list of specific APIs available for each permission are listed in the following:
+ Read permission (`AmazonRAMPermissionSageMaker AIHubRead`): The read privilege allows resource consumer accounts to read contents in the shared hubs and view details and metadata.
+ `DescribeHub`: Retrieves details about a hub and its configuration
+ `DescribeHubContent`: Retrieves details about a model available in a specific hub
+ `ListHubContent`: Lists all models available in a hub
+ `ListHubContentVersions`: Lists the version of all models available in a hub
+ Read and use permission (`AmazonRAMPermissionSageMaker AIHubReadAndUse`): The read and use privilege allows resource consumer accounts to read contents in the shared hubs and deploy available models for inference.
+ `DescribeHub`: Retrieves details about a hub and its configuration
+ `DescribeHubContent`: Retrieves details about a model available in a specific hub
+ `ListHubContent`: Lists all models available in a hub
+ `ListHubContentVersions`: Lists the version of all models available in a hub
+ `DeployHubModel`: Allows access to deploy available open-weight hub models for inference
+ Full access permission (`AmazonRAMPermissionSageMaker AIHubFullAccessPolicy`): The full access privilege allows resource consumer accounts to read contents in the shared hubs, add and remove hub content, and deploy available models for inference.
+ `DescribeHub`: Retrieves details about a hub and its configuration
+ `DescribeHubContent`: Retrieves details about a model available in a specific hub
+ `ListHubContent`: Lists all models available in a hub
+ `ListHubContentVersions`: Lists the version of all models available in a hub
+ `ImportHubContent`: Imports hub content
+ `DeleteHubContent`: Deletes hub content
+ `CreateHubContentReference`: Creates a hub content reference that shares a model from the SageMaker AI **Public models** hub to a private hub
+ `DeleteHubContentReference`: Delete a hub content reference that shares a model from the SageMaker AI **Public models** hub to a private hub
+ `DeployHubModel`: Allows access to deploy available open-weight hub models for inference
`DeployHubModel` permissions are not required for proprietary models.
# Set up cross-account hub sharing
SageMaker uses [Amazon Resource Access Manager (Amazon RAM)](https://docs.amazonaws.cn/ram/latest/userguide/what-is.html) to help you securely share your private hubs across accounts. Set up cross-account hub sharing using the following instructions along with the [Sharing your Amazon resources](https://docs.amazonaws.cn/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-create) instructions in the *Amazon RAM User Guide*.
**Create a resource share**
1. Select **Create resource share** through the [Amazon RAM console](https://console.aws.amazon.com/ram/home).
1. When specifying resource share details, choose the **SageMaker Hubs** resource type and select one more more private hubs that you want to share. When you share a hub with any other account, all of its contents are also shared implicitly.
1. Associate permissions with your resources share. For more information about managed permissions, see [Managed permissions for curated private hubs](jumpstart-curated-hubs-ram.md#jumpstart-curated-hubs-ram-permissions)
1. Use Amazon account IDs to specify the accounts to which you want to grant access to your shared resources.
1. Review your resource share configuration and select **Create resource share**. It may take a few minutes for the resource share and principal associations to complete.
For more information, see [Sharing your Amazon resources](https://docs.amazonaws.cn/ram/latest/userguide/getting-started-sharing.html) in the *Amazon Resource Access Manager User Guide*.
After the resource share and principal associations are set, the specified Amazon accounts receive an invitation to join the resource share. The Amazon accounts must accept the invite to gain access to any shared resources.
For more information on accepting a resource share invite through Amazon RAM, see [Using shared Amazon resources ](https://docs.amazonaws.cn/ram/latest/userguide/getting-started-shared.html)in the *Amazon Resource Access Manager User Guide*.
# Delete models from a private hub
You can delete models from a private hub used by your organization by specifying the model ARN in the `hub.delete_model_reference()` method. This removes access to the model from the private hub.
```
hub.delete_model_reference(model-name)
```
# Restrict access to JumpStart gated models
Amazon SageMaker JumpStart provides access to both publicly available and proprietary foundation models. There are certain gated models in private Amazon S3 buckets that require you to have accepted the model's EULA (end user license agreement) in order to access them. For more information, see [EULA acceptance with the SageMaker Python SDK](jumpstart-foundation-models-choose.md#jumpstart-foundation-models-choose-eula-python-sdk).
The current default behavior is that if a user accepts a model's EULA, then the user can access the model and create [ fine-tuning training jobs](jumpstart-foundation-models-use-python-sdk-estimator-class.md). However, if you're an administrator and would like to restrict fine-tuning access to these gated models, you can set a policy that denies permissions to use the `CreateTrainingJob` action whenever the request is to a gated model.
The following is an example Amazon Identity and Access Management (IAM) policy that an administrator can add to a user's IAM role:
```
{
"Effect": "Deny",
"Action": "sagemaker:CreateTrainingJob",
"Resource": "*",
"Condition": {
"Bool": {
"sagemaker:DirectGatedModelAccess": "true"
}
}
}
```
If you want to grant users access to specific models without providing unrestricted access to the gated models, set up a curated hub and add the specific models to the hub. For more information, see [Private curated hubs for foundation model access control in JumpStart](jumpstart-curated-hubs.md).
# Remove access to the SageMaker **Public models** hub
In addition to adding a private curated hub to JumpStart in Studio, you can also remove access to the SageMaker **Public models** hub for your users. The SageMaker **Public models** hub has access to all available JumpStart foundation models.
If you remove access to the SageMaker **Public models** hub and a user has access to only one private hub, then the user is taken directly into that private hub when they choose **JumpStart** in the left navigation pane in Studio. If a user has access to multiple private hubs, then the user is taken to a **Hubs** menu page when they choose **JumpStart** in the left navigation pane in Studio.
Remove access to the SageMaker **Public models** hub for your users with the following inline policy:
**Note**
You can specify any additional Amazon S3 buckets that you want your hub to access in the policy below. Be sure to replace *`REGION`* with the Region of your hub.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": "s3:*",
"Effect": "Deny",
"NotResource": [
"arn:aws-cn:s3:::jumpstart-cache-prod-us-east-1/*.ipynb",
"arn:aws-cn:s3:::jumpstart-cache-prod-us-east-1/*eula*",
"arn:aws-cn:s3:::amzn-s3-demo-bucket/*"
]
},
{
"Action": "sagemaker:*",
"Effect": "Deny",
"Resource": [
"arn:aws-cn:sagemaker:us-east-1:aws:hub/SageMakerPublicHub",
"arn:aws-cn:sagemaker:us-east-1:aws:hub-content/SageMakerPublicHub/*/*"
]
}
]
}
```
------
# Delete a private hub
You can delete a private hub from your admin account. Before deleting a private hub, you must first remove any content in that hub. Delete hub contents and hubs with the following commands:
```
# List the model references in the private hub
response = hub.list_models()
models = response["hub_content_summaries"]
while response["next_token"]:
response = hub.list_models(next_token=response["next_token"])
models.extend(response["hub_content_summaries"])
# Delete all model references in the hub
for model in models:
hub.delete_model_reference(model_name=model.get('HubContentName'))
# Delete the private hub
hub.delete()
```
# Troubleshooting
The following sections give information about IAM permissions issues that might arise when creating a private model hub, as well as informationa bout how to resolve those issues.
**`ValidationException` when calling the `CreateModel` operation: Could not access model data**
This exception arises when you do not have the appropriate Amazon S3 permissions configured for your **Admin** role. For more information on the Amazon S3 permissions needed to create a private hub, see **Step 3** in [Create a private model hub](jumpstart-curated-hubs-admin-guide-create.md).
**`Access Denied` or `Forbidden` when calling `create()`**
You are denied access when creating a private hub if you do not have the appropriate permissions to access the Amazon S3 bucket associated with the SageMaker **Public models** hub. For more information on the Amazon S3 permissions needed to create a private hub, see **Step 3** in [Create a private model hub](jumpstart-curated-hubs-admin-guide-create.md).
# User guide
The following topics cover accessing and using models in your Amazon SageMaker JumpStart curated model hubs. Learn how to access your curated hub models through the Amazon SageMaker Studio interface or programmatically with the SageMaker Python SDK. Additionally, learn how to fine-tune curated hub models to adapt them for your specific use cases and business needs.
**Topics**
+ [
# Access curated model hubs in Amazon SageMaker JumpStart
](jumpstart-curated-hubs-access-hubs.md)
+ [
# Fine-tune curated hub models
](jumpstart-curated-hubs-fine-tune.md)
# Access curated model hubs in Amazon SageMaker JumpStart
You can access a private model hub either through Studio or through the SageMaker Python SDK.
## Access your private model hub in Studio
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the updated Studio experience. For information about using the Studio Classic application, see [Amazon SageMaker Studio Classic](studio.md).
In Amazon SageMaker Studio, open the JumpStart landing page either through the **Home** page or the **Home** menu on the left-side panel. This opens the **SageMaker JumpStart** landing page where you can explore model hubs and search for models.
+ From the **Home** page, choose **JumpStart** in the **Prebuilt and automated solutions** pane.
+ From the **Home** menu in the left panel, navigate to the **JumpStart** node.
For more information on getting started with Amazon SageMaker Studio, see [Amazon SageMaker Studio](studio-updated.md).
From the **SageMaker JumpStart** landing page in Studio, you can explore any private model hubs that include allow-listed models for your organization. If you only have access to one model hub, then the **SageMaker JumpStart** landing page takes you directly into that hub. If you have access to multiple hubs, you are taken to the **Hubs **page.
For more information on fine-tuning, deploying, and evaluating models that you have access to in Studio, see [Use foundation models in Studio](jumpstart-foundation-models-use-studio-updated.md).
## Access your private model hub using the SageMaker Python SDK
You can access your private model hub using the SageMaker Python SDK. Your access to read, use, or edit your curated hub is provided by your administrator.
**Note**
If a hub is shared across accounts, then the `HUB_NAME` must be the hub ARN. If a hub is not shared across accounts, then the `HUB_NAME` can be the hub name.
1. Install the SageMaker Python SDK and import the necessary Python packages.
```
# Install the SageMaker Python SDK
!pip3 install sagemaker --force-reinstall --quiet
# Import the necessary Python packages
import boto3
from sagemaker import Session
from sagemaker.jumpstart.hub.hub import Hub
from sagemaker.jumpstart.model import JumpStartModel
from sagemaker.jumpstart.estimator import JumpStartEstimator
```
1. Initalize a SageMaker AI session and connect to your private hub using the hub name and Region.
```
# If a hub is shared across accounts, then the HUB_NAME must be the hub ARN
HUB_NAME="Example-Hub-ARN"
REGION="us-west-2"
# Initialize a SageMaker session
sm_client = boto3.client('sagemaker')
sm_runtime_client = boto3.client('sagemaker-runtime')
session = Session(sagemaker_client=sm_client,
sagemaker_runtime_client=sm_runtime_client)
# Initialize the private hub
hub = Hub(hub_name=HUB_NAME, sagemaker_session=session)
```
1. After connecting to a private hub, you can list all available models in that hub using the following commands:
```
response = hub.list_models()
models = response["hub_content_summaries"]
while response["next_token"]:
response = hub.list_models(next_token=response["next_token"])
models.extend(response["hub_content_summaries"])
print(models)
```
1. You can get more information about a specific model using the model name with the following command:
```
response = hub.describe_model(model_name="example-model")
print(response)
```
For more information on fine-tuning and deploying models that you have access to using the SageMaker Python SDK, see [Use foundation models with the SageMaker Python SDK](jumpstart-foundation-models-use-python-sdk.md).
# Fine-tune curated hub models
In your private curated model hub, you can run fine-tuning training jobs using your model references. Model references point to a publicly available JumpStart model in the SageMaker AI public hub, but you can fine-tune the model on your own data for your specific use case. After the fine-tuning job, you have access to the model weights that you can then use or deploy to an endpoint.
You can fine-tune curated hub models in just a few lines of code using the SageMaker Python SDK. For more general information on fine-tuning publicly available JumpStart models, see [Foundation models and hyperparameters for fine-tuning](jumpstart-foundation-models-fine-tuning.md).
## Prerequisites
In order to fine-tune a JumpStart model reference in your curated hub, do the following:
1. Make sure that your user's IAM role has the SageMaker AI `TrainHubModel` permission attached. For more information, see [ Adding and removing IAM identity permissions](https://docs.amazonaws.cn/IAM/latest/UserGuide/access_policies_manage-attach-detach.html) in the *Amazon IAM User Guide*.
You should attach a policy like the following example to your user's IAM role:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "sagemaker:TrainHubModel",
"Resource": "arn:aws-cn:sagemaker:*:111122223333:hub/*"
}
]
}
```
------
**Note**
If your curated hub is shared across accounts and the hub content is owned by another account, make sure that your `HubContent` (the model reference resource) has a resource-based IAM policy that also grants the `TrainHubModel` permission to the requesting account, as shown in the following example.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "AllowCrossAccountSageMakerAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws-cn:iam::111122223333:root"
},
"Action": [
"sagemaker:TrainHubModel"
],
"Resource": [
"arn:aws-cn:sagemaker:*:111122223333:hub/*"
]
}
]
}
```
1. Have a private curated hub with a model reference to a JumpStart model that you want to fine-tune. For more information about creating a private hub, see [Create a private model hub](jumpstart-curated-hubs-admin-guide-create.md). To learn how to add publicly available JumpStart models to your private hub, see [Add models to a private hub](jumpstart-curated-hubs-admin-guide-add-models.md).
**Note**
The JumpStart model you choose should be fine-tunable. You can verify whether a model is fine-tunable by checking the [ Built-in Algorithms with Pre-trained Models Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html).
1. Have a training dataset that you want to use for fine-tuning the model. The dataset should be in the appropriate training format for the model that you want to fine-tune.
## Fine-tune a curated hub model reference
The following procedure shows you how to fine-tune a model reference in your private curated hub using the SageMaker Python SDK.
1. Make sure that you have the latest version (at least `2.242.0`) of the SageMaker Python SDK installed. For more information, see [ Use Version 2.x of the SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/v2.html).
```
!pip install --upgrade sagemaker
```
1. Import the Amazon SDK for Python (Boto3) and the modules you'll need from the SageMaker Python SDK.
```
import boto3
from sagemaker.jumpstart.estimator import JumpStartEstimator
from sagemaker.session import Session
```
1. Initialize a Boto3 session, a SageMaker AI client, and a SageMaker Python SDK session.
```
sagemaker_client = boto3.Session(region_name=).client("sagemaker")
sm_session = Session(sagemaker_client=sagemaker_client)
```
1. Create a `JumpStartEstimator` and provide the JumpStart model ID, the name of your hub that contains the model reference, and your SageMaker Python SDK session. For a list of model IDs, see the [ Built-in Algorithms with Pre-trained Models Table](https://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html).
Optionally, you can specify the `instance_type` and `instance_count` fields when creating the estimator. If you don't, the training job uses the default instance type and count for the model you're using.
You can also optionally specify the `output_path` to the Amazon S3 location where you want to store the fine-tuned model weights. If you don't specify the `output_path`, then uses a default SageMaker AI Amazon S3 bucket for the region in your account, named with the following format: `sagemaker--`.
```
estimator = JumpStartEstimator(
model_id="meta-textgeneration-llama-3-2-1b",
hub_name=,
sagemaker_session=sm_session, # If you don't specify an existing session, a default one is created for you
# Optional: specify your desired instance type and count for the training job
# instance_type = "ml.g5.2xlarge"
# instance_count = 1
# Optional: specify a custom S3 location to store the fine-tuned model artifacts
# output_path: "s3://"
)
```
1. Create a dictionary with the `training` key where you specify the location of your fine-tuning dataset. This example points to an Amazon S3 URI. If you have additional considerations, such as using local mode or multiple training data channels, see [ JumpStartEstimator.fit()](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.jumpstart.estimator.JumpStartEstimator.fit) in the SageMaker Python SDK documentation for more information.
```
training_input = {
"training": "s3://"
}
```
1. Call the estimator's `fit()` method and pass in your training data and your EULA acceptance (if applicable).
**Note**
The following example sets `accept_eula=False.` You should manually change the value to `True` in order to accept the EULA.
```
estimator.fit(inputs=training_input, accept_eula=False)
```
Your fine-tuning job should now begin.
You can check on your fine-tuning job by viewing your training jobs, either in the SageMaker AI console or by using the [ListTrainingJobs](https://docs.amazonaws.cn/sagemaker/latest/APIReference/API_ListTrainingJobs.html) API.
You can access your fine-tuned model artifacts at the Amazon S3 `output_path` that was specified in the `JumpStartEstimator` object (either the default SageMaker AI Amazon S3 bucket for the region, or a custom Amazon S3 path you specified, if applicable).
# Amazon SageMaker JumpStart in Studio Classic
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
The following JumpStart features are only available in Amazon SageMaker Studio Classic.
+ [Task-Specific Models](jumpstart-models.md)
+ [Shared Models and Notebooks](jumpstart-content-sharing.md)
+ [End-to-end JumpStart solution templates](jumpstart-solutions.md)
+ [Amazon SageMaker JumpStart Industry: Financial](studio-jumpstart-industry.md)
# Task-Specific Models
JumpStart supports task-specific models across fifteen of the most popular problem types. Of the supported problem types, Vision and NLP-related types total thirteen. There are eight problem types that support incremental training and fine-tuning. For more information about incremental training and hyper-parameter tuning, see [SageMaker AI Automatic Model Tuning](https://docs.amazonaws.cn/sagemaker/latest/dg/automatic-model-tuning.html). JumpStart also supports four popular algorithms for tabular data modeling.
You can search and browse models from the JumpStart landing page in Studio or Studio Classic. When you select a model, the model detail page provides information about the model, and you can train and deploy your model in a few steps. The description section describes what you can do with the model, the expected types of inputs and outputs, and the data type needed for fine-tuning your model.
You can also programmatically utilize models with the [SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/overview.html#use-prebuilt-models-with-sagemaker-jumpstart). For a list of all available models, see the [JumpStart Available Model Table](https://sagemaker.readthedocs.io/en/v2.132.0/doc_utils/pretrainedmodels.html).
The list of problem types and links to their example Jupyter notebooks are summarized in the following table.
| Problem types | Supports inference with pre-trained models | Trainable on a custom dataset | Supported frameworks | Example Notebooks |
| --- | --- | --- | --- | --- |
| Image classification | Yes | Yes | PyTorch, TensorFlow | [Introduction to JumpStart - Image Classification](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_image_classification/Amazon_JumpStart_Image_Classification.ipynb) |
| Object detection | Yes | Yes | PyTorch, TensorFlow, MXNet | [Introduction to JumpStart - Object Detection](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_object_detection/Amazon_JumpStart_Object_Detection.ipynb) |
| Semantic segmentation | Yes | Yes | MXNet | [Introduction to JumpStart - Semantic Segmentation](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_semantic_segmentation/Amazon_JumpStart_Semantic_Segmentation.ipynb) |
| Instance segmentation | Yes | Yes | MXNet | [Introduction to JumpStart - Instance Segmentation](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_instance_segmentation/Amazon_JumpStart_Instance_Segmentation.ipynb) |
| Image embedding | Yes | No | TensorFlow, MXNet | [Introduction to JumpStart - Image Embedding](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_image_embedding/Amazon_JumpStart_Image_Embedding.ipynb) |
| Text classification | Yes | Yes | TensorFlow | [Introduction to JumpStart - Text Classification](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_text_classification/Amazon_JumpStart_Text_Classification.ipynb) |
| Sentence pair classification | Yes | Yes | TensorFlow, Hugging Face | [Introduction to JumpStart - Sentence Pair Classification](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_sentence_pair_classification/Amazon_JumpStart_Sentence_Pair_Classification.ipynb) |
| Question answering | Yes | Yes | PyTorch, Hugging Face | [Introduction to JumpStart – Question Answering](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_question_answering/Amazon_JumpStart_Question_Answering.ipynb) |
| Named entity recognition | Yes | No | Hugging Face | [Introduction to JumpStart - Named Entity Recognition](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_named_entity_recognition/Amazon_JumpStart_Named_Entity_Recognition.ipynb) |
| Text summarization | Yes | No | Hugging Face | [Introduction to JumpStart - Text Summarization](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_text_summarization/Amazon_JumpStart_Text_Summarization.ipynb) |
| Text generation | Yes | No | Hugging Face | [Introduction to JumpStart - Text Generation](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_text_generation/Amazon_JumpStart_Text_Generation.ipynb) |
| Machine translation | Yes | No | Hugging Face | [Introduction to JumpStart - Machine Translation](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_machine_translation/Amazon_JumpStart_Machine_Translation.ipynb) |
| Text embedding | Yes | No | TensorFlow, MXNet | [Introduction to JumpStart - Text Embedding](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_text_embedding/Amazon_JumpStart_Text_Embedding.ipynb) |
| Tabular classification | Yes | Yes | LightGBM, CatBoost, XGBoost, AutoGluon-Tabular, TabTransformer, Linear Learner | [Introduction to JumpStart - Tabular Classification - LightGBM, CatBoost](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/lightgbm_catboost_tabular/Amazon_Tabular_Classification_LightGBM_CatBoost.ipynb) [Introduction to JumpStart - Tabular Classification - XGBoost, Linear Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/xgboost_linear_learner_tabular/Amazon_Tabular_Classification_XGBoost_LinearLearner.ipynb) [Introduction to JumpStart - Tabular Classification - AutoGluon Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/autogluon_tabular/Amazon_Tabular_Classification_AutoGluon.ipynb) [Introduction to JumpStart - Tabular Classification - TabTransformer Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/tabtransformer_tabular/Amazon_Tabular_Classification_TabTransformer.ipynb) |
| Tabular regression | Yes | Yes | LightGBM, CatBoost, XGBoost, AutoGluon-Tabular, TabTransformer, Linear Learner | [Introduction to JumpStart - Tabular Regression - LightGBM, CatBoost](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/lightgbm_catboost_tabular/Amazon_Tabular_Regression_LightGBM_CatBoost.ipynb) [Introduction to JumpStart – Tabular Regression - XGBoost, Linear Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/xgboost_linear_learner_tabular/Amazon_Tabular_Regression_XGBoost_LinearLearner.ipynb) [Introduction to JumpStart – Tabular Regression - AutoGluon Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/autogluon_tabular/Amazon_Tabular_Regression_AutoGluon.ipynb) [Introduction to JumpStart – Tabular Regression - TabTransformer Learner](https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/tabtransformer_tabular/Amazon_Tabular_Regression_TabTransformer.ipynb) |
# Deploy a Model
When you deploy a model from JumpStart, SageMaker AI hosts the model and deploys an endpoint that you can use for inference. JumpStart also provides an example notebook that you can use to access the model after it's deployed.
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
**Note**
Fore more information on JumpStart model deployment in Studio, see [Deploy a model in Studio](jumpstart-foundation-models-use-studio-updated-deploy.md)
## Model deployment configuration
After you choose a model, the model's tab opens. In the **Deploy Model** pane, choose **Deployment Configuration** to configure your model deployment.
![\[The Deploy Model pane.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy.png)
The default instance type for deploying a model depends on the model. The instance type is the hardware that the training job runs on. In the following example, the `ml.p2.xlarge` instance is the default for this particular BERT model.
You can also change the endpoint name, add `key;value` resource tags, activate or deactive the `jumpstart-` prefix for any JumpStart resources related to the model, and specify an Amazon S3 bucket for storing model artifacts used by your SageMaker AI endpoint.
![\[JumpStart Deploy Model pane with Deployment Configuration open to select its settings.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-config.png)
Choose **Security Settings** to specify the Amazon Identity and Access Management (IAM ) role, Amazon Virtual Private Cloud (Amazon VPC), and encryption keys for the model.
![\[JumpStart Deploy Model pane with Security Settings open to select its settings.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security.png)
## Model deployment security
When you deploy a model with JumpStart, you can specify an IAM role, Amazon VPC, and encryption keys for the model. If you don't specify any values for these entries: The default IAM role is your Studio Classic runtime role; default encryption is used; no Amazon VPC is used.
### IAM role
You can select an IAM role that is passed as part of training jobs and hosting jobs. SageMaker AI uses this role to access training data and model artifacts. If you don't select an IAM role, SageMaker AI deploys the model using your Studio Classic runtime role. For more information about IAM roles, see [Amazon Identity and Access Management for Amazon SageMaker AI](security-iam.md).
The role that you pass must have access to the resources that the model needs, and must include all of the following.
+ For training jobs: [CreateTrainingJob API: Execution Role Permissions](https://docs.amazonaws.cn//sagemaker/latest/dg/sagemaker-roles.html#sagemaker-roles-createtrainingjob-perms).
+ For hosting jobs: [CreateModel API: Execution Role Permissions](https://docs.amazonaws.cn//sagemaker/latest/dg/sagemaker-roles.html#sagemaker-roles-createmodel-perms).
**Note**
You can scope down the Amazon S3 permissions granted in each of the following roles. Do this by using the ARN of your Amazon Simple Storage Service (Amazon S3) bucket and the JumpStart Amazon S3 bucket.
```
[
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::jumpstart-cache-prod-/*",
"arn:aws:s3:::jumpstart-cache-prod-",
"arn:aws:s3:::/*"
]
},
{
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricData",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:CreateLogGroup",
"logs:DescribeLogStreams",
"ecr:GetAuthorizationToken"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer"
],
"Resource": [
"*"
]
},
]
}
```
**Find IAM role**
If you select this option, you must select an existing IAM role from the dropdown list.
![\[JumpStart Security Settings IAM section with Find IAM role selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-findiam.png)
**Input IAM role**
If you select this option, you must manually enter the ARN for an existing IAM role. If your Studio Classic runtime role or Amazon VPC block the `iam:list* `call, you must use this option to use an existing IAM role.
![\[JumpStart Security Settings IAM section with Input IAM role selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-inputiam.png)
### Amazon VPC
All JumpStart models run in network isolation mode. After the model container is created, no more calls can be made. You can select an Amazon VPC that is passed as part of training jobs and hosting jobs. SageMaker AI uses this Amazon VPC to push and pull resources from your Amazon S3 bucket. This Amazon VPC is different from the Amazon VPC that limits access to the public internet from your Studio Classic instance. For more information about the Studio Classic Amazon VPC, see [Connect Studio notebooks in a VPC to external resources](studio-notebooks-and-internet-access.md).
The Amazon VPC that you pass does not need access to the public internet, but it does need access to Amazon S3. The Amazon VPC endpoint for Amazon S3 must allow access to at least the following resources that the model needs.
```
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListMultipartUploadParts",
"s3:ListBucket"
],
"Resources": [
"arn:aws:s3:::jumpstart-cache-prod-/*",
"arn:aws:s3:::jumpstart-cache-prod-",
"arn:aws:s3:::bucket/*"
]
}
```
If you do not select an Amazon VPC, no Amazon VPC is used.
**Find VPC**
If you select this option, you must select an existing Amazon VPC from the dropdown list. After you select an Amazon VPC, you must select a subnet and security group for your Amazon VPC. For more information about subnets and security groups, see [Overview of VPCs and subnets](https://docs.amazonaws.cn//vpc/latest/userguide/VPC_Subnets.html).
![\[JumpStart Security Settings VPC section with Find VPC selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-findvpc.png)
**Input VPC**
If you select this option, you must manually select the subnet and security group that compose your Amazon VPC. If your Studio Classic runtime role or Amazon VPC blocks the `ec2:list*` call, you must use this option to select the subnet and security group.
![\[JumpStart Security Settings VPC section with Input VPC selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-inputvpc.png)
### Encryption keys
You can select an Amazon KMS key that is passed as part of training jobs and hosting jobs. SageMaker AI uses this key to encrypt the Amazon EBS volume for the container, and the repackaged model in Amazon S3 for hosting jobs and the output for training jobs. For more information about Amazon KMS keys, see [Amazon KMS keys](https://docs.amazonaws.cn//kms/latest/developerguide/concepts.html#kms_keys).
The key that you pass must trust the IAM role that you pass. If you do not specify an IAM role, the Amazon KMS key must trust your Studio Classic runtime role.
If you do not select an Amazon KMS key, SageMaker AI provides default encryption for the data in the Amazon EBS volume and the Amazon S3 artifacts.
**Find encryption keys**
If you select this option, you must select existing Amazon KMS keys from the dropdown list.
![\[JumpStart Security Settings encryption section with Find encryption keys selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-findencryption.png)
**Input encryption keys**
If you select this option, you must manually enter the Amazon KMS keys. If your Studio Classic execution role or Amazon VPC block the `kms:list* `call, you must use this option to select existing Amazon KMS keys.
![\[JumpStart Security Settings encryption section with Input encryption keys selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-deploy-security-inputencryption.png)
## Configure default values for JumpStart models
You can configure default values for parameters such as IAM roles, VPCs, and KMS keys to pre-populate for JumpStart model deployment and training. After configuring default values, the Studio Classic UI automatically provides your specified security settings and tags to JumpStart models to simplify deployment and training workflows. Administrators and end-users can initialize default values specified in a configuration file in YAML format.
By default, the SageMaker Python SDK uses two configuration files: one for the administrator and one for the user. Using the admininistrator configuration file, administrators can define a set of default values. End-users can override values set in the administrator configuration file and set additional default values using the end-user configuration file. For more information, see [Default configuration file location](https://sagemaker.readthedocs.io/en/stable/overview.html#default-configuration-file-location).
The following code sample lists the default locations of the configuration files when using the SageMaker Python SDK in Amazon SageMaker Studio Classic.
```
# Location of the admin config file
/etc/xdg/sagemaker/config.yaml
# Location of the user config file
/root/.config/sagemaker/config.yaml
```
Values specified in the user configuration file override values set in the administrator configuration file. The configuration file is unique to each user profile within an Amazon SageMaker AI domain. The user profile's Studio Classic application is directly associated with the user profile. For more information, see [Domain user profiles](domain-user-profile.md).
Administrators can optionally set configuration defaults for JumpStart model training and deployment through `JupyterServer` lifecycle configurations. For more information, see [Create and Associate a Lifecycle Configuration with Amazon SageMaker Studio Classic](studio-lcc-create.md).
### Default value configuration YAML file
Your configuration file should adhere to the SageMaker Python SDK [configuration file structure](https://sagemaker.readthedocs.io/en/stable/overview.html#configuration-file-structure). Note that specific fields in the `TrainingJob`, `Model`, and `EndpointConfig` configurations apply to JumpStart model training and deployment default values.
```
SchemaVersion: '1.0'
SageMaker:
TrainingJob:
OutputDataConfig:
KmsKeyId: example-key-id
ResourceConfig:
# Training configuration - Volume encryption key
VolumeKmsKeyId: example-key-id
# Training configuration form - IAM role
RoleArn: arn:aws:iam::123456789012:role/SageMakerExecutionRole
VpcConfig:
# Training configuration - Security groups
SecurityGroupIds:
- sg-1
- sg-2
# Training configuration - Subnets
Subnets:
- subnet-1
- subnet-2
# Training configuration - Custom resource tags
Tags:
- Key: Example-key
Value: Example-value
Model:
EnableNetworkIsolation: true
# Deployment configuration - IAM role
ExecutionRoleArn: arn:aws:iam::123456789012:role/SageMakerExecutionRole
VpcConfig:
# Deployment configuration - Security groups
SecurityGroupIds:
- sg-1
- sg-2
# Deployment configuration - Subnets
Subnets:
- subnet-1
- subnet-2
EndpointConfig:
AsyncInferenceConfig:
OutputConfig:
KmsKeyId: example-key-id
DataCaptureConfig:
# Deployment configuration - Volume encryption key
KmsKeyId: example-key-id
KmsKeyId: example-key-id
# Deployment configuration - Custom resource tags
Tags:
- Key: Example-key
Value: Example-value
```
# Fine-Tune a Model
Fine-tuning trains a pretrained model on a new dataset without training from scratch. This process, also known as transfer learning, can produce accurate models with smaller datasets and less training time. You can fine-tune a model if its card shows a **fine-tunable** attribute set to **Yes**.
![\[JumpStart fine-tunable Image Classification - TensorFlow model\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-finetune-model.png)
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
**Note**
For more information on JumpStart model fine-tuning in Studio, see [Fine-tune a model in Studio](jumpstart-foundation-models-use-studio-updated-fine-tune.md)
## Fine-Tuning data source
When you fine-tune a model, you can use the default dataset or choose your own data, which is located in an Amazon S3 bucket.
To browse the buckets available to you, choose **Find S3 bucket**. These buckets are limited by the permissions used to set up your Studio Classic account. You can also specify an Amazon S3 URI by choosing **Enter Amazon S3 bucket location**.
![\[JumpStart data source settings with default dataset selected.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-dataset.png)
**Tip**
To find out how to format the data in your bucket, choose **Learn more**. The description section for the model has detailed information about inputs and outputs.
For text models:
+ The bucket must have a data.csv file.
+ The first column must be a unique integer for the class label. For example: `1`, `2`, `3`, `4`, `n`
+ The second column must be a string.
+ The second column should have the corresponding text that matches the type and language for the model.
For vision models:
+ The bucket must have as many subdirectories as the number of classes.
+ Each subdirectory should contain images that belong to that class in .jpg format.
**Note**
The Amazon S3 bucket must be in the same Amazon Web Services Region where you're running SageMaker Studio Classic because SageMaker AI doesn't allow cross-Region requests.
## Fine-Tuning deployment configuration
The p3 family is recommended as the fastest for deep learning training, and this is recommended for fine-tuning a model. The following chart shows the number of GPUs in each instance type. There are other available options that you can choose from, including p2 and g4 instance types.
| Instance type | GPUs |
| --- | --- |
| p3.2xlarge | 1 |
| p3.8xlarge | 4 |
| p3.16xlarge | 8 |
| p3dn.24xlarge | 8 |
## Hyperparameters
You can customize the hyperparameters of the training job that are used to fine-tune the model. The hyperparameters available for each fine-tunable model differ depending on the model. For information on each available hyperparameter, reference the hyperparameters documentation for the model of your choosing in [Built-in algorithms and pretrained models in Amazon SageMaker](algos.md). For example, see [Image Classification - TensorFlow Hyperparameters](IC-TF-Hyperparameter.md) for details on the fine-tunable Image Classification - TensorFlow hyperparameters.
If you use the default dataset for text models without changing the hyperparameters, you get a nearly identical model as a result. For vision models, the default dataset is different from the dataset used to train the pretrained models, so your model is different as a result.
The following hyperparameters are common among models:
+ **Epochs** – One epoch is one cycle through the entire dataset. Multiple intervals complete a batch, and multiple batches eventually complete an epoch. Multiple epochs are run until the accuracy of the model reaches an acceptable level, or when the error rate drops below an acceptable level.
+ **Learning rate** – The amount that values should be changed between epochs. As the model is refined, its internal weights are being nudged and error rates are checked to see if the model improves. A typical learning rate is 0.1 or 0.01, where 0.01 is a much smaller adjustment and could cause the training to take a long time to converge, whereas 0.1 is much larger and can cause the training to overshoot. It is one of the primary hyperparameters that you might adjust for training your model. Note that for text models, a much smaller learning rate (5e-5 for BERT) can result in a more accurate model.
+ **Batch size** – The number of records from the dataset that is to be selected for each interval to send to the GPUs for training.
In an image example, you might send out 32 images per GPU, so 32 would be your batch size. If you choose an instance type with more than one GPU, the batch is divided by the number of GPUs. Suggested batch size varies depending on the data and the model that you are using. For example, how you optimize for image data differs from how you handle language data.
In the instance type chart in the deployment configuration section, you can see the number of GPUs per instance type. Start with a standard recommended batch size (for example, 32 for a vision model). Then, multiply this by the number of GPUs in the instance type that you selected. For example, if you're using a `p3.8xlarge`, this would be 32(batch size) multiplied by 4 (GPUs), for a total of 128, as your batch size adjusts for the number of GPUs. For a text model like BERT, try starting with a batch size of 64, and then reduce as needed.
## Training output
When the fine-tuning process is complete, JumpStart provides information about the model: parent model, training job name, training job ARN, training time, and output path. The output path is where you can find your new model in an Amazon S3 bucket. The folder structure uses the model name that you provided and the model file is in an `/output` subfolder and it's always named `model.tar.gz`.
Example: `s3://bucket/model-name/output/model.tar.gz`
## Configure default values for model training
You can configure default values for parameters such as IAM roles, VPCs, and KMS keys to pre-populate for JumpStart model deployment and training. For more information, see, [Configure default values for JumpStart models](jumpstart-deploy.md#jumpstart-config-defaults).
# Share Models
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
You can share JumpStart models through the Studio Classic UI directly from the **Launched JumpStart assets** page using the following procedure:
1. Open Amazon SageMaker Studio Classic and choose **Launched JumpStart assets** in the **JumpStart** section of the lefthand navigation pane.
1. Select the **Training jobs** tab to view the list of your model training jobs.
1. Under the **Training jobs** list, select the training job that you want to share. This opens the training job details page. You cannot share more than one training job at a time.
1. In the header for the training job, choose **Share**, and select **Share with my organization**.
For more information about sharing models with your organization, see [Shared Models and Notebooks](jumpstart-content-sharing.md).
# Shared Models and Notebooks
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
Share your models and notebooks to centralize model artifacts, facilitate discoverability, and increase the reuse of models within your organization. When sharing your models, you can provide training and inference environment information and allow collaborators to use these environments for their own training and inference jobs.
All models that you share and models that are shared with you are searchable in a centralized location directly in Amazon SageMaker Studio Classic. For information on the onboarding steps to sign into Amazon SageMaker Studio Classic, see [Onboard to Amazon SageMaker AI Domain](https://docs.amazonaws.cn/sagemaker/latest/dg/gs-studio-onboard.html).
**Topics**
+ [
# Model and notebook sharing
](jumpstart-content-sharing-access.md)
+ [
# Access shared content
](jumpstart-content-sharing-access-filter.md)
+ [
# Add a model
](jumpstart-content-sharing-add-model.md)
# Model and notebook sharing
To share models and notebooks, navigate to the **Shared models** section in Amazon SageMaker Studio Classic, choose **Shared by my organization**, and then select the **Add** dropdown list. Choose to either add a model or add a notebook.
![\[The menu to add shared models or notebooks to JumpStart.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-shared-models.png)
# Access shared content
From the Amazon SageMaker Studio Classic UI, you can access shared content and filter what you see.
There are three main options for filtering shared models and notebooks:
1. **Shared by me** – Models and notebooks that you shared to JumpStart.
1. **Shared with me** – Models and notebooks shared with you
1. **Shared by my organization** – All models and notebooks that are shared to anyone in your organization
You can also sort your models and notebooks based on the time they were last updated or by ascending or descending alphabetical order. Choose the filter icon (![\[Funnel or filter icon representing data filtering or narrowing down options.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-filter-icon.png)) to further sort your selections.
# Add a model
To add a model, choose **Shared by my organization**, and then select **Add model** from the **Add** dropdown list. Enter the basic information for your model, and add any training or inference information you want to share with collaborators to train or deploy your model. After you enter all the necessary information, choose **Add model** in the lower right corner of the screen.
**Topics**
+ [
# Add basic information
](jumpstart-content-sharing-info.md)
+ [
# Enable training
](jumpstart-content-sharing-training.md)
+ [
# Enable deployment
](jumpstart-content-sharing-deployment.md)
+ [
# Add a notebook
](jumpstart-content-sharing-notebooks.md)
# Add basic information
Adding a model in JumpStart involves providing some basic information about the model you want to train. This information helps define the characteristics and capabilities of your model, as well as improving its discoverability and searchability. To create a new model, follow these steps:
1. Add a title for this model. Adding a title automatically populates a unique identifier in the ID field based on the model title.
1. Add a description of the model.
1. Select a data type from the options: *text*, *vision*, *tabular*, or *audio*.
1. Select a machine learning task from the list of available tasks, such as *image classification* or *text generation*.
1. Select a machine learning framework.
1. Add metadata information with keywords or phrases to use when searching for a model. Use commas to separate keywords. Any spaces are automatically replaced with commas.
# Enable training
When adding a model to share, you can optionally provide a training environment and allow collaborators in your organization to train the shared model.
**Note**
If you are adding a tabular model, you also need to specify a column format and target column to enable training.
After providing the basic details about your model, you'll need to configure the settings for the training job that will be used to train your model. This involves specifying the container environment, code scripts, datasets, output locations, and various other parameters to control how the training job is executed. To configure the training job settings, follow these steps:
1. Add a container to use for model training. You can select a container used for an existing training job, bring your own container in Amazon ECR, or use an Amazon SageMaker Deep Learning Container.
1. Add environment variables.
1. Provide a training script location.
1. Provide a script mode entry point.
1. Provide an Amazon S3 URI for model artifacts generated during training.
1. Provide the Amazon S3 URI to the default training dataset.
1. Provide a model output path. The model output path should be the Amazon S3 URI path for any model artifacts generated from training. SageMaker AI saves the model artifacts as a single compressed TAR file in Amazon S3.
1. Provide a validation dataset to use for evaluating your model during training. Validation datasets must contain the same number of columns and the same feature headers as the training dataset.
1. Turn on network isolation. Network isolation isolates the model container so that no inbound or outbound network calls can be made to or from the model container.
1. Provide training channels through which SageMaker AI can access your data. For example, you might specify input channels named `train` or `test`. For each channel, specify a channel name and a URI to the location of your data. Choose **Browse** to search for Amazon S3 locations.
1. Provide hyperparameters. Add any hyperparameters with which collaborators should experiment during training. Provide a range of valid values for these hyperparameters. This range is used for training job hyperparameter validation. You can define ranges based on the datatype of the hyperparameter.
1. Select an instance type. We recommend a GPU instance with more memory for training with large batch sizes. For a comprehensive list of SageMaker training instances across Amazon Regions, see the **On-Demand Pricing** table in [Amazon SageMaker Pricing.](https://www.amazonaws.cn/sagemaker/pricing/)
1. Provide metrics. Define metrics for a training job by specifying a name and a regular expression for each metric that your training monitors. Design the regular expressions to capture the values of metrics that your algorithm emits. For example, the metric `loss` might have the regular expression `"Loss =(.*?);"`.
# Enable deployment
When adding a model to share, you can optionally provide an inference environment in which collaborators in your organization can deploy the shared model for inference.
After training your machine learning model, you'll need to deploy it to an Amazon SageMaker AI endpoint for inference. This involves providing a container environment, an inference script, the model artifacts generated during training, and selecting an appropriate compute instance type. Configuring these settings properly is crucial for ensuring your deployed model can make accurate predictions and handle inference requests efficiently. To set up your model for inference, follow these steps:
1. Add a container to use for inference. You can bring your own container in Amazon ECR or use an Amazon SageMaker Deep Learning Container.
1. Provide the Amazon S3 URI to an inference script. Custom inference scripts run inside your chosen container. Your inference script should include a function for model loading, and optionally functions generating predictions, and input and output processing. For more information on creating inference scripts for the framework of your choice, see [Frameworks](https://sagemaker.readthedocs.io/en/stable/frameworks/index.html) in the SageMaker Python SDK documentation. For example, for TensorFlow, see [How to implement the pre- and/or post-processing handler(s)](https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/deploying_tensorflow_serving.html#how-to-implement-the-pre-and-or-post-processing-handler-s).
1. Provide an Amazon S3 URI for model artifacts. Model artifacts are the output that results from training a model, and typically consist of trained parameters, a model definition that describes how to compute inferences, and other metadata. If you trained your model in SageMaker AI, the model artifacts are saved as a single compressed TAR file in Amazon S3. If you trained your model outside SageMaker AI, you need to create this single compressed TAR file and save it in an Amazon S3 location.
1. Select an instance type. We recommend a GPU instance with more memory for training with large batch sizes. For a comprehensive list of SageMaker training instances across Amazon Regions, see the **On-Demand Pricing** table in [Amazon SageMaker Pricing](https://www.amazonaws.cn/sagemaker/pricing/).
# Add a notebook
To add a notebook, choose **Shared by my organization**, and then select **Add notebook** from the **Add** dropdown list. Enter the basic information for your notebook and provide an Amazon S3 URI for the location of that notebook.
First, add the basic descriptive information about your notebook. This information is used to improve the searchability of your notebook.
1. Add a title for this notebook. Adding a title automatically populates a unique identifier in the ID field based on the notebook title.
1. Add a description of the notebook.
1. Select a data type from the options: *text*, *vision*, *tabular*, or *audio*.
1. Select an ML task from the list of available tasks, such as *image classification* or *text generation*.
1. Select an ML framework.
1. Add metadata information with keywords or phrases to use when searching for a notebook. Use commas to separate keywords. Any spaces are automatically replaced with commas.
After you've specified the basic information, you can provide an Amazon S3 URI for the location of that notebook. You can choose **Browse** to search through your Amazon S3 buckets for your notebook file location. After you find your notebook, copy the Amazon S3 URI, choose **Cancel**, and then add the Amazon S3 URI to the **Notebook Location** field.
After you enter all the necessary information, choose **Add notebook** in the lower right corner.
# End-to-end JumpStart solution templates
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
**Note**
JumpStart Solutions are only available in Studio Classic.
SageMaker JumpStart provides one-click, end-to-end solutions that are designed to address common machine learning use cases. They use proven algorithms for their domains and provide a complete workflow which typically includes data processing, model training, deployment, inference, and monitoring. Explore the following use cases for more information on available solution templates.
+ [Demand forecasting](#jumpstart-solutions-demand-forecasting)
+ [Credit rating prediction](#jumpstart-solutions-credit-prediction)
+ [Fraud detection](#jumpstart-solutions-fraud-detection)
+ [Computer vision](#jumpstart-solutions-computer-vision)
+ [Extract and analyze data from documents](#jumpstart-solutions-documents)
+ [Predictive maintenance](#jumpstart-solutions-predictive-maintenance)
+ [Churn prediction](#jumpstart-solutions-churn-prediction)
+ [Personalized recommendations](#jumpstart-solutions-recommendations)
+ [Reinforcement learning](#jumpstart-solutions-reinforcement-learning)
+ [Healthcare and life sciences](#jumpstart-solutions-healthcare-life-sciences)
+ [Financial pricing](#jumpstart-solutions-financial-pricing)
+ [Causal inference](#jumpstart-solutions-causal-inference)
Choose the solution template that best fits your use case from the JumpStart landing page. When you choose a solution template, JumpStart opens a new tab showing a description of the solution and a **Launch** button. When you select **Launch**, JumpStart creates all of the resources that you need to run the solution, including training and model hosting instances. For more information on launching a JumpStart solution, see [Launch a Solution](jumpstart-solutions-launch.md).
After launching the solution, you can explore solution features and any generated artifacts in JumpStart. Use the **Launched JumpStart assets** menu to find your solution. In your solution's tab, select **Open Notebook** to use provided notebooks and explore the solution’s features. When artifacts are generated during launch or after running the provided notebooks, they're listed in the **Generated Artifacts** table. You can delete individual artifacts with the trash icon (![\[The trash icon for JumpStart.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-trash.png)). You can delete all of the solution’s resources by choosing **Delete solution resources**.
## Demand forecasting
Demand forecasting uses historical time series data in order to make future estimations in relation to customer demand over a specific period and streamline the supply-demand decision-making process across businesses.
Demand forecasting use cases include predicting ticket sales in the transportation industry, stock prices, number of hospital visits, number of customer representatives to hire for multiple locations in the next month, product sales across multiple regions in the next quarter, cloud server usage for the next day for a video streaming service, electricity consumption for multiple regions over the next week, number of IoT devices and sensors such as energy consumption, and more.
Time series data is categorized as *univariate* and *multi-variate*. For example, the total electricity consumption for a single household is a univariate time series over a period of time. When multiple univariate time series are stacked on each other, it’s called a multi-variate time series. For example, the total electricity consumption of 10 different (but correlated) households in a single neighborhood make up a multi-variate time series dataset.
| Solution name | Description | Get started |
| --- | --- | --- |
| Demand forecasting | Demand forecasting for multivariate time series data using three state-of-the-art time series forecasting algorithms: [LSTNet](https://ts.gluon.ai/stable/api/gluonts/gluonts.mx.model.lstnet.html), [Prophet](https://facebook.github.io/prophet/), and [SageMaker AI DeepAR](https://docs.amazonaws.cn/sagemaker/latest/dg/deepar.html). | [GitHub »](https://github.com/awslabs/sagemaker-deep-demand-forecast) |
## Credit rating prediction
Use JumpStart's credit rating prediction solutions to predict corporate credit ratings or to explain credit prediction decisions made by machine learning models. Compared to traditional credit rating modeling methods, machine learning models can automate and improve the accuracy of credit prediction.
| Solution name | Description | Get started |
| --- | --- | --- |
| Corporate credit rating prediction | Multimodal (long text and tabular) machine learning for quality credit predictions using Amazon [AutoGluon Tabular](https://auto.gluon.ai/scoredebugweight/tutorials/tabular_prediction/index.html). | [GitHub »](https://github.com/awslabs/sagemaker-corporate-credit-rating) |
| Graph-based credit scoring | Predict corporate credit ratings using tabular data and a corporate network by training a [Graph Neural Network GraphSAGE](https://cs.stanford.edu/people/jure/pubs/graphsage-nips17.pdf) and Amazon [AutoGluon Tabular](https://auto.gluon.ai/scoredebugweight/tutorials/tabular_prediction/index.html) model. | Find in Amazon SageMaker Studio Classic. |
| Explain credit decisions | Predict credit default in credit applications and provide explanations using [LightGBM](https://lightgbm.readthedocs.io/en/latest/) and [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/en/latest/index.html). | [GitHub »](https://github.com/awslabs/sagemaker-explaining-credit-decisions) |
## Fraud detection
Many businesses lose billions annually to fraud. Machine learning based fraud detection models can help systematically identify likely fraudulent activities from a tremendous amount of data. The following solutions use transaction and user identity datasets to identify fraudulent transactions.
| Solution name | Description | Get started |
| --- | --- | --- |
| Detect malicious users and transactions | Automatically detect potentially fraudulent activity in transactions using [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html) with the over-sampling technique [Synthetic Minority Over-sampling](https://arxiv.org/abs/1106.1813) (SMOTE). | [GitHub »](https://github.com/awslabs/fraud-detection-using-machine-learning) |
| Fraud detection in financial transactions using deep graph library | Detect fraud in financial transactions by training a [graph convolutional network](https://arxiv.org/pdf/1703.06103.pdf) with the [deep graph library](https://www.dgl.ai/) and a [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html) model. | [GitHub »](https://github.com/awslabs/sagemaker-graph-fraud-detection) |
| Financial payment classification | Classify financial payments based on transaction information using [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html). Use this solution template as an intermediate step in fraud detection, personalization, or anomaly detection. | Find in Amazon SageMaker Studio Classic. |
## Computer vision
With the rise of business use cases such as autonomous vehicles, smart video surveillance, healthcare monitoring and various object counting tasks, fast and accurate object detection systems are rising in demand. These systems involve not only recognizing and classifying every object in an image, but localizing each one by drawing the appropriate bounding box around it. In the last decade, the rapid advances of deep learning techniques greatly accelerated the momentum of object detection.
| Solution name | Description | Get started |
| --- | --- | --- |
| Visual product defect detection | Identify defective regions in product images either by training an [object detection model from scratch](https://ieeexplore.ieee.org/document/8709818) or fine-tuning pretrained SageMaker AI models. | [GitHub »](https://github.com/awslabs/sagemaker-defect-detection) |
| Handwriting recognition | Recognize handwritten text in images by training an [object detection model](https://mxnet.apache.org/versions/1.0.0/api/python/gluon/model_zoo.html#mxnet.gluon.model_zoo.vision.resnet34_v1) and [handwriting recognition model](https://arxiv.org/abs/1910.00663). Label your own data using [SageMaker Ground Truth](https://www.amazonaws.cn/sagemaker/data-labeling/). | [GitHub »](https://github.com/awslabs/sagemaker-handwritten-text-recognition) |
| Object detection for bird species | Identify birds species in a scene using a [SageMaker AI object detection model](https://docs.amazonaws.cn/sagemaker/latest/dg/object-detection.html). | Find in Amazon SageMaker Studio Classic. |
## Extract and analyze data from documents
JumpStart provides solutions for you to uncover valuable insights and connections in business-critical documents. Use cases include text classification, document summarization, handwriting recognition, relationship extraction, question and answering, and filling in missing values in tabular records.
| Solution name | Description | Get started |
| --- | --- | --- |
| Privacy for sentiment classification | [Anonymize text ](https://www.amazon.science/blog/preserving-privacy-in-analyses-of-textual-data) to better preserve user privacy in sentiment classification. | [GitHub »](https://github.com/awslabs/sagemaker-privacy-for-nlp) |
| Document understanding | Document summarization, entity, and relationship extraction using the [transformers](https://huggingface.co/docs/transformers/index) library in PyTorch. | [GitHub »](https://github.com/awslabs/sagemaker-document-understanding) |
| Handwriting recognition | Recognize handwritten text in images by training an [object detection model](https://mxnet.apache.org/versions/1.0.0/api/python/gluon/model_zoo.html#mxnet.gluon.model_zoo.vision.resnet34_v1) and [handwriting recognition model](https://arxiv.org/abs/1910.00663). Label your own data using [SageMaker Ground Truth](https://www.amazonaws.cn/sagemaker/data-labeling/). | [GitHub »](https://github.com/awslabs/sagemaker-handwritten-text-recognition) |
| Filling in missing values in tabular records | Fill missing values in tabular records by training a [SageMaker Autopilot](https://www.amazonaws.cn/sagemaker/autopilot/) model. | [GitHub »](https://github.com/awslabs/filling-in-missing-values-in-tabular-records) |
## Predictive maintenance
Predictive maintenance aims to optimize the balance between corrective and preventative maintenance by facilitating the timely replacement of components. The following solutions use sensor data from industrial assets to predict machine failures, unplanned downtime, and repair costs.
| Solution name | Description | Get started |
| --- | --- | --- |
| Predictive maintenance for vehicle fleets | Predict vehicle fleet failures using vehicle sensor and maintenance information with a convolutional neural network model. | [GitHub »](https://github.com/awslabs/aws-fleet-predictive-maintenance/) |
| Predictive maintenance for manufacturing | Predict the remaining useful life for each sensor by training a [stacked Bidirectional LSTM neural network](https://arxiv.org/pdf/1801.02143.pdf) model using historical sensor readings. | [GitHub »](https://github.com/awslabs/predictive-maintenance-using-machine-learning) |
## Churn prediction
Customer churn, or rate of attrition, is a costly problem faced by a wide range of companies. In an effort to reduce churn, companies can identify customers that are likely to leave their service in order to focus their efforts on customer retention. Use a JumpStart churn prediction solution to analyze data sources such as user behavior and customer support chat logs to identify customers that are at a high risk of cancelling a subscription or service.
| Solution name | Description | Get started |
| --- | --- | --- |
| Churn prediction with text | Predict churn using numerical, categorical, and textual features with [BERT encoder](https://huggingface.co/) and [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html). | [GitHub »](https://github.com/awslabs/sagemaker-churn-prediction-text) |
| Churn prediction for mobile phone customers | Identify unhappy mobile phone customers using [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html). | Find in Amazon SageMaker Studio Classic. |
## Personalized recommendations
You can use JumpStart solutions to analyze customer identity graphs or user sessions to better understand and predict customer behavior. Use the following solutions for personalized recommendations to model customer identity across multiple devices, to determine the likelihood of a customer making a purchase, or to create a custom movie recommender based on past customer behavior.
| Solution name | Description | Get started |
| --- | --- | --- |
| Entity resolution in identity graphs with deep graph library | Perform cross-device entity linking for online advertising by training a [graph convolutional network](https://arxiv.org/pdf/1703.06103.pdf) with [deep graph library](https://www.dgl.ai/). | [GitHub »](https://github.com/awslabs/sagemaker-graph-entity-resolution) |
| Purchase modeling | Predict whether a customer will make a purchase by training a [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html) model. | [GitHub »](https://github.com/awslabs/sagemaker-purchase-modelling) |
| Customized recommender system | Train and deploy a custom recommender system that generates movie suggestions for a customer based on past behavior using Neural Collaborative Filtering in SageMaker AI. | Find in Amazon SageMaker Studio Classic. |
## Reinforcement learning
Reinforcement learning (RL) is a type of learning that is based on interaction with the environment. This type of learning is used by an agent that must learn behavior through trial-and-error interactions with a dynamic environment in which the goal is to maximize the long-term rewards that the agent receives as a result of its actions. Rewards are maximized by trading off exploring actions that have uncertain rewards with exploiting actions that have known rewards.
RL is well-suited for solving large, complex problems, such as supply chain management, HVAC systems, industrial robotics, game artificial intelligence, dialog systems, and autonomous vehicles.
| Solution name | Description | Get started |
| --- | --- | --- |
| Reinforcement learning for Battlesnake AI competitions | Provide a reinforcement learning workflow for training and inference with the [BattleSnake](https://play.battlesnake.com/) AI competitions. | [GitHub »](https://github.com/awslabs/sagemaker-battlesnake-ai) |
| Distributed reinforcement learning for Procgen challenge | Distributed reinforcement learning starter kit for [NeurIPS 2020 Procgen](https://www.aicrowd.com/challenges/neurips-2020-procgen-competition) Reinforcement learning challenge. | [GitHub »](https://github.com/aws-samples/sagemaker-rl-procgen-ray) |
## Healthcare and life sciences
Clinicians and researchers can use JumpStart solutions to analyze medical imagery, genomic information, and clinical health records.
| Solution name | Description | Get started |
| --- | --- | --- |
| Lung cancer survival prediction | Predict non-small cell lung cancer patient survival status with 3-dimensional lung computerized tomography (CT) scans, genomic data, and clinical health records using [SageMaker AI XGBoost](https://docs.amazonaws.cn/sagemaker/latest/dg/xgboost.html). | [GitHub »](https://github.com/aws-samples/machine-learning-pipelines-for-multimodal-health-data/tree/sagemaker-soln-lcsp) |
## Financial pricing
Many businesses dynamically adjust pricing on a regular basis in order to maximize their returns. Use the following JumpStart solutions for price optimization, dynamic pricing, option pricing, or portfolio optimization use cases.
| Solution name | Description | Get started |
| --- | --- | --- |
| Price optimization | Estimate price elasticity using Double Machine Learning (ML) for causal inference and the [Prophet](https://facebook.github.io/prophet/) forecasting procedure. Use these estimates to optimize daily prices. | Find in Amazon SageMaker Studio Classic. |
## Causal inference
Researchers can use machine learning models such as Bayesian networks to represent causal dependencies and draw causal conclusions based on data. Use the following JumpStart solution to understand the causal relationship between Nitrogen-based fertilizer application and corn crop yields.
| Solution name | Description | Get started |
| --- | --- | --- |
| Crop yield counterfactuals | Generate a counterfactual analysis of corn response to nitrogen. This solution learns the crop phenology cycle in its entirety using multi-spectral satellite imagery and [ground-level observations](https://www.sciencedirect.com/science/article/pii/S2352340921010283#tbl0001). | Find in Amazon SageMaker Studio Classic. |
# Launch a Solution
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
**Note**
JumpStart Solutions are only available in Studio Classic.
First, choose a solution through the SageMaker JumpStart landing page in the Amazon SageMaker Studio Classic UI. For information on the onboarding steps to sign in to Amazon SageMaker Studio Classic, see [Onboard to Amazon SageMaker AI domain](https://docs.amazonaws.cn/sagemaker/latest/dg/gs-studio-onboard.html). For details on getting to the SageMaker JumpStart landing page, see [Open and use JumpStart in Studio Classic](studio-jumpstart.md#jumpstart-open-use).
After you choose a solution, a solution's tab opens showing a description of the solution and a `Launch` button. To launch a solution, select `Launch` in the **Launch Solution** section. JumpStart then creates all of the resources needed to run the solution. This includes training and model hosting instances.
## Advanced parameters
The solution that you choose may have advanced parameters that you can select. Choose **Advanced Parameters** to specify the Amazon Identity and Access Management role for the solution.
Solutions are able to launch resources across 9 Amazon services that interact with each other. For the solution to work as expected, newly created components from one service must be able to act on newly created components from another service. We recommend that you use the default IAM role to ensure that all needed permissions are added. For more information about IAM roles, see [Amazon Identity and Access Management for Amazon SageMaker AI](security-iam.md).
**Default IAM role**
If you select this option, the default IAM roles that are required by this solution are used. Each solution requires different resources. The following list describes the default roles that are used for the solutions based on the service needed. For a description of the permissions required for each service, see [Amazon Managed Policies for SageMaker Projects and JumpStart](security-iam-awsmanpol-sc.md).
+ **API Gateway** – AmazonSageMakerServiceCatalogProductsApiGatewayRole
+ **CloudFormation** – AmazonSageMakerServiceCatalogProductsCloudformationRole
+ **CodeBuild** – AmazonSageMakerServiceCatalogProductsCodeBuildRole
+ **CodePipeline** – AmazonSageMakerServiceCatalogProductsCodePipelineRole
+ **Events** – AmazonSageMakerServiceCatalogProductsEventsRole
+ **Firehose** – AmazonSageMakerServiceCatalogProductsFirehoseRole
+ **Glue** – AmazonSageMakerServiceCatalogProductsGlueRole
+ **Lambda** – AmazonSageMakerServiceCatalogProductsLambdaRole
+ **SageMaker AI** – AmazonSageMakerServiceCatalogProductsExecutionRole
If you are using a new SageMaker AI domain with JumpStart project templates enabled, these roles are automatically created in your account.
If you are using an existing SageMaker AI domain, these roles may not exist in your account. If this is the case, you will receive the following error when launching the solution.
```
Unable to locate the updated roles required to launch this solution, a general role '/service-role/AmazonSageMakerServiceCatalogProductsUseRole' will be used. Please update your studio domain to generate these roles.
```
You can still launch a solution without the needed role, but the legacy default role `AmazonSageMakerServiceCatalogProductsUseRole` is used in place of the needed role. The legacy default role has trust relationships with all of the services that JumpStart solutions need to interact with. For the best security, we recommend that you update your domain to have the newly created default roles for each Amazon service.
If you have already onboarded to a SageMaker AI domain, you can update your domain to generate the default roles using the following procedure.
1. Open the Amazon SageMaker AI console at [https://console.amazonaws.cn/sagemaker/](https://console.amazonaws.cn/sagemaker/).
1. Choose ** Control Panel ** at the top left of the page.
1. From the **domain** page, choose the **Settings** icon (![\[Black square icon representing a placeholder or empty image.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/icons/Settings_squid.png)) to edit the domain settings.
1. On **General Settings** choose **Next**.
1. Under **SageMaker Projects and JumpStart**, select **Enable Amazon SageMaker project templates and Amazon SageMaker JumpStart for this account ** and **Enable Amazon SageMaker project templates and Amazon SageMaker JumpStart for Studio Classic users**, choose **Next**.
1. Select **Submit**.
You should be able to see the default roles listed in **Projects - Amazon SageMaker project templates enabled for this account** under the **Apps - Studio** tab.
**Find IAM role**
If you select this option, you must select an existing IAM role from the dropdown list for each of the required services. The selected role must have at least the minimum permissions required for the corresponding service. For a description of the permissions required for each service, see [Amazon Managed Policies for SageMaker Projects and JumpStart](security-iam-awsmanpol-sc.md).
**Input IAM role**
If you select this option, you must manually enter the ARN for an existing IAM role. The selected role must have at least the minimum permissions required for the corresponding service. For a description of the permissions required for each service, see [Amazon Managed Policies for SageMaker Projects and JumpStart](security-iam-awsmanpol-sc.md).
# Amazon SageMaker JumpStart Industry: Financial
Use SageMaker JumpStart Industry: Financial solutions, models, and example notebooks to learn about SageMaker AI features and capabilities through curated one-step solutions and example notebooks of industry-focused machine learning (ML) problems. The notebooks also walk through how to use the SageMaker JumpStart Industry Python SDK to enhance industry text data and fine-tune pretrained models.
**Topics**
+ [
## Amazon SageMaker JumpStart Industry Python SDK
](#studio-jumpstart-industry-pysdk)
+ [
## Amazon SageMaker JumpStart Industry: Financial Solution
](#studio-jumpstart-industry-solutions)
+ [
## Amazon SageMaker JumpStart Industry: Financial Models
](#studio-jumpstart-industry-models)
+ [
## Amazon SageMaker JumpStart Industry: Financial Example Notebooks
](#studio-jumpstart-industry-examples)
+ [
## Amazon SageMaker JumpStart Industry: Financial Blog Posts
](#studio-jumpstart-industry-blogs)
+ [
## Amazon SageMaker JumpStart Industry: Financial Related Research
](#studio-jumpstart-industry-research)
+ [
## Amazon SageMaker JumpStart Industry: Financial Additional Resources
](#studio-jumpstart-industry-resources)
## Amazon SageMaker JumpStart Industry Python SDK
SageMaker Runtime JumpStart provides processing tools for curating industry datasets and fine-tuning pretrained models through its client library called SageMaker JumpStart Industry Python SDK. For detailed API documentation of the SDK, and to learn more about processing and enhancing industry text datasets for improving the performance of state-of-the-art models on SageMaker JumpStart, see the [SageMaker JumpStart Industry Python SDK open source documentation](https://sagemaker-jumpstart-industry-pack.readthedocs.io).
## Amazon SageMaker JumpStart Industry: Financial Solution
SageMaker JumpStart Industry: Financial provides the following solution notebooks:
+ **Corporate Credit Rating Prediction**
This SageMaker JumpStart Industry: Financial solution provides a template for a text-enhanced corporate credit rating model. It shows how to take a model based on numeric features (in this case, Altman's famous 5 financial ratios) combined with texts from SEC filings to achieve an improvement in the prediction of credit ratings. In addition to the 5 Altman ratios, you can add more variables as needed or set custom variables. This solution notebook shows how SageMaker JumpStart Industry Python SDK helps process Natural Language Processing (NLP) scoring of texts from SEC filings. Furthermore, the solution demonstrates how to train a model using the enhanced dataset to achieve a best-in-class model, deploy the model to a SageMaker AI endpoint for production, and receive improved predictions in real time.
+ **Graph-Based Credit Scoring **
Credit ratings are traditionally generated using models that use financial statement data and market data, which is tabular only (numeric and categorical). This solution constructs a network of firms using [SEC filings](https://www.sec.gov/edgar/searchedgar/companysearch.html)and shows how to use the network of firm relationships with tabular data to generate accurate rating predictions. This solution demonstrates a methodology to use data on firm linkages to extend the traditionally tabular-based credit scoring models, which have been used by the ratings industry for decades, to the class of machine learning models on networks.
**Note**
The solution notebooks are for demonstration purposes only. They should not be relied on as financial or investment advice.
You can find these financial services solutions through the SageMaker JumpStart page in Studio Classic.
**Important**
As of November 30, 2023, the previous Amazon SageMaker Studio experience is now named Amazon SageMaker Studio Classic. The following section is specific to using the Studio Classic application. For information about using the updated Studio experience, see [Amazon SageMaker Studio](studio-updated.md).
Studio Classic is still maintained for existing workloads but is no longer available for onboarding. You can only stop or delete existing Studio Classic applications and cannot create new ones. We recommend that you [migrate your workload to the new Studio experience](studio-updated-migrate.md).
**Note**
The SageMaker JumpStart Industry: Financial solutions, model cards, and example notebooks are hosted and runnable only through SageMaker Studio Classic. Log in to the [SageMaker AI console](https://console.aws.amazon.com/sagemaker), and launch SageMaker Studio Classic. For more information about how to find the solution card, see the previous topic at [SageMaker JumpStart](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-jumpstart.html).
## Amazon SageMaker JumpStart Industry: Financial Models
SageMaker JumpStart Industry: Financial provides the following pretrained [Robustly Optimized BERT approach (RoBERTa)](https://arxiv.org/pdf/1907.11692.pdf) models:
+ **Financial Text Embedding (RoBERTa-SEC-Base) **
+ **RoBERTa-SEC-WIKI-Base **
+ **RoBERTa-SEC-Large **
+ **RoBERTa-SEC-WIKI-Large **
The RoBERTa-SEC-Base and RoBERTa-SEC-Large models are the text embedding models based on [GluonNLP's RoBERTa model](https://nlp.gluon.ai/api/model.html#gluonnlp.model.RoBERTaModel) and pretrained on S&P 500 SEC 10-K/10-Q reports of the decade of the 2010's (from 2010 to 2019). In addition to these, SageMaker AI JumpStart Industry: Financial provides two more RoBERTa variations, RoBERTa-SEC-WIKI-Base and RoBERTa-SEC-WIKI-Large, which are pretrained on the SEC filings and common texts of Wikipedia.
You can find these models in SageMaker JumpStart by navigating to the **Text Models** node, choosing **Explore All Text Models**, and then filtering for the ML Task **Text Embedding**. You can access any corresponding notebooks after selecting the model of your choice. The paired notebooks will walk you through how the pretrained models can be fine-tuned for specific classification tasks on multimodal datasets, which are enhanced by the SageMaker JumpStart Industry Python SDK.
**Note**
The model notebooks are for demonstration purposes only. They should not be relied on as financial or investment advice.
The following screenshot shows the pretrained model cards provided through the SageMaker AI JumpStart page on Studio Classic.
![\[The pretrained model cards provided through the SageMaker AI JumpStart page on Studio Classic.\]](http://docs.amazonaws.cn/en_us/sagemaker/latest/dg/images/jumpstart/jumpstart-finance-models.png)
**Note**
The SageMaker JumpStart Industry: Financial solutions, model cards, and example notebooks are hosted and runnable only through SageMaker Studio Classic. Log in to the [SageMaker AI console](https://console.aws.amazon.com/sagemaker), and launch SageMaker Studio Classic. For more information about how to find the model cards, see the previous topic at [SageMaker JumpStart](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-jumpstart.html).
## Amazon SageMaker JumpStart Industry: Financial Example Notebooks
SageMaker JumpStart Industry: Financial provides the following example notebooks to demonstrate solutions to industry-focused ML problems:
+ **Financial TabText Data Construction** – This example introduces how to use the SageMaker JumpStart Industry Python SDK for processing the SEC filings, such as text summarization and scoring texts based on NLP score types and their corresponding word lists. To preview the content of this notebook, see [Simple Construction of a Multimodal Dataset from SEC Filings and NLP Scores](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/finance/notebook1/SEC_Retrieval_Summarizer_Scoring.html).
+ **Multimodal ML on TabText Data** – This example shows how to merge different types of datasets into a single dataframe called TabText and perform multimodal ML. To preview the content of this notebook, see [Machine Learning on a TabText Dataframe – An Example Based on the Paycheck Protection Program](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/finance/notebook2/PPP_TabText_ML.html).
+ **Multi-category ML on SEC filings data** – This example shows how to train an AutoGluon NLP model over the multimodal (TabText) datasets curated from SEC filings for a multiclass classification task. [Classify SEC 10K/Q Filings to Industry Codes Based on the MDNA Text Column](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/finance/notebook3/SEC_MNIST_ML.html).
**Note**
The example notebooks are for demonstrative purposes only. They should not be relied on as financial or investment advice.
**Note**
The SageMaker JumpStart Industry: Financial solutions, model cards, and example notebooks are hosted and runnable only through SageMaker Studio Classic. Log in to the [SageMaker AI console](https://console.aws.amazon.com/sagemaker), and launch SageMaker Studio Classic. For more information about how to find the example notebooks, see the previous topic at [SageMaker JumpStart](https://docs.amazonaws.cn/sagemaker/latest/dg/studio-jumpstart.html).
To preview the content of the example notebooks, see [Tutorials – Finance](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/index.html) in the *SageMaker JumpStart Industry Python SDK documentation*.
## Amazon SageMaker JumpStart Industry: Financial Blog Posts
For thorough applications of using SageMaker JumpStart Industry: Financial solutions, models, examples, and the SDK, see the following blog posts:
+ [Use pre-trained financial language models for transfer learning in Amazon SageMaker JumpStart](https://www.amazonaws.cn/blogs/machine-learning/use-pre-trained-financial-language-models-for-transfer-learning-in-amazon-sagemaker-jumpstart/)
+ [Use SEC text for ratings classification using multimodal ML in Amazon SageMaker JumpStart](https://www.amazonaws.cn/blogs/machine-learning/use-sec-text-for-ratings-classification-using-multimodal-ml-in-amazon-sagemaker-jumpstart/)
+ [Create a dashboard with SEC text for financial NLP in Amazon SageMaker JumpStart](https://www.amazonaws.cn/blogs/machine-learning/create-a-dashboard-with-sec-text-for-financial-nlp-in-amazon-sagemaker-jumpstart/)
+ [Build a corporate credit ratings classifier using graph machine learning in Amazon SageMaker JumpStart](https://www.amazonaws.cn/blogs/machine-learning/build-a-corporate-credit-ratings-classifier-using-graph-machine-learning-in-amazon-sagemaker-jumpstart/)
+ [Domain-adaptation Fine-tuning of Foundation Models in Amazon SageMaker JumpStart on Financial data](https://www.amazonaws.cn/blogs/machine-learning/domain-adaptation-fine-tuning-of-foundation-models-in-amazon-sagemaker-jumpstart-on-financial-data/)
## Amazon SageMaker JumpStart Industry: Financial Related Research
For research related to SageMaker JumpStart Industry: Financial solutions, see the following papers:
+ [Context, Language Modeling, and Multimodal Data in Finance](https://www.pm-research.com/content/iijjfds/3/3/52)
+ [Multimodal Machine Learning for Credit Modeling](https://www.amazon.science/publications/multimodal-machine-learning-for-credit-modeling)
+ [On the Lack of Robust Interpretability of Neural Text Classifiers](https://www.amazon.science/publications/on-the-lack-of-robust-interpretability-of-neural-text-classifiers)
+ [FinLex: An Effective Use of Word Embeddings for Financial Lexicon Generation](https://www.sciencedirect.com/science/article/pii/S2405918821000131)
## Amazon SageMaker JumpStart Industry: Financial Additional Resources
For additional documentation and tutorials, see the following resources:
+ [The SageMaker JumpStart Industry: Financial Python SDK](https://pypi.org/project/smjsindustry/)
+ [SageMaker JumpStart Industry: Financial Python SDK Tutorials](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/index.html#)
+ [The SageMaker JumpStart Industry: Financial GitHub repository](https://github.com/aws/sagemaker-jumpstart-industry-pack/)
+ [Getting started with Amazon SageMaker AI - Machine Learning Tutorials](https://www.amazonaws.cn/sagemaker/getting-started/)