View model candidates in the model leaderboard

When you do a Standard build for tabular and time series forecasting models in Amazon SageMaker Canvas, SageMaker trains multiple model candidates (different iterations of the model) and by default selects the one with the highest value for the optimization metric. For tabular models, Canvas builds up to 250 different model candidates using various algorithms and hyperparameter settings. For time series forecasting models, Canvas builds 7 different models—one for each of the supported forecasting algorithms and one ensemble model that averages the predictions of the other models to try to optimize accuracy.

The default model candidate is the only version that you can use in Canvas for actions like making predictions, registering to the model registry, or deploying to an endpoint. However, you might want to review all of the model candidates and select a different candidate to be the default model. You can view all of the model candidates and more details about each candidate on the Model leaderboard in Canvas.

To view the Model leaderboard, do the following:

The Model leaderboard page opens, which for tabular models looks like the following screenshot.

For time series forecasting models, you see 7 models, which include one for each of the time series forecasting algorithms supported by Canvas and one ensemble model. For more information about the algorithms, see Advanced time series forecasting model settings.

In the preceding screenshot, you can see that the first model candidate listed is marked as the Default model. This is the model candidate with which you can make predictions or deploy to endpoints.

To view more detailed metrics information about the model candidates to compare them, you can choose the More options icon ( ) and choose View model details.

The model candidate opens in the Analyze tab, and the metrics shown are specific to that model candidate. When you’re done reviewing the model candidate’s metrics, you can go back or exit the view to return to the Model leaderboard.

If you’d like to set the Default model to a different candidate, you can choose the More options icon ( ) and choose Change to the default model. Changing the default model for a model trained using HPO mode might take several minutes.