Attach EI to a Notebook Instance - Amazon SageMaker
Setting up to use Elastic Inference Using EI in Local Mode in SageMaker
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Attach EI to a Notebook Instance

To test and evaluate inference performance using EI, you can attach EI to a notebook instance when you create or update a notebook instance. You can then use EI in local mode to host a model at an endpoint hosted on the notebook instance. You should test various sizes of notebook instances and EI accelerators to evaluate the configuration that works best for your use case.

Set Up to Use EI

To use EI locally in a notebook instance, create a notebook instance with an EI instance.

To create a notebook instance with an EI instance

  1. Open the Amazon SageMaker console at https://console.amazonaws.cn/sagemaker/.

  2. In the navigation pane, choose Notebook instances.

  3. Choose Create notebook instance.

  4. For Notebook instance name, provide a unique name for your notebook instance.

  5. For notebook instance type, choose a CPU instance such as ml.t2.medium.

  6. For Elastic Inference (EI), choose an instance from the list, such as ml.eia2.medium.

  7. For IAM role, choose an IAM role that has the required permissions to use SageMaker and EI.

  8. (Optional) For VPC - Optional, if you want the notebook instance to use a VPC, choose one from the available list. Otherwise, leave it as No VPC. If you use a VPC follow the instructions at Use a Custom VPC to Connect to EI.

  9. (Optional) For Lifecycle configuration - optional, either leave it as No configuration or choose a lifecycle configuration. For more information, see Customize a Notebook Instance Using a Lifecycle Configuration Script.

  10. (Optional) For Encryption key - optional, Optional) If you want SageMaker to use an Amazon Key Management Service (Amazon KMS) key to encrypt data in the ML storage volume attached to the notebook instance, specify the key.

  11. (Optional) For Volume Size In GB - optional, leave the default value of 5.

  12. (Optional) For Tags, add tags to the notebook instance. A tag is a label you assign to help manage your notebook instances. A tag consists of a key and a value, both of which you define.

  13. Choose Create Notebook Instance.

After you create your notebook instance with EI attached, you can create a Jupyter notebook and set up an EI endpoint that is hosted locally on the notebook instance.

Use EI in Local Mode in SageMaker

To use EI locally in an endpoint hosted on a notebook instance, use local mode with the Amazon SageMaker Python SDK versions of either the TensorFlow, MXNet, or PyTorch estimators or models. For more information about local mode support in the SageMaker Python SDK, see https://github.com/aws/sagemaker-python-sdk#sagemaker-python-sdk-overview.

Use EI in Local Mode with SageMaker TensorFlow Estimators and Models

To use EI with TensorFlow in local mode, specify local for instance_type and local_sagemaker_notebook for accelerator_type when you call the deploy method of an estimator or a model object. For more information about Amazon SageMaker Python SDK TensorFlow estimators and models, see https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/index.html.

The following code shows how to use local mode with an estimator object. To call the deploy method, you must have previously either:

  • Trained the model by calling the fit method of an estimator.

  • Pass a model artifact when you initialize the model object.

# Deploys the model to a local endpoint
tf_predictor = tf_model.deploy(initial_instance_count=1,
                               instance_type='local',
                               accelerator_type='local_sagemaker_notebook')

Use EI in Local Mode with SageMaker Apache MXNet Estimators and Models

To use EI with MXNet in local mode, specify local for instance_type and local_sagemaker_notebook for accelerator_type when you call the deploy method of an estimator or a model object. For more information about Amazon SageMaker Python SDK MXNet estimators and models, see https://sagemaker.readthedocs.io/en/stable/frameworks/mxnet/index.html.

The following code shows how to use local mode with an estimator object. You must have previously called the fit method of the estimator to train the model.

# Deploys the model to a local endpoint
mxnet_predictor = mxnet_estimator.deploy(initial_instance_count=1,
                                         instance_type='local',
                                         accelerator_type='local_sagemaker_notebook')

For a complete example of using EI in local mode with MXNet, see the sample notebook at https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-python-sdk/mxnet_mnist/mxnet_mnist_elastic_inference_local.html .

Use EI in Local Mode with SageMaker PyTorch Estimators and Models

To use EI with PyTorch in local mode, when you call the deploy method of an estimator or a model object, specify local for instance_type and local_sagemaker_notebook for accelerator_type. For more information about Amazon SageMaker Python SDK PyTorch estimators and models, see SageMaker PyTorch Estimators and Models .

The following code shows how to use local mode with an estimator object. You must have previously called the fit method of the estimator to train the model.

# Deploys the model to a local endpoint
pytorch_predictor = pytorch_estimator.deploy(initial_instance_count=1,
                                             instance_type='local',
                                             accelerator_type='local_sagemaker_notebook')