Use Amazon SageMaker AI Edge Manager on Greengrass core devices - Amazon IoT Greengrass
How it worksRequirementsGet started with SageMaker AI Edge Manager
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Use Amazon SageMaker AI Edge Manager on Greengrass core devices

Important

SageMaker AI Edge Manager was discontinued on April 26th, 2024. For more information about continuing to deploy your models to edge devices, see SageMaker AI Edge Manager end of life.

Amazon SageMaker AI Edge Manager is a software agent that runs on edge devices. SageMaker AI Edge Manager provides model management for edge devices so that you can package and use Amazon SageMaker AI Neo-compiled models directly on Greengrass core devices. By using SageMaker AI Edge Manager, you can also sample model input and output data from your core devices, and send that data to the Amazon Web Services Cloud for monitoring and analysis. Because SageMaker AI Edge Manager uses SageMaker AI Neo to optimize your models for your target hardware, you don't need to install the DLR runtime directly on your device. On Greengrass devices, SageMaker AI Edge Manager doesn't load local Amazon IoT certificates or call the Amazon IoT credential provider endpoint directly. Instead, SageMaker AI Edge Manager uses the token exchange service to fetch temporary credential from a TES endpoint.

This section describes how SageMaker AI Edge Manager works on Greengrass core devices.

How SageMaker AI Edge Manager works on Greengrass devices

To deploy the SageMaker AI Edge Manager agent to your core devices, create a deployment that includes the aws.greengrass.SageMakerEdgeManager component. Amazon IoT Greengrass manages the installation and lifecycle of the Edge Manager agent on your devices. When a new version of the agent binary is available, deploy the updated version of the aws.greengrass.SageMakerEdgeManager component to upgrade the version of the agent that is installed on your device.

When you use SageMaker AI Edge Manager with Amazon IoT Greengrass, your workflow includes the following high-level steps:

  1. Compile models with SageMaker AI Neo.

  2. Package your SageMaker AI Neo-compiled models using SageMaker AI edge packaging jobs. When you run an edge packaging job for your model, you can choose to create a model component with the packaged model as an artifact that can be deployed to your Greengrass core device.

  3. Create a custom inference component. You use this inference component to interact with the Edge Manager agent to perform inference on the core device. These operations include loading models, invoke prediction requests to run inference, and unloading models when the component shuts down.

  4. Deploy the SageMaker AI Edge Manager component, the packaged model component, and the inference component to run your model on the SageMaker AI inference engine (Edge Manager agent) on your device.

For more information about creating edge packaging jobs and inference components that work with SageMaker AI Edge Manager, see Deploy Model Package and Edge Manager Agent with Amazon IoT Greengrass in the Amazon SageMaker AI Developer Guide.

The Tutorial: Get started with SageMaker AI Edge Manager tutorial shows you how to set up and use the SageMaker AI Edge Manager agent on an existing Greengrass core device, using Amazon-provided example code that you can use to create sample inference and model components.

When you use SageMaker AI Edge Manager on Greengrass core devices, you can also use the capture data feature to upload sample data to the Amazon Web Services Cloud. Capture data is a SageMaker AI feature that you use to upload inference input, inference results, and additional inference data to an S3 bucket or a local directory for future analysis. For more information about using capture data with SageMaker AI Edge Manager, see Manage Model in the Amazon SageMaker AI Developer Guide.

Requirements

You must meet the following requirements to use the SageMaker AI Edge Manager agent on Greengrass core devices.

  • A Greengrass core device running on Amazon Linux 2, a Debian-based Linux platform (x86_64 or Armv8), or Windows (x86_64). If you don't have one, see Tutorial: Getting started with Amazon IoT Greengrass V2.

  • Python 3.6 or later, including pip for your version of Python, installed on your core device.

  • The Greengrass device role configured with the following:

    • A trust relationship that allows credentials.iot.amazonaws.com and sagemaker.amazonaws.com to assume the role, as shown in the following IAM policy example.

      { 
  "Version": "2012-10-17",
  "Statement": [ 
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "credentials.iot.amazonaws.com"
       }, 
      "Action": "sts:AssumeRole" 
    },
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "sagemaker.amazonaws.com"
      }, 
      "Action": "sts:AssumeRole" 
    } 
  ] 
}

    • The AmazonSageMakerEdgeDeviceFleetPolicy IAM managed policy.

    • The s3:PutObject action, as shown in the following IAM policy example.

      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "*"
      ],
      "Effect": "Allow"
    }
  ]
}

  • An Amazon S3 bucket created in the same Amazon Web Services account and Amazon Web Services Region as your Greengrass core device. SageMaker AI Edge Manager requires an S3 bucket to create an edge device fleet, and to store sample data from running inference on your device. For information about creating S3 buckets, see Getting started with Amazon S3.

  • A SageMaker AI edge device fleet that uses the same Amazon IoT role alias as your Greengrass core device. For more information, see Create an edge device fleet.

  • Your Greengrass core device registered as an edge device in your SageMaker AI Edge device fleet. The edge device name must match the Amazon IoT thing name for your core device. For more information, see Register your Greengrass core device.

Get started with SageMaker AI Edge Manager

You can complete a tutorial to get started using SageMaker AI Edge Manager. The tutorial shows you how to get started using SageMaker AI Edge Manager with Amazon-provided sample components on an existing core device. These sample components use the SageMaker AI Edge Manager component as a dependency to deploy the Edge Manager agent, and perform inference using pre-trained models that were compiled using SageMaker AI Neo. For more information, see Tutorial: Get started with SageMaker AI Edge Manager.