Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Perform machine learning inference

With Amazon IoT Greengrass, you can perform machine learning (ML) inference on your edge devices on locally generated data using cloud-trained models. You benefit from the low latency and cost savings of running local inference, yet still take advantage of cloud computing power for training models and complex processing.

Amazon IoT Greengrass makes the steps required to perform inference more efficient. You can train your inference models anywhere and deploy them locally as machine learning components. For example, you can build and train deep-learning models in Amazon SageMaker or computer vision models in Amazon Lookout for Vision. Then, you can store these models in an Amazon S3 bucket, so you can use these models as artifacts in your components to perform inference on your core devices.

How Amazon IoT Greengrass ML inference works

Amazon provides machine learning components that you can use to create one-step deployments to perform machine learning inference on your device. You can also use these components as templates to create custom components to meet your specific requirements.

Each deployment that you create to perform machine learning inference consists of at least one component that runs your inference application, installs the machine learning framework, and downloads your machine learning models. To perform sample inference with Amazon-provided components, you deploy an inference component to your core device, which automatically includes the corresponding model and runtime components as dependencies. To customize your deployments, you can plug in or swap out the sample model components with custom model components, or you can use the component recipes for the Amazon-provided components as templates to create your own custom inference, model, and runtime components.

To perform machine learning inference by using custom components:

To get started with Amazon-provided components, see Tutorial: Perform sample image classification inference using TensorFlow Lite.

For information about creating custom machine learning components, see Customize your machine learning components.

What's different in Amazon IoT Greengrass Version 2?

Amazon IoT Greengrass consolidates functional units for machine learning—such as models, runtimes, and inference code— into components that enable you to use a one-step process to install the machine learning runtime, download your trained models, and perform inference on your device.

By using the Amazon-provided machine learning components, you have the flexibility to start performing machine learning inference with sample inference code and pre-trained models. You can plug in custom model components to use your own custom-trained models with the inference and runtime components that Amazon provides. For a completely customized machine learning solution, you can use the public components as templates to create custom components and use any runtime, model, or inference type that you want.

Requirements

To create and use machine learning components, you must have the following:

Supported model sources

Amazon IoT Greengrass supports using custom-trained machine learning models that are stored in Amazon S3. You can also use Amazon SageMaker edge packaging jobs to directly create model components for your SageMaker Neo-compiled models. For information about using SageMaker Edge Manager with Amazon IoT Greengrass, see Use Amazon SageMaker Edge Manager on Greengrass core devices. You can also use Amazon Lookout for Vision model packaging jobs to create model components for your Lookout for Vision models. For more information about using Lookout for Vision with Amazon IoT Greengrass, see Use Amazon Lookout for Vision on Greengrass core devices.

The S3 buckets that contain your models must meet the following requirements:

Supported machine learning runtimes

Amazon IoT Greengrass enables you to create custom components to use any machine learning runtime of your choice to perform machine learning inference with your custom-trained models. For information about creating custom machine learning components, see Customize your machine learning components.

To make the process of getting started with machine learning more efficient, Amazon IoT Greengrass provides sample inference, model, and runtime components that use the following machine learning runtimes:

Amazon-provided machine learning components

The following table lists the Amazon-provided components used for machine learning.