Using Amazon CloudFormation to set up remote inference for semantic search - Amazon OpenSearch Service
Available Amazon CloudFormation templatesPrerequisites
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Using Amazon CloudFormation to set up remote inference for semantic search

Starting with OpenSearch version 2.9, you can use remote inference with semantic search to host your own machine learning (ML) models. Remote inference uses the ML Commons plugin.

With Remote inference, you can host your model inferences remotely on ML services, such as Amazon SageMaker AI and Amazon Bedrock, and connect them to Amazon OpenSearch Service with ML connectors.

To ease the setup of remote inference, Amazon OpenSearch Service provides an Amazon CloudFormation template in the console. CloudFormation is an Amazon Web Services service where you can, provision, and manage Amazon and third-party resources by treating infrastructure as code.

The OpenSearch CloudFormation template automates the model provisioning process for you, so that you can easily create a model in your OpenSearch Service domain and then use the model ID to ingest data and run neural search queries.

When you use neural sparse encoders with OpenSearch Service version 2.12 and onwards, we recommend that you use the tokenizer model locally instead of deploying remotely. For more information, see Sparse encoding models in the OpenSearch documentation.

Topics

Available Amazon CloudFormation templates

The following Amazon CloudFormation machine learning (ML) templates are available for use:

Amazon Bedrock templates
Amazon Titan Text Embeddings Integration

Connects to Amazon Bedrock's hosted ML models, eliminates the need for separate model deployment, and uses predetermined Amazon Bedrock endpoints. For more information, see Amazon Titan Text Embeddings in the Amazon Bedrock User Guide.

Cohere Embed Integration

Provides access to Cohere Embed models, and is optimized for specific text processing workflows. For more information, see Embed on the Cohere docs website.

Amazon Titan Multimodal Embeddings

Supports both text and image embeddings, and enables multimodal search capabilities. For more information, see Amazon Titan Multimodal Embeddings in the Amazon Bedrock User Guide.

MCP server integration templates
MCP server integration

Deploys an Amazon Bedrock AgentCore Runtime, provides an agent endpoint, handles inbound and outbound authentication, and supports OAuth for enterprise authentication.

Amazon SageMaker templates
Integration with text embedding models through Amazon SageMaker

Deploys text embedding models in Amazon SageMaker Runtime, creates IAM roles for model artifact access, and establishes ML connectors for semantic search.

Integration with Sparse Encoders through SageMaker

Sets up sparse encoding models for neural search, creates Amazon Lambda functions for connector management, and returns model IDs for immediate use.

Prerequisites

To use a CloudFormation template with OpenSearch Service, complete the following prerequisites.

Set up an OpenSearch Service domain

Before you can use a CloudFormation template, you must set up an Amazon OpenSearch Service domain with version 2.9 or later and fine-grained access control enabled. Create an OpenSearch Service backend role to give the ML Commons plugin permission to create your connector for you.

The CloudFormation template creates a Lambda IAM role for you with the default name LambdaInvokeOpenSearchMLCommonsRole, which you can override if you want to choose a different name. After the template creates this IAM role, you need to give the Lambda function permission to call your OpenSearch Service domain. To do so, map the role named ml_full_access to your OpenSearch Service backend role with the following steps:

  1. Navigate to the OpenSearch Dashboards plugin for your OpenSearch Service domain. You can find the Dashboards endpoint on your domain dashboard on the OpenSearch Service console.

  2. From the main menu choose Security, Roles, and select the ml_full_access role.

  3. Choose Mapped users, Manage mapping.

  4. Under Backend roles, add the ARN of the Lambda role that needs permission to call your domain.

    arn:aws:iam::account-id:role/role-name

  5. Select Map and confirm the user or role shows up under Mapped users.

After you've mapped the role, navigate to the security configuration of your domain and add the Lambda IAM role to your OpenSearch Service access policy.

Enable permissions on your Amazon Web Services account

Your Amazon Web Services account must have permission to access CloudFormation and Lambda, along with whichever Amazon Web Services service you choose for your template – either SageMaker Runtime or Amazon Bedrock.

If you're using Amazon Bedrock, you must also register your model. See Model access in the Amazon Bedrock User Guide to register your model.

If you're using your own Amazon S3 bucket to provide model artifacts, you must add the CloudFormation IAM role to your S3 access policy. For more information, see Adding and removing IAM identity permissions in the IAM User Guide.