# Real-time vector embedding blueprints
Real-time vector embedding blueprints

Amazon MSK (Managed Streaming for Apache Kafka) supports Amazon Managed Service for Apache Flink blueprints to generate vector-embeddings using Amazon Bedrock, streamlining the process to build real-time AI applications powered by up-to-date, contextual data. The MSF blueprint simplifies the process of incorporating the latest data from your Amazon MSK streaming pipelines into your generative AI models, eliminating the need to write custom code to integrate real-time data streams, vector databases, and large language models.

You can configure the MSF blueprint to continuously generate vector embeddings using Bedrock's embedding models, then index those embeddings in OpenSearch Service for their Amazon MSK data streams. This allows you to combine the context from real-time data with Bedrock's powerful large language models to generate accurate, up-to-date AI responses without writing custom code. You can also choose to improve the efficiency of data retrieval using built-in support for data chunking techniques from LangChain, an open-source library, supporting high-quality inputs for model ingestion. The blueprint manages the data integration and processing between MSK, the chosen embedding model, and the OpenSearch vector store, allowing you to focus on building your AI applications, rather than managing the underlying integration.

Real-time vector embedding blueprints is available in the following Amazon Regions:
+ N. Virginia - us-east-1
+ Ohio - us-east-2
+ Oregon - us-west-2
+ Mumbai - ap-south-1
+ Seoul - ap-northeast-2
+ Singapore - ap-southeast-1
+ Sydney - ap-southeast-2
+ Tokyo - ap-northeast-1
+ Canada Central - ca-central-1
+ Frankfurt - eu-central-1
+ Ireland - eu-west-1
+ London - eu-west-2
+ Paris - eu-west-3
+ Sao Paulo - sa-east-1

**Topics**
+ [

# Logging and observability
](ai-vector-embedding-integration-logging-observability.md)
+ [

# Notes before enabling real-time vector embedding blueprints
](ai-vector-embedding-integration-notes.md)
+ [

# Deploy streaming data vectorization blueprint
](ai-vector-embedding-integration-deploy.md)

# Logging and observability
Logging and observability

All logs and metrics for real-time vector embedding blueprints can be enabled using CloudWatch logs.

All metrics that are available for a regular MSF application and Amazon Bedrock can monitor your [application](https://docs.amazonaws.cn/managed-flink/latest/java/metrics-dimensions.html) and [Bedrock metrics](https://docs.amazonaws.cn/bedrock/latest/userguide/monitoring.html#runtime-cloudwatch-metrics).

There are two additional metrics for monitoring performance of generating embeddings. These metrics are part of the EmbeddingGeneration operation name in CloudWatch.
+ **BedrockTitanEmbeddingTokenCount**: monitors the number of tokens present in a single request to Bedrock.
+ **BedrockEmbeddingGenerationLatencyMs**: reports the time taken to send and receive a response from Bedrock for generating embeddings in milliseconds.

For OpenSearch Service, you can use the following metrics:
+ **OpenSearch Serverless collection metrics**: see [Monitoring OpenSearch Serverless with Amazon CloudWatch](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/monitoring-cloudwatch.html) in the *Amazon OpenSearch Service Developer Guide*.
+ **OpenSearch provisioned metrics**: see [Monitoring OpenSearch cluster metrics with Amazon CloudWatch](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/managedomains-cloudwatchmetrics.html) in the *Amazon OpenSearch Service Developer Guide*.

# Notes before enabling real-time vector embedding blueprints
Notes before enabling real-time vector embedding blueprints

The Managed Service for Apache Flink application will only support unstructured text or JSON data in the input stream.

Two modes of input processing are supported:
+ When input data is **unstructured text**, the entire text message is embedded. The vector DB contains the original text and the generated embedding.
+ When the input data is in **JSON** format, the application gives you the ability to configure and specify one or more keys within the JSON object value to use for the embedding process. If there is more than one key, all keys are vectorized together and indexed in the vector DB. The vector DB will contain the original message and the generated embedding.

**Embedding Generation**: The application supports all text embedding models exclusively provided by Bedrock.

**Persist in vector DB store**: The application uses an existing OpenSearch cluster (provisioned or Serverless) in the customer’s account as a destination for persisting embedded data. When using Opensearch Serverless to create a vector index, always use the vector field name `embedded_data`.

Similar to MSF blueprints, you are expected to manage the infrastructure to run the code associated with the real-time vector embedding blueprint.

Similar to MSF Blueprints, once an MSF application is created, it must be exclusively started in the Amazon account using the console or CLI. Amazon will not start the MSF application for you. You have to call the StartApplication API (through CLI or console) to get the application running.

**Cross-account movement of data**: The application does not allow you to move data between input stream and vector destinations that live in different Amazon accounts.

# Deploy streaming data vectorization blueprint
Deploy streaming data vectorization blueprint

This topic describes how to deploy a streaming data vectorization blueprint.

**Deploy streaming data vectorization blueprint**

1. Ensure following resources are setup correctly:

   1. Provisioned or Serverless MSK cluster with one or more topics containing data.

1. Bedrock Setup: [Access to desired Bedrock Model](https://docs.amazonaws.cn/bedrock/latest/userguide/model-access.html). Currently supported Bedrock models are:
   + Amazon Titan Embeddings G1 - Text
   + Amazon Titan Text Embeddings V2
   + Amazon Titan Multimodal Embeddings G1
   + Cohere Embed English
   + Cohere Embed Multilingual

1. Amazon OpenSearch collection:
   + You may use a provisioned or Serverless OpenSearch Service collection.
   + The OpenSearch Service collection must have at least one index.
   + If you plan to use an **OpenSearch Serverless collection**, make sure to create a vector search collection. For details on how to setup a vector index, see [Prerequisites for your own vector store for a knowledge base](https://docs.amazonaws.cn/bedrock/latest/userguide/knowledge-base-setup.html). To learn more about vectorization, see [Amazon OpenSearch Service’s vector database capabilities explained](https://amazonaws-china.com/blogs/big-data/amazon-opensearch-services-vector-database-capabilities-explained/).
**Note**  
When creating a vector index, you must use the vector field name `embedded_data`.
   + If you plan to use an **OpenSearch Provisioned collection**, you need to add the MSF application role (that contains the Opensearch access policy) that was created by the blueprint, as a master user to your OpenSearch collection. Also, confirm that the access policy in OpenSearch is set to "Allow" actions. This is needed to [enable fine grain access control](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/fgac.html#fgac-enabling).
   + Optionally, you can enable access to the OpenSearch dashboard to view results. Refer to [enable fine grain access control](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/fgac.html#fgac-enabling).

1. Login using a role that allows [aws:CreateStack](https://docs.amazonaws.cn/systems-manager/latest/userguide/automation-action-createstack.html) permissions.

1. Go to the MSF console dashboard and select **Create Streaming Application**.

1. In **Choose a method to setup the stream processing application** select **Use a Blueprint**.

1. Select **Real-time AI application blueprint** from the blueprints drop-down menu.

1. Provide desired configurations. See [Create page configurations](#ai-vector-embedding-integration-create-page-configs).

1. Select **Deploy Blueprint** to start a CloudFormation deployment.

1. Once the CloudFormation deployment is complete, go to the deployed Flink application. Check Runtime properties of the application.

1. You can choose to change/add runtime properties to your application. See [Runtime Properties Configuration](https://docs.amazonaws.cn/managed-flink/latest/java/troubleshooting-blueprints.html) for details to configure these properties.
**Note**  
Note:  
If you are using OpenSearch provisioned, please ensure you enabled [fine grain access control](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/fgac.html#fgac-enabling).  
If your provisioned cluster is private, add `https://` to your OpenSearch Provisioned VPC endpoint URL and change `sink.os.endpoint` to point to this endpoint.  
If your provisioned cluster is public, ensure your MSF application can access the internet. For more information, see [>>>>>> express-brokers-publication-merge type="documentation" url="managed-flink/latest/java/vpc-internet.html" >Internet and service access for a VPC-connected Managed Service for Apache Flink application]().

1. Once you are satisfied with all the configurations, select `Run`. The application will start running.

1. Pump messages in your MSK cluster.

1. Navigate to the Opensearch cluster and go to the OpenSearch dashboard.

1. On the dashboard, select **Discover** in the left menu. You should see persisted documents along with their vector embeddings.

1. Refer to [Working with vector search collections](https://docs.amazonaws.cn/opensearch-service/latest/developerguide/serverless-vector-search.html) to see how you can use the vectors stored in the index.

## Create page configurations
Create page configurations

This topic describes create page configurations to refer to when specifying configurations for real-time AI application blueprints.

**Application name**  
Existing field in MSF, give any name to your application.

**MSK cluster**  
Select the MSK cluster you created during setup from the dropdown list.

**Topics**  
Add the name of the topic(s) which you created in the setup.

**Input stream data type**  
Choose **String** if you will supply string input to the MSK stream.  
Choose **JSON** if the input in the MSK stream is JSON. In **JSON keys embedded**, write the names of the fields in your input JSON whose value you want to send to Bedrock for generating embeddings.

**Bedrock embedding model**  
Select one from the list. Ensure that you have model access for the model you choose, otherwise the stack might fail. See [Add or remove access to Amazon Bedrock foundation models](https://docs.amazonaws.cn/bedrock/latest/userguide/model-access-modify.html).

**OpenSearch cluster**  
Select the cluster you created from the dropdown.

**OpenSearch vector index name**  
Select the vector index that you created in the above step.