View a markdown version of this page

Prerequisites - Amazon Bedrock AgentCore
SetupConsole and API operations
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Prerequisites

Before you run on-demand evaluations, make sure the following are in place.

Setup

Set up the following before you run on-demand evaluations:

  • An agent built with a supported framework and instrumentation library. For more information about supported frameworks and instrumentation libraries, see Supported agent frameworks.

  • An agent deployed on AgentCore Runtime with observability enabled, or an agent built with a supported framework configured with AgentCore Observability, including Transaction Search. For more information about telemetry setup, see Telemetry setup and delivery.

  • An agent invoked with telemetry data in CloudWatch Logs. Wait 2–5 minutes for CloudWatch to ingest the telemetry before starting an on-demand evaluation.

Your IAM user or role also needs the following permissions to run on-demand evaluations:

Console and API operations

{
"Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock-agentcore:Evaluate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:Converse",
                "bedrock:InvokeModelWithResponseStream",
                "bedrock:ConverseStream"
            ],
            "Resource": [
                "arn:aws:bedrock:*::foundation-model/*",
                "arn:aws:bedrock:*:*:inference-profile/*"
            ]
        },
        {
            "Sid": "LambdaInvokeForCodeBasedEvaluators",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "lambda:GetFunction"
            ],
            "Resource": "arn:aws:lambda:*:*:function:*"
        }
    ]
}
Note

The Lambda permissions are only required if you use Custom code-based evaluator. You can scope the Lambda resource ARN to specific functions as needed.