Exporting application metrics to Amazon Managed Service for Prometheus - Amazon Elastic Container Service
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Exporting application metrics to Amazon Managed Service for Prometheus

The Amazon Distro for OpenTelemetry (ADOT) metrics collection feature is in preview. The preview is open to all Amazon accounts. Features may be added or changed before announcing General Availability.

Amazon ECS supports exporting your task-level CPU, memory, network, and storage metrics and your custom application metrics to Amazon Managed Service for Prometheus. This is done by adding the Amazon Distro for OpenTelemetry sidecar container to your task definition. The new Amazon ECS console experience simplifies this process by adding the Use metric collection option when creating a new task definition. For more information, see Creating a task definition using the new console.

The metrics are exported to Amazon Managed Service for Prometheus and can be viewed using the Amazon Managed Grafana dashboard. Your application must be instrumented with either Prometheus libraries or with the OpenTelemetry SDK. For more information about instrumenting your application with the OpenTelemetry SDK, see Introduction to Amazon Distro for OpenTelemetry in the Amazon Distro for OpenTelemetry documentation.

When using the Prometheus libraries, your application must expose a /metrics endpoint which is used to scrape the metrics data. For more information about instrumenting your application with Prometheus libraries, see Prometheus client libraries in the Prometheus documentation.

Considerations

The following should be considered when using the Amazon ECS on Fargate integration with Amazon Distro for OpenTelemetry to send application metrics to Amazon CloudWatch.

  • The Amazon Distro for OpenTelemetry integration is only supported for Amazon ECS workloads hosted on Fargate. Amazon ECS workloads hosted on Amazon EC2 instances or external instances aren't supported currently.

  • Amazon ECS doesn't send any task-level metrics to CloudWatch Container Insights. You can enable Container Insights at the Amazon ECS cluster level to receive those metrics. For more information, see Amazon ECS CloudWatch Container Insights.

  • By default, Amazon Distro for OpenTelemetry includes all available task-level dimensions for your application metrics when exporting to Amazon Managed Service for Prometheus. You can also instrument your application to add additional dimensions. For more information, see Getting Started with Prometheus Remote Write Exporter for Amazon Managed Service for Prometheus in the Amazon Distro for OpenTelemetry documentation.

Required IAM permissions for Amazon Distro for OpenTelemetry integration with Amazon Managed Service for Prometheus

The Amazon ECS integration with Amazon Managed Service for Prometheus using the Amazon Distro for OpenTelemetry sidecar requires that you create a task IAM role and specify the role in your task definition. This task IAM role must be created manually using the steps below prior to registering your task definition.

We recommend that the Amazon Distro for OpenTelemetry sidecar also be configured to route container logs to CloudWatch Logs which requires a task execution IAM role be created and specified in your task definition as well. The new Amazon ECS console experience takes care of the task execution IAM role on your behalf, but the task IAM role must be created manually. For more information about creating a task execution IAM role, see Amazon ECS task execution IAM role.

Important

If you're also collecting application trace data using the Amazon Distro for OpenTelemetry integration, ensure your task IAM role also contains the permissions necessary for that integration. For more information, see Collecting application trace data.

To create a task IAM role for Amazon Distro for OpenTelemetry integration with Amazon Managed Service for Prometheus

  1. Open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Roles, Create role.

  3. In the Select type of trusted entity section, choose Amazon service, Elastic Container Service.

  4. For Select your use case, choose Elastic Container Service Task, then choose Next: Permissions.

  5. In the Attach permissions policy section, search for the AmazonPrometheusRemoteWriteAccess policy, select the policy, and then choose Next: Tags.

  6. For Add tags (optional), specify any custom tags to associate with the policy and then choose Next: Review.

  7. For Role name, specify AmazonECS_OpenTelemetryPrometheusRole and choose Create role.

Specifying the Amazon Distro for OpenTelemetry sidecar in your task definition

The new Amazon ECS console experience simplifies the experience of creating the Amazon Distro for OpenTelemetry sidecar container by using the Use metric collection option. For more information, see Creating a task definition using the new console.

If you're not using the Amazon ECS console, you can add the Amazon Distro for OpenTelemetry sidecar container to your task definition manually. The following task definition example shows the container definition for adding the Amazon Distro for OpenTelemetry sidecar for Amazon Managed Service for Prometheus integration.

{ "family": "otel-using-cloudwatch", "taskRoleArn": "arn:aws:iam::111122223333:role/AmazonECS_OpenTelemetryCloudWatchRole", "executionRoleArn": "arn:aws:iam::111122223333:role/ecsTaskExecutionRole", "containerDefinitions": [{ "name": "aws-otel-emitter", "image": "application-image", "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "/ecs/aws-otel-emitter", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "ecs" } }, "dependsOn": [{ "containerName": "aws-otel-collector", "condition": "START" }] }, { "name": "aws-otel-collector", "image": "public.ecr.aws/aws-observability/aws-otel-collector:v0.17.0", "essential": true, "command": [ "--config=/etc/ecs/ecs-cloudwatch.yaml" ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "True", "awslogs-group": "/ecs/ecs-aws-otel-sidecar-collector", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "ecs" } } } ], "networkMode": "awsvpc", "requiresCompatibilities": [ "FARGATE" ], "cpu": "1024", "memory": "3072" }