Deploying the Amazon Distro for OpenTelemetry to collect EC2 instance-level metrics on Amazon ECS clusters - Amazon CloudWatch
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Deploying the Amazon Distro for OpenTelemetry to collect EC2 instance-level metrics on Amazon ECS clusters

Use the steps in this section to use Amazon Distro for OpenTelemetry to collect EC2 instance-level metrics on an Amazon ECS cluster. For more information about the Amazon Distro for OpenTelemetry, see Amazon Distro for OpenTelemetry.

These steps assume that you already have a cluster running Amazon ECS. This cluster must be deployed with the EC2 launch type. For more information about using Amazon Distro for Open Telemetry with Amazon ECS and setting up an Amazon ECS cluster for this purpose, see Setting up Amazon Distro for OpenTelemetry Collector in Amazon Elastic Container Service for ECS EC2 instance level metrics.

Quick setup using Amazon CloudFormation

Download the Amazon CloudFormation template file for installing the Amazon Distro for OpenTelemetry collector for Amazon ECS on EC2. Run the following curl command.

curl -O https://raw.githubusercontent.com/aws-observability/aws-otel-collector/main/deployment-template/ecs/aws-otel-ec2-instance-metrics-daemon-deployment-cfn.yaml

After you download the template file, open it and replace PATH_TO_CloudFormation_TEMPLATE with the path where you saved the template file. Then export the following parameters and run the Amazon CloudFormation command, as shown in the following command.

  • Cluster_Name– The Amazon ECS cluster name

  • AWS_Region– The Region where the data will be sent

  • PATH_TO_CloudFormation_TEMPLATE– The path where you saved the Amazon CloudFormation template file.

  • command– To enable the Amazon Distro for OpenTelemetry collector to collect the instance-level metrics for Amazon ECS on Amazon EC2, you must specify --config=/etc/ecs/otel-instance-metrics-config.yaml for this parameter.

ClusterName=Cluster_Name Region=AWS_Region command=--config=/etc/ecs/otel-instance-metrics-config.yaml aws cloudformation create-stack --stack-name AOCECS-${ClusterName}-${Region} \ --template-body file://PATH_TO_CloudFormation_TEMPLATE \ --parameters ParameterKey=ClusterName,ParameterValue=${ClusterName} \ ParameterKey=CreateIAMRoles,ParameterValue=True \ ParameterKey=command,ParameterValue=${command} \ --capabilities CAPABILITY_NAMED_IAM \ --region ${Region}

After running this command, use the Amazon ECS console to see if the task is running.

Troubleshooting the quick setup

To check the status of the Amazon CloudFormation stack, enter the following command.

ClusterName=cluster-name Region=cluster-region aws cloudformation describe-stack --stack-name AOCECS-$ClusterName-$Region --region $Region

If the value of StackStatus is anything other than CREATE_COMPLETE or CREATE_IN_PROGRESS, check the stack events to find the error. Enter the following command.

ClusterName=cluster-name Region=cluster-region aws cloudformation describe-stack-events --stack-name AOCECS-$ClusterName-$Region --region $Region

To check the status of the AOCECS daemon service, enter the following command. In the output, you should see that runningCount is equal to the desiredCount in the deployment section. If it isn't equal, check the failures section in the output.

ClusterName=cluster-name Region=cluster-region aws ecs describe-services --services AOCECS-daemon-service --cluster $ClusterName --region $Region

You can also use the CloudWatch Logs console to check the agent log. Look for the /aws/ecs/containerinsights/{ClusterName}/performance log group.

Manual and custom setup

Follow the steps in this section to manually deploy the Amazon Distro for OpenTelemetry to collect instance-level metrics from your Amazon ECS clusters that are hosted on Amazon EC2 instances.

Step 1: Necessary roles and policies

Two IAM roles are required. You must create them if they don't already exist. For more information about these roles, see Create IAM policy and Create IAM role.

Step 2: Create the task definition

Create a task definition and use it to launch the Amazon Distro for OpenTelemetry as a daemon service.

To use the task definition template to create the task definition, follow the instructions in Create ECS EC2 Task Definition for EC2 instance with Amazon OTel Collector.

To use the Amazon ECS console to create the task definition, follow the instructions in Install Amazon OTel Collector by creating Task Definition through Amazon console for Amazon ECS EC2 instance metrics.

Step 3: Launch the daemon service

To launch the Amazon Distro for OpenTelemetry as a daemon service, follow the instructions in Run your task on the Amazon Elastic Container Service (Amazon ECS) using daemon service.

(Optional) Advanced configuration

Optionally, you can use SSM to specify other configuration options for the Amazon Distro for OpenTelemetry in your Amazon ECS clusters that are hosted on Amazon EC2 instances. For more information, about creating a configuration file, see Custom OpenTelemetry Configuration. For more information about the options that you can use in the configuration file, see Amazon Container Insights Receiver.