Use a custom setup to enable Application Signals on Amazon ECS - Amazon CloudWatch
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Use a custom setup to enable Application Signals on Amazon ECS

Use these custom setup instructions to onboard your applications on Amazon ECS to CloudWatch Application Signals. You install and configure the CloudWatch agent and Amazon Distro for OpenTelemetry yourself.

Important

Only the awsvpc network mode is supported. Both the EC2 and Fargate launch types are supported.

On Amazon ECS clusters, Application Signals doesn't autodiscover the names of your services. You must specify your service names during the custom setup, and the names that you specify are what is displayed on Application Signals dashboards.

Step 1: Enable Application Signals in your account

If you haven't enabled Application Signals in this account yet, you must grant Application Signals the permissions it needs to discover your services. To do so, do the following. You need to do this only once for your account.

To enable Application Signals for your applications
  1. Open the CloudWatch console at https://console.amazonaws.cn/cloudwatch/.

  2. In the navigation pane, choose Services.

  3. Choose Start discovering your Services.

  4. Select the check box and choose Start discovering Services.

    Completing this step for the first time in your account creates the AWSServiceRoleForCloudWatchApplicationSignals service-linked role. This role grants Application Signals the following permissions:

    • xray:GetServiceGraph

    • logs:StartQuery

    • logs:GetQueryResults

    • cloudwatch:GetMetricData

    • cloudwatch:ListMetrics

    • tag:GetResources

    For more information about this role, see Service-linked role permissions for CloudWatch Application Signals.

Step 2: Create IAM roles

You must create an IAM role. If you already have created this role, you might need to add permissions to it.

  • ECS task role— Containers use this role to run. The permissions should be whatever your applications need, plus CloudWatchAgentServerPolicy.

For more information about creating IAM roles, see Creating IAM Roles.

Step 3: Prepare CloudWatch agent configuration

First, prepare the agent configuration with Application Signals enabled. To do this, create a local file named /tmp/ecs-cwagent.json.

{ "traces": { "traces_collected": { "application_signals": {} } }, "logs": { "metrics_collected": { "application_signals": {} } } }

Then upload this configuration to the SSM Parameter Store. To do this, enter the following command. In the file, replace $REGION with your actual Region name.

aws ssm put-parameter \ --name "ecs-cwagent" \ --type "String" \ --value "`cat /tmp/ecs-cwagent.json`" \ --region "$REGION"

Step 4: Instrument your application with the CloudWatch agent

The next step is to instrument your application for CloudWatch Application Signals.

Java
To instrument your application on Amazon ECS with the CloudWatch agent
  1. First, specify a bind mount. The volume will be used to share files across containers in the next steps. You will use this bind mount later in this procedure.

    "volumes": [ { "name": "opentelemetry-auto-instrumentation" } ]
  2. Add a CloudWatch agent sidecar definition. To do this, append a new container called ecs-cwagent to your application's task definition. Replace $REGION with your actual Region name. Replace $IMAGEwith the path to the latest CloudWatch container image on Amazon Elastic Container Registry. For more information, see cloudwatch-agent on Amazon ECR.

    { "name": "ecs-cwagent", "image": "$IMAGE", "essential": true, "secrets": [ { "name": "CW_CONFIG_CONTENT", "valueFrom": "ecs-cwagent" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "/ecs/ecs-cwagent", "awslogs-region": "$REGION", "awslogs-stream-prefix": "ecs" } } }
  3. Append a new container init to your application's task definition. Replace $IMAGE with the latest image from the Amazon Distro for OpenTelemetry Amazon ECR image repository.

    { "name": "init", "image": "$IMAGE", "essential": false, "command": [ "cp", "/javaagent.jar", "/otel-auto-instrumentation/javaagent.jar" ], "mountPoints": [ { "sourceVolume": "opentelemetry-auto-instrumentation", "containerPath": "/otel-auto-instrumentation", "readOnly": false } ] }
  4. Add the following environment variables to your application container. You must be using version 1.32.2 or later of the Amazon Distro for OpenTelemetry auto-instrumentation agent for Java

    Environment variable Setting to enable Application Signals

    OTEL_RESOURCE_ATTRIBUTES

    Specify the following information as key-value pairs:

    • service.name sets the name of the service. This will be diplayed as the service name for your application in Application Signals dashboards. If you don't provide a value for this key, the default of UnknownService is used.

    • deployment.environment sets the environment that the application runs in. This will be diplayed as the Hosted In environment of your application in Application Signals dashboards. If you don't specify this, the default of generic:default is used.

    This attribute key is used only by Application Signals, and is converted into X-Ray trace annotations and CloudWatch metric dimensions.

    (Optional) To enable trace log correlation, set an additional environment variable aws.log.group.names to be the log group name for your application log. By doing so, the traces from your application can be correlated with the relevant log entries from this specific log group. For this variable, replace $YOUR_APPLICATION_LOG_GROUP with the log group name for your application. You'll also need to change the logging configuration in your application. For more information, see Enable trace log correlation.

    OTEL_AWS_APPLICATION_SIGNALS_ENABLED

    Set to true to have your container start sending X-Ray traces and CloudWatch metrics to Application Signals.

    OTEL_METRICS_EXPORTER

    Set to none to disable other metrics exporters.

    OTEL_LOGS_EXPORTER

    Set to none to disable other logs exporters.

    OTEL_EXPORTER_OTLP_PROTOCOL

    Set to http/protobuf to send metrics and traces to Application Signals using HTTP.

    OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT

    Set to http://localhost:4316/v1/metrics to send metrics to the CloudWatch sidecar.

    OTEL_EXPORTER_OTLP_TRACES_ENDPOINT

    Set to http://localhost:4316/v1/traces to send traces to the CloudWatch sidecar.

    OTEL_TRACES_SAMPLER

    Set this to xray to set X-Ray as the traces sampler.

    OTEL_PROPAGATORS

    Set xray as one of the propagators.

    JAVA_TOOL_OPTIONS

    Set to " -javaagent:$AWS_ADOT_JAVA_INSTRUMENTATION_PATH" Replace AWS_ADOT_JAVA_INSTRUMENTATION_PATH with the path where the Amazon Distro for OpenTelemetry Java auto-instrumentation agent is stored. For example, /otel-auto-instrumentation/javaagent.jar

  5. Mount the volume opentelemetry-auto-instrumentation that you defined in step 1 of this procedure.

    For a Java application, use the following.

    { "name": "my-app", ... "environment": [ { "name": "OTEL_RESOURCE_ATTRIBUTES", "value": "aws.log.group.names=$YOUR_APPLICATION_LOG_GROUP,service.name=$SVC_NAME" }, { "name": "OTEL_LOGS_EXPORTER", "value": "none" }, { "name": "OTEL_METRICS_EXPORTER", "value": "none" }, { "name": "OTEL_EXPORTER_OTLP_PROTOCOL", "value": "http/protobuf" }, { "name": "OTEL_AWS_APPLICATION_SIGNALS_ENABLED", "value": "true" }, { "name": "JAVA_TOOL_OPTIONS", "value": " -javaagent:/otel-auto-instrumentation/javaagent.jar" }, { "name": "OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT", "value": "http://localhost:4316/v1/metrics" }, { "name": "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "value": "http://localhost:4316/v1/traces" }, { "name": "OTEL_TRACES_SAMPLER", "value": "xray" }, { "name": "OTEL_PROPAGATORS", "value": "tracecontext,baggage,b3,xray" } ], "mountPoints": [ { "sourceVolume": "opentelemetry-auto-instrumentation", "containerPath": "/otel-auto-instrumentation", "readOnly": false } ] }
Python

Before you enable Application Signals for your Python applications, be aware of the following considerations.

  • In some containerized applications, a missing PYTHONPATH environment variable can sometimes cause the application to fail to start. To resolve this, ensure that you set the PYTHONPATH environment variable to the location of your application’s working directory. This is due to a known issue with OpenTelemetry auto-instrumentation. For more information about this issue, see Python autoinstrumentation setting of PYTHONPATH is not compliant.

  • For Django applications, there are additional required configurations, which are outlined in the OpenTelemetry Python documentation.

    • Use the --noreload flag to prevent automatic reloading.

    • Set the DJANGO_SETTINGS_MODULE environment variable to the location of your Django application’s settings.py file. This ensures that OpenTelemetry can correctly access and integrate with your Django settings.

To instrument your Python application on Amazon ECS with the CloudWatch agent
  1. First, specify a bind mount. The volume will be used to share files across containers in the next steps. You will use this bind mount later in this procedure.

    "volumes": [ { "name": "opentelemetry-auto-instrumentation-python" } ]
  2. Add a CloudWatch agent sidecar definition. To do this, append a new container called ecs-cwagent to your application's task definition. Replace $REGION with your actual Region name. Replace $IMAGEwith the path to the latest CloudWatch container image on Amazon Elastic Container Registry. For more information, see cloudwatch-agent on Amazon ECR.

    { "name": "ecs-cwagent", "image": "$IMAGE", "essential": true, "secrets": [ { "name": "CW_CONFIG_CONTENT", "valueFrom": "ecs-cwagent" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "/ecs/ecs-cwagent", "awslogs-region": "$REGION", "awslogs-stream-prefix": "ecs" } } }
  3. Append a new container init to your application's task definition. Replace $IMAGE with the latest image from the Amazon Distro for OpenTelemetry Amazon ECR image repository.

    { "name": "init", "image": "$IMAGE", "essential": false, "command": [ "cp", "-a", "/autoinstrumentation/.", "/otel-auto-instrumentation-python" ], "mountPoints": [ { "sourceVolume": "opentelemetry-auto-instrumentation-python", "containerPath": "/otel-auto-instrumentation-python", "readOnly": false } ] }
  4. Add the following environment variables to your application container.

    Environment variable Setting to enable Application Signals

    OTEL_RESOURCE_ATTRIBUTES

    Specify the following information as key-value pairs:

    • service.name sets the name of the service. This will be diplayed as the service name for your application in Application Signals dashboards. If you don't provide a value for this key, the default of UnknownService is used.

    • deployment.environment sets the environment that the application runs in. This will be diplayed as the Hosted In environment of your application in Application Signals dashboards. If you don't specify this, the default of generic:default is used.

    This attribute key is used only by Application Signals, and is converted into X-Ray trace annotations and CloudWatch metric dimensions.

    (Optional) To enable trace log correlation, set an additional environment variable aws.log.group.names to be the log group name for your application log. By doing so, the traces from your application can be correlated with the relevant log entries from this specific log group. For this variable, replace $YOUR_APPLICATION_LOG_GROUP with the log group name for your application. You'll also need to change the logging configuration in your application. For more information, see Enable trace log correlation.

    OTEL_AWS_APPLICATION_SIGNALS_ENABLED

    Set to true to have your container start sending X-Ray traces and CloudWatch metrics to Application Signals.

    OTEL_METRICS_EXPORTER

    Set to none to disable other metrics exporters.

    OTEL_EXPORTER_OTLP_PROTOCOL

    Set to http/protobuf to send metrics and traces to CloudWatch using HTTP.

    OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT

    Set to http://127.0.0.1:4316/v1/metrics to send metrics to the CloudWatch sidecar.

    OTEL_EXPORTER_OTLP_TRACES_ENDPOINT

    Set to http://127.0.0.1:4316/v1/traces to send traces to the CloudWatch sidecar.

    OTEL_TRACES_SAMPLER

    Set this to xray to set X-Ray as the traces sampler.

    OTEL_PROPAGATORS

    Add xray as one of the propagators.

    OTEL_PYTHON_DISTRO

    Set to aws_distro to use the ADOT Python instrumentation.

    OTEL_PYTHON_CONFIGURATOR

    Set to aws_configuration to use the ADOT Python configuration.

    PYTHONPATH

    Replace $APP_PATH with the location of the application’s working directory within the container. This is required for the Python interpreter to find your application modules.

    DJANGO_SETTINGS_MODULE

    Required only for Django applications. Set it to the location of your Django application’s settings.py file. Replace $PATH_TO_SETTINGS.

  5. Mount the volume opentelemetry-auto-instrumentation-python that you defined in step 1 of this procedure. In the following example, the value for OTEL_RESOURCE_ATTRIBUTES includes an optional aws.log.group.names that helps enable trace log correlation. For more information, see the table in the previous step.

    For a Python application, use the following.

    { "name": "my-app", ... "environment": [ { "name": "PYTHONPATH", "value": "/otel-auto-instrumentation-python/opentelemetry/instrumentation/auto_instrumentation:$APP_PATH:/otel-auto-instrumentation-python" }, { "name": "OTEL_EXPORTER_OTLP_PROTOCOL", "value": "http/protobuf" }, { "name": "OTEL_TRACES_SAMPLER", "value": "xray" }, { "name": "OTEL_TRACES_SAMPLER_ARG", "value": "endpoint=http://localhost:2000" }, { "name": "OTEL_LOGS_EXPORTER", "value": "none" }, { "name": "OTEL_PYTHON_DISTRO", "value": "aws_distro" }, { "name": "OTEL_PYTHON_CONFIGURATOR", "value": "aws_configurator" }, { "name": "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "value": "http://localhost:4316/v1/traces" }, { "name": "OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT", "value": "http://localhost:4316/v1/metrics" }, { "name": "OTEL_METRICS_EXPORTER", "value": "none" }, { "name": "OTEL_AWS_APPLICATION_SIGNALS_ENABLED", "value": "true" }, { "name": "OTEL_RESOURCE_ATTRIBUTES", "value": "aws.log.group.names=$YOUR_APPLICATION_LOG_GROUP,service.name=$SVC_NAME" }, { "name": "DJANGO_SETTINGS_MODULE", "value": "$PATH_TO_SETTINGS.settings" } ], "mountPoints": [ { "sourceVolume": "opentelemetry-auto-instrumentation-python", "containerPath": "/otel-auto-instrumentation-python", "readOnly": false } ] }

Step 5: Deploy your application

Create a new revision of your task definition and deploy it to your application cluster. You should see three containers in the newly created task:

  • init

  • ecs-cwagent

  • my-app