Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Create a custom connector to a data source

To connect a custom data source to CloudWatch, you have two options:

Use a template

Using a template creates a sample Lambda function, and can help you get your custom connector built faster. These sample functions provide sample code for many common scenarios involved with building a custom connector. You can examine the Lambda code after you create a connector with a template, then modify it to use to connect to your data source.

Additionally, if you use the template, CloudWatch takes care of creating the Lambda permissions policy and attaching resource tags to the Lambda function.

Create a custom data source from scratch

Follow the steps in this section to create a Lambda function that connects CloudWatch to a data source.

Step 1: Create the function

A custom data source connector must support GetMetricData events from CloudWatch. Optionally, you can also implement a DescribeGetMetricData event to provide documentation to users in the CloudWatch console for how to use the connector. The DescribeGetMetricData response can also be used to set defaults that are used in the CloudWatch custom query builder.

CloudWatch provides code snippets as samples to help you get started. For more information, see the samples repository at https://github.com/aws-samples/cloudwatch-data-source-samples.

Constraints

GetMetricData event

Request payload

The following is an example of a GetMetricData request payload sent as input to the Lambda function.

{
  "EventType": "GetMetricData",
  "GetMetricDataRequest": {
    "StartTime": 1697060700,
    "EndTime": 1697061600,
    "Period": 300,
    "Arguments": ["serviceregistry_external_http_requests{host_cluster!=\"prod\"}"] 
  }
}

Response payload

The following is an example of a GetMetricData response payload returned by the Lambda function.

{
   "MetricDataResults": [
      {
         "StatusCode": "Complete",
         "Label": "CPUUtilization",
         "Timestamps": [ 1697060700, 1697061000, 1697061300 ],
         "Values": [ 15000, 14000, 16000 ]
      }
   ]
}

The response payload will contain either a MetricDataResults field or an Error field, but not both.

A MetricDataResults field is a list of time-series fields of type MetricDataResult. Each of those time-series fields can include the following fields.

For more information about Error objects, see the following sections.

Error response formats

You can optionally use the error response to provide more information about errors. We recommend that you return an error with Code Validation when a validation error occurs, such as when a parameter is missing or is the wrong type.

The following is an example of the response when the Lambda function wants to raise a GetMetricData validation exception.

{
   "Error": {
      "Code": "Validation",
      "Value": "Invalid Prometheus cluster"
   }
}

The following is an example of the response when the Lambda function indicates that it's unable to return data because of an access issue. The response is translated into a single time series with a status code of Forbidden.

{
   "Error": {
      "Code": "Forbidden",
      "Value": "Unable to access ..."
   }
}

The following is an example of when the Lambda function raises an overall InternalError exception, which is translated into a single time series with a status code of InternalError and a message. Whenever an error code has a value other than Validation or Forbidden, CloudWatch assumes that it's a generic internal error.

{
   "Error": {
      "Code": "PrometheusClusterUnreachable",
      "Value": "Unable to communicate with the cluster"
   }
}

DescribeGetMetricData event

Request payload

The following is an example of a DescribeGetMetricData request payload.

{
  "EventType": "DescribeGetMetricData"
}

Response payload

The following is an example of a DescribeGetMetricData response payload.

{
    "Description": "Data source connector",
    "ArgumentDefaults": [{
        Value: "default value"
     }]
}

Important considerations for CloudWatch alarms

If you are going to use the data source to set CloudWatch alarms, you should set it up to report data with timestamps every minute to CloudWatch. For more information and other considerations for creating alarms on metrics from connected data sources, see Create an alarm based on a connected data source.

(Optional) Use Amazon Secrets Manager to store credentials

If your Lambda function needs to use credentials to access the data source, we recommend using Amazon Secrets Manager to store these credentials instead of hardcoding them into your Lambda function. For more information about using Amazon Secrets Manager with Lambda, see Use Amazon Secrets Manager secrets in Amazon Lambda functions.

(Optional) Connect to a data source in a VPC

If your data source is in a VPC managed by Amazon Virtual Private Cloud, you must configure your Lambda function to access it. For more information, see Connecting outbound networking to resources in a VPC.

You might also need to configure VPC service endpoints to access services such as Amazon Secrets Manager. For more information, see Access an Amazon service using an interface VPC endpoint.

Step 2: Create a Lambda permissions policy

You must use create a policy statement that grants CloudWatch permission to use the Lambda function that you created. You can use the Amazon CLI or the Lambda console to create the policy statement.

Step 3: Attach a resource tag to the Lambda function

The CloudWatch console determines which of your Lambda functions are data source connectors by using a tag. When you create a data source using one of the wizards, the tag is automatically applied by the Amazon CloudFormation stack that configures it. When you create a data source yourself, you can use the following tag for your Lambda function. This makes your connector appear in the Data source dropdown in the CloudWatch console when you query metrics.