Set up a metric stream to an Amazon service (data lake scenario)

Open the CloudWatch console at https://console.amazonaws.cn/cloudwatch/.

In the navigation pane, choose Metrics, Streams. Then choose Create metric stream.

(Optional) If you are signed in to an account that is set up as a monitoring account in CloudWatch cross-account observability, you can choose whether to include metrics from linked source accounts in this metric stream. To include metrics from source accounts, choose Include source account metrics.

Choose the CloudWatch metric namespaces to include in the metric stream.

• To include all or most of your metric namespaces in the metric stream, choose All namespaces.

  Then, if you want to exclude some metric namespaces from the stream, choose Exclude metric namespaces and select the namespaces to exclude. If you specify a namespace in Exclude metric namespaces, all metrics in that namespace are excluded from the stream.

• To include only a few metric namespaces in the metric stream, choose Selected namespaces and then select the namespaces to include.

(Optional) Use the Select metrics for the metric stream option to create additional filters for specific metrics.

• If you chose to stream All namespaces, you can use this section to exclude specific metrics from the stream while still including other metrics from that same namespace in the stream. To do so, do the following:

  1. Choose Exclude metrics by metric name

  2. Specify each metric to exclude by selecting its namespace and then its metric name.

     After you select one metric, you can repeat this step to exclude another metric from the same namespace or a different one.

• If you chose to stream Selected namespaces, you can use this section to include only a subset of metrics from those namespace in the metric stream. To do so, do the following:

  1. Choose Include metrics by metric name

  2. Specify each metric to include by selecting its namespace and then its metric name.

     After you select one metric, you can repeat this step to include another metric from the same namespace or a different one.

     If you don't select any metrics from one of your included namespaces, then all metric from that namespace are included in the metric stream.

Choose one of the following:

• For a quick setup, choose Quick S3 setup. CloudWatch will create all necessary resources including the Kinesis Data Firehose delivery stream and the necessary IAM roles. The default format for this option is JSON, but you can change the format later in this procedure.

• To use a Kinesis Data Firehose delivery stream that already exists, choose Select an existing Firehose owned by your account. The Kinesis Data Firehose delivery stream must be in the same account. The default format for this option is OpenTelemetry, but you can change the format later in this procedure.

  Then select the Kinesis Data Firehose delivery stream to use under Select your Kinesis Data Firehose delivery stream.

(Optional) If you're using Quick S3 Setup, you can choose Select existing resources to use an existing S3 bucket or existing IAM roles instead of having CloudWatch create new ones for you.

(Optional) If you're using Select an existing Firehose owned by your account, you can choose Select existing service role to use an existing IAM role instead of having CloudWatch create a new one for you.

(Optional) To change the output format from the default format for your scenario, choose Change output format. The supported formats are JSON and OpenTelemetry 0.7.0.

(Optional) To stream additional statistics for some of these metrics beyond Minimum, Maximum, Sample count, and Sum, choose Add additional statistics. Either choose Add recommended metrics to add some commonly used statistics, or manually select the namespace and metric name to stream additional statistics for. Next, select the additional statistics to stream.

To then choose another group of metrics to stream a different set of additional statistics for, choose Add additional statistics. Each metric can include as many as 20 additional statistics, and as many as 100 metrics within a metric stream can include additional statistics.

Streaming additional statistics incurs more charges. For more information, see Statistics that can be streamed.

For definitions of the additional statistics, see CloudWatch statistics definitions.

(Optional) Customize the name of the new metric stream under Metric stream name.

Choose Create metric stream.

If you're streaming to Amazon S3, first create the bucket. For more information, see Creating a bucket.

Create the Kinesis Data Firehose delivery stream. For more information, see Creating an Amazon Kinesis Data Firehose Delivery Stream.

Create an IAM role that enables CloudWatch to write to the Kinesis Data Firehose delivery stream. For more information about the contents of this role, see Trust between CloudWatch and Kinesis Data Firehose.

Use the aws cloudwatch put-metric-stream CLI command or the PutMetricStream API to create the CloudWatch metric stream.

If you're streaming to Amazon S3, first create the bucket. For more information, see Creating a bucket.

Create the Kinesis Data Firehose delivery stream. For more information, see Creating an Amazon Kinesis Data Firehose Delivery Stream.

Create an IAM role that enables CloudWatch to write to the Kinesis Data Firehose delivery stream. For more information about the contents of this role, see Trust between CloudWatch and Kinesis Data Firehose.

Create the stream in Amazon CloudFormation. For more information, see AWS::CloudWatch::MetricStream.

If you're streaming to Amazon S3, first create the bucket. For more information, see Creating a bucket.

Create the Kinesis Data Firehose delivery stream. For more information, see Creating an Amazon Kinesis Data Firehose Delivery Stream.

Create an IAM role that enables CloudWatch to write to the Kinesis Data Firehose delivery stream. For more information about the contents of this role, see Trust between CloudWatch and Kinesis Data Firehose.

Create the metric stream. The metric stream resource is available in Amazon CDK as a Level 1 (L1) Construct named CfnMetricStream. For more information, see Using L1 constructs.