CloudWatch pipelines IAM policies and permissions

This section provides IAM requirements for CloudWatch pipelines. Permissions vary based on your data source and integration method.

The following table helps you identify which IAM sections apply to your use case.

API caller permissions

The IAM principal that calls CreateTelemetryPipeline needs iam:PassRole permission for any roles referenced in the pipeline configuration.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PassRoleForPipelineSource",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::111122223333:role/your-source-role",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "service-principal"
                    ],
                    "iam:AssociatedResourceARN": [
                        "arn:aws:observabilityadmin:us-east-1:111122223333:telemetry-pipeline/*"
                    ]
                }
            }
        }
    ]
}

Replace service-principal with the value from the following table based on your use case.

Pipeline rule permissions (CloudWatch Logs sources only)

When using cloudwatch_logs as a source, the API caller also needs permissions for pipeline rule operations. The logs:PutPipelineRule permission is required for CreateTelemetryPipeline and UpdateTelemetryPipeline operations. The logs:DeletePipelineRule permission is required for DeleteTelemetryPipeline operations.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PipelineRuleForCloudWatchLogs",
            "Effect": "Allow",
            "Action": [
                "logs:PutPipelineRule",
                "logs:DeletePipelineRule"
            ],
            "Resource": "*"
        }
    ]
}

Pipeline rule permissions for CloudWatch Metrics sources

When you use cloudwatch_metrics as a source, you need permissions for pipeline rule operations. To create or update a pipeline, grant the cloudwatch:PutPipelineRule permission. To delete a pipeline, grant the cloudwatch:DeletePipelineRule permission. Metrics pipelines do not require iam:PassRole or CloudWatch Logs resource policies. You can scope these actions down to the dataset/default resource.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PipelineRuleForCloudWatchMetrics",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutPipelineRule",
                "cloudwatch:DeletePipelineRule"
            ],
            "Resource": "arn:aws:cloudwatch:us-east-1:111122223333:dataset/default"
        }
    ]
}

Reducing scope with condition keys

Source role policies

Each pipeline requires a dedicated IAM role that the service assumes to read your data. The following subsections provide the complete policies (permission and trust) for each use case.

Third-party sources (API Pull)

This section applies to Microsoft Office 365, Microsoft Entra ID, Okta SSO, Palo Alto NGFW, and other vendor API integrations that store credentials in Amazon Secrets Manager.

Permission policy

The following policy allows the role to retrieve your stored API credentials.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "secretsmanageraccess",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:your-secret-name*"
        },
        {
            "Sid": "kmsaccess",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:us-east-1:111122223333:key/your-key-id"
        }
    ]
}

Trust policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Complete IAM setup for API Pull pipelines

The following example shows all the IAM policies needed to create a third-party API Pull pipeline end to end.

Caller identity policy — attach to the principal calling CreateTelemetryPipeline:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CreateApiPullPipeline",
            "Effect": "Allow",
            "Action": "observabilityadmin:CreateTelemetryPipeline",
            "Resource": "*"
        },
        {
            "Sid": "PassRoleForApiPullPipeline",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::111122223333:role/your-source-role",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "telemetry-pipelines.observabilityadmin.amazonaws.com"
                    ],
                    "iam:AssociatedResourceARN": [
                        "arn:aws:observabilityadmin:us-east-1:111122223333:telemetry-pipeline/*"
                    ]
                }
            }
        }
    ]
}

Source role permission policy — attach to the role the pipeline assumes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "SecretsManagerAccess",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:your-secret-name*"
        }
    ]
}

Source role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Third-party sources (S3 delivery)

This section applies to any third-party vendor that delivers log files to your S3 bucket (for example, CrowdStrike Falcon, Wiz, or Cisco Umbrella).

Permission policy

The following policy allows the role to read objects from S3 and consume SQS notifications.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "s3access",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        },
        {
            "Sid": "sqsaccess",
            "Effect": "Allow",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage",
                "sqs:ChangeMessageVisibility"
            ],
            "Resource": "arn:aws:sqs:us-east-1:111122223333:your-queue-name"
        },
        {
            "Sid": "kmsaccess",
            "Effect": "Allow",
            "Action": "kms:Decrypt",
            "Resource": "arn:aws:kms:us-east-1:111122223333:key/your-key-id"
        }
    ]
}

Trust policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Complete IAM setup for S3 delivery pipelines

The following example shows all the IAM policies needed to create an S3 delivery pipeline end to end.

Caller identity policy — attach to the principal calling CreateTelemetryPipeline:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CreateS3Pipeline",
            "Effect": "Allow",
            "Action": "observabilityadmin:CreateTelemetryPipeline",
            "Resource": "*"
        },
        {
            "Sid": "PassRoleForS3Pipeline",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::111122223333:role/your-source-role",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "telemetry-pipelines.observabilityadmin.amazonaws.com"
                    ],
                    "iam:AssociatedResourceARN": [
                        "arn:aws:observabilityadmin:us-east-1:111122223333:telemetry-pipeline/*"
                    ]
                }
            }
        }
    ]
}

Source role permission policy — attach to the role the pipeline assumes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3Access",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        },
        {
            "Sid": "SqsAccess",
            "Effect": "Allow",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage",
                "sqs:ChangeMessageVisibility"
            ],
            "Resource": "arn:aws:sqs:us-east-1:111122223333:your-queue-name"
        }
    ]
}

Source role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Custom data from S3

This section applies to your own applications or infrastructure writing log files to an S3 bucket.

The IAM setup is identical to Third-party sources (S3 delivery). Use the same permission policy and trust policy. The pipeline reads from your bucket through SQS event notifications regardless of who wrote the data.

Custom data from CloudWatch Logs

This section applies to your own applications logging to a CloudWatch Logs log group (for example, Lambda functions, ECS containers, or custom EC2 applications).

Permission policy

The following policy allows the role to process logs from your specified log groups.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "logsprocessingaccess",
            "Effect": "Allow",
            "Action": [
                "logs:processWithPipeline"
            ],
            "Resource": [
                "arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-01",
                "arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-02"
            ]
        }
    ]
}

You can scope down this permission by using the logs:data_source_name and logs:data_source_type condition keys to restrict which pipeline sources can invoke transformations. The logs:data_source_name value corresponds to the data_source_name in your pipeline configuration, and logs:data_source_type corresponds to the data_source_type in your pipeline configuration.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowProcessWithPipelineScopedDown",
            "Effect": "Allow",
            "Action": "logs:ProcessWithPipeline",
            "Resource": "arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-name",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceAccount": "111122223333",
                    "logs:data_source_name": "your-source-name",
                    "logs:data_source_type": "your-source-type"
                }
            }
        }
    ]
}

Trust policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Complete IAM setup for CloudWatch Logs pipelines

The following example shows all the IAM policies needed to create a CloudWatch Logs pipeline end to end.

Caller identity policy — attach to the principal calling CreateTelemetryPipeline:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CreateLogsPipeline",
            "Effect": "Allow",
            "Action": [
                "observabilityadmin:CreateTelemetryPipeline",
                "logs:PutPipelineRule",
                "logs:DeletePipelineRule"
            ],
            "Resource": "*"
        },
        {
            "Sid": "PassRoleForLogsPipeline",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::111122223333:role/your-source-role",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "logs.amazonaws.com"
                    ],
                    "iam:AssociatedResourceARN": [
                        "arn:aws:observabilityadmin:us-east-1:111122223333:telemetry-pipeline/*"
                    ]
                }
            }
        }
    ]
}

Source role permission policy — attach to the role the pipeline assumes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LogsProcessing",
            "Effect": "Allow",
            "Action": [
                "logs:processWithPipeline"
            ],
            "Resource": [
                "arn:aws:logs:us-east-1:111122223333:log-group:your-log-group"
            ]
        }
    ]
}

Source role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Vended Amazon service logs

This section applies to Amazon service logs delivered to CloudWatch Logs, such as VPC Flow Logs, Route 53 query logs, and other Amazon vended log types.

The IAM setup is identical to Custom data from CloudWatch Logs. Use the same permission policy (logs:processWithPipeline scoped to the log group) and the same trust policy (logs.amazonaws.com).

Because this uses the cloudwatch_logs source type, the caller also needs logs:PutPipelineRule and logs:DeletePipelineRule permissions. See Pipeline rule permissions (CloudWatch Logs sources only).

Resource policies

CloudWatch Logs resource policies are required for the direct CloudWatch Logs destination of an S3-based or third-party API pipeline. The direct destination is the single sink, or the named default route when conditional routing is configured.

For a CloudWatch Logs source, the @original destination does not require a resource policy. Named conditional and templatized destinations use the pipeline source role instead of the CloudWatch pipelines service principal. For more information, see Permissions for routed destinations.

After calling CreateTelemetryPipeline, you receive a pipeline ARN. You must then call logs:PutResourcePolicy to allow the CloudWatch pipelines service principal to write to the direct destination log group. The principal that creates the policy needs logs:PutResourcePolicy. To inspect or update an existing policy, the principal also needs logs:DescribeResourcePolicies.

Use a resource-scoped policy for the destination log group. Resource-scoped policies do not count toward the quota of 10 account-level resource policies per account per Region. The destination log group must exist before you create its resource-scoped policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-name:*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceArn": "arn:aws:observabilityadmin:us-east-1:111122223333:telemetry-pipeline/your-pipeline-id"
                }
            }
        }
    ]
}

Permissions for routed destinations

Additional conditional and templatized destinations are authorized by identity-based permissions on the IAM role in the pipeline source configuration, not by an additional CloudWatch Logs resource policy.

The pipeline source role must allow logs:CreateLogGroup for log groups that the pipeline creates automatically. It must also allow logs:CreateLogStream and logs:PutLogEvents for every named routed destination. A resource policy does not grant logs:CreateLogGroup.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CreateRoutedLogGroups",
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "*"
        },
        {
            "Sid": "WriteRoutedLogEvents",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:your-region:your-account-id:log-group:your-routed-log-group-prefix*:*"
        }
    ]
}

Scope the write statement to the exact destination log groups or the narrowest prefix that contains them. Include logs:CreateLogGroup only when the pipeline must create conditional or templatized destination log groups.

The role trust policy must allow each service that assumes the role:

Managing resource policies

Use the Amazon CLI to create or update CloudWatch Logs resource policies for CloudWatch pipelines.

To check for existing policies

aws logs describe-resource-policies \
    --policy-scope RESOURCE \
    --resource-arn arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-name

To create a new policy

aws logs put-resource-policy \
    --region us-east-1 \
    --resource-arn arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-name \
    --policy-document file://policy.json

To merge with an existing policy

Each log group can have one resource-scoped policy. If a policy already exists, add the new statement to the existing Statement array, then call put-resource-policy with the merged file and the current revision ID.

Confirm the policy was created or updated successfully:

aws logs describe-resource-policies \
    --policy-scope RESOURCE \
    --resource-arn arn:aws:logs:us-east-1:111122223333:log-group:your-log-group-name

Replace the following values in the policy:

Pipeline condition keys

CloudWatch pipelines supports IAM condition keys that let you restrict who can create pipelines based on the source name and type. Use these condition keys to enforce governance policies across your organization.

Use these condition keys in identity policies to control which pipelines a principal can create.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPipelineCreationForSpecificSourceType",
            "Effect": "Allow",
            "Action": "observabilityadmin:CreateTelemetryPipeline",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "observabilityadmin:SourceType": "cloudwatch_logs"
                }
            }
        }
    ]
}

Source role trust policy conditions

Use these condition keys in the trust policy of your source role to restrict which account can assume the role. This prevents confused deputy attacks where the service might act on behalf of a different account.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "111122223333"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:logs:us-east-1:111122223333:log-group:*"
                }
            }
        }
    ]
}

AI-assisted processor configuration permissions

To use AI-assisted processor configuration in the CloudWatch pipelines console, the IAM principal must have the logs:GeneratePipeline permission. This permission authorizes the generation of processor configurations from natural language descriptions.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowGeneratePipeline",
            "Effect": "Allow",
            "Action": "logs:GeneratePipeline",
            "Resource": "*"
        }
    ]
}

Configuring lookup processor permissions

If your pipeline configuration includes a Lookup processor, the source role needs to have the logs:GetLookupTable permission on the lookup table ARN. The role must also be assumable by logs.amazonaws.com.

Add the following statement to the source role's permissions policy, for all source types.

{
    "Sid": "LookupTableAccess",
    "Effect": "Allow",
    "Action": "logs:GetLookupTable",
    "Resource": "arn:aws:logs:us-east-1:111122223333:lookup-table:your-lookup-table-name"
}

Replace the following values in the policy:

The source role trust policy requirement differs by source type.

Custom data from CloudWatch Logs and vended Amazon service logs

For Custom data from CloudWatch Logs and Vended Amazon service logs, the role already trusts logs.amazonaws.com, so no trust policy change is required. The following trust policy shows the existing configuration.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Third-party sources

For Third-party sources (API Pull) and Third-party sources (S3 delivery), add a second statement to the trust policy so that logs.amazonaws.com can assume the role in addition to the existing telemetry service principal. The following trust policy shows both statements.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "telemetry-pipelines.observabilityadmin.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "111122223333"
                }
            }
        }
    ]
}