Configure IAM permissions for Amazon Glue Data Quality
This topic provides information to help you understand the actions and resources that you an IAM administrator can use in an Amazon Identity and Access Management (IAM) policy for Amazon Glue Data Quality. It also includes sample IAM policies with the minimum permissions you need to use Amazon Glue Data Quality with the Amazon Glue Data Catalog.
For additional information about security in Amazon Glue, see Security in Amazon Glue.
IAM permissions for Amazon Glue Data Quality
The following table lists the permissions that a user needs in order to perform specific
Amazon Glue Data Quality operations. To set fine-grained authorization for Amazon Glue Data Quality, you can specify these
actions in the Action element of an IAM policy statement.
| Action | Description | Resource types |
|---|---|---|
glue:CreateDataQualityRuleset |
Grants permission to create a data quality ruleset. | ::dataQualityRuleset/<name> |
glue:DeleteDataQualityRuleset |
Grants permission to delete a data quality ruleset. | ::dataQualityRuleset/<name> |
glue:GetDataQualityRuleset |
Grants permission to retrieve a data quality ruleset. | ::dataQualityRuleset/<name> |
glue:ListDataQualityRulesets |
Grants permission to retrieve all data quality rulesets. | ::dataQualityRuleset/* |
glue:UpdateDataQualityRuleset |
Grants permission to update a data quality ruleset. | ::dataQualityRuleset/<name> |
glue:GetDataQualityResult |
Grants permission to retrieve a data quality task run result. This IAM action also provides permissions to the following
APIS:
|
::dataQualityRuleset/<name> |
glue:ListDataQualityResults |
Grants permission to retrieve all data quality task run results. | ::dataQualityRuleset/* |
glue:CancelDataQualityRuleRecommendationRun |
Grants permission to stop an in-progress data quality recommendation task run. | ::dataQualityRuleset/* |
glue:GetDataQualityRuleRecommendationRun |
Grants permission to retrieve a data quality recommendation task run. | ::dataQualityRuleset/* |
glue:ListDataQualityRuleRecommendationRuns |
Grants permission to retrieve all data quality recommendation task runs. | ::dataQualityRuleset/* |
glue:StartDataQualityRuleRecommendationRun |
Grants permission to start a data quality recommendation task run. | ::dataQualityRuleset/* |
glue:CancelDataQualityRulesetEvaluationRun |
Grants permission to stop an in-progress data quality task run. | ::dataQualityRuleset/* |
glue:GetDataQualityRulesetEvaluationRun |
Grants permission to retrieve a data quality task run. | ::dataQualityRuleset/* |
glue:ListDataQualityRulesetEvaluationRuns |
Grants permission to retrieve all data quality task runs. | ::dataQualityRuleset/* |
glue:StartDataQualityRulesetEvaluationRun |
Grants permission to start a data quality task run. | ::dataQualityRuleset/<name> |
glue:PublishDataQuality |
Grants permission to publish data quality results. | ::dataQualityRuleset/<name> |
glue:GetDataQualityModel |
Grants permission to retrieve Data Quality Model. | ::dataQualityRuleset/<name>,
::job/<name>
|
glue:GetDataQualityModelResult |
Grants permission to retrieve Data Quality Model Results. | ::dataQualityRuleset/<name>,
::job/<name>
|
glue:PutDataQualityStatisticAnnotation |
Grants permission to add annotations to Statistics. This IAM action also provides permissions to the following
APIS:
|
::dataQualityRuleset/<name>,
::job/<name>
|
glue:PutDataQualityProfileAnnotation |
Grants permission to put annotations to all Statistics in a Profile. | ::dataQualityRuleset/<name>,
::job/<name>
|
glue:CreateDataQualityEvaluationConfig |
Grants permission to create a data quality evaluation configuration. If you specify tags, you also need glue:TagResource. |
::dataQualityEvaluationConfig/<name> |
glue:GetDataQualityEvaluationConfig |
Grants permission to retrieve a data quality evaluation configuration. | ::dataQualityEvaluationConfig/<name> |
glue:UpdateDataQualityEvaluationConfig |
Grants permission to update a data quality evaluation configuration. | ::dataQualityEvaluationConfig/<name> |
glue:DeleteDataQualityEvaluationConfig |
Grants permission to delete a data quality evaluation configuration. | ::dataQualityEvaluationConfig/<name> |
glue:ListDataQualityEvaluationConfigs |
Grants permission to retrieve all data quality evaluation configurations. | ::dataQualityEvaluationConfig/* |
IAM setup required for scheduling evaluation runs
IAM permissions
To run scheduled Data Quality evaluation runs, you must add the IAM:PassRole action to the permissions policy.
| Action | Description | Resource types |
|---|---|---|
iam:PassRole |
Grants permission for IAM to allow the user to pass the approved roles. | ARN of the role used to call StartDataQualityRulesetEvaluationRun |
Without these permissions the following error occurs:
"errorCode": "AccessDenied" "errorMessage": "User: arn:aws:sts::account_id:assumed-role/AWSGlueServiceRole is not authorized to perform: iam:PassRole on resource: arn:aws:iam::account_id:role/service-role/AWSGlueServiceRole because no identity-based policy allows the iam:PassRole action"
IAM trusted entities
The Amazon Glue and Amazon EventBridge Scheduler services need to be listed in the trusted entities in order to create and run a scheduled StartDataQualityEvaluationRun.
IAM permissions for data quality evaluation configurations
A data quality evaluation configuration involves two sets of permissions:
-
Caller permissions – The permissions that you need to create and manage configurations, pass the configuration's role, and read the results.
-
Role permissions – The permissions that the configuration's role needs so that Amazon Glue Data Quality can find the tables in scope, start an evaluation run for each table, and read each table's data during the run.
For an overview of evaluation configurations, see Data quality monitoring at the catalog and database level.
Permissions to manage evaluation configurations
The following identity-based policy allows a user to manage evaluation configurations and read their results. It includes the following permissions:
-
The evaluation configuration actions. Each action authorizes the configuration's ARN, so you can limit access to specific configurations by name or by tag.
-
glue:TagResource, which you need to create a configuration with tags and to tag it later. -
glue:GetCatalog,glue:GetDatabase, andglue:GetTable. When you create or update a configuration, Amazon Glue checks that you can describe the catalog, database, or table that it applies to. -
iam:PassRolefor the configuration's role. You need this permission whenever you specify the role in a create or update request. -
Actions to read evaluation runs and results.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ManageEvaluationConfigs", "Effect": "Allow", "Action": [ "glue:CreateDataQualityEvaluationConfig", "glue:GetDataQualityEvaluationConfig", "glue:UpdateDataQualityEvaluationConfig", "glue:DeleteDataQualityEvaluationConfig", "glue:ListDataQualityEvaluationConfigs", "glue:TagResource", "glue:UntagResource", "glue:GetTags" ], "Resource": "arn:aws-cn:glue:us-east-1:111122223333:dataQualityEvaluationConfig/*" }, { "Sid": "DescribeConfigurationTargets", "Effect": "Allow", "Action": [ "glue:GetCatalog", "glue:GetDatabase", "glue:GetTable" ], "Resource": [ "arn:aws-cn:glue:us-east-1:111122223333:catalog", "arn:aws-cn:glue:us-east-1:111122223333:database/database-name", "arn:aws-cn:glue:us-east-1:111122223333:table/database-name/*" ] }, { "Sid": "ReadEvaluationResults", "Effect": "Allow", "Action": [ "glue:ListDataQualityResults", "glue:GetDataQualityResult", "glue:ListDataQualityRulesetEvaluationRuns", "glue:GetDataQualityRulesetEvaluationRun" ], "Resource": "arn:aws-cn:glue:us-east-1:111122223333:dataQualityRuleset/*" }, { "Sid": "PassEvaluationConfigRole", "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws-cn:iam::111122223333:role/DataQualityEvaluationConfigRole", "Condition": { "StringEquals": { "iam:PassedToService": "glue.amazonaws.com" } } } ] }
If you use your own rulesets, also allow glue:CreateDataQualityRuleset
and glue:UpdateDataQualityRuleset. If you use Lake Formation, you also need the
DESCRIBE permission on the catalog, database, or table to create, get, or
update a configuration that applies to it.
If a configuration applies to an Amazon S3 Tables catalog, also allow
lakeformation:GetDataAccess, and add the Amazon S3 Tables forms of the catalog,
database, and table ARNs to the DescribeConfigurationTargets statement. For example, for every table bucket in
the account, add the following resources:
"arn:aws-cn:glue:us-east-1:111122223333:catalog/s3tablescatalog", "arn:aws-cn:glue:us-east-1:111122223333:catalog/s3tablescatalog/*", "arn:aws-cn:glue:us-east-1:111122223333:database/s3tablescatalog/*", "arn:aws-cn:glue:us-east-1:111122223333:table/s3tablescatalog/*"
To control access with tags, use the aws:RequestTag/
and tag-keyaws:TagKeys condition keys with
glue:CreateDataQualityEvaluationConfig, and the
aws:ResourceTag/ condition key with the other
actions. For more information, see Specifying Amazon Glue resource ARNs.tag-key
Trust policy for the configuration role
Amazon Glue assumes the configuration's role to find tables and to start each evaluation run, and the evaluation run assumes the role to read the table. Use the following trust policy to allow Amazon Glue to assume the role for your account's catalog and tables.
Amazon Glue sets aws:SourceArn to your catalog's ARN or a table's ARN, and aws:SourceAccount to your account ID. The trust policy must allow both ARN forms. When you create or update a configuration, Amazon Glue checks that it can assume the role with each form.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "glue.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "aws:SourceAccount": "111122223333" }, "ArnLike": { "aws:SourceArn": [ "arn:aws-cn:glue:us-east-1:111122223333:catalog", "arn:aws-cn:glue:us-east-1:111122223333:table/*" ] } } } ] }
Permissions policy for the configuration role
The configuration's role needs permissions for each of the following tasks:
-
Find the tables in scope –
glue:GetCatalog,glue:GetDatabase, andglue:GetTableto describe the catalog, database, or table, andglue:SearchTablesto list the tables in a catalog or database. -
Prepare the ruleset –
glue:GetDataQualityRulesetto read the ruleset, andglue:CreateDataQualityRulesetandglue:UpdateDataQualityRulesetto create and maintain the built-in rulesets in your account. -
Start evaluation runs –
glue:StartDataQualityRulesetEvaluationRun, andiam:PassRoleon the role itself, because Amazon Glue starts each run with the same role. -
Evaluate the data – The permissions that any data quality evaluation run needs:
glue:GetPartitions,glue:GetDataQualityRulesetEvaluationRun,glue:PublishDataQuality, read access to the table's data in Amazon S3, and permissions to write logs to Amazon CloudWatch Logs.
Attach the following permissions policy to the role. Replace the bucket with the locations of your tables' data.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "FindTables", "Effect": "Allow", "Action": [ "glue:GetCatalog", "glue:GetDatabase", "glue:GetTable", "glue:SearchTables", "glue:GetPartitions" ], "Resource": [ "arn:aws-cn:glue:us-east-1:111122223333:catalog", "arn:aws-cn:glue:us-east-1:111122223333:catalog/*", "arn:aws-cn:glue:us-east-1:111122223333:database/*", "arn:aws-cn:glue:us-east-1:111122223333:table/*" ] }, { "Sid": "PrepareRulesetsAndStartRuns", "Effect": "Allow", "Action": [ "glue:GetDataQualityRuleset", "glue:CreateDataQualityRuleset", "glue:UpdateDataQualityRuleset", "glue:StartDataQualityRulesetEvaluationRun", "glue:GetDataQualityRulesetEvaluationRun", "glue:PublishDataQuality" ], "Resource": "arn:aws-cn:glue:us-east-1:111122223333:dataQualityRuleset/*" }, { "Sid": "PassThisRoleToGlue", "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws-cn:iam::111122223333:role/DataQualityEvaluationConfigRole", "Condition": { "StringEquals": { "iam:PassedToService": "glue.amazonaws.com" } } }, { "Sid": "ReadTableData", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:ListBucket" ], "Resource": [ "arn:aws-cn:s3:::amzn-s3-demo-bucket", "arn:aws-cn:s3:::amzn-s3-demo-bucket/*" ] }, { "Sid": "WriteRunLogs", "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:aws-cn:logs:us-east-1:111122223333:log-group:/aws-glue/*" } ] }
If a condition in this list applies, add the related permissions to the role:
-
If a catalog-level or database-level configuration applies to an Amazon S3 Tables catalog, add
s3tables:ListTableBucketsands3tables:ListTables. Amazon Glue uses these actions instead ofglue:SearchTablesto find the tables in a table bucket. -
If you use Lake Formation, grant the role the
DESCRIBEpermission on the tables to evaluate. The configuration evaluates only the tables that the role can describe. -
If Lake Formation manages the tables' data, as it does for all Amazon S3 Tables, add
lakeformation:GetDataAccessand grant the role theSELECTpermission on each table, without column or data filters. In the Lake Formation data lake settings, also allow external engines to access data with full table access. For more information, see Supported source types. -
If you set
CloudWatchMetricsEnabledin the run options, addcloudwatch:PutMetricData. -
If you write results to Amazon S3 with
ResultsS3Prefix, adds3:PutObjecton that location. For information about writing results to Amazon Glue Data Catalog tables, see Prerequisites. -
If your data is encrypted with an Amazon KMS key, add
kms:Decrypton that key.
Example IAM policies
An IAM role for Amazon Glue Data Quality needs the following types of permissions:
-
Permissions for Amazon Glue Data Quality operations so that you can get recommended data quality rules and run a data quality task against a table in the Amazon Glue Data Catalog. The example IAM policies in this section include the minimum permissions required for Amazon Glue Data Quality operations.
-
Permissions that grant access to your Data Catalog table and the underlying data. These permissions vary depending on your use case. For example, for data that you catalog in Amazon S3, the permissions should include access to Amazon S3.
NoteYou must configure Amazon S3 permissions in addition to the permissions described in this section.
Minimum permissions to get recommended data quality rules
This example policy includes the permissions you need in order to generate recommended data quality rules.
Minimum permissions to get advanced data quality rule recommendations
You must have iam:PassRole permission for the recommendation role. The
role does not need Amazon Bedrock permissions.
Trust policy
Use this trust policy to allow Amazon Glue to assume the recommendation role for your source table.
Set aws:SourceArn to the full source table ARN. Use the account ID from that ARN for aws:SourceAccount.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "glue.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "aws:SourceAccount": "111122223333" }, "ArnEquals": { "aws:SourceArn": "arn:aws-cn:glue:us-east-1:111122223333:table/database-name/table-name" } } } ] }
Permissions policy
On the first ADVANCED run, Amazon Glue creates the following resources in your
account:
-
The
glue-dataquality-samplingAthena workgroup to run queries for rule recommendations. -
The
aws-glue-dataquality-sampling-Amazon S3 bucket to store query results.account-id-region
For subsequent runs, Amazon Glue reuses the existing workgroup and bucket. For details, see Data protection for advanced data quality rule recommendations.
Attach the following permissions policy to your recommendation role.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowCatalogRead", "Effect": "Allow", "Action": [ "glue:GetDatabase", "glue:GetPartitions", "glue:GetTable" ], "Resource": [ "arn:aws-cn:glue:us-east-1:111122223333:catalog", "arn:aws-cn:glue:us-east-1:111122223333:database/database-name", "arn:aws-cn:glue:us-east-1:111122223333:table/database-name/table-name" ] }, { "Sid": "AllowDataQualityRecommendationResults", "Effect": "Allow", "Action": "glue:PublishDataQuality", "Resource": "arn:aws-cn:glue:us-east-1:111122223333:dataQualityRuleset/*" }, { "Sid": "AllowDataAccess", "Effect": "Allow", "Action": [ "athena:CreateWorkGroup", "athena:GetQueryExecution", "athena:GetQueryResults", "athena:StartQueryExecution" ], "Resource": "arn:aws-cn:athena:us-east-1:111122223333:workgroup/glue-dataquality-sampling" }, { "Sid": "AllowSourceRead", "Effect": "Allow", "Action": [ "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Resource": [ "arn:aws-cn:s3:::source-bucket", "arn:aws-cn:s3:::source-bucket/*" ] }, { "Sid": "AllowSamplingBucket", "Effect": "Allow", "Action": [ "s3:AbortMultipartUpload", "s3:CreateBucket", "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket", "s3:PutLifecycleConfiguration", "s3:PutObject" ], "Resource": [ "arn:aws-cn:s3:::aws-glue-dataquality-sampling-111122223333-us-east-1", "arn:aws-cn:s3:::aws-glue-dataquality-sampling-111122223333-us-east-1/athena-results/*" ] } ] }
If the bucket exists, you can remove s3:CreateBucket from this policy.
If you delete the bucket, Amazon Glue needs this permission again. All other permissions in
this policy remain necessary for subsequent runs.
Additional permissions
If a condition in this list applies, add the related permissions to your recommendation role.
-
If Lake Formation controls table access, add
lakeformation:GetDataAccess. Grant the roleDESCRIBEon the database andDESCRIBEandSELECTon the table. -
If you specify a data quality security configuration, add
glue:GetSecurityConfigurationpermission. -
If you specify
CreatedRulesetName, addglue:CreateDataQualityRulesetpermission. -
If you specify a data quality security configuration with a customer managed key, add the following statement for that key.
{ "Sid": "AllowDataQualityEncryption", "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:DescribeKey", "kms:GenerateDataKey", "kms:GenerateDataKeyWithoutPlaintext", "kms:ReEncrypt*" ], "Resource": "arn:aws-cn:kms:us-east-1:111122223333:key/key-id" }Your recommendation role needs these actions for query result encryption and Amazon Glue Data Quality asset encryption. The key policy must also allow these actions for your role. For details, see Key policy.
-
If the source objects use SSE-KMS, add the following statement.
{ "Sid": "AllowSourceDataDecryption", "Effect": "Allow", "Action": "kms:Decrypt", "Resource": "arn:aws-cn:kms:us-east-1:111122223333:key/source-key-id" }The source key policy must also allow the recommendation role to decrypt the source objects.
Minimum permissions to run a data quality task
This example policy includes the permissions you need in order to run a data quality evaluation task.
The following policy statements are optional, depending on your use case:
-
AllowCloudWatchPutMetricDataToPublishTaskMetrics- Required if you want to publish data quality run metrics to Amazon CloudWatch. -
AllowS3PutObjectToWriteTaskResults- Required if you want to write data quality run results to Amazon S3.
Minimum permissions to run an data quality ETL job
This example policy includes the permissions you need in order to run an data quality ETL Job.