AWSPremiumSupport-CollectAWSGlueMetadata

Description

The AWSPremiumSupport-CollectAWSGlueMetadata runbook gathers configuration metadata for your Amazon Glue resources, including databases, crawlers, ETL jobs, development endpoints, triggers, workflows, and security configurations. The runbook writes this metadata to a JSON report in the Amazon S3 bucket that you choose. Your Technical Account Manager (TAM) or Specialist Technical Account Manager (STAM) reviews the report as part of the Amazon Glue Operational Review. This review is a proactive assessment of your Amazon Glue environment against Amazon best practices.

How it works

A Python application collects the Amazon Glue metadata. The application runs on a temporary Amazon EC2 instance. The runbook creates this temporary instance in a new isolated Amazon VPC using an embedded Amazon CloudFormation template. The Amazon EC2 instance runs on a private-facing subnet with internet access through a Network Address Translation (NAT) gateway. The runbook accesses Amazon services, including Amazon Glue, Amazon EC2 Systems Manager, and Amazon S3, through Amazon VPC endpoints in the same Region. The NAT gateway retrieves metadata from Amazon S3 buckets that are associated with your Amazon Glue resources in other Amazon Regions.

When the runbook finishes, it deletes the Amazon CloudFormation stack and uploads the report to the Amazon S3 bucket of your choice. The runbook then attaches the report to the Amazon Support case for this Operational Review. The JSON file is in the root folder of your Amazon S3 bucket. Contact your TAM or STAM if you have questions about the Operational Review and the collected metadata.

Run this Automation (console)

Document type

Automation

Owner

Amazon

Platforms

/

Document parameters

Required IAM permissions to collect the Amazon Glue metadata

The Amazon Glue data collector application requires the following actions to successfully run this runbook and upload the file to the private Amazon S3 bucket of your choice. An IAM role automatically grants these permissions to the temporary Amazon EC2 instance.

Required IAM permissions to run this runbook

To run this runbook, the AutomationAssumeRole or your IAM user requires the following actions. These permissions are required to create a new temporary Amazon VPC and the Amazon EC2 instance using Amazon CloudFormation.

The following example policy shows the least-privilege permissions required for the AutomationAssumeRole. Replace REGION, ACCOUNTID, and S3_BUCKET_NAME with your own values:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CloudFormationStackAccess",
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:DeleteStack",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DescribeStacks",
                "cloudformation:UpdateStack"
            ],
            "Resource": "arn:aws:cloudformation:REGION:ACCOUNTID:stack/AWSPremiumSupport-CollectAWSGlueMetadata-*/*"
        },
        {
            "Sid": "EC2NetworkingAccess",
            "Effect": "Allow",
            "Action": [
                "ec2:AllocateAddress",
                "ec2:AssociateRouteTable",
                "ec2:AttachInternetGateway",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CreateInternetGateway",
                "ec2:CreateNatGateway",
                "ec2:CreateRoute",
                "ec2:CreateRouteTable",
                "ec2:CreateSecurityGroup",
                "ec2:CreateSubnet",
                "ec2:CreateTags",
                "ec2:CreateVpc",
                "ec2:CreateVpcEndpoint",
                "ec2:DeleteInternetGateway",
                "ec2:DeleteNatGateway",
                "ec2:DeleteRoute",
                "ec2:DeleteRouteTable",
                "ec2:DeleteSecurityGroup",
                "ec2:DeleteSubnet",
                "ec2:DeleteTags",
                "ec2:DeleteVpc",
                "ec2:DeleteVpcEndpoints",
                "ec2:DescribeAddresses",
                "ec2:DescribeImages",
                "ec2:DescribeInstances",
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeNatGateways",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcs",
                "ec2:DetachInternetGateway",
                "ec2:DisassociateRouteTable",
                "ec2:ModifySubnetAttribute",
                "ec2:ModifyVpcAttribute",
                "ec2:RebootInstances",
                "ec2:ReleaseAddress",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RunInstances",
                "ec2:TerminateInstances"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "REGION"
                }
            }
        },
        {
            "Sid": "IAMRoleAccess",
            "Effect": "Allow",
            "Action": [
                "iam:AddRoleToInstanceProfile",
                "iam:AttachRolePolicy",
                "iam:CreateInstanceProfile",
                "iam:CreateRole",
                "iam:DeleteInstanceProfile",
                "iam:DeleteRole",
                "iam:DeleteRolePolicy",
                "iam:DetachRolePolicy",
                "iam:GetInstanceProfile",
                "iam:GetRole",
                "iam:GetRolePolicy",
                "iam:PutRolePolicy",
                "iam:RemoveRoleFromInstanceProfile"
            ],
            "Resource": [
                "arn:aws:iam::ACCOUNTID:role/AWSPremiumSupport-CollectAWSGlueMetadata-*",
                "arn:aws:iam::ACCOUNTID:instance-profile/AWSPremiumSupport-CollectAWSGlueMetadata-*"
            ]
        },
        {
            "Sid": "PassRoleToEC2",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::ACCOUNTID:role/AWSPremiumSupport-CollectAWSGlueMetadata-*",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": "ec2.amazonaws.com"
                }
            }
        },
        {
            "Sid": "S3BucketAccess",
            "Effect": "Allow",
            "Action": [
                "s3:GetAccountPublicAccessBlock",
                "s3:GetBucketAcl",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketPublicAccessBlock",
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::S3_BUCKET_NAME"
        },
        {
            "Sid": "SSMAutomationAccess",
            "Effect": "Allow",
            "Action": [
                "ssm:DescribeAutomationExecutions",
                "ssm:DescribeInstanceInformation",
                "ssm:GetDocument",
                "ssm:GetParameters",
                "ssm:ListCommandInvocations",
                "ssm:ListCommands",
                "ssm:SendCommand"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:RequestedRegion": "REGION"
                }
            }
        },
        {
            "Sid": "SupportCaseAccess",
            "Effect": "Allow",
            "Action": [
                "support:AddAttachmentsToSet",
                "support:AddCommunicationToCase",
                "support:DescribeCases"
            ],
            "Resource": "*"
        }
    ]
}
        

Document steps