Setting up encryption in Amazon Glue - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Setting up encryption in Amazon Glue

The following example workflow highlights the options to configure when you use encryption with Amazon Glue. The example demonstrates the use of specific Amazon Key Management Service (Amazon KMS) keys, but you might choose other settings based on your particular needs. This workflow highlights only the options that pertain to encryption when setting up Amazon Glue.

  1. If the user of the Amazon Glue console doesn't use a permissions policy that allows all Amazon Glue API operations (for example, "glue:*"), confirm that the following actions are allowed:

    • "glue:GetDataCatalogEncryptionSettings"

    • "glue:PutDataCatalogEncryptionSettings"

    • "glue:CreateSecurityConfiguration"

    • "glue:GetSecurityConfiguration"

    • "glue:GetSecurityConfigurations"

    • "glue:DeleteSecurityConfiguration"

  2. Any client that accesses or writes to an encrypted catalog—that is, any console user, crawler, job, or development endpoint—needs the following permissions.

    { "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "kms:GenerateDataKey", "kms:Decrypt", "kms:Encrypt" ], "Resource": "<key-arns-used-for-data-catalog>" } }
  3. Any user or role that accesses an encrypted connection password needs the following permissions.

    { "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": "<key-arns-used-for-password-encryption>" } }
  4. The role of any extract, transform, and load (ETL) job that writes encrypted data to Amazon S3 needs the following permissions.

    { "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:Encrypt", "kms:GenerateDataKey" ], "Resource": "<key-arns-used-for-s3>" } }
  5. Any ETL job or crawler that writes encrypted Amazon CloudWatch Logs requires the following permissions in the key and IAM policies.

    In the key policy (not the IAM policy):

    { "Effect": "Allow", "Principal": { "Service": "logs.region.amazonaws.com" }, "Action": [ "kms:Encrypt*", "kms:Decrypt*", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:Describe*" ], "Resource": "<arn of key used for ETL/crawler cloudwatch encryption>" }

    For more information about key policies, see Using Key Policies in Amazon KMS in the Amazon Key Management Service Developer Guide.

    In the IAM policy attach the logs:AssociateKmsKey permission:

    { "Effect": "Allow", "Principal": { "Service": "logs.region.amazonaws.com" }, "Action": [ "logs:AssociateKmsKey" ], "Resource": "<arn of key used for ETL/crawler cloudwatch encryption>" }
  6. Any ETL job that uses an encrypted job bookmark needs the following permissions.

    { "Version": "2012-10-17", "Statement": { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:Encrypt" ], "Resource": "<key-arns-used-for-job-bookmark-encryption>" } }
  7. On the Amazon Glue console, choose Settings in the navigation pane.

    1. On the Data catalog settings page, encrypt your Data Catalog by selecting Metadata encryption. This option encrypts all the objects in the Data Catalog with the Amazon KMS key that you choose.

    2. For Amazon KMS key, choose aws/glue. You can also choose a Amazon KMS key that you created.

    Important

    Amazon Glue supports only symmetric customer master keys (CMKs). The Amazon KMS key list displays only symmetric keys. However, if you select Choose a Amazon KMS key ARN, the console lets you enter an ARN for any key type. Ensure that you enter only ARNs for symmetric keys.

    When encryption is enabled, the client that is accessing the Data Catalog must have Amazon KMS permissions.

  8. In the navigation pane, choose Security configurations. A security configuration is a set of security properties that can be used to configure Amazon Glue processes. Then choose Add security configuration. In the configuration, choose any of the following options:

    1. Select S3 encryption. For Encryption mode, choose SSE-KMS. For the Amazon KMS key, choose aws/s3 (ensure that the user has permission to use this key). This enables data written by the job to Amazon S3 to use the Amazon managed Amazon Glue Amazon KMS key.

    2. Select CloudWatch logs encryption, and choose a CMK. (Ensure that the user has permission to use this key). For more information, see Encrypt Log Data in CloudWatch Logs Using Amazon KMS in the Amazon Key Management Service Developer Guide.

      Important

      Amazon Glue supports only symmetric customer master keys (CMKs). The Amazon KMS key list displays only symmetric keys. However, if you select Choose a Amazon KMS key ARN, the console lets you enter an ARN for any key type. Ensure that you enter only ARNs for symmetric keys.

    3. Choose Advanced properties, and select Job bookmark encryption. For the Amazon KMS key, choose aws/glue (ensure that the user has permission to use this key). This enables encryption of job bookmarks written to Amazon S3 with the Amazon Glue Amazon KMS key.

  9. In the navigation pane, choose Connections.

    1. Choose Add connection to create a connection to the Java Database Connectivity (JDBC) data store that is the target of your ETL job.

    2. To enforce that Secure Sockets Layer (SSL) encryption is used, select Require SSL connection, and test your connection.

  10. In the navigation pane, choose Jobs.

    1. Choose Add job to create a job that transforms data.

    2. In the job definition, choose the security configuration that you created.

  11. On the Amazon Glue console, run your job on demand. Verify that any Amazon S3 data written by the job, the CloudWatch Logs written by the job, and the job bookmarks are all encrypted.