Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Granting cross-account access

Granting access to Data Catalog resources across accounts enables your extract, transform, and load (ETL) jobs to query and join data from different accounts.

Methods for granting cross-account access in Amazon Glue

You can grant access to your data to external Amazon accounts by using Amazon Glue methods or by using Amazon Lake Formation cross-account grants. The Amazon Glue methods use Amazon Identity and Access Management (IAM) policies to achieve fine-grained access control. Lake Formation uses a simpler GRANT/REVOKE permissions model similar to the GRANT/REVOKE commands in a relational database system.

This section describes using the Amazon Glue methods. For information about using Lake Formation cross-account grants, see Granting Lake Formation Permissions in the Amazon Lake Formation Developer Guide.

There are two Amazon Glue methods for granting cross-account access to a resource:

Granting cross-account access using a resource policy

The following are the general steps for granting cross-account access using a Data Catalog resource policy:

Granting cross-account access using an IAM role

The following are the general steps for granting cross-account access using an IAM role:

For more information about using IAM to delegate permissions, see Access management in the IAM User Guide. For more information about users, groups, roles, and permissions, see Identities (users, groups, and roles) in the IAM User Guide.

For a comparison of these two approaches, see How IAM roles differ from resource-based policies in the IAM User Guide. Amazon Glue supports both options, with the restriction that a resource policy can grant access only to Data Catalog resources.

For example, to give the Dev role in Account B access to database db1 in Account A, attach the following resource policy to the catalog in Account A.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "glue:GetDatabase"
      ],
      "Principal": {"AWS": [
        "arn:aws-cn:iam::account-B-id:role/Dev"
      ]},
      "Resource": [
        "arn:aws:glue:us-east-1:account-A-id:catalog",      
        "arn:aws:glue:us-east-1:account-A-id:database/db1"
      ]
    }
  ]
}

In addition, Account B would have to attach the following IAM policy to the Dev role before it would actually get access to db1 in Account A.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "glue:GetDatabase"
      ],
      "Resource": [
        "arn:aws:glue:us-east-1:account-A-id:catalog",      
        "arn:aws:glue:us-east-1:account-A-id:database/db1"
      ]
    }
  ]
}

Adding or updating the Data Catalog resource policy

You can add or update the Amazon Glue Data Catalog resource policy using the console, API, or Amazon Command Line Interface (Amazon CLI).

Making a cross-account API call

All Amazon Glue Data Catalog operations have a CatalogId field. If the required permissions have been granted to enable cross-account access, a caller can make Data Catalog API calls across accounts. The caller does this by passing the target Amazon account ID in CatalogId so as to access the resource in that target account.

If no CatalogId value is provided, Amazon Glue uses the caller's own account ID by default, and the call is not cross-account.

Making a cross-account ETL call

Some Amazon Glue PySpark and Scala APIs have a catalog ID field. If all the required permissions have been granted to enable cross-account access, an ETL job can make PySpark and Scala calls to API operations across accounts by passing the target Amazon account ID in the catalog ID field to access Data Catalog resources in a target account.

If no catalog ID value is provided, Amazon Glue uses the caller's own account ID by default, and the call is not cross-account.

For PySpark APIs that support catalog_id, see GlueContext class. For Scala APIs that support catalogId, see Amazon Glue Scala GlueContext APIs.

The following example shows the permissions required by the grantee to run an ETL job. In this example, grantee-account-id is the catalog-id of the client running the job and grantor-account-id is the owner of the resource. This example grants permission to all catalog resources in the grantor's account. To limit the scope of resources granted, you can provide specific ARNs for the catalog, database, table, and connection.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "glue:GetConnection",
                "glue:GetDatabase",
                "glue:GetTable",
                "glue:GetPartition" 
            ],
            "Principal": {"AWS": ["arn:aws-cn:iam::grantee-account-id:root"]},
            "Resource": [
                "arn:aws:glue:us-east-1:grantor-account-id:*"
            ]
        }
    ]
}

Given that the client in Account A already has permission to create and run ETL jobs, the following are the basic steps to set up an ETL job for cross-account access:

Cross-account CloudTrail logging

When an Amazon Glue extract, transform, and load (ETL) job accesses the underlying data of a Data Catalog table shared through Amazon Lake Formation cross-account grants, there is additional Amazon CloudTrail logging behavior.

For purposes of this discussion, the Amazon account that shared the table is the owner account, and the account that the table was shared with is the recipient account. When an ETL job in the recipient account accesses data in the table in the owner account, the data-access CloudTrail event that is added to the logs for the recipient account gets copied to the owner account's CloudTrail logs. This is so owner accounts can track data accesses by the various recipient accounts. By default, the CloudTrail events do not include a human-readable principal identifier (principal ARN). An administrator in the recipient account can opt in to include the principal ARN in the logs.

For more information, see Cross-account CloudTrail logging in the Amazon Lake Formation Developer Guide.

Cross-account resource ownership and billing

When a user in one Amazon account (Account A) creates a new resource such as a database in a different account (Account B), that resource is then owned by Account B, the account where it was created. An administrator in Account B automatically gets full permissions to access the new resource, including reading, writing, and granting access permissions to a third account. The user in Account A can access the resource that they just created only if they have the appropriate permissions granted by Account B.

Storage costs and other costs that are directly associated with the new resource are billed to Account B, the resource owner. The cost of requests from the user who created the resource are billed to the requester's account, Account A.

For more information about Amazon Glue billing and pricing, see How Amazon Pricing Works.

Cross-account access limitations

Amazon Glue cross-account access has the following limitations: