Enabling cross-account federated queries - Amazon Athena
Considerations and limitationsRequired permissionsSharing a data source in Account A with Account BAdding a shared data source from Account A to Account BTroubleshooting
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Enabling cross-account federated queries

Federated query allows you to query data sources other than Amazon S3 using data source connectors deployed on Amazon Lambda. The cross-account federated query feature allows the Lambda function and the data sources that are to be queried to be located in different accounts.

As a data administrator, you can enable cross-account federated queries by sharing your data connector with a data analyst's account or, as a data analyst, by using a shared Lambda ARN from a data administrator to add to your account. When configuration changes are made to a connector in the originating account, the updated configuration is automatically applied to the shared instances of the connector in other user's accounts.

Considerations and limitations

  • The cross-account federated query feature is available for non-Hive metastore data connectors that use a Lambda-based data source.

  • The feature is not available for the Amazon Glue Data Catalog data source type. For information about cross-account access to Amazon Glue Data Catalogs, see Cross-account access to Amazon Glue data catalogs.

  • If the response from your connector's Lambda function exceeds the Lambda response size limit of 6MB, Athena automatically encrypts, batches, and spills the response to an Amazon S3 bucket that you configure. The entity running the Athena query must have access to the spill location in order for Athena to read the spilled data. We recommend that you set an Amazon S3 lifecycle policy to delete objects from the spill location since the data is not needed after the query completes.

  • Using federated queries across Amazon Web Services Regions is not supported.

Required permissions

  • For data administrator Account A to share a Lambda function with data analyst Account B, Account B requires Lambda invoke function and spill bucket access. Accordingly, Account A should add a resource-based policy to the Lambda function and principal access to its spill bucket in Amazon S3.

    1. The following policy grants Lambda invoke function permissions to Account B on a Lambda function in Account A.

      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CrossAccountInvocationStatement",
      "Effect": "Allow",
      "Principal": {
        "AWS": ["arn:aws:iam::account-B-id:user/username"]
      }, 
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:aws-region:account-A-id:function:lambda-function-name"
    }
  ]
}

    2. The following policy allows spill bucket access to the principal in Account B.

      {
    "Version": "2008-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": ["arn:aws:iam::account-B-id:user/username"]
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
             ],
            "Resource": [
                "arn:aws:s3:::spill-bucket",
                "arn:aws:s3:::spill-bucket/*"
            ]
        }
     ]
 }

    3. If the Lambda function is encrypting the spill bucket with a Amazon KMS key instead of the default encryption offered by the federation SDK, the Amazon KMS key policy in Account A must grant access to the user in Account B, as in the following example.

      { 
    "Sid": "Allow use of the key", 
    "Effect": "Allow", 
    "Principal": 
    { 
       "AWS": ["arn:aws:iam::account-B-id:user/username"] 
    }, 
    "Action": [ "kms:Decrypt" ], 
    "Resource": "*" // Resource policy that gets placed on the KMS key. 
 }

  • For Account A to share its connector with Account B, Account B must create a role called AthenaCrossAccountCreate-account-A-id that Account A assumes by calling the Amazon Security Token Service AssumeRole API action.

    The following policy, which allows the CreateDataCatalog action, should be created in Account B and added to the AthenaCrossAccountCreate-account-A-id role that Account B creates for Account A.

    {
 "Effect": "Allow",
 "Action": "athena:CreateDataCatalog",
 "Resource": "arn:aws:athena:*:account-B-id:datacatalog/*"
}

Sharing a data source in Account A with Account B

After permissions are in place, you can use the Data sources page in the Athena console to share a data connector in your account (Account A) with another account (Account B). Account A retains full control and ownership of the connector. When Account A makes configuration changes to the connector, the updated configuration applies to the shared connector in Account B.

To share a Lambda data source in Account A with Account B

  1. Open the Athena console at https://console.amazonaws.cn/athena/.

  2. If the console navigation pane is not visible, choose the expansion menu on the left.

    Choose the expansion menu.

  3. Choose Data sources.

  4. On the Data sources page, choose the link of the connector that you want to share.

  5. On the details page for a Lambda data source, choose the Share option in the top right corner.

    Choose Share.

  6. In the Share Lambda-name with another account dialog box, enter the required information.

    • For Data source name, enter the name of the copied data source as you want it to appear in the other account.

    • For Account ID, enter the ID of the account with which you want to share your data source (in this case, Account B).

  7. Choose Share. The shared data connector that you specified is created in Account B. Configuration changes to the connector in Account A apply to the connector in Account B.

Adding a shared data source from Account A to Account B

As a data analyst, you may be given the ARN of a connector to add to your account from a data administrator. You can use the Data sources page of the Athena console to add the Lambda ARN provided by your administrator to your account.

To add the Lambda ARN of a shared data connector to your account

  1. Open the Athena console at https://console.amazonaws.cn/athena/.

  2. If you are using the new console experience and the navigation pane is not visible, choose the expansion menu on the left.

  3. Choose Data sources.

  4. On the Data sources page, choose Connect data source.

  5. Choose Custom or shared connector.

  6. In the Lambda function section, make sure that the option Use an existing Lambda function is selected.

    Specifying the Lambda ARN of another account.

  7. For Choose or enter a Lambda function, enter the Lambda ARN of Account A.

  8. Choose Connect data source.

Troubleshooting

If you receive an error message that Account A does not have the permissions to assume a role in Account B, make sure that the name of the role created in Account B is spelled correctly and that it has the proper policy attached.