Enable Lake Formation with S3 Tables integration with Data Catalog - Amazon Lake Formation
PrerequisitesUsing Amazon CLI
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Enable Lake Formation with S3 Tables integration with Data Catalog

This section describes the workflow to migrate access control from IAM privileges to IAM with Amazon Lake Formation grants for Amazon S3 Tables integrated with the Amazon Glue Data Catalog.

Important

Enabling Amazon Lake Formation access control will revoke all existing IAM-based access to your S3 Tables resources. After completing Step 1, users and roles that previously accessed data through IAM permissions will immediately lose access. You must grant Lake Formation permissions in Step 2 before users can query data again. Plan this migration during a maintenance window and coordinate with your data team.

Prerequisites

For read/write access to S3 Tables, in addition to Lake Formation permissions, principals also need the lakeformation:GetDataAccess IAM permission. With this permission, Lake Formation grants the request for temporary credentials to access the data.

Using Amazon CLI

  1. Step 1: Register bucket with Lake Formation using IAM role

    Register the S3 Tables resource with Lake Formation.

    Note

    If you have an existing role, ensure hybrid access is false.

    aws lakeformation register-resource \
  --resource-arn "arn:aws:s3tables:us-east-1:AWSAccountID:bucket/*" \
  --role-arn "arn:aws:iam::AWSAccountID:role/service-role/LFAccessRole" \
  --with-federation

  2. Step 2: Update Amazon Glue catalog to enable Lake Formation access control

    Update the catalog with empty CreateDatabaseDefaultPermissions and CreateTableDefaultPermissions (set to []) and set OverwriteChildResourcePermissionsWithDefault to Accept. This removes IAM-based access from all existing child resources and allows the catalog and its objects to be managed using Lake Formation grants.

    aws glue update-catalog \
  --catalog-id "s3tablescatalog" \
  --catalog-input '{
    "FederatedCatalog": {
        "Identifier": "arn:aws:s3tables:us-east-1:AWSAccountID:bucket/*",
        "ConnectionName": "aws:s3tables"
    },
    "CreateDatabaseDefaultPermissions": [],
    "CreateTableDefaultPermissions": [],
    "OverwriteChildResourcePermissionsWithDefault": "Accept",
    "AllowFullTableExternalDataAccess": "True"
  }'

  3. Step 3: Grant Lake Formation permissions to your data team

    Grant Lake Formation permissions to the principals (roles, users, or groups) that need access. For example, to grant full-table read access to a role:

    aws lakeformation grant-permissions \
  --principal DataLakePrincipalIdentifier=arn:aws:iam::AWSAccountID:role/DataTeamRole \
  --resource '{
    "Table": {
        "CatalogId": "AWSAccountID",
        "DatabaseName": "s3tablescatalog/table-bucket-name/namespace",
        "TableWildcard": {}
    }
  }' \
  --permissions "SELECT" "DESCRIBE"

    Repeat for each principal and resource combination as needed.