Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Accessing Amazon S3 tables using the Amazon Glue Iceberg REST endpoint

Once your S3 table buckets are integrated with the Amazon Glue Data Catalog you can use the Amazon Glue Iceberg REST endpoint to connect to your S3 tables from Apache Iceberg-compatible clients, such as PyIceberg or Spark. The Amazon Glue Iceberg REST endpoint implements the Iceberg REST Catalog Open API specification which provides a standardized interface for interacting with Iceberg tables. To access S3 tables using the endpoint you need to configure permissions through a combination of IAM policies and Amazon Lake Formation grants. The following sections explain how to set up access, including creating the necessary IAM role, defining the required policies, and establishing Lake Formation permissions for both database and table-level access.

For an end to end walkthrough using PyIceberg, see Access data in Amazon S3 Tables using PyIceberg through the Amazon Glue Iceberg REST endpoint.

Authentication methods

The Amazon Glue Iceberg REST endpoint supports two authentication methods for reaching your Amazon S3 tables:

Create an IAM role for your client

To access tables through Amazon Glue endpoints, you need to create an IAM role with permissions to Amazon Glue and Lake Formation actions. This procedure explains how to create this role and configure its permissions.

Define access in Lake Formation

Lake Formation provides fine-grained access control for your data lake tables. When you integrated your S3 bucket with the Amazon Glue Data Catalog, your tables were automatically registered as resources in Lake Formation. To access these tables, you must grant specific Lake Formation permissions to your IAM identity, in addition to its IAM policy permissions.

The following steps explain how to apply Lake Formation access controls to allow your Iceberg client to connect to your tables. You must sign in as a data lake administrator to apply these permissions.

Allow external engines to access table data

In Lake Formation, you must enable full table access for external engines to access data. This allows third-party applications to get temporary credentials from Lake Formation when using an IAM role that has full permissions on the requested table.

Grant Lake Formation permissions on your table resources

Next, grant Lake Formation permissions to the IAM role you created for your Iceberg-compatible client. These permissions will allow the role to create and manage tables in your namespace. You need to provide both database and table-level permissions. For more information, see Granting Lake Formation permission on a table or database.

Set up your environment to use the endpoint

After you have setup the IAM role with the permissions required for table access you can use it to run Iceberg clients from your local machine by configuring the Amazon CLI with your role, using the following command:

aws sts assume-role --role-arn "arn:aws:iam::<accountid>:role/<glue-irc-role>" --role-session-name <glue-irc-role>

To access tables through the Amazon Glue REST endpoint, you need to initialize a catalog in your Iceberg-compatible client. This initialization requires specifying custom properties, including sigv4 properties, the endpoint URI and the warehouse location. Specify these properties as follows:

The following example shows how to initialize a pyIceberg catalog.

rest_catalog = load_catalog(
        s3tablescatalog,
**{
"type": "rest",
"warehouse": "<accountid>:s3tablescatalog/<table-bucket-name>",
"uri": "https://glue.<region>.amazonaws.com/iceberg",
"rest.sigv4-enabled": "true",
"rest.signing-name": "glue",
"rest.signing-region": region
        }
)

For additional information about the Amazon Glue Iceberg REST endpoint implementation, see Connecting to the Data Catalog using Amazon Glue Iceberg REST endpoint in the Amazon Glue User Guide.

Accessing Amazon S3 tables with an OAuth 2.0 bearer token

External integration partners and third-party query engines can access your Amazon S3 tables using an OAuth 2.0 bearer token instead of SigV4. The client obtains a short-lived, Glue-scoped access token from Amazon Sign-In through a token exchange defined in RFC 8693 on the IETF Datatracker website, then presents it to the Amazon Glue Iceberg REST endpoint. Amazon Glue validates the token and enforces the same Lake Formation permissions described earlier.

Before a client can use this method, complete the identity-federation setup (IAM OIDC identity provider, an IAM role with a web-identity trust policy, and Lake Formation grants). The partner exchanges an end user's JSON Web Token (JWT) for the access token. For the full setup and token exchange details, see Configure OAuth access to Amazon Glue Iceberg REST Catalog in the Amazon Sign-In User Guide.