Federate to Snowflake Iceberg Catalog

Sign in to the the console and open the Lake Formation console at https://console.amazonaws.cn/lakeformation/.

Choose the preferred Amazon region in the top-right page section.

In the left navigation pane, choose Catalogs.

Choose Create Catalog to open the Create Catalog Workflow.

In Choose data source step, select Snowflake from the available options.

In Set catalog details step, you provide three information - catalog details, connection details, and registration details.

In catalog details container, provide a unique name to your Amazon Glue federated catalog and enter the name of the existing Snowflake catalog.

In connections details container, you can either choose from an existing connection that you have access or provide configuration to create a new connector.

New connection configurations include:

Create an IAM role that Amazon Glue and Lake Formation service principals can use to access secret in Amazon Secrets Manager and Amazon S3 locations of remote Iceberg tables respectively. Select the IAM role in the registration dropdown. Refer to step 2 and 3 in following CLI section for IAM policy details.

Select Test Connection to test whether your connection properties and IAM role access are configured correctly.

Select Next to review your settings.

Select Create Catalog in review page.

Create an Amazon Secrets Manager secret

The Amazon Glue connector supports two authentication types - OAuth2 and Custom. When using OAuth2 option, use Amazon Secrets Manager to store client secret of the Snowflake service principal. You will later use this secret when creating the Amazon Glue Connection. For Custom authentication, use Amazon Secrets Manager to store and retrieve the access token.

In the following example, replace your-snowflake-secret, client_secret, region with your own information.

aws secretsmanager create-secret \
--name your-snowflake-secret \
--description "Snowflake secret" \
--secret-string '{
"USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET": "client_secret"
}' \
--region region

Create an IAM role which gives Amazon Glue connection object access to secret created in previous step

The Amazon Glue connection object requires access to the Amazon Secrets Manager secret when you use Amazon Secrets Manager to store, retrieve, and refresh your OAuth secret token. The Amazon Glue connection object also requires access to create, describe, and use the Amazon VPC network interfaces when you use a Amazon VPC endpoint to restrict connectivity to your Snowflake account.

Create an IAM policy and attach it to an IAM role. Add Amazon Glue service principal to the trust policy.

In the following example, replace your-secrets-manager-ARN, your-vpc-id, and your-subnet-id1 with your own information.

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [{
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret",
                "secretsmanager:PutSecretValue"
            ],
            "Resource": [
                "your-secrets-manager-ARN"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateNetworkInterface",
                "ec2:DeleteNetworkInterface",
                "ec2:DescribeNetworkInterfaces"
            ],
            "Resource": "*",
            "Condition": {
                "ArnEquals": {
                    "ec2:Vpc": "arn:aws:ec2:region:account-id:vpc/your-vpc-id",
                    "ec2:Subnet": ["arn:aws:ec2:region:account-id:subnet/your-subnet-id1"]
                }
            }
        }
    ]
}

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [{
        "Effect": "Allow",
        "Principal": {
            "Service": "glue.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }]
}

Create an IAM policy that gives Lake Formation read access to catalog's Amazon S3 location

As the catalog owner of a federated catalog in Data Catalog, you use Lake Formation to grant coarse-grained table access, fine-grained - column-level, row-level, and cell-level - access, and tag-based access to your data teams. Lake Formation uses an IAM role that gives it access to the underlying Amazon S3 locations of your remote Iceberg tables. This access allows Lake Formation to vend scoped access credentials to analytics engines querying remote tables.

Create IAM policy and attach to an IAM role. Add Lake Formation service principal to the role trust policy.

In the following example, replace amzn-s3-demo-bucketN and your-kms-key with your own information.

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [{
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket1/*",
                "arn:aws:s3:::amzn-s3-demo-bucket2/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket1",
                "arn:aws:s3:::amzn-s3-demo-bucket2"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:Encrypt"
            ],
            "Resource": [
                "your-kms-key"
            ]
        }
    ]
}

{
    "Version": "2012-10-17",		 	 	 
    "Statement": [{
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "Service": "lakeformation.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }]
}

Create an Amazon Glue connection object

Amazon Glue connector supports OAuth2 and Custom authentication methods. Data Catalog connection object supports CATALOG_CASING_FILTER configuration that allows you to bring either lowercase or uppercase objects in your remote catalog to Data Catalog.

The following example uses OAuth2 authentication configuration to create an Amazon Glue connection. Replace highlighted sections with your information.

aws glue create-connection \
    --connection-input '{
"Name": "your-glue-connection-to-snowflake-account",
"ConnectionType": "SNOWFLAKEICEBERGRESTCATALOG",
"ConnectionProperties": {
    "INSTANCE_URL": "your-snowflake-account-URL",
    "ROLE_ARN": "your-IAM-role-for-secrets-and-VPC-access",
    "CATALOG_CASING_FILTER": "LOWERCASE_ONLY"
},
"AuthenticationConfiguration": {
    "AuthenticationType": "OAUTH2",
    "OAuth2Properties": {
        "OAuth2GrantType": "CLIENT_CREDENTIALS",
        "TokenUrl": "your-internal-or-external-token-server-url",
        "OAuth2ClientApplication": {
            "UserManagedClientApplicationClientId": "our-client-id"
        },
        "TokenUrlParametersMap": {
            "Scope": "all-apis"
        }
    },
    "SecretArn": "arn:aws:secretsmanager:your-aws-region:your-aws-account-id:secret:snowflake-secret"
}
}'

Register Amazon Glue connection as a Lake Formation resource

Using the Amazon Glue connection object (created in Step 4) and IAM role (created in Step 3), you can now register the Amazon Glue connection object as a Lake Formation managed resource.

Replace your-glue-connector-arn and your-IAM-role-ARN-having-LF-access with your information.

aws lakeformation register-resource \
    --resource-arn your-glue-connector-arn \
    --role-arn your-IAM-role-ARN-having-LF-access \
    --with-federation \
    --with-privileged-access

Create a federated catalog in Data Catalog

After creating an Amazon Glue connection object and registering it with Lake Formation, you can create a federated catalog in the Data Catalog:

Provide the federated catalog a unique name at your-federated-catalog-name, reference the catalog in Snowflake at catalog-name-in-Snowflake, and input connection name created earlier at your-glue-connection-name.

aws glue create-catalog \
    --name your-federated-catalog-name \
    --catalog-input '{
    "FederatedCatalog": {
        "Identifier": catalog-name-in-Snowflake",
        "ConnectionName": your-glue-connection-name"
    },
    "CreateTableDefaultPermissions": [],
    "CreateDatabaseDefaultPermissions": []
}'