Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Troubleshooting Lake Formation

If you encounter issues when working with Amazon Lake Formation, consult the topics in this section.

General troubleshooting

Use the information here to help you diagnose and fix various Lake Formation issues.

Error: Insufficient Lake Formation permissions on <Amazon S3 location>

An attempt was made to create or alter a Data Catalog resource without data location permissions on the Amazon S3 location pointed to by the resource.

If a Data Catalog database or table points to an Amazon S3 location, when you grant the Lake Formation permissions CREATE_TABLE or ALTER, you must also grant the DATA_LOCATION_ACCESS permission on the location. If you are granting these permissions to external accounts or to organizations, you must include the grant option.

After these permissions are granted to an external account, the data lake administrator in that account must then grant the permissions to principals (users or roles) in the account. When granting the DATA_LOCATION_ACCESS permission that was received from another account, you must specify the catalog ID (Amazon account ID) of the owner account. The owner account is the account that registered the location.

For more information, see Underlying data access control and Granting data location permissions.

Error: "Insufficient encryption key permissions for Glue API"

An attempt was made to grant Lake Formation permissions without Amazon Identity and Access Management (IAM) permissions on the Amazon KMS encryption key for an encrypted Data Catalog.

My Amazon Athena or Amazon Redshift query that uses manifests is failing

Lake Formation does not support queries that use manifests.

Error: "Insufficient Lake Formation permission(s): Required create tag on catalog"

The user/role must be a data lake administrator.

Error when deleting invalid data lake administrators

You should delete all invalid data lake administrators (deleted IAM roles that are defined as data lake administrators) simultaneously. If you try to delete invalid data lake administrators separately, Lake Formation throws invalid principal error.

Troubleshooting cross-account access

Use the information here to help you diagnose and fix cross-account access issues.

I granted a cross-account Lake Formation permission but the recipient can't see the resource

Principals in the recipient account can see the Data Catalog resource but can't access the underlying data

Principals in the recipient account must have the required Amazon Identity and Access Management (IAM) permissions. For details, see Accessing the underlying data of a shared table.

Error: "Association failed because the caller was not authorized" when accepting a Amazon RAM resource share invitation

After granting access to a resource to a different account, when the receiving account attempts to accept the resource share invitation, the action fails.

$ aws ram get-resource-share-associations --association-type PRINCIPAL --resource-share-arns arn:aws-cn:ram:aws-region:444444444444:resource-share/e1d1f4ba-xxxx-xxxx-xxxx-xxxxxxxx5d8d
{
    "resourceShareAssociations": [
        {
            "resourceShareArn": "arn:aws-cn:ram:aws-region:444444444444:resource-share/e1d1f4ba-xxxx-xxxx-xxxx-xxxxxxxx5d8d
",
            "resourceShareName": "LakeFormation-MMCC0XQBH3Y",
            "associatedEntity": "5815803XXXXX",
            "associationType": "PRINCIPAL",
            "status": "FAILED",
            "statusMessage": "Association failed because the caller was not authorized.",
            "creationTime": "2021-07-12T02:20:10.267000+00:00",
            "lastUpdatedTime": "2021-07-12T02:20:51.830000+00:00",
            "external": true
        }
    ]
}

The error occurs because the glue:PutResourcePolicy is invoked by Amazon Glue when the receiving account accepts the resource share invitation. To resolve the issue, allow the glue:PutResourcePolicy action by the assumed role used by the producer/grantor account.

Error: "Not authorized to grant permissions for the resource"

An attempt was made to grant cross-account permissions on a database or table that is owned by another account. When a database or table is shared with your account, as a data lake administrator, you can grant permissions on it only to users in your account.

Error: "Access denied to retrieve Amazon Organization information"

Your account is an Amazon Organizations management account and you do not have the required permissions to retrieve organization information, such as organizational units in the account.

For more information, see Required permissions for cross-account grants.

Error: "Organization <organization-ID> not found"

An attempt was made to share a resource with an organization, but sharing with organizations is not enabled. Enable resource sharing with organizations.

For more information, see Enable Sharing with Amazon Organizations in the Amazon RAM User Guide.

Error: "Insufficient Lake Formation permissions: Illegal combination"

A user shared a Data Catalog resource while Lake Formation permissions were granted to the IAMAllowedPrincipals group for the resource. The user must revoke all Lake Formation permissions from IAMAllowedPrincipals before sharing the resource.

ConcurrentModificationException on grant/revoke requests to external accounts

When users make multiple concurrent grant and/or revoke permission requests for a principal on LF-Tag policies, then Lake Formation throws ConcurrentModificationException. Users need to catch the exception and retry the failed the grant/revoke request. Using batch versions of the GrantPermissions/RevokePermissions API operations - BatchGrantPermissionsand BatchRevokePermissions alleviates this problem to an extent by reducing the number of concurrent grant/revoke requests.

Error when using Amazon EMR to access data shared via cross-account

When you use Amazon EMR to access data shared with you from another account, some Spark libraries will attempt to call Glue:GetUserDefinedFunctions API operation. Since versions 1 and 2 of the Amazon RAM managed permissions does not support this action, you receive the following error message:

"ERROR: User: arn:aws:sts::012345678901:assumed-role/my-spark-role/i-06ab8c2b59299508a is not authorized to perform: glue:GetUserDefinedFunctions on resource: arn:exampleCatalogResource because no resource-based policy allows the glue:GetUserDefinedFunctions action"

To resolve this error, the data lake administrator who created the resource share must update the Amazon RAM managed permissions attached to the resource share. Version 3 of the Amazon RAM managed permissions allows principals to perform the glue:GetUserDefinedFunctions action.

If you create a new resource share, Lake Formation applies the latest version of the Amazon RAM managed permission by default, and no action is required by you. To enable cross-account data access for existing resource shares, you need to update the Amazon RAM managed permissions to version 3.

You can view the Amazon RAM permissions assigned to resources shared with you in Amazon RAM. The following permissions are included in version 3:

Databases
  AWSRAMPermissionGlueDatabaseReadWriteForCatalog 
  AWSRAMPermissionGlueDatabaseReadWrite
    
Tables
  AWSRAMPermissionGlueTableReadWriteForCatalog
  AWSRAMPermissionGlueTableReadWriteForDatabase
    
AllTables
  AWSRAMPermissionGlueAllTablesReadWriteForCatalog
  AWSRAMPermissionGlueAllTablesReadWriteForDatabase
   

To update Amazon RAM managed permissions version of existing resource shares

You (data lake administrator) can either update Amazon RAM managed permissions to a newer version by following instructions in the Amazon RAM User Guide or you can revoke all existing permissions for the resource type and regrant them. If you revoke permissions, Amazon RAM deletes the Amazon RAM resource share associated with the resource type. When you regrant permissions, Amazon RAM creates new resource shares attaching the latest version of Amazon RAM managed permissions.

Troubleshooting blueprints and workflows

Use the information here to help you diagnose and fix blueprint and workflow issues.

My blueprint failed with "User: <user-ARN> is not authorized to perform: iam:PassRole on resource: <role-ARN>"

An attempt was made to create a blueprint by a user who does not have sufficient permissions to pass the chosen role.

Update the user’s IAM policy to be able to pass the role, or ask the user to choose a different role with the required passrole permissions.

For more information, see Lake Formation personas and IAM permissions reference.

My workflow failed with "User: <user-ARN> is not authorized to perform: iam:PassRole on resource: <role-ARN>"

The role that you specified for the workflow did not have an inline policy allowing the role to pass itself.

For more information, see (Optional) Create an IAM role for workflows.

A crawler in my workflow failed with "Resource does not exist or requester is not authorized to access requested permissions"

One possible cause is that the passed role did not have sufficient permissions to create a table in the target database. Grant the role the CREATE_TABLE permission on the database.

A crawler in my workflow failed with "An error occurred (AccessDeniedException) when calling the CreateTable operation..."

One possible cause is that the workflow role did not have data location permissions on the target storage location. Grant data location permissions to the role.

For more information, see DATA_LOCATION_ACCESS.