How to specify configuration options for a Delta Lake data store - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

How to specify configuration options for a Delta Lake data store

When you configure a crawler for a Delta Lake data store, you specify these configuration parameters:

Connection

Optionally select or add a Network connection to use with this Amazon S3 target. For information about connections, see Connecting to data.

Create tables for querying

Select how you want to create the Delta Lake tables:

  • Create Native tables: Allow integration with query engines that support querying of the Delta transaction log directly.

  • Create Symlink tables: Create a symlink manifest folder with manifest files partitioned by the partition keys, based on the specified configuration parameters.

Enable write manifest (configurable only you've selected to Create Symlink tables for a Delta Lake source

Select whether to detect table metadata or schema changes in the Delta Lake transaction log; it regenerates the manifest file. You should not choose this option if you configured an automatic manifest update with Delta Lake SET TBLPROPERTIES.

Include delta lake table path(s)

Specify one or more Amazon S3 paths to Delta tables as s3://bucket/prefix/object.

Specifying crawling a Delta Lake data store.