Setting the partition index crawler configuration option - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Setting the partition index crawler configuration option

The Data Catalog supports partition indexes to provide efficient lookup for specific partitions. For more information, see Working with partition indexes in Amazon Glue. The Amazon Glue crawler creates partition indexes for Amazon S3 and Delta Lake targets by deafult.

When you define a cralwer, the option to Create partition indexes automatically is enabled by default under Advanced options on the Set output and scheduling page.

To disable this option, you can unselect the checkbox Create partition indexes automatically in the console. You can also disable this option by using the crawler API, set the CreatePartitionIndex in the Configuration. The default value is true.

Usage notes for partition indexes

  • Tables created by the crawler do not have the variable partition_filtering.enabled by default. For more information, see Amazon Glue partition indexing and filtering.

  • Creating partition indexes for encrypted partitions is not supported.