Configuring a crawler to populate the Amazon Glue Data Catalog

Navigate to the Amazon Glue console.

In the navigation pane, choose Crawlers.

Choose Add crawler.

On the Add crawler page, do the following:

1. Enter a name for your crawler, such as IoTSiteWiseDataCrawler, and then choose Next.

2. For Crawler source type, choose Data stores, and then choose Next.

3. On the Add a data store page, do the following:

   1. For Choose a data store, choose S3.

   2. In Include path, enter s3://DOC-EXAMPLE-BUCKET1 to add your asset data bucket as a data store. Replace DOC-EXAMPLE-BUCKET1 with the bucket name that you chose when you created the stack.

   3. Choose Next.

      

4. On the Add another data store page, choose No, and then choose Next.

5. On the Choose an IAM role page, do the following:

   1. To create a new service role that allows Amazon Glue to access the S3 bucket, choose Create an IAM role.

   2. Enter a suffix for your role's name, such as IoTSiteWiseDataCrawler.

   3. Choose Next.

6. For Frequency, choose Hourly, and then choose Next. The crawler updates the tables with new data each time it runs, so you can choose any frequency that fits your use case.

7. On the Configure the crawler's output page, do the following:

   1. Choose Add database to create an Amazon Glue database for your asset data.

   2. Enter a name for the database, such as iot_sitewise_asset_database.

   3. Choose Create.

   4. Choose Next.

8. Review the crawler details, and then choose Finish.

   

On the Crawlers page, select the check box for your new crawler, and then choose Run crawler.

Wait until the crawler finishes and has a status of Ready.

The crawler can take several minutes to run, and its status updates automatically.

In the navigation pane, choose Tables.

You should see two new tables: asset_metadata and asset_property_updates.