Tutorial: Using the Amazon Glue Connector for Elasticsearch
Elasticsearch is a popular open-source search and analytics engine for use cases such as log
analytics, real-time application monitoring, and clickstream analysis. You can use OpenSearch as a
data store for your extract, transform, and load (ETL) jobs by configuring the
Amazon Glue Connector for Elasticsearch in Amazon Glue Studio. This connector
is available for free from
Amazon Web Services Marketplace
The Amazon Web Services Marketplace
Elasticsearch Spark Connector
In this tutorial, we will show how to connect to your Amazon OpenSearch Service nodes with a minimal number of steps.
Topics
- Prerequisites
- Step 1: (Optional) Create an Amazon secret for your OpenSearch cluster information
- Step 2: Subscribe to the connector
- Step 3: Activate the connector in Amazon Glue Studio and create a connection
- Step 4: Configure an IAM role for your ETL job
- Step 5: Create a job that uses the OpenSearch connection
- Step 6: Run the job
Prerequisites
To use this tutorial, you must have the following:
-
Access to Amazon Glue Studio
-
Access to an OpenSearch cluster in the Amazon Cloud
-
Configured access to the Amazon VPC that contains your data store, as described in Configure a VPC for your ETL job.
-
Configured permissions according to Review IAM permissions needed for ETL jobs
-
(Optional) Access to Amazon Secrets Manager.