Developing and testing Amazon Glue job scripts locally - Amazon Glue
Developing using Amazon Glue StudioDeveloping using interactive sessions
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Developing and testing Amazon Glue job scripts locally

When you develop and test your Amazon Glue for Spark job scripts, there are multiple available options:

  • Amazon Glue Studio console

    • Visual editor

    • Script editor

    • Amazon Glue Studio notebook

  • Interactive sessions

    • Jupyter notebook

  • Docker image

    • Local development

    • Remote development

You can choose any of the above options based on your requirements.

If you prefer no code or less code experience, the Amazon Glue Studio visual editor is a good choice.

If you prefer an interactive notebook experience, Amazon Glue Studio notebook is a good choice. For more information, see Using Notebooks with Amazon Glue Studio and Amazon Glue. If you want to use your own local environment, interactive sessions is a good choice. For more information, see Using interactive sessions with Amazon Glue.

If you prefer local/remote development experience, the Docker image is a good choice. This helps you to develop and test Amazon Glue for Spark job scripts anywhere you prefer without incurring Amazon Glue cost.

If you prefer local development without Docker, installing the Amazon Glue ETL library directory locally is a good choice.

Developing using Amazon Glue Studio

The Amazon Glue Studio visual editor is a graphical interface that makes it easy to create, run, and monitor extract, transform, and load (ETL) jobs in Amazon Glue. You can visually compose data transformation workflows and seamlessly run them on Amazon Glue's Apache Spark-based serverless ETL engine. You can inspect the schema and data results in each step of the job. For more information, see the Amazon Glue Studio User Guide.

Developing using interactive sessions

Interactive sessions allow you to build and test applications from the environment of your choice. For more information, see Using interactive sessions with Amazon Glue.