Adding jobs in Amazon Glue - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Adding jobs in Amazon Glue

There are three types of jobs in Amazon Glue: Spark, Streaming ETL, and Python shell.

  • A Spark job is run in an Apache Spark environment managed by Amazon Glue. It processes data in batches.

  • A streaming ETL job is similar to a Spark job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.

  • A Python shell job runs Python scripts as a shell and supports a Python version that depends on the Amazon Glue version you are using. You can use these jobs to schedule and run tasks that don't require an Apache Spark environment.