Amazon Glue Spark and PySpark jobs - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Glue Spark and PySpark jobs

Amazon Glue support Spark and PySpark jobs. A Spark job is run in an Apache Spark environment managed by Amazon Glue. It processes data in batches. A streaming ETL job is similar to a Spark job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.

The following sections provide information on Amazon Glue Spark and PySpark jobs.

Topics