Migrating Amazon Glue for Ray from preview to the Ray2.4 runtime environment - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Migrating Amazon Glue for Ray from preview to the Ray2.4 runtime environment

Warning

When you save your Amazon Glue for Ray (preview) job in Amazon Glue Studio, it will be automatically upgraded to the Ray2.4 runtime. If you experience compatibility issues with your script, please contact support.

You should migrate Amazon Glue jobs that were created during the Amazon Glue for Ray (preview) to Amazon Glue for Ray. This will involve a few concurrent changes to your job configuration.

  • In the Runtime field, provide the Ray2.4 runtime value. This will upgrade the underlying Ray version from 2.0.0 to 2.4.0.

  • Certain Python libraries included by default in the preview are no longer provided. If your job takes advantage of the Amazon SDK for pandas (awswrangler), dask, modin, or pymars, you will need to include these as additional libraries. For more information on including additional Python libraries, see Additional Python modules for Ray jobs.

  • If you're using the --additional-python-modules parameter, the parameters used to support this workflow have been broken out into --pip-install and --s3-py-modules. For more information about these parameters, see Additional Python modules for Ray jobs.

  • If you're using the --auto-scaling-ray-min-workers parameter, it has been renamed --min-workers.