Migrating Amazon Glue for Ray from preview to the Ray2.4 runtime environment
Warning
When you save your Amazon Glue for Ray (preview) job in Amazon Glue Studio, it will be automatically upgraded to the
Ray2.4
runtime. If you experience compatibility issues with your script, please contact
support.
You should migrate Amazon Glue jobs that were created during the Amazon Glue for Ray (preview) to Amazon Glue for Ray. This will involve a few concurrent changes to your job configuration.
In the
Runtime
field, provide theRay2.4
runtime value. This will upgrade the underlying Ray version from 2.0.0 to 2.4.0.-
Certain Python libraries included by default in the preview are no longer provided. If your job takes advantage of the Amazon SDK for pandas (awswrangler), dask, modin, or pymars, you will need to include these as additional libraries. For more information on including additional Python libraries, see Additional Python modules for Ray jobs.
-
If you're using the
--additional-python-modules
parameter, the parameters used to support this workflow have been broken out into--pip-install
and--s3-py-modules
. For more information about these parameters, see Additional Python modules for Ray jobs. -
If you're using the
--auto-scaling-ray-min-workers
parameter, it has been renamed--min-workers
.