Setting up Amazon CloudWatch alarms on Amazon Glue job profiles - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Setting up Amazon CloudWatch alarms on Amazon Glue job profiles

Amazon Glue metrics are also available in Amazon CloudWatch. You can set up alarms on any Amazon Glue metric for scheduled jobs.

A few common scenarios for setting up alarms are as follows:

  • Jobs running out of memory (OOM): Set an alarm when the memory usage exceeds the normal average for either the driver or an executor for an Amazon Glue job.

  • Straggling executors: Set an alarm when the number of executors falls below a certain threshold for a large duration of time in an Amazon Glue job.

  • Data backlog or reprocessing: Compare the metrics from individual jobs in a workflow using a CloudWatch math expression. You can then trigger an alarm on the resulting expression value (such as the ratio of bytes written by a job and bytes read by a following job).

For detailed instructions on setting alarms, see Create or Edit a CloudWatch Alarm in the Amazon CloudWatch Events User Guide.

For monitoring and debugging scenarios using CloudWatch, see Job monitoring and debugging.