Monitoring Amazon Glue
Monitoring is an important part of maintaining the reliability, availability, and performance of Amazon Glue and your other Amazon solutions. Amazon provides monitoring tools that you can use to watch Amazon Glue, report when something is wrong, and take action automatically when appropriate:
You can use the following automated monitoring tools to watch Amazon Glue and report when something is wrong:
-
Amazon CloudWatch Events delivers a near real-time stream of system events that describe changes in Amazon resources. CloudWatch Events enables automated event-driven computing. You can write rules that watch for certain events and trigger automated actions in other Amazon services when these events occur. For more information, see the Amazon CloudWatch Events User Guide.
-
Amazon CloudWatch Logs enables you to monitor, store, and access your log files from Amazon EC2 instances, Amazon CloudTrail, and other sources. CloudWatch Logs can monitor information in the log files and notify you when certain thresholds are met. You can also archive your log data in highly durable storage. For more information, see the Amazon CloudWatch Logs User Guide.
-
Amazon CloudTrail captures API calls and related events made by or on behalf of your Amazon account and delivers the log files to an Amazon S3 bucket that you specify. You can identify which users and accounts call Amazon, the source IP address from which the calls are made, and when the calls occur. For more information, see the Amazon CloudTrail User Guide.
Additionally, you have access to the following insights in the Amazon Glue console to help you debug and profile jobs:
Spark jobs – you can see a visualization of selected CloudWatch metrics series, and newer jobs have access to the Spark UI. For more information, see Monitoring Amazon Glue Spark jobs.
Ray jobs – you can see a visualization of selected CloudWatch metrics series. For more information, see Monitoring Ray jobs with metrics.