Amazon EMR
管理指南
AWS 文档中描述的 AWS 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅 Amazon AWS 入门

步骤 2:检查环境

Amazon EMR 作为 Web 服务的生态系统的一部分和开源软件运行。影响那些依赖项的事情会影响 Amazon EMR 的性能。

检查服务中断

Amazon EMR uses several Amazon Web Services internally. It runs virtual servers on Amazon EC2, stores data and scripts on Amazon S3, indexes log files in Amazon SimpleDB, and reports metrics to CloudWatch. Events that disrupt these services are rare — but when they occur — can cause issues in Amazon EMR.

Before you go further, check the Service Health Dashboard. Check the region where you launched your cluster to see whether there are disruption events in any of these services.

检查使用限制

If you are launching a large cluster, have launched many clusters simultaneously, or you are an IAM user sharing an AWS account with other users, the cluster may have failed because you exceeded an AWS service limit.

Amazon EC2 limits the number of virtual server instances running on a single AWS region to 20 on-demand or reserved instances. If you launch a cluster with more than 20 nodes, or launch a cluster that causes the total number of EC2 instances active on your AWS account to exceed 20, the cluster will not be able to launch all of the EC2 instances it requires and may fail. When this happens, Amazon EMR returns an EC2 QUOTA EXCEEDED error. You can request that AWS increase the number of EC2 instances that you can run on your account by submitting a Request to Increase Amazon EC2 Instance Limit application.

Another thing that may cause you to exceed your usage limits is the delay between when a cluster is terminated and when it releases all of its resources. Depending on its configuration, it may take up to 5-20 minutes for a cluster to fully terminate and release allocated resources. If you are getting an EC2 QUOTA EXCEEDED error when you attempt to launch a cluster, it may be because resources from a recently terminated cluster may not yet have been released. In this case, you can either request that your Amazon EC2 quota be increased, or you can wait twenty minutes and re-launch the cluster.

Amazon S3 limits the number of buckets created on an account to 100. If your cluster creates a new bucket that exceeds this limit, the bucket creation will fail and may cause the cluster to fail.

检查发行版

比较用于启动集群的版本标签和最新的 Amazon EMR 版本。Amazon EMR 的每个版本都包含改进,例如新的应用程序、功能、补丁和错误修复。影响集群的问题可能已经在最新的发行版中得到修复。如果可能,请使用最新版本的 重新运行cluster。

检查 Amazon VPC 子网配置

If your cluster was launched in a Amazon VPC subnet, the subnet needs to be configured as described in 计划和配置联网. In addition, check that the subnet you launch the cluster into has enough free elastic IP addresses to assign one to each node in the cluster.