Managing Amazon EMR clusters with the console
The console offers an updated interface that provides you with an intuitive way to manage your Amazon EMR environment and gives you convenient access to documentation, product information, and other resources.
Console capabilities
The Amazon EMR console is available at the following URL:
-
Console URL – https://console.amazonaws.cn/emr
The following table lists the main Amazon EMR console components status.
| Amazon EMR console component | Console |
|---|---|
|
EMR Studio |
✔ |
|
Create and manage clusters |
✔ |
|
Block public access |
✔ |
|
Monitor Amazon CloudWatch Events |
✔ |
|
Security configurations |
✔ |
|
Virtual clusters (Amazon EMR on EKS) |
✔ |
| View and manage your Amazon Virtual Private Cloud subnets1 | ✔ |
|
Notebooks2 |
✔ |
1 In the console, you can view and manage your Amazon VPC subnets within the Networking section when you create a cluster.
2 EMR Notebooks are available as EMR Studio Workspaces in the console. The Create Workspace button in the console lets you create new notebooks. To access or create Workspaces, EMR Notebooks users need additional IAM role permissions. For more information, see Amazon EMR Notebooks are Amazon EMR Studio Workspaces in the console and Amazon EMR console.
Summary of differences
This section outlines the capabilities of the Amazon EMR console experience. These capabilities fall into the following categories:
Cluster compatibility in the console
In some cases, a cluster that you created might not be compatible with the console. The following list describes compatibility requirements for the Amazon EMR console.
-
The console supports clusters created in Amazon EMR releases 5.20.1 and later.
-
You can clone clusters that use automatic scaling in the console, but you can only create new clusters if you want to manually scale them or use managed scaling.
To create and work with clusters of release 5.20.1 and earlier, you can use the Amazon Command Line Interface (Amazon CLI) or the Amazon SDK.
Creating clusters
| Capability | Console |
|---|---|
|
Terminology: Amazon EMR cluster node types |
Primary, core, task |
|
Amazon EMR release 5.20.1 and later |
|
|
Use the Create cluster button under the Summary panel. Your cluster name can't contain the characters <, >, $, |, or ` (backtick). |
|
Define a timeout period for provisioning instances for each fleet in your cluster. |
|
|
The console doesn't create default roles; you must create
roles with the IAM
Console |
|
| From within the Amazon EMR console, you can't make a cluster visible to all a users; your IAM policy determines cluster access | |
| You must configure Amazon S3 endpoints and NAT gateways
from their respective Amazon S3 |
|
|
With the release of Amazon S3 strong read-after-write consistency on December 1, 2020, you don't need to use EMRFS CV with your EMR clusters |
|
| You can debug jobs using the Application UI interface on the cluster details page |
1 You can't create or edit clusters using releases earlier than Amazon EMR 5.20.1 in the console, but any existing clusters created using releases earlier than 5.20.1 will continue to work. To create and edit clusters with Amazon EMR releases earlier than 5.20.1, use the API or CLI. You can view all clusters using the console, but consoles created earlier than 5.20.1 might not be compatible newer features.
Viewing and searching for clusters
The following table highlights how you can use the Amazon EMR console to view view and search for clusters.
Note
Applying a data filter to the cluster list queries the entire database. But when you enter a text string into the search box, the search only applies to the results that the list has loaded client side.
| Capability | Console |
|---|---|
|
Viewing cluster details |
You can select the Cluster ID to view exhaustive cluster details like configuration options, persistent application UIs, and logs. |
|
Searching for clusters |
Use a single search field to enter text search queries and to create and apply data filters like "Status = Any active status". |
|
Finding failed clusters |
To search for failed clusters, apply the filter Status = Terminated with errors. |
Viewing or editing cluster details
| Capability | Console |
|---|---|
|
Viewing the instances in your instance groups and instance fleets, along with scaling, provisioning, resizing, and termination options |
View instance options and details in the Instances tab. View termination options in the Properties tab. |
|
Viewing app UIs, logs, and configurations (Apache Spark |
View cluster configurations in the Configurations tab. Launch a live, persistent, application UI to see the logs for an application from the Applications tab. |
|
Exporting a cluster to CLI |
Option available from cluster detail and list view Actions menus as "View command for cloning cluster" |
Differences when you work with security configurations
| Capability | Console |
|---|---|
|
Cloning security configurations |
✔ |
|
✔ |
|
|
✔ |
|
|
Amazon S3 access points |
|
|
Amazon Lake Formation access controls |
Runtime roles |
1 To pass a role during step submission, your cluster must use a security configuration with an IAM permissions policy attached so that the a user can pass only the approved roles and your jobs can access Amazon EMR resources. For more information, see Runtime roles for Amazon EMR steps.