Adding steps to a cluster with the Amazon EMR Management Console - Amazon EMR
During cluster creationRunning clusterModify step concurrencyAdd step arguments
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Adding steps to a cluster with the Amazon EMR Management Console

Use the following procedures to add steps to a cluster with the Amazon Web Services Management Console. For detailed information about how to submit steps for specific big data applications, see the following sections of the Amazon EMR Release Guide:

Add steps during cluster creation

From the Amazon Web Services Management Console, you can add steps when you create a cluster.

Note

We’ve redesigned the Amazon EMR console to make it easier to use. See Amazon EMR console to learn about the differences between the old and new console experiences.

New console
To add steps when you create a cluster with the new console

  1. Sign in to the Amazon Web Services Management Console, and open the Amazon EMR console at https://console.amazonaws.cn/emr.

  2. Under EMR on EC2 in the left navigation pane, choose Clusters, and then choose Create cluster.

  3. Under Steps, choose Add step. Enter appropriate values in the fields in the Add step dialog. For information on formatting your step arguments, see Add step arguments. Options differ depending on the step type. To add your step and exit the dialog, select Add step.

  4. Choose any other options that apply to your cluster.

  5. To launch your cluster, choose Create cluster.

Old console
To add steps when you create a cluster with the old console

  1. Open the Amazon EMR console at https://console.amazonaws.cn/elasticmapreduce/home. Choose Create Cluster - Advanced Options.

  2. On the Step 1: Software and Steps page, for Steps (optional), select Run multiple steps in parallel to improve cluster utilization and save cost. The default value for the concurrency level is 10. You can choose between 2 and 256 steps that can run in parallel.

    Note

    Running multiple steps in parallel is only supported with Amazon EMR version 5.28.0 and later.

  3. For After last step completes, choose Cluster enters waiting state or Auto-terminate the cluster.

  4. Choose Step type, then Add step.

  5. Type appropriate values in the fields in the Add Step dialog. For information on formatting your step arguments, see Add step arguments. Options differ depending on the step type. If you have enabled Run multiple steps in parallel to improve cluster utilization and save cost, the only option for Action on failure is Continue. Next, choose Add.

Add steps to a running cluster

With the Amazon Web Services Management Console, you can add steps to a cluster with the auto-terminate option disabled.

New console
To add steps to a running cluster with the new console

  1. Sign in to the Amazon Web Services Management Console, and open the Amazon EMR console at https://console.amazonaws.cn/emr.

  2. Under EMR on EC2 in the left navigation pane, choose Clusters, and select the cluster that you want to update.

  3. On the Steps tab on the cluster details page, select Add step. To clone an existing step, choose the Actions dropdown menu and select Clone step.

  4. Enter appropriate values in the fields in the Add step dialog. Options differ depending on the step type. To add your step and exit the dialog, choose Add step.

Old console
To add steps to a running cluster with the old console

  1. Open the Amazon EMR console at https://console.amazonaws.cn/elasticmapreduce/home. On the Cluster List page, select the link for your cluster.

  2. On the Cluster Details page, choose the Steps tab.

  3. On the Steps tab, choose Add step.

  4. Type appropriate values in the fields in the Add Step dialog, and then choose Add. The options differ depending on the step type.

Modify the step concurrency level in a running cluster

With the Amazon Web Services Management Console, you can modify the step concurrency level in a running cluster.

Note

You can only run multiple steps in parallel with Amazon EMR version 5.28.0 and later.

New console
To modify step concurrency in a running cluster with the new console

  1. Sign in to the Amazon Web Services Management Console, and open the Amazon EMR console at https://console.amazonaws.cn/emr.

  2. Under EMR on EC2 in the left navigation pane, choose Clusters, and select the cluster that you want to update. The cluster must be running to change its concurrency attribute.

  3. On the Steps tab on the cluster details page, find the Attributes section. Select Edit to change the concurrency. Enter a value between 1 and 256.

Old console
To modify step concurrency in a running cluster with the old console

  1. Open the Amazon EMR console at https://console.amazonaws.cn/elasticmapreduce/home. On the Cluster List page, select the link for your cluster.

  2. On the Cluster Details page, choose the Steps tab.

  3. For Concurrency, choose Change. Select a new value for the step concurrency level and then save.

Add step arguments

When you use the Amazon Web Services Management Console to add a step to your cluster, you can specify arguments for that step in the Arguments field. You must separate arguments with whitespace and surround string arguments that consist of characters and whitespace with quotation marks.

Example : Correct arguments

The following example arguments are formatted correctly for the Amazon Web Services Management Console, with quotation marks around the final string argument.

bash -c "aws s3 cp s3://DOC-EXAMPLE-BUCKET/my-script.sh ."

You can also put each argument on a separate line for readability as shown in the following example.

bash 
-c 
"aws s3 cp s3://DOC-EXAMPLE-BUCKET/my-script.sh ."
Example : Incorrect arguments

The following example arguments are improperly formatted for the Amazon Web Services Management Console. Notice that the final string argument, aws s3 cp s3://DOC-EXAMPLE-BUCKET/my-script.sh ., contains whitespace and is not surrounded by quotation marks.

bash -c aws s3 cp s3://DOC-EXAMPLE-BUCKET/my-script.sh .