Configure the discoverability of Amazon EMR clusters (for administrators) - Amazon SageMaker
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Configure the discoverability of Amazon EMR clusters (for administrators)

This section provides details about how administrators can configure the discoverability of existing Amazon EMR clusters from SageMaker Studio Classic. The clusters can be deployed in the same Amazon account as Studio Classic (Single Account tab) or in separate accounts (Cross Accounts tab).

Single Account

Attach the following permissions to the SageMaker Studio Classic execution role accessing your cluster.

The following list provides a breakdown of the permissions required.

  • AllowSagemakerProjectManagement enables the creation of SageMaker projects. In Studio Classic, access to the Amazon Service Catalog is granted through Projects.

  • AllowClusterDetailsDiscovery and AllowClusterDiscovery allow the discovery and connection to Amazon EMR clusters.

  • AllowPresignedUrl allows the creation of pre-signed URLs to access Spark UI.

The following is a comprehensive JSON that includes these permissions. 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPresignedUrl",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:DescribeCluster",
                "elasticmapreduce:ListInstanceGroups",
                "elasticmapreduce:CreatePersistentAppUI",
                "elasticmapreduce:DescribePersistentAppUI",
                "elasticmapreduce:GetPersistentAppUIPresignedURL",
                "elasticmapreduce:GetOnClusterAppUIPresignedURL"
            ],
            "Resource": [
                "arn:aws:elasticmapreduce:region:account-id:cluster/*"
            ]
        },
        {
            "Sid": "AllowClusterDetailsDiscovery",
            "Effect": "Allow",
            "Action": [
               "elasticmapreduce:DescribeCluster",
               "elasticmapreduce:ListInstances",
               "elasticmapreduce:ListInstanceGroups",
               "elasticmapreduce:DescribeSecurityConfiguration"
            ],
            "Resource": [
                "arn:aws:elasticmapreduce:region:account-id:cluster/*"
            ]
        },
        {
            "Sid": "AllowClusterDiscovery",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:ListClusters"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowSagemakerProjectManagement",
            "Effect": "Allow",
            "Action": [
                "sagemaker:CreateProject",
                "sagemaker:DeleteProject"
            ],
            "Resource": "arn:aws:sagemaker:region:account-id:project/*"
        }
    ]
}
Cross Accounts

If your Amazon EMR clusters and SageMaker Studio Classic are deployed in separate Amazon accounts, you configure the permissions in multiple steps.

  • On the trusting account (the account in which Amazon EMR is deployed ), create a custom IAM role (referred to as ASSUMABLE-ROLE in this page) with the following permissions and trust relationship.

    For information about creating a role on an Amazon account, see Creating an IAM role (console).

    1. Add a policy defining the following permissions.

      • AllowClusterDetailsDiscovery and AllowClusterDiscovery to allow the discovery and connection to Amazon EMR clusters.

      • AllowPresignedUrl to allow the creation of pre-signed URLs to access Spark UI.

      The following is a comprehensive JSON that includes these permissions.

      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPresignedUrl",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:DescribeCluster",
                "elasticmapreduce:ListInstanceGroups",
                "elasticmapreduce:CreatePersistentAppUI",
                "elasticmapreduce:DescribePersistentAppUI",
                "elasticmapreduce:GetPersistentAppUIPresignedURL",
                "elasticmapreduce:GetOnClusterAppUIPresignedURL"
            ],
            "Resource": [
                "arn:aws:elasticmapreduce:emr-region:emr-account:cluster/*"
            ]
        },
        {
            "Sid": "AllowClusterDetailsDiscovery",
            "Effect": "Allow",
            "Action": [
               "elasticmapreduce:DescribeCluster",
               "elasticmapreduce:ListInstances",
               "elasticmapreduce:ListInstanceGroups",
               "elasticmapreduce:DescribeSecurityConfiguration"
            ],
            "Resource": [
                "arn:aws:elasticmapreduce:emr-region:emr-account:cluster/*"
            ]
        },
        {
            "Sid": "AllowClusterDiscovery",
            "Effect": "Allow",
            "Action": [
                "elasticmapreduce:ListClusters"
            ],
            "Resource": "*"
        }
    ]
}

    2. To grant the trusted account (the account in which SageMaker Studio Classic's account is deployed ) the permission to assume a role in the trusting account, add the following trust relationship.

      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::studio-account:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

  • On the trusted account (the account in which SageMaker Studio Classic is deployed), add the following trust relationship to the Studio Classic execution role.

    {
  "Version": "2012-10-17",
  "Statement": [
  { 
            "Sid": "AllowRoleAssumptionForCrossAccountDiscovery", 
            "Effect": "Allow", 
            "Action": "sts:AssumeRole", 
            "Resource":  ["arn:aws:iam::emr-account:role/ASSUMABLE-ROLE" ]
  }]
}

  • Last, see Additional Configuration for cross accounts use cases (for administrators) to learn how to provide the ARN of the ASSUMABLE-ROLE to the Studio Classic execution role. The ARN is loaded by the Studio Classic Jupyter server at launch. The Studio Classic execution role assumes that cross-account role to discover and connect to Amazon EMR clusters in the trusting account.

Visit Discover Amazon EMR clusters from SageMaker Studio Classic to learn about how to discover and connect to Amazon EMR clusters from Studio Classic notebooks.