启用 Amazon EMR on EKS 的集群访问 - Amazon EMR
Amazon Web Services 文档中描述的 Amazon Web Services 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅 中国的 Amazon Web Services 服务入门 (PDF)

启用 Amazon EMR on EKS 的集群访问

您必须通过执行以下操作来允许 Amazon EMR on EKS 访问集群中的特定命名空间:创建 Kubernetes 角色、将角色绑定到 Kubernetes 用户以及将 Kubernetes 用户映射为服务关联角色 AWSServiceRoleForAmazonEMRContainers。当 IAM 身份映射命令与 emr-containers 一起作为服务名称时,这些操作在 eksctl 中自动执行。您可以使用以下命令轻松地执行这些操作。

eksctl create iamidentitymapping \ --cluster my_eks_cluster \ --namespace kubernetes_namespace \ --service-name "emr-containers"

用 Amazon EKS 集群的名称来替换 my_eks_cluster,并使用创建用来运行 Amazon EMR 工作负载的 Kubernetes 命名空间来替换 kubernetes_namespace

重要

您必须使用前一步骤安装 eksctl下载最新的 eksctl 以使用此功能。

执行手动步骤以启用 Amazon EMR on EKS 的集群访问

您还可以使用以下手动步骤来启用 Amazon EMR on EKS 的集群访问。

  1. 在特定命名空间中创建 Kubernetes 角色

    Amazon EKS 1.22 - 1.27

    使用 Amazon EKS 1.22-1.27,运行以下命令以在特定命名空间中创建 Kubernetes 角色。此角色向 Amazon EMR on EKS 授予必要的 RBAC 权限。

    namespace=my-namespace cat - >>EOF | kubectl apply -f - >>namespace "${namespace}" apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: emr-containers namespace: ${namespace} rules: - apiGroups: [""] resources: ["namespaces"] verbs: ["get"] - apiGroups: [""] resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] - apiGroups: [""] resources: ["secrets"] verbs: ["create", "patch", "delete", "watch"] - apiGroups: ["apps"] resources: ["statefulsets", "deployments"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["batch"] resources: ["jobs"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["extensions", "networking.k8s.io"] resources: ["ingresses"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["rbac.authorization.k8s.io"] resources: ["roles", "rolebindings"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] EOF
    Amazon EKS 1.21 and below

    使用 Amazon EKS 1.21 及更低版本,运行以下命令以在特定命名空间中创建 Kubernetes 角色。此角色向 Amazon EMR on EKS 授予必要的 RBAC 权限。

    namespace=my-namespace cat - >>EOF | kubectl apply -f - >>namespace "${namespace}" apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: emr-containers namespace: ${namespace} rules: - apiGroups: [""] resources: ["namespaces"] verbs: ["get"] - apiGroups: [""] resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] - apiGroups: [""] resources: ["secrets"] verbs: ["create", "patch", "delete", "watch"] - apiGroups: ["apps"] resources: ["statefulsets", "deployments"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["batch"] resources: ["jobs"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["extensions"] resources: ["ingresses"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["rbac.authorization.k8s.io"] resources: ["roles", "rolebindings"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] EOF
  2. 创建作用域为命名空间的 Kubernetes 角色绑定

    运行以下命令以创建绑定在特定命名空间中的 Kubernetes 角色。此角色绑定将在上一步中创建的角色中定义的权限授予名为 emr-containers 的用户。此用户确定 Amazon EMR on EKS 的服务相关角色,因此允许 Amazon EMR on EKS 执行由您创建的角色所定义的操作。

    namespace=my-namespace cat - <<EOF | kubectl apply -f - --namespace "${namespace}" apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: emr-containers namespace: ${namespace} subjects: - kind: User name: emr-containers apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: emr-containers apiGroup: rbac.authorization.k8s.io EOF
  3. 更新 Kubernetes aws-auth 配置映射

    您可以使用以下选项之一将 Amazon EMR on EKS 与服务关联的角色映射到 emr-containers 用户,并且该用户在上一步中绑定了 Kubernetes 角色。

    选项 1:使用 eksctl

    运行以下 eksctl 命令:将 Amazon EMR on EKS 服务相关角色映射到 emr-containers 用户。

    eksctl create iamidentitymapping \ --cluster my-cluster-name \ --arn "arn:aws:iam::my-account-id:role/AWSServiceRoleForAmazonEMRContainers" \ --username emr-containers

    选项 2:不使用 eksctl

    1. 运行以下命令可在文本编辑器中打开 aws-auth 配置映射。

      kubectl edit -n kube-system configmap/aws-auth
      注意

      如果您收到指示 Error from server (NotFound): configmaps "aws-auth" not found 的错误,请参考《Amazon EKS 用户指南》中的添加用户角色的步骤,以应用库存 ConfigMap。

    2. data 下,将 Amazon EMR on EKS 服务相关角色详细信息添加到 ConfigMapmapRoles 部分。如果此部分在文件中尚不存在,请添加它。已更新的 mapRoles 部分类似于以下示例。

      apiVersion: v1 data: mapRoles: | - rolearn: arn:aws:iam::<your-account-id>:role/AWSServiceRoleForAmazonEMRContainers username: emr-containers - ... <other previously existing role entries, if there's any>.
    3. 保存文件并退出文本编辑器。