Setting up the Flink Kubernetes operator for Amazon EMR on EKS - Amazon EMR
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Setting up the Flink Kubernetes operator for Amazon EMR on EKS

Complete the following tasks to get set up before you install the Flink Kubernetes operator on Amazon EKS. If you've already signed up for Amazon Web Services (Amazon) and have used Amazon EKS, you are almost ready to use Amazon EMR on EKS. Complete the following tasks to get set up for the Flink operator on Amazon EKS. If you've already completed any of the prerequisites, you can skip those and move on to the next one.

  • Install the Amazon CLI – If you've already installed the Amazon CLI, confirm that you have the latest version.

  • Install eksctl – eksctl is a command line tool that you use to communicate with Amazon EKS.

  • Install Helm – The Helm package manager for Kubernetes helps you install and manage applications on your Kubernetes cluster.

  • Set up an Amazon EKS cluster – Follow the steps to create a new Kubernetes cluster with nodes in Amazon EKS.

  • Choose an Amazon EMR release label (release 6.13.0 or higher) – the Flink Kubernetes operator is supported with Amazon EMR releases 6.13.0 and higher.

  • Enable IAM Roles for Service Accounts (IRSA) on the Amazon EKS cluster.

  • Create a job execution role.

  • Update the trust policy of the job execution role .

  • Create an operator execution role. This step is optional. You can use the same role for Flink jobs and operator. If you want to have a different IAM role for your operator, you can create a separate role.

  • Update the trust policy of the operator execution role. You must explicitly add one trust policy entry for the roles you want to use for the Amazon EMR Flink Kubernetes operator service account. You can follow this example format:

    {
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Principal": {
               "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/OIDC_PROVIDER"
           },
           "Action": "sts:AssumeRoleWithWebIdentity",
           "Condition": {
               "StringLike": {
                   "OIDC_PROVIDER:sub": "system:serviceaccount:NAMESPACE:emr-containers-sa-flink-operator"
               }
           }
       }
   ]
}