AWS CloudFormation
User Guide (API 版本 2010-05-15)
AWS 文档中描述的 AWS 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅 Amazon AWS 入门

AWS::EMR::Cluster

AWS::EMR::Cluster 资源创建 Amazon EMR 集群。此集群是 EC2 实例的集合,您可以在这些实例上运行大数据框架以处理和分析大量数据。有关更多信息,请参阅 Amazon EMR 管理指南 中的规划 Amazon EMR 集群

语法

要在 AWS CloudFormation 模板中声明此实体,请使用以下语法:

JSON

{ "Type" : "AWS::EMR::Cluster", "Properties" : { "AdditionalInfo" : JSON object, "Applications" : [ Applications, ... ], "AutoScalingRole" : String, "BootstrapActions" [ Bootstrap Actions, ... ], "Configurations" : [ Configurations, ... ], "CustomAmiId" : String, "EbsRootVolumeSize" : Integer, "Instances" : JobFlowInstancesConfig, "JobFlowRole" : String, "LogUri" : String, "Name" : String, "ReleaseLabel" : String, "ScaleDownBehavior" : String, "SecurityConfiguration" : String, "ServiceRole" : String, "Tags" : [ Resource Tag, ... ], "VisibleToAllUsers" : Boolean } }

YAML

Type: "AWS::EMR::Cluster" Properties: AdditionalInfo: JSON object Applications: - Applications AutoScalingRole: String BootstrapActions: - Bootstrap Actions Configurations: - Configurations CustomAmiId: String EbsRootVolumeSize: Integer Instances: JobFlowInstancesConfig JobFlowRole: String LogUri: String Name: String ReleaseLabel: String ScaleDownBehavior: String SecurityConfiguration: String ServiceRole: String Tags: - Resource Tag VisibleToAllUsers: Boolean

属性

注意

有关每个属性的约束和有效值的更多信息,请参阅 Amazon EMR API Reference 中的 Cluster 数据类型。

AdditionalInfo

(仅适用于高级使用。)要选择的其他功能。这是有关第三方供应商用于测试目的的第三方应用程序的元信息。

Required: No

Type: JSON object

更新要求替换

Applications

要在群集上部署的软件应用程序,以及 Amazon EMR 传递给这些应用程序的参数。

Required: No

Type: Amazon EMR 群集应用程序 属性类型的列表

更新要求替换

AutoScalingRole

自动扩展策略的 AWS Identity and Access Management (IAM) 角色。默认角色是 EMR_AutoScaling_DefaultRole。IAM 角色提供自动扩展功能在实例组中启动和终止 Amazon EC2 实例所需的权限。

Required: No

Type: String

更新要求替换

BootstrapActions

Amazon EMR 在群集上启动应用程序之前运行的引导操作的列表。

Required: No

Type: Amazon EMR Cluster BootstrapActionConfig 属性类型的列表

更新要求替换

Configurations

Amazon EMR 群集的软件配置。

Required: No

Type: Amazon EMR Cluster Configurations 属性类型的列表

更新要求替换

CustomAmiId

集群的自定义 Amazon Linux AMI (而不是 EMR 拥有的 AMI)。有关更多信息,请参阅 Amazon EMR 管理指南 中的使用自定义 AMI

Required: No

Type: String

更新要求替换

示例: "CustomAmiId" : "ami-7fb3bc69"

EbsRootVolumeSize

用于每个 EC2 实例的 Linux AMI 的 EBS 根设备卷的大小 (GiB)。

目前,AWS CloudFormation 只支持 Amazon EMR 4.0 及更高软件版本。

必需:否

类型:整数

更新要求替换

Instances

配置在 Amazon EMR 集群中运行任务的 EC2 实例。

Required: Yes

Type: Amazon EMR Cluster JobFlowInstancesConfig

更新要求时而中断

JobFlowRole

(也称为实例配置文件EC2 角色。)接受与您要使用的角色关联的实例配置文件。群集中的所有 EC2 实例将代入此角色。有关更多信息,请参阅 Amazon EMR 管理指南 中的为 Amazon EMR 创建和使用 IAM 角色

Required: Yes

Type: String

更新要求替换

LogUri

Amazon EMR 从任务流将日志文件写入到的 S3 存储桶的位置。如果不指定值,则 Amazon EMR 不写入任何日志文件。

Required: No

Type: String

更新要求替换

Name

Amazon EMR 群集的名称。

Required: Yes

Type: String

更新要求替换

ReleaseLabel

Amazon EMR 软件版本标签。版本是您可以在 Amazon EMR 群集上安装和配置的一组软件应用程序和组件。有关更多信息,请参阅 Amazon EMR 版本指南 中的关于 Amazon EMR 版本

目前,AWS CloudFormation 只支持 Amazon EMR 4.0 及更高软件版本。

Required: Conditional。如果您指定了 Applications 属性,则必须指定该属性。

Type: String

更新要求替换

ScaleDownBehavior

指示出现自动缩减活动或调整实例组大小时,各个 EC2 实例将如何终止。有关更多信息,请参阅 Amazon EMR API Reference 中的集群

Required: No

Type: String

更新要求替换

SecurityConfiguration

应用于集群的安全配置的名称。

Required: No

Type: String

更新要求替换

ServiceRole

Amazon EMR 代您访问 AWS 资源时所担任的 IAM 角色。有关更多信息,请参阅 Amazon EMR 管理指南 中的为 Amazon EMR 配置 IAM 角色

Required: Yes

Type: String

更新要求替换

Tags

帮助您标识 Amazon EMR 群集的任意标签组(键/值对)。

Required: No

Type: AWS CloudFormation 资源标签

更新要求无需中断

VisibleToAllUsers

指示群集中的实例是否对 AWS 账户中的所有 IAM 用户可见。如果指定 true,则所有 IAM 用户都能查看和(如果有权限)管理这些实例。如果指定 false,则只有创建群集的 IAM 用户能够查看和管理它。

Required: No

Type: Boolean

更新要求无需中断

默认值: false

返回值

Ref

如果向 Ref 内部函数提供此资源的逻辑 ID,Ref 将返回此群集 ID,如 j-1ABCD123AB1A

有关使用 Ref 功能的更多信息,请参阅参考

Fn::GetAtt

Fn::GetAtt 返回一个此类型指定属性的值。以下为可用属性和示例返回值。

MasterPublicDNS

主节点(实例)的公有 DNS 名称,如 ec2-12-123-123-123.us-west-2.compute.amazonaws.com

有关使用 Fn::GetAtt 的更多信息,请参见 Fn::GetAtt

示例

创建具有两个核心节点的集群

下面的示例创建具有一个主节点和两个核心节点的 Amazon EMR 群集。指定的 IAM 角色是 Amazon EMR 提供的默认角色。此示例还假定集群是在具有默认 VPC 和子网的 AWS 区域中启动的。如果您没有 VPC 和子网,可使用 Ec2SubnetId 属性为群集指定 VPC 和子网。否则,AWS CloudFormation 将无法启动群集,并返回以下状态消息:ElasticMapReduce Cluster failed to stabilize

JSON

"TestCluster": { "Type": "AWS::EMR::Cluster", "Properties": { "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Master" }, "CoreInstanceGroup": { "InstanceCount": 2, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Core" }, "TerminationProtected" : true }, "Name": "TestCluster", "JobFlowRole": "EMR_EC2_DefaultRole", "ServiceRole": "EMR_DefaultRole", "ReleaseLabel": "emr-4.2.0", "Tags": [ { "Key": "IsTest", "Value": "True" } ] } }

YAML

TestCluster: Type: "AWS::EMR::Cluster" Properties: Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Master" CoreInstanceGroup: InstanceCount: 2 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Core" TerminationProtected: true Name: "TestCluster" JobFlowRole: "EMR_EC2_DefaultRole" ServiceRole: "EMR_DefaultRole" ReleaseLabel: "emr-4.2.0" Tags: - Key: "IsTest" Value: "True"

创建具有引导操作的集群

下面的示例使用引导操作创建 Amazon EMR 群集。

JSON

"TestCluster": { "Type": "AWS::EMR::Cluster", "Properties": { "BootstrapActions": [{ "Name": "SomeBootStrapAction", "ScriptBootstrapAction": { "Path": "/path/to/s3" } }], "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Master" }, "CoreInstanceGroup": { "InstanceCount": 2, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Core" }, "TerminationProtected": true }, "Name": "TestCluster", "JobFlowRole": "EMR_EC2_DefaultRole", "ScaleDownBehavior": "TERMINATE_AT_TASK_COMPLETION", "ServiceRole": "EMR_DefaultRole", "ReleaseLabel": "emr-4.2.0", "Tags": [ { "Key": "IsTest", "Value": "True" } ] } }

YAML

TestCluster: Type: "AWS::EMR::Cluster" Properties: BootstrapActions: - Name: "SomeBootStrapAction" ScriptBootstrapAction: Path: "/path/to/s3" Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Master" CoreInstanceGroup: InstanceCount: 2 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Core" TerminationProtected: true Name: "TestCluster" JobFlowRole: "EMR_EC2_DefaultRole" ScaleDownBehavior: "TERMINATE_AT_TASK_COMPLETION" ServiceRole: "EMR_DefaultRole" ReleaseLabel: "emr-4.2.0" Tags: - Key: "IsTest" Value: "True"

使用自定义 AMI 创建集群

以下示例模板允许您在创建 Amazon EMR 集群时指定自定义 Amazon Linux AMI。

JSON

{ "AWSTemplateFormatVersion": "2010-09-09", "Parameters" : { "CustomAmiId" : { "Type" : "String" }, "InstanceType" : { "Type" : "String" }, "ReleaseLabel" : { "Type" : "String" }, "SubnetId" : { "Type" : "String" }, "TerminationProtected" : { "Type" : "String", "Default" : "false" }, "ElasticMapReducePrincipal" : { "Type" : "String" }, "Ec2Principal" : { "Type" : "String" } }, "Resources": { "cluster": { "Type": "AWS::EMR::Cluster", "Properties": { "CustomAmiId" : {"Ref" : "CustomAmiId"}, "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": {"Ref" : "InstanceType"}, "Market": "ON_DEMAND", "Name": "cfnMaster" }, "CoreInstanceGroup": { "InstanceCount": 1, "InstanceType": {"Ref" : "InstanceType"}, "Market": "ON_DEMAND", "Name": "cfnCore" }, "TerminationProtected" : {"Ref" : "TerminationProtected"}, "Ec2SubnetId" : {"Ref" : "SubnetId"} }, "Name": "CFNtest", "JobFlowRole" : {"Ref": "emrEc2InstanceProfile"}, "ServiceRole" : {"Ref": "emrRole"}, "ReleaseLabel" : {"Ref" : "ReleaseLabel"}, "VisibleToAllUsers" : true, "Tags": [ { "Key": "key1", "Value": "value1" } ] } }, "emrRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2008-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": {"Ref" : "ElasticMapReducePrincipal"} }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole"] } }, "emrEc2Role": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2008-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": {"Ref" : "Ec2Principal"} }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role"] } }, "emrEc2InstanceProfile": { "Type": "AWS::IAM::InstanceProfile", "Properties": { "Path": "/", "Roles": [ { "Ref": "emrEc2Role" } ] } } } }

YAML

AWSTemplateFormatVersion: 2010-09-09 Parameters: CustomAmiId: Type: String InstanceType: Type: String ReleaseLabel: Type: String SubnetId: Type: String TerminationProtected: Type: String Default: 'false' ElasticMapReducePrincipal: Type: String Ec2Principal: Type: String Resources: cluster: Type: 'AWS::EMR::Cluster' Properties: CustomAmiId: !Ref CustomAmiId Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: !Ref InstanceType Market: ON_DEMAND Name: cfnMaster CoreInstanceGroup: InstanceCount: 1 InstanceType: !Ref InstanceType Market: ON_DEMAND Name: cfnCore TerminationProtected: !Ref TerminationProtected Ec2SubnetId: !Ref SubnetId Name: CFNtest JobFlowRole: !Ref emrEc2InstanceProfile ServiceRole: !Ref emrRole ReleaseLabel: !Ref ReleaseLabel VisibleToAllUsers: true Tags: - Key: key1 Value: value1 emrRole: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2008-10-17 Statement: - Sid: '' Effect: Allow Principal: Service: !Ref ElasticMapReducePrincipal Action: 'sts:AssumeRole' Path: / ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole' emrEc2Role: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2008-10-17 Statement: - Sid: '' Effect: Allow Principal: Service: !Ref Ec2Principal Action: 'sts:AssumeRole' Path: / ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role' emrEc2InstanceProfile: Type: 'AWS::IAM::InstanceProfile' Properties: Path: / Roles: - !Ref emrEc2Role

指定根卷大小

以下示例模板允许您指定 Amazon EMR 集群的 EBS 根卷的大小。

JSON

{ "AWSTemplateFormatVersion": "2010-09-09", "Parameters" : { "InstanceType" : { "Type" : "String" }, "ReleaseLabel" : { "Type" : "String" }, "SubnetId" : { "Type" : "String" }, "TerminationProtected" : { "Type" : "String", "Default" : "false" }, "EbsRootVolumeSize" : { "Type" : "String" } }, "Resources": { "cluster": { "Type": "AWS::EMR::Cluster", "Properties": { "EbsRootVolumeSize" : {"Ref" : "EbsRootVolumeSize"}, "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": {"Ref" : "InstanceType"}, "Market": "ON_DEMAND", "Name": "cfnMaster" }, "CoreInstanceGroup": { "InstanceCount": 1, "InstanceType": {"Ref" : "InstanceType"}, "Market": "ON_DEMAND", "Name": "cfnCore" }, "TerminationProtected" : {"Ref" : "TerminationProtected"}, "Ec2SubnetId" : {"Ref" : "SubnetId"} }, "Name": "CFNtest", "JobFlowRole" : {"Ref": "emrEc2InstanceProfile"}, "ServiceRole" : {"Ref": "emrRole"}, "ReleaseLabel" : {"Ref" : "ReleaseLabel"}, "VisibleToAllUsers" : true, "Tags": [ { "Key": "key1", "Value": "value1" } ] } }, "emrRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2008-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "elasticmapreduce.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole"] } }, "emrEc2Role": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2008-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "ManagedPolicyArns": ["arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role"] } }, "emrEc2InstanceProfile": { "Type": "AWS::IAM::InstanceProfile", "Properties": { "Path": "/", "Roles": [ { "Ref": "emrEc2Role" } ] } } } }

YAML

AWSTemplateFormatVersion: 2010-09-09 Parameters: InstanceType: Type: String ReleaseLabel: Type: String SubnetId: Type: String TerminationProtected: Type: String Default: 'false' EbsRootVolumeSize: Type: String Resources: cluster: Type: 'AWS::EMR::Cluster' Properties: EbsRootVolumeSize: !Ref EbsRootVolumeSize Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: !Ref InstanceType Market: ON_DEMAND Name: cfnMaster CoreInstanceGroup: InstanceCount: 1 InstanceType: !Ref InstanceType Market: ON_DEMAND Name: cfnCore TerminationProtected: !Ref TerminationProtected Ec2SubnetId: !Ref SubnetId Name: CFNtest JobFlowRole: !Ref emrEc2InstanceProfile ServiceRole: !Ref emrRole ReleaseLabel: !Ref ReleaseLabel VisibleToAllUsers: true Tags: - Key: key1 Value: value1 emrRole: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2008-10-17 Statement: - Sid: '' Effect: Allow Principal: Service: elasticmapreduce.amazonaws.com Action: 'sts:AssumeRole' Path: / ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole' emrEc2Role: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2008-10-17 Statement: - Sid: '' Effect: Allow Principal: Service: ec2.amazonaws.com Action: 'sts:AssumeRole' Path: / ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role' emrEc2InstanceProfile: Type: 'AWS::IAM::InstanceProfile' Properties: Path: / Roles: - !Ref emrEc2Role

本页内容: