Attribute-based instance type selection for EC2 Fleet - Amazon Elastic Compute Cloud
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Attribute-based instance type selection for EC2 Fleet

When you create an EC2 Fleet, you must specify one or more instance types for configuring the On-Demand Instances and Spot Instances in the fleet. As an alternative to manually specifying the instance types, you can specify the attributes that an instance must have, and Amazon EC2 will identify all the instance types with those attributes. This is known as attribute-based instance type selection. For example, you can specify the minimum and maximum number of vCPUs required for your instances, and EC2 Fleet will launch the instances using any available instance types that meet those vCPU requirements.

Attribute-based instance type selection is ideal for workloads and frameworks that can be flexible about what instance types they use, such as when running containers or web fleets, processing big data, and implementing continuous integration and deployment (CI/CD) tooling.

Benefits

Attribute-based instance type selection has the following benefits:

  • Easily use the right instance types – With so many instance types available, finding the right instance types for your workload can be time consuming. When you specify instance attributes, the instance types will automatically have the required attributes for your workload.

  • Simplified configuration – To manually specify multiple instance types for an EC2 Fleet, you must create a separate launch template override for each instance type. But with attribute-based instance type selection, to provide multiple instance types, you need only specify the instance attributes in the launch template or in a launch template override.

  • Automatic use of new instance types – When you specify instance attributes rather than instance types, your fleet can use newer generation instance types as they’re released, "future proofing" the fleet's configuration.

  • Instance type flexibility – When you specify instance attributes rather than instance types, EC2 Fleet can select from a wide range of instance types for launching Spot Instances, which adheres to the Spot best practice of instance type flexibility.

How attribute-based instance type selection works

To use attribute-based instance type selection in your fleet configuration, you replace the list of instance types with a list of instance attributes that your instances require. EC2 Fleet will launch instances on any available instance types that have the specified instance attributes.

Types of instance attributes

There are several instance attributes that you can specify to express your compute requirements, such as:

  • vCPU count – The minimum and maximum number of vCPUs per instance.

  • Memory – The minimum and maximum GiBs of memory per instance.

  • Local storage – Whether to use EBS or instance store volumes for local storage.

  • Burstable performance – Whether to use the T instance family, including T4g, T3a, T3, and T2 types.

For a description of each attribute and the default values, see InstanceRequirements in the Amazon EC2 API Reference.

Where to configure attribute-based instance type selection

Depending on whether you use the console or the Amazon CLI, you can specify the instance attributes for attribute-based instance type selection as follows:

In the console, you can specify the instance attributes in the following fleet configuration component:

  • In a launch template, and then reference the launch template in the fleet request

In the Amazon CLI, you can specify the instance attributes in one or all of the following fleet configuration components:

  • In a launch template, and then reference the launch template in the fleet request

  • In a launch template override

    If you want a mix of instances that use different AMIs, you can specify instance attributes in multiple launch template overrides. For example, different instance types can use x86 and Arm-based processors.

  • In a launch specification

How EC2 Fleet uses attribute-based instance type selection when provisioning a fleet

EC2 Fleet provisions a fleet in the following way:

  • EC2 Fleet identifies the instance types that have the specified attributes.

  • EC2 Fleet uses price protection to determine which instance types to exclude.

  • EC2 Fleet determines the capacity pools from which it will consider launching the instances based on the Amazon Regions or Availability Zones that have matching instance types.

  • EC2 Fleet applies the specified allocation strategy to determine from which capacity pools to launch the instances.

    Note that attribute-based instance type selection does not pick the capacity pools from which to provision the fleet; that's the job of the allocation strategies. There might be a large number of instance types with the specified attributes, and some of them might be expensive. The default allocation strategy of lowest-price for Spot and On-Demand guarantees that EC2 Fleet will launch instances from the least expensive capacity pools.

    If you specify an allocation strategy, EC2 Fleet will launch instances according to the specified allocation strategy.

    • For Spot Instances, attribute-based instance type selection supports the price-capacity-optimized, capacity-optimized, and lowest-price allocation strategies.

    • For On-Demand Instances, attribute-based instance type selection supports the lowest-price allocation strategy.

  • If there is no capacity for the instance types with the specified instance attributes, no instances can be launched, and the fleet returns an error.

Price protection

Price protection is a feature that prevents your EC2 Fleet from using instance types that you would consider too expensive even if they happen to fit the attributes that you specified. To use price protection, you set a price threshold. Then, when Amazon EC2 selects instance types with your attributes, it excludes instance types priced above your threshold.

The way that Amazon EC2 calculates the price threshold is as follows:

  • Amazon EC2 first identifies the lowest priced instance type from those that match your attributes.

  • Amazon EC2 then takes the value (expressed as a percentage) that you specified for the price protection parameter and multiplies it with the price of the identified instance type. The result is the price that is used as the price threshold.

There are separate price thresholds for On-Demand Instances and Spot Instances.

When you create a fleet with attribute-based instance type selection, price protection is enabled by default. You can keep the default values, or you can specify your own.

You can also turn off price protection. To indicate no price protection threshold, specify a high percentage value, such as 999999.

How the lowest priced instance type is identified

Amazon EC2 determines the price to base the price threshold on by identifying the instance type with the lowest price from those that match your specified attributes. It does this in the following way:

  • It first looks at the current generation C, M, or R instance types that match your attributes. If it finds any matches, it identifies the lowest priced instance type.

  • If there is no match, it then looks at any current generation instance types that match your attributes. If it finds any matches, it identifies the lowest priced instance type.

  • If there is no match, it then looks at any previous generation instance types that match your attributes, and identifies the lowest priced instance type.

On-Demand Instance price protection

The price protection threshold for On-Demand instance types is calculated as a percentage higher than the identified lowest priced On-Demand instance type (OnDemandMaxPricePercentageOverLowestPrice). You specify the percentage higher that you're willing to pay. If you don't specify this parameter, then a default value of 20 is used to calculate a price protection threshold of 20% higher than the identified price.

For example, if the identified On-Demand instance price is 0.4271, and you specify 25, then the price threshold is 25% more than 0.4271. It is calculated as follows: 0.4271 * 1.25 = 0.533875. The calculated price is the maximum you're willing to pay for On-Demand Instances, and, in this example, Amazon EC2 will exclude any On-Demand instance types that cost more than 0.533875.

Spot Instance price protection

By default, Amazon EC2 will automatically apply optimal Spot Instance price protection to consistently select from a wide range of instance types. You can also manually set the price protection yourself. However, letting Amazon EC2 do it for you can improve the likelihood that your Spot capacity is fulfilled.

You can manually specify the price protection using one of the following options. If you manually set the price protection, we recommend using the first option.

  • A percentage of the identified lowest priced On-Demand instance type [MaxSpotPriceAsPercentageOfOptimalOnDemandPrice]

    For example, if the identified On-Demand instance type price is 0.4271, and you specify 60, then the price threshold is 60% of 0.4271. It is calculated as follows: 0.4271 * 0.60 = 0.25626. The calculated price is the maximum you're willing to pay for Spot Instances, and, in this example, Amazon EC2 will exclude any Spot instance types that cost more than 0.25626.

  • A percentage higher than the identified lowest priced Spot instance type [SpotMaxPricePercentageOverLowestPrice]

    For example, if the identified Spot instance type price is 0.1808, and you specify 25, then the price threshold is 25% more than 0.1808. It is calculated as follows: 0.1808 * 1.25 = 0.226. The calculated price is the maximum you're willing to pay for Spot Instances, and, in this example, Amazon EC2 will exclude any Spot instance types that cost more than 0.266. We do not recommend using this parameter because Spot prices can fluctuate, and therefore your price protection threshold might also fluctuate.

Specify the price protection threshold

To specify the price protection threshold

While creating the EC2 Fleet, configure the fleet for attribute-based instance type selection, and then do the following:

  • To specify the On-Demand Instance price protection threshold, in the JSON configuration file, in the InstanceRequirements structure, for OnDemandMaxPricePercentageOverLowestPrice, enter the price protection threshold as a percentage.

  • To specify the Spot Instance price protection threshold, in the JSON configuration file, in the InstanceRequirements structure, specify one of the following parameters:

    • For MaxSpotPriceAsPercentageOfOptimalOnDemandPrice, enter the price protection threshold as a percentage.

    • For SpotMaxPricePercentageOverLowestPrice, enter the price protection threshold as a percentage.

For more information about creating the fleet, see Create an EC2 Fleet with attribute-based instance type selection.

Note

When creating the EC2 Fleet, if you set TargetCapacityUnitType to vcpu or memory-mib, the price protection threshold is applied based on the per-vCPU or per-memory price instead of the per-instance price.

Considerations

  • You can specify either instance types or instance attributes in an EC2 Fleet, but not both at the same time.

    When using the CLI, the launch template overrides will override the launch template. For example, if the launch template contains an instance type and the launch template override contains instance attributes, the instances that are identified by the instance attributes will override the instance type in the launch template.

  • When using the CLI, when you specify instance attributes as overrides, you can't also specify weights or priorities.

  • You can specify a maximum of four InstanceRequirements structures in a request configuration.

Create an EC2 Fleet with attribute-based instance type selection

You can configure a fleet to use attribute-based instance type selection by using the Amazon CLI.

To create an EC2 Fleet with attribute-based instance type selection (Amazon CLI)

Use the create-fleet (Amazon CLI) command to create an EC2 Fleet. Specify the fleet configuration in a JSON file.

aws ec2 create-fleet \ --region us-east-1 \ --cli-input-json file://file_name.json

Example file_name.json file

The following example contains the parameters that configure an EC2 Fleet to use attribute-based instance type selection, and is followed by a text explanation.

{ "SpotOptions": { "AllocationStrategy": "price-capacity-optimized" }, "LaunchTemplateConfigs": [{ "LaunchTemplateSpecification": { "LaunchTemplateName": "my-launch-template", "Version": "1" }, "Overrides": [{ "InstanceRequirements": { "VCpuCount": { "Min": 2 }, "MemoryMiB": { "Min": 4 } } }] }], "TargetCapacitySpecification": { "TotalTargetCapacity": 20, "DefaultTargetCapacityType": "spot" }, "Type": "instant" }

The attributes for attribute-based instance type selection are specified in the InstanceRequirements structure. In this example, two attributes are specified:

  • VCpuCount – A minimum of 2 vCPUs is specified. Because no maximum is specified, there is no maximum limit.

  • MemoryMiB – A minimum of 4 MiB of memory is specified. Because no maximum is specified, there is no maximum limit.

Any instance types that have 2 or more vCPUs and 4 MiB or more of memory will be identified. However, price protection and the allocation strategy might exclude some instance types when EC2 Fleet provisions the fleet.

For a list and descriptions of all the possible attributes that you can specify, see InstanceRequirements in the Amazon EC2 API Reference.

Note

When InstanceRequirements is included in the fleet configuration, InstanceType and WeightedCapacity must be excluded; they cannot determine the fleet configuration at the same time as instance attributes.

The JSON also contains the following fleet configuration:

  • "AllocationStrategy": "price-capacity-optimized" – The allocation strategy for the Spot Instances in the fleet.

  • "LaunchTemplateName": "my-launch-template", "Version": "1" – The launch template contains some instance configuration information, but if any instance types are specified, they will be overridden by the attributes that are specified in InstanceRequirements.

  • "TotalTargetCapacity": 20 – The target capacity is 20 instances.

  • "DefaultTargetCapacityType": "spot" – The default capacity is Spot Instances.

  • "Type": "instant" – The request type for the fleet is instant.

Examples of configurations that are valid and not valid

If you use the Amazon CLI to create an EC2 Fleet, you must make sure that your fleet configuration is valid. The following examples show configurations that are valid and not valid.

Configurations are considered not valid when they contain the following:

  • A single Overrides structure with both InstanceRequirements and InstanceType

  • Two Overrides structures, one with InstanceRequirements and the other with InstanceType

  • Two InstanceRequirements structures with overlapping attribute values within the same LaunchTemplateSpecification

Valid configuration: Single launch template with overrides

The following configuration is valid. It contains one launch template and one Overrides structure containing one InstanceRequirements structure. A text explanation of the example configuration follows.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "My-launch-template", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 2, "Max": 8 }, "MemoryMib": { "Min": 0, "Max": 10240 }, "MemoryGiBPerVCpu": { "Max": 10000 }, "RequireHibernateSupport": true } } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 5000, "DefaultTargetCapacityType": "spot", "TargetCapacityUnitType": "vcpu" } } }
InstanceRequirements

To use attribute-based instance selection, you must include the InstanceRequirements structure in your fleet configuration, and specify the desired attributes for the instances in the fleet.

In the preceding example, the following instance attributes are specified:

  • VCpuCount – The instance types must have a minimum of 2 and a maximum of 8 vCPUs.

  • MemoryMiB – The instance types must have a maximum of 10240 MiB of memory. A minimum of 0 indicates no minimum limit.

  • MemoryGiBPerVCpu – The instance types must have a maximum of 10,000 GiB of memory per vCPU. The Min parameter is optional. By omitting it, you indicate no minimum limit.

TargetCapacityUnitType

The TargetCapacityUnitType parameter specifies the unit for the target capacity. In the example, the target capacity is 5000 and the target capacity unit type is vcpu, which together specify a desired target capacity of 5,000 vCPUs. EC2 Fleet will launch enough instances so that the total number of vCPUs in the fleet is 5,000 vCPUs.

Valid configuration: Single launch template with multiple InstanceRequirements

The following configuration is valid. It contains one launch template and one Overrides structure containing two InstanceRequirements structures. The attributes specified in InstanceRequirements are valid because the values do not overlap—the first InstanceRequirements structure specifies a VCpuCount of 0-2 vCPUs, while the second InstanceRequirements structure specifies 4-8 vCPUs.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } }, { "InstanceRequirements": { "VCpuCount": { "Min": 4, "Max": 8 }, "MemoryMiB": { "Min": 0 } } } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Valid configuration: Two launch templates, each with overrides

The following configuration is valid. It contains two launch templates, each with one Overrides structure containing one InstanceRequirements structure. This configuration is useful for arm and x86 architecture support in the same fleet.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "armLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } }, { "LaunchTemplateSpecification": { "LaunchTemplateName": "x86LaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Valid configuration: Only InstanceRequirements specified, no overlapping attribute values

The following configuration is valid. It contains two LaunchTemplateSpecification structures, each with a launch template and an Overrides structure containing an InstanceRequirements structure. The attributes specified in InstanceRequirements are valid because the values do not overlap—the first InstanceRequirements structure specifies a VCpuCount of 0-2 vCPUs, while the second InstanceRequirements structure specifies 4-8 vCPUs.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } } ] }, { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyOtherLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 4, "Max": 8 }, "MemoryMiB": { "Min": 0 } } } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Configuration not valid: Overrides contain InstanceRequirements and InstanceType

The following configuration is not valid. The Overrides structure contains both InstanceRequirements and InstanceType. For the Overrides, you can specify either InstanceRequirements or InstanceType, but not both.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } }, { "InstanceType": "m5.large" } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Configuration not valid: Two Overrides contain InstanceRequirements and InstanceType

The following configuration is not valid. The Overrides structures contain both InstanceRequirements and InstanceType. You can specify either InstanceRequirements or InstanceType, but not both, even if they're in different Overrides structures.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } } ] }, { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyOtherLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceType": "m5.large" } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Configuration not valid: Overlapping attribute values

The following configuration is not valid. The two InstanceRequirements structures each contain "VCpuCount": {"Min": 0, "Max": 2}. The values for these attributes overlap, which will result in duplicate capacity pools.

{ "LaunchTemplateConfigs": [ { "LaunchTemplateSpecification": { "LaunchTemplateName": "MyLaunchTemplate", "Version": "1" }, "Overrides": [ { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } }, { "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 2 }, "MemoryMiB": { "Min": 0 } } } } ] } ], "TargetCapacitySpecification": { "TotalTargetCapacity": 1, "DefaultTargetCapacityType": "spot" } } }

Preview instance types with specified attributes

You can use the get-instance-types-from-instance-requirements Amazon CLI command to preview the instance types that match the attributes that you specify. This is especially useful for working out what attributes to specify in your request configuration without launching any instances. Note that the command does not consider available capacity.

To preview a list of instance types by specifying attributes using the Amazon CLI
  1. (Optional) To generate all of the possible attributes that can be specified, use the get-instance-types-from-instance-requirements command and the --generate-cli-skeleton parameter. You can optionally direct the output to a file to save it by using input > attributes.json.

    aws ec2 get-instance-types-from-instance-requirements \ --region us-east-1 \ --generate-cli-skeleton input > attributes.json

    Expected output

    { "DryRun": true, "ArchitectureTypes": [ "i386" ], "VirtualizationTypes": [ "hvm" ], "InstanceRequirements": { "VCpuCount": { "Min": 0, "Max": 0 }, "MemoryMiB": { "Min": 0, "Max": 0 }, "CpuManufacturers": [ "intel" ], "MemoryGiBPerVCpu": { "Min": 0.0, "Max": 0.0 }, "ExcludedInstanceTypes": [ "" ], "InstanceGenerations": [ "current" ], "SpotMaxPricePercentageOverLowestPrice": 0, "OnDemandMaxPricePercentageOverLowestPrice": 0, "BareMetal": "included", "BurstablePerformance": "included", "RequireHibernateSupport": true, "NetworkInterfaceCount": { "Min": 0, "Max": 0 }, "LocalStorage": "included", "LocalStorageTypes": [ "hdd" ], "TotalLocalStorageGB": { "Min": 0.0, "Max": 0.0 }, "BaselineEbsBandwidthMbps": { "Min": 0, "Max": 0 }, "AcceleratorTypes": [ "gpu" ], "AcceleratorCount": { "Min": 0, "Max": 0 }, "AcceleratorManufacturers": [ "nvidia" ], "AcceleratorNames": [ "a100" ], "AcceleratorTotalMemoryMiB": { "Min": 0, "Max": 0 }, "NetworkBandwidthGbps": { "Min": 0.0, "Max": 0.0 }, "AllowedInstanceTypes": [ "" ] }, "MaxResults": 0, "NextToken": "" }
  2. Create a JSON configuration file using the output from the previous step, and configure it as follows:

    Note

    You must provide values for ArchitectureTypes, VirtualizationTypes, VCpuCount, and MemoryMiB. You can omit the other attributes; when omitted, the default values are used.

    For a description of each attribute and their default values, see get-instance-types-from-instance-requirements in the Amazon EC2 Command Line Reference.

    1. For ArchitectureTypes, specify one or more types of processor architecture.

    2. For VirtualizationTypes, specify one or more types of virtualization.

    3. For VCpuCount, specify the minimum and maximum number of vCPUs. To specify no minimum limit, for Min, specify 0. To specify no maximum limit, omit the Max parameter.

    4. For MemoryMiB, specify the minimum and maximum amount of memory in MiB. To specify no minimum limit, for Min, specify 0. To specify no maximum limit, omit the Max parameter.

    5. You can optionally specify one or more of the other attributes to further constrain the list of instance types that are returned.

  3. To preview the instance types that have the attributes that you specified in the JSON file, use the get-instance-types-from-instance-requirements command, and specify the name and path to your JSON file by using the --cli-input-json parameter. You can optionally format the output to appear in a table format.

    aws ec2 get-instance-types-from-instance-requirements \ --cli-input-json file://attributes.json \ --output table

    Example attributes.json file

    In this example, the required attributes are included in the JSON file. They are ArchitectureTypes, VirtualizationTypes, VCpuCount, and MemoryMiB. In addition, the optional InstanceGenerations attribute is also included. Note that for MemoryMiB, the Max value can be omitted to indicate that there is no limit.

    { "ArchitectureTypes": [ "x86_64" ], "VirtualizationTypes": [ "hvm" ], "InstanceRequirements": { "VCpuCount": { "Min": 4, "Max": 6 }, "MemoryMiB": { "Min": 2048 }, "InstanceGenerations": [ "current" ] } }

    Example output

    ------------------------------------------ |GetInstanceTypesFromInstanceRequirements| +----------------------------------------+ || InstanceTypes || |+--------------------------------------+| || InstanceType || |+--------------------------------------+| || c4.xlarge || || c5.xlarge || || c5a.xlarge || || c5ad.xlarge || || c5d.xlarge || || c5n.xlarge || || d2.xlarge || ...
  4. After identifying instance types that meet your needs, make note of the instance attributes that you used so that you can use them when configuring your fleet request.