AWS CLI Command Reference

Note:

You are viewing the documentation for an older major version of the AWS CLI (version 1).

AWS CLI version 2, the latest major version of AWS CLI, is now stable and recommended for general use. To view this page for the AWS CLI version 2, click here. For more information see the AWS CLI version 2 installation instructions and migration guide.

[ aws . sagemaker ]

create-auto-ml-job

Description

Creates an Autopilot job also referred to as Autopilot experiment or AutoML job.

Note

We recommend using the new versions CreateAutoMLJobV2 and DescribeAutoMLJobV2 , which offer backward compatibility.

CreateAutoMLJobV2 can manage tabular problem types identical to those of its previous version CreateAutoMLJob , as well as time-series forecasting, non-tabular problem types such as image or text classification, and text generation (LLMs fine-tuning).

Find guidelines about how to migrate a CreateAutoMLJob to CreateAutoMLJobV2 in Migrate a CreateAutoMLJob to CreateAutoMLJobV2 .

You can find the best-performing model after you run an AutoML job by calling DescribeAutoMLJobV2 (recommended) or DescribeAutoMLJob .

See also: AWS API Documentation

Synopsis

  create-auto-ml-job
--auto-ml-job-name <value>
--input-data-config <value>
--output-data-config <value>
[--problem-type <value>]
[--auto-ml-job-objective <value>]
[--auto-ml-job-config <value>]
--role-arn <value>
[--generate-candidate-definitions-only | --no-generate-candidate-definitions-only]
[--tags <value>]
[--model-deploy-config <value>]
[--cli-input-json <value>]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]

Options

--auto-ml-job-name (string)

Identifies an Autopilot job. The name must be unique to your account and is case insensitive.

--input-data-config (list)

An array of channel objects that describes the input data and its location. Each channel is a named input source. Similar to InputDataConfig supported by HyperParameterTrainingJobDefinition . Format(s) supported: CSV, Parquet. A minimum of 500 rows is required for the training dataset. There is not a minimum number of rows required for the validation dataset.

(structure)

A channel is a named input source that training algorithms can consume. The validation dataset size is limited to less than 2 GB. The training dataset size must be less than 100 GB. For more information, see Channel .

Note

A validation dataset must contain the same headers as the training dataset.

DataSource -> (structure)

The data source for an AutoML channel.

S3DataSource -> (structure)

The Amazon S3 location of the input data.

S3DataType -> (string)

The data type.

  • If you choose S3Prefix , S3Uri identifies a key name prefix. SageMaker uses all objects that match the specified key name prefix for model training. The S3Prefix should have the following format: s3://DOC-EXAMPLE-BUCKET/DOC-EXAMPLE-FOLDER-OR-FILE
  • If you choose ManifestFile , S3Uri identifies an object that is a manifest file containing a list of object keys that you want SageMaker to use for model training. A ManifestFile should have the format shown below: [ {"prefix": "s3://DOC-EXAMPLE-BUCKET/DOC-EXAMPLE-FOLDER/DOC-EXAMPLE-PREFIX/"}, "DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-1", "DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-2", ... "DOC-EXAMPLE-RELATIVE-PATH/DOC-EXAMPLE-FOLDER/DATA-N" ]
  • If you choose AugmentedManifestFile , S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile is available for V2 API jobs only (for example, for jobs created by calling CreateAutoMLJobV2 ). Here is a minimal, single-record example of an AugmentedManifestFile : {"source-ref": "s3://DOC-EXAMPLE-BUCKET/DOC-EXAMPLE-FOLDER/cats/cat.jpg", "label-metadata": {"class-name": "cat" } For more information on AugmentedManifestFile , see Provide Dataset Metadata to Training Jobs with an Augmented Manifest File .

S3Uri -> (string)

The URL to the Amazon S3 data source. The Uri refers to the Amazon S3 prefix or ManifestFile depending on the data type.

CompressionType -> (string)

You can use Gzip or None . The default value is None .

TargetAttributeName -> (string)

The name of the target variable in supervised learning, usually represented by 'y'.

ContentType -> (string)

The content type of the data from the input source. You can use text/csv;header=present or x-application/vnd.amazon+parquet . The default value is text/csv;header=present .

ChannelType -> (string)

The channel type (optional) is an enum string. The default value is training . Channels for training and validation must share the same ContentType and TargetAttributeName . For information on specifying training and validation channel types, see How to specify training and validation datasets .

SampleWeightAttributeName -> (string)

If specified, this column name indicates which column of the dataset should be treated as sample weights for use by the objective metric during the training, evaluation, and the selection of the best model. This column is not considered as a predictive feature. For more information on Autopilot metrics, see Metrics and validation .

Sample weights should be numeric, non-negative, with larger values indicating which rows are more important than others. Data points that have invalid or no weight value are excluded.

Support for sample weights is available in Ensembling mode only.

Shorthand Syntax:

DataSource={S3DataSource={S3DataType=string,S3Uri=string}},CompressionType=string,TargetAttributeName=string,ContentType=string,ChannelType=string,SampleWeightAttributeName=string ...

JSON Syntax:

[
  {
    "DataSource": {
      "S3DataSource": {
        "S3DataType": "ManifestFile"|"S3Prefix"|"AugmentedManifestFile",
        "S3Uri": "string"
      }
    },
    "CompressionType": "None"|"Gzip",
    "TargetAttributeName": "string",
    "ContentType": "string",
    "ChannelType": "training"|"validation",
    "SampleWeightAttributeName": "string"
  }
  ...
]

--output-data-config (structure)

Provides information about encryption and the Amazon S3 output path needed to store artifacts from an AutoML job. Format(s) supported: CSV.

KmsKeyId -> (string)

The Key Management Service encryption key ID.

S3OutputPath -> (string)

The Amazon S3 output path. Must be 128 characters or less.

Shorthand Syntax:

KmsKeyId=string,S3OutputPath=string

JSON Syntax:

{
  "KmsKeyId": "string",
  "S3OutputPath": "string"
}

--problem-type (string)

Defines the type of supervised learning problem available for the candidates. For more information, see SageMaker Autopilot problem types .

Possible values:

  • BinaryClassification
  • MulticlassClassification
  • Regression

--auto-ml-job-objective (structure)

Specifies a metric to minimize or maximize as the objective of a job. If not specified, the default objective metric depends on the problem type. See AutoMLJobObjective for the default values.

MetricName -> (string)

The name of the objective metric used to measure the predictive quality of a machine learning system. During training, the model's parameters are updated iteratively to optimize its performance based on the feedback provided by the objective metric when evaluating the model on the validation dataset.

The list of available metrics supported by Autopilot and the default metric applied when you do not specify a metric name explicitly depend on the problem type.

  • For tabular problem types:

    • List of available metrics:
      • Regression: MAE , MSE , R2 , RMSE
      • Binary classification: Accuracy , AUC , BalancedAccuracy , F1 , Precision , Recall
      • Multiclass classification: Accuracy , BalancedAccuracy , F1macro , PrecisionMacro , RecallMacro

    For a description of each metric, see Autopilot metrics for classification and regression .

    • Default objective metrics:
      • Regression: MSE .
      • Binary classification: F1 .
      • Multiclass classification: Accuracy .

  • For image or text classification problem types:

  • For time-series forecasting problem types:

  • For text generation problem types (LLMs fine-tuning): Fine-tuning language models in Autopilot does not require setting the AutoMLJobObjective field. Autopilot fine-tunes LLMs without requiring multiple candidates to be trained and evaluated. Instead, using your dataset, Autopilot directly fine-tunes your target model to enhance a default objective metric, the cross-entropy loss. After fine-tuning a language model, you can evaluate the quality of its generated text using different metrics. For a list of the available metrics, see Metrics for fine-tuning LLMs in Autopilot .

Shorthand Syntax:

MetricName=string

JSON Syntax:

{
  "MetricName": "Accuracy"|"MSE"|"F1"|"F1macro"|"AUC"|"RMSE"|"BalancedAccuracy"|"R2"|"Recall"|"RecallMacro"|"Precision"|"PrecisionMacro"|"MAE"|"MAPE"|"MASE"|"WAPE"|"AverageWeightedQuantileLoss"
}

--auto-ml-job-config (structure)

A collection of settings used to configure an AutoML job.

CompletionCriteria -> (structure)

How long an AutoML job is allowed to run, or how many candidates a job is allowed to generate.

MaxCandidates -> (integer)

The maximum number of times a training job is allowed to run.

For text and image classification, time-series forecasting, as well as text generation (LLMs fine-tuning) problem types, the supported value is 1. For tabular problem types, the maximum value is 750.

MaxRuntimePerTrainingJobInSeconds -> (integer)

The maximum time, in seconds, that each training job executed inside hyperparameter tuning is allowed to run as part of a hyperparameter tuning job. For more information, see the StoppingCondition used by the CreateHyperParameterTuningJob action.

For job V2s (jobs created by calling CreateAutoMLJobV2 ), this field controls the runtime of the job candidate.

For TextGenerationJobConfig problem types, the maximum time defaults to 72 hours (259200 seconds).

MaxAutoMLJobRuntimeInSeconds -> (integer)

The maximum runtime, in seconds, an AutoML job has to complete.

If an AutoML job exceeds the maximum runtime, the job is stopped automatically and its processing is ended gracefully. The AutoML job identifies the best model whose training was completed and marks it as the best-performing model. Any unfinished steps of the job, such as automatic one-click Autopilot model deployment, are not completed.

SecurityConfig -> (structure)

The security configuration for traffic encryption or Amazon VPC settings.

VolumeKmsKeyId -> (string)

The key used to encrypt stored data.

EnableInterContainerTrafficEncryption -> (boolean)

Whether to use traffic encryption between the container layers.

VpcConfig -> (structure)

The VPC configuration.

SecurityGroupIds -> (list)

The VPC security group IDs, in the form sg-xxxxxxxx . Specify the security groups for the VPC that is specified in the Subnets field.

(string)

Subnets -> (list)

The ID of the subnets in the VPC to which you want to connect your training job or model. For information about the availability of specific instance types, see Supported Instance Types and Availability Zones .

(string)

CandidateGenerationConfig -> (structure)

The configuration for generating a candidate for an AutoML job (optional).

FeatureSpecificationS3Uri -> (string)

A URL to the Amazon S3 data source containing selected features from the input data source to run an Autopilot job. You can input FeatureAttributeNames (optional) in JSON format as shown below:

{ "FeatureAttributeNames":["col1", "col2", ...] } .

You can also specify the data type of the feature (optional) in the format shown below:

{ "FeatureDataTypes":{"col1":"numeric", "col2":"categorical" ... } }

Note

These column keys may not include the target column.

In ensembling mode, Autopilot only supports the following data types: numeric , categorical , text , and datetime . In HPO mode, Autopilot can support numeric , categorical , text , datetime , and sequence .

If only FeatureDataTypes is provided, the column keys (col1 , col2 ,..) should be a subset of the column names in the input data.

If both FeatureDataTypes and FeatureAttributeNames are provided, then the column keys should be a subset of the column names provided in FeatureAttributeNames .

The key name FeatureAttributeNames is fixed. The values listed in ["col1", "col2", ...] are case sensitive and should be a list of strings containing unique values that are a subset of the column names in the input data. The list of columns provided must not include the target column.

AlgorithmsConfig -> (list)

Stores the configuration information for the selection of algorithms used to train the model candidates.

The list of available algorithms to choose from depends on the training mode set in ` AutoMLJobConfig.Mode https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_AutoMLJobConfig.html`__ .

  • AlgorithmsConfig should not be set in AUTO training mode.
  • When AlgorithmsConfig is provided, one AutoMLAlgorithms attribute must be set and one only. If the list of algorithms provided as values for AutoMLAlgorithms is empty, AutoMLCandidateGenerationConfig uses the full set of algorithms for the given training mode.
  • When AlgorithmsConfig is not provided, AutoMLCandidateGenerationConfig uses the full set of algorithms for the given training mode.

For the list of all algorithms per training mode, see AutoMLAlgorithmConfig .

For more information on each algorithm, see the Algorithm support section in Autopilot developer guide.

(structure)

The collection of algorithms run on a dataset for training the model candidates of an Autopilot job.

AutoMLAlgorithms -> (list)

The selection of algorithms run on a dataset to train the model candidates of an Autopilot job.

Note

Selected algorithms must belong to the list corresponding to the training mode set in AutoMLJobConfig.Mode (ENSEMBLING or HYPERPARAMETER_TUNING ). Choose a minimum of 1 algorithm.
  • In ENSEMBLING mode:
    • "catboost"
    • "extra-trees"
    • "fastai"
    • "lightgbm"
    • "linear-learner"
    • "nn-torch"
    • "randomforest"
    • "xgboost"
  • In HYPERPARAMETER_TUNING mode:
    • "linear-learner"
    • "mlp"
    • "xgboost"

(string)

DataSplitConfig -> (structure)

The configuration for splitting the input training dataset.

Type: AutoMLDataSplitConfig

ValidationFraction -> (float)

The validation fraction (optional) is a float that specifies the portion of the training dataset to be used for validation. The default value is 0.2, and values must be greater than 0 and less than 1. We recommend setting this value to be less than 0.5.

Mode -> (string)

The method that Autopilot uses to train the data. You can either specify the mode manually or let Autopilot choose for you based on the dataset size by selecting AUTO . In AUTO mode, Autopilot chooses ENSEMBLING for datasets smaller than 100 MB, and HYPERPARAMETER_TUNING for larger ones.

The ENSEMBLING mode uses a multi-stack ensemble model to predict classification and regression tasks directly from your dataset. This machine learning mode combines several base models to produce an optimal predictive model. It then uses a stacking ensemble method to combine predictions from contributing members. A multi-stack ensemble model can provide better performance over a single model by combining the predictive capabilities of multiple models. See Autopilot algorithm support for a list of algorithms supported by ENSEMBLING mode.

The HYPERPARAMETER_TUNING (HPO) mode uses the best hyperparameters to train the best version of a model. HPO automatically selects an algorithm for the type of problem you want to solve. Then HPO finds the best hyperparameters according to your objective metric. See Autopilot algorithm support for a list of algorithms supported by HYPERPARAMETER_TUNING mode.

JSON Syntax:

{
  "CompletionCriteria": {
    "MaxCandidates": integer,
    "MaxRuntimePerTrainingJobInSeconds": integer,
    "MaxAutoMLJobRuntimeInSeconds": integer
  },
  "SecurityConfig": {
    "VolumeKmsKeyId": "string",
    "EnableInterContainerTrafficEncryption": true|false,
    "VpcConfig": {
      "SecurityGroupIds": ["string", ...],
      "Subnets": ["string", ...]
    }
  },
  "CandidateGenerationConfig": {
    "FeatureSpecificationS3Uri": "string",
    "AlgorithmsConfig": [
      {
        "AutoMLAlgorithms": ["xgboost"|"linear-learner"|"mlp"|"lightgbm"|"catboost"|"randomforest"|"extra-trees"|"nn-torch"|"fastai", ...]
      }
      ...
    ]
  },
  "DataSplitConfig": {
    "ValidationFraction": float
  },
  "Mode": "AUTO"|"ENSEMBLING"|"HYPERPARAMETER_TUNING"
}

--role-arn (string)

The ARN of the role that is used to access the data.

--generate-candidate-definitions-only | --no-generate-candidate-definitions-only (boolean)

Generates possible candidates without training the models. A candidate is a combination of data preprocessors, algorithms, and algorithm parameter settings.

--tags (list)

An array of key-value pairs. You can use tags to categorize your Amazon Web Services resources in different ways, for example, by purpose, owner, or environment. For more information, see Tagging Amazon Web ServicesResources . Tag keys must be unique per resource.

(structure)

A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.

You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags .

For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources . For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy .

Key -> (string)

The tag key. Tag keys must be unique per resource.

Value -> (string)

The tag value.

Shorthand Syntax:

Key=string,Value=string ...

JSON Syntax:

[
  {
    "Key": "string",
    "Value": "string"
  }
  ...
]

--model-deploy-config (structure)

Specifies how to generate the endpoint name for an automatic one-click Autopilot model deployment.

AutoGenerateEndpointName -> (boolean)

Set to True to automatically generate an endpoint name for a one-click Autopilot model deployment; set to False otherwise. The default value is False .

Note

If you set AutoGenerateEndpointName to True , do not specify the EndpointName ; otherwise a 400 error is thrown.

EndpointName -> (string)

Specifies the endpoint name to use for a one-click Autopilot model deployment if the endpoint name is not generated automatically.

Note

Specify the EndpointName if and only if you set AutoGenerateEndpointName to False ; otherwise a 400 error is thrown.

Shorthand Syntax:

AutoGenerateEndpointName=boolean,EndpointName=string

JSON Syntax:

{
  "AutoGenerateEndpointName": true|false,
  "EndpointName": "string"
}

--cli-input-json (string) Performs service operation based on the JSON string provided. The JSON string follows the format provided by --generate-cli-skeleton. If other arguments are provided on the command line, the CLI values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally.

--generate-cli-skeleton (string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value input, prints a sample input JSON that can be used as an argument for --cli-input-json. If provided with the value output, it validates the command inputs and returns a sample output JSON for that command.

Global Options

--debug (boolean)

Turn on debug logging.

--endpoint-url (string)

Override command's default URL with the given URL.

--no-verify-ssl (boolean)

By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

--no-paginate (boolean)

Disable automatic pagination.

--output (string)

The formatting style for command output.

  • json
  • text
  • table

--query (string)

A JMESPath query to use in filtering the response data.

--profile (string)

Use a specific profile from your credential file.

--region (string)

The region to use. Overrides config/env settings.

--version (string)

Display the version of this tool.

--color (string)

Turn on/off color output.

  • on
  • off
  • auto

--no-sign-request (boolean)

Do not sign requests. Credentials will not be loaded if this argument is provided.

--ca-bundle (string)

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.

--cli-read-timeout (int)

The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.

--cli-connect-timeout (int)

The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.

Output

AutoMLJobArn -> (string)

The unique ARN assigned to the AutoML job when it is created.