Tune an IP Insights Model
Automatic model tuning, also called hyperparameter tuning, finds the best version of a model by running many jobs that test a range of hyperparameters on your dataset. You choose the tunable hyperparameters, a range of values for each, and an objective metric. You choose the objective metric from the metrics that the algorithm computes. Automatic model tuning searches the hyperparameters chosen to find the combination of values that result in the model that optimizes the objective metric.
For more information about model tuning, see Perform automatic model tuning with SageMaker.
Metrics Computed by the IP Insights Algorithm
The Amazon SageMaker IP Insights algorithm is an unsupervised learning algorithm that learns
associations between IP addresses and entities. The algorithm trains a discriminator
model , which learns to separate observed data points (positive
samples) from randomly generated data points (negative
samples). Automatic model tuning on IP Insights helps you find the
model that can most accurately distinguish between unlabeled validation data and
automatically generated negative samples. The model accuracy on the validation
dataset is measured by the area under the receiver operating characteristic
curve. This validation:discriminator_auc metric can take values between
0.0 and 1.0, where 1.0 indicates perfect accuracy.
The IP Insights algorithm computes a validation:discriminator_auc
metric during validation, the value of which is used as the objective function to
optimize for hyperparameter tuning.
| Metric Name | Description | Optimization Direction |
|---|---|---|
validation:discriminator_auc |
Area under the receiver operating characteristic curve on the validation dataset. The validation dataset is not labeled. Area Under the Curve (AUC) is a metric that describes the model's ability to discriminate validation data points from randomly generated data points. |
Maximize |
Tunable IP Insights Hyperparameters
You can tune the following hyperparameters for the SageMaker IP Insights algorithm.
| Parameter Name | Parameter Type | Recommended Ranges |
|---|---|---|
epochs |
IntegerParameterRange |
MinValue: 1, MaxValue: 100 |
learning_rate |
ContinuousParameterRange |
MinValue: 1e-4, MaxValue: 0.1 |
mini_batch_size |
IntegerParameterRanges |
MinValue: 100, MaxValue: 50000 |
num_entity_vectors |
IntegerParameterRanges |
MinValue: 10000, MaxValue: 1000000 |
num_ip_encoder_layers |
IntegerParameterRanges |
MinValue: 1, MaxValue: 10 |
random_negative_sampling_rate |
IntegerParameterRanges |
MinValue: 0, MaxValue: 10 |
shuffled_negative_sampling_rate |
IntegerParameterRanges |
MinValue: 0, MaxValue: 10 |
vector_dim |
IntegerParameterRanges |
MinValue: 8, MaxValue: 256 |
weight_decay |
ContinuousParameterRange |
MinValue: 0.0, MaxValue: 1.0 |