Class InferenceConfiguration
- All Implemented Interfaces:
Serializable
,SdkPojo
,ToCopyableBuilder<InferenceConfiguration.Builder,
InferenceConfiguration>
Contains inference parameters to use when the agent invokes a foundation model in the part of the agent sequence
defined by the promptType
. For more information, see Inference parameters for foundation
models.
- See Also:
-
Nested Class Summary
-
Method Summary
Modifier and TypeMethodDescriptionbuilder()
final boolean
final boolean
equalsBySdkFields
(Object obj) Indicates whether some other object is "equal to" this one by SDK fields.final <T> Optional
<T> getValueForField
(String fieldName, Class<T> clazz) final int
hashCode()
final boolean
For responses, this returns true if the service returned a value for the StopSequences property.final Integer
The maximum number of tokens to allow in the generated response.static Class
<? extends InferenceConfiguration.Builder> A list of stop sequences.final Float
The likelihood of the model selecting higher-probability options while generating a response.Take this object and create a builder that contains all of the current property values of this object.final Integer
topK()
While generating a response, the model determines the probability of the following token at each point of generation.final Float
topP()
While generating a response, the model determines the probability of the following token at each point of generation.final String
toString()
Returns a string representation of this object.Methods inherited from interface software.amazon.awssdk.utils.builder.ToCopyableBuilder
copy
-
Method Details
-
maximumLength
The maximum number of tokens to allow in the generated response.
- Returns:
- The maximum number of tokens to allow in the generated response.
-
hasStopSequences
public final boolean hasStopSequences()For responses, this returns true if the service returned a value for the StopSequences property. This DOES NOT check that the value is non-empty (for which, you should check theisEmpty()
method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified. -
stopSequences
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.
This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the
hasStopSequences()
method.- Returns:
- A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
-
temperature
The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.
- Returns:
- The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.
-
topK
While generating a response, the model determines the probability of the following token at each point of generation. The value that you set for
topK
is the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you settopK
to 50, the model selects the next token from among the top 50 most likely choices.- Returns:
- While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for
topK
is the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you settopK
to 50, the model selects the next token from among the top 50 most likely choices.
-
topP
While generating a response, the model determines the probability of the following token at each point of generation. The value that you set for
Top P
determines the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you settopP
to 80, the model only selects the next token from the top 80% of the probability distribution of next tokens.- Returns:
- While generating a response, the model determines the probability of the following token at each point of
generation. The value that you set for
Top P
determines the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you settopP
to 80, the model only selects the next token from the top 80% of the probability distribution of next tokens.
-
toBuilder
Description copied from interface:ToCopyableBuilder
Take this object and create a builder that contains all of the current property values of this object.- Specified by:
toBuilder
in interfaceToCopyableBuilder<InferenceConfiguration.Builder,
InferenceConfiguration> - Returns:
- a builder for type T
-
builder
-
serializableBuilderClass
-
hashCode
public final int hashCode() -
equals
-
equalsBySdkFields
Description copied from interface:SdkPojo
Indicates whether some other object is "equal to" this one by SDK fields. An SDK field is a modeled, non-inherited field in anSdkPojo
class, and is generated based on a service model.If an
SdkPojo
class does not have any inherited fields,equalsBySdkFields
andequals
are essentially the same.- Specified by:
equalsBySdkFields
in interfaceSdkPojo
- Parameters:
obj
- the object to be compared with- Returns:
- true if the other object equals to this object by sdk fields, false otherwise.
-
toString
Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value. -
getValueForField
-
sdkFields
-