InferenceTargetConfiguration
The configuration for an inference target. An inference target routes requests to a large language model (LLM) provider, either through a built-in connector or an explicitly configured provider.
Contents
Important
This data type is a UNION, so only one of the following members can be specified when used or returned.
- connector
-
The connector-based inference configuration. Use this option to route requests to an LLM provider through a built-in connector that includes predefined provider rules.
Type: InferenceConnectorTargetConfiguration object
Required: No
- provider
-
The provider-based inference configuration. Use this option to explicitly configure the endpoint, model mapping, and operations for an LLM provider.
Type: InferenceProviderTargetConfiguration object
Required: No
See Also
For more information about using this API in one of the language-specific Amazon SDKs, see the following: