Namespace Amazon.CDK.AWS.OpenSearchService
Amazon OpenSearch Service Construct Library
See Migrating to OpenSearch for migration instructions from aws-cdk-lib/aws-elasticsearch
to this module, aws-cdk-lib/aws-opensearchservice
.
Quick start
Create a development cluster by simply specifying the version:
var devDomain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0
});
To perform version upgrades without replacing the entire domain, specify the enableVersionUpgrade
property.
var devDomain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
EnableVersionUpgrade = true
});
Create a cluster with GP3 volumes:
var gp3Domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_2_5,
Ebs = new EbsOptions {
VolumeSize = 30,
VolumeType = EbsDeviceVolumeType.GP3,
Throughput = 125,
Iops = 3000
}
});
Create a production grade cluster by also specifying things like capacity and az distribution
var prodDomain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
Capacity = new CapacityConfig {
MasterNodes = 5,
DataNodes = 20
},
Ebs = new EbsOptions {
VolumeSize = 20
},
ZoneAwareness = new ZoneAwarenessConfig {
AvailabilityZoneCount = 3
},
Logging = new LoggingOptions {
SlowSearchLogEnabled = true,
AppLogEnabled = true,
SlowIndexLogEnabled = true
}
});
This creates an Amazon OpenSearch Service cluster and automatically sets up log groups for logging the domain logs and slow search logs.
A note about SLR
Some cluster configurations (e.g VPC access) require the existence of the AWSServiceRoleForAmazonElasticsearchService
Service-Linked Role.
When performing such operations via the AWS Console, this SLR is created automatically when needed. However, this is not the behavior when using CloudFormation. If an SLR is needed, but doesn't exist, you will encounter a failure message similar to:
Before you can proceed, you must enable a service-linked role to give Amazon OpenSearch Service...
To resolve this, you need to create the SLR. We recommend using the AWS CLI:
aws iam create-service-linked-role --aws-service-name es.amazonaws.com
You can also create it using the CDK, but note that only the first application deploying this will succeed:
var slr = new CfnServiceLinkedRole(this, "Service Linked Role", new CfnServiceLinkedRoleProps {
AwsServiceName = "es.amazonaws.com"
});
Importing existing domains
Using a known domain endpoint
To import an existing domain into your CDK application, use the Domain.fromDomainEndpoint
factory method.
This method accepts a domain endpoint of an already existing domain:
var domainEndpoint = "https://my-domain-jcjotrt6f7otem4sqcwbch3c4u.us-east-1.es.amazonaws.com";
var domain = Domain.FromDomainEndpoint(this, "ImportedDomain", domainEndpoint);
Using the output of another CloudFormation stack
To import an existing domain with the help of an exported value from another CloudFormation stack,
use the Domain.fromDomainAttributes
factory method. This will accept tokens.
var domainArn = Fn.ImportValue("another-cf-stack-export-domain-arn");
var domainEndpoint = Fn.ImportValue("another-cf-stack-export-domain-endpoint");
var domain = Domain.FromDomainAttributes(this, "ImportedDomain", new DomainAttributes {
DomainArn = domainArn,
DomainEndpoint = domainEndpoint
});
Permissions
IAM
Helper methods also exist for managing access to the domain.
Function fn;
Domain domain;
// Grant write access to the app-search index
domain.GrantIndexWrite("app-search", fn);
// Grant read access to the 'app-search/_search' path
domain.GrantPathRead("app-search/_search", fn);
Encryption
The domain can also be created with encryption enabled:
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
Ebs = new EbsOptions {
VolumeSize = 100,
VolumeType = EbsDeviceVolumeType.GENERAL_PURPOSE_SSD
},
NodeToNodeEncryption = true,
EncryptionAtRest = new EncryptionAtRestOptions {
Enabled = true
}
});
This sets up the domain with node to node encryption and encryption at rest. You can also choose to supply your own KMS key to use for encryption at rest.
VPC Support
Domains can be placed inside a VPC, providing a secure communication between Amazon OpenSearch Service and other services within the VPC without the need for an internet gateway, NAT device, or VPN connection.
Visit <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/vpc.html">VPC Support for Amazon OpenSearch Service Domains</a> for more details.
var vpc = new Vpc(this, "Vpc");
var domainProps = new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
RemovalPolicy = RemovalPolicy.DESTROY,
Vpc = vpc,
// must be enabled since our VPC contains multiple private subnets.
ZoneAwareness = new ZoneAwarenessConfig {
Enabled = true
},
Capacity = new CapacityConfig {
// must be an even number since the default az count is 2.
DataNodes = 2
}
};
new Domain(this, "Domain", domainProps);
In addition, you can use the vpcSubnets
property to control which specific subnets will be used, and the securityGroups
property to control
which security groups will be attached to the domain. By default, CDK will select all private subnets in the VPC, and create one dedicated security group.
Metrics
Helper methods exist to access common domain metrics for example:
Domain domain;
var freeStorageSpace = domain.MetricFreeStorageSpace();
var masterSysMemoryUtilization = domain.Metric("MasterSysMemoryUtilization");
This module is part of the AWS Cloud Development Kit project.
Fine grained access control
The domain can also be created with a master user configured. The password can be supplied or dynamically created if not supplied.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
EnforceHttps = true,
NodeToNodeEncryption = true,
EncryptionAtRest = new EncryptionAtRestOptions {
Enabled = true
},
FineGrainedAccessControl = new AdvancedSecurityOptions {
MasterUserName = "master-user"
}
});
var masterUserPassword = domain.MasterUserPassword;
SAML authentication
You can enable SAML authentication to use your existing identity provider to offer single sign-on (SSO) for dashboards on Amazon OpenSearch Service domains running OpenSearch or Elasticsearch 6.7 or later. To use SAML authentication, fine-grained access control must be enabled.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
EnforceHttps = true,
NodeToNodeEncryption = true,
EncryptionAtRest = new EncryptionAtRestOptions {
Enabled = true
},
FineGrainedAccessControl = new AdvancedSecurityOptions {
MasterUserName = "master-user",
SamlAuthenticationEnabled = true,
SamlAuthenticationOptions = new SAMLOptionsProperty {
IdpEntityId = "entity-id",
IdpMetadataContent = "metadata-content-with-quotes-escaped"
}
}
});
Using unsigned basic auth
For convenience, the domain can be configured to allow unsigned HTTP requests that use basic auth. Unless the domain is configured to be part of a VPC this means anyone can access the domain using the configured master username and password.
To enable unsigned basic auth access the domain is configured with an access policy that allows anonymous requests, HTTPS required, node to node encryption, encryption at rest and fine grained access control.
If the above settings are not set they will be configured as part of enabling unsigned basic auth. If they are set with conflicting values, an error will be thrown.
If no master user is configured a default master user is created with the
username admin
.
If no password is configured a default master user password is created and
stored in the AWS Secrets Manager as secret. The secret has the prefix
<domain id>MasterUser
.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
UseUnsignedBasicAuth = true
});
var masterUserPassword = domain.MasterUserPassword;
Custom access policies
If the domain requires custom access control it can be configured either as a constructor property, or later by means of a helper method.
For simple permissions the accessPolicies
constructor may be sufficient:
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
AccessPolicies = new [] {
new PolicyStatement(new PolicyStatementProps {
Actions = new [] { "es:*ESHttpPost", "es:ESHttpPut*" },
Effect = Effect.ALLOW,
Principals = new [] { new AccountPrincipal("123456789012") },
Resources = new [] { "*" }
}) }
});
For more complex use-cases, for example, to set the domain up to receive data from a
cross-account Kinesis Firehose the addAccessPolicies
helper method
allows for policies that include the explicit domain ARN.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0
});
domain.AddAccessPolicies(
new PolicyStatement(new PolicyStatementProps {
Actions = new [] { "es:ESHttpPost", "es:ESHttpPut" },
Effect = Effect.ALLOW,
Principals = new [] { new AccountPrincipal("123456789012") },
Resources = new [] { domain.DomainArn, $"{domain.domainArn}/*" }
}),
new PolicyStatement(new PolicyStatementProps {
Actions = new [] { "es:ESHttpGet" },
Effect = Effect.ALLOW,
Principals = new [] { new AccountPrincipal("123456789012") },
Resources = new [] { $"{domain.domainArn}/_all/_settings", $"{domain.domainArn}/_cluster/stats", $"{domain.domainArn}/index-name*/_mapping/type-name", $"{domain.domainArn}/roletest*/_mapping/roletest", $"{domain.domainArn}/_nodes", $"{domain.domainArn}/_nodes/stats", $"{domain.domainArn}/_nodes/*/stats", $"{domain.domainArn}/_stats", $"{domain.domainArn}/index-name*/_stats", $"{domain.domainArn}/roletest*/_stat" }
}));
Audit logs
Audit logs can be enabled for a domain, but only when fine grained access control is enabled.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
EnforceHttps = true,
NodeToNodeEncryption = true,
EncryptionAtRest = new EncryptionAtRestOptions {
Enabled = true
},
FineGrainedAccessControl = new AdvancedSecurityOptions {
MasterUserName = "master-user"
},
Logging = new LoggingOptions {
AuditLogEnabled = true,
SlowSearchLogEnabled = true,
AppLogEnabled = true,
SlowIndexLogEnabled = true
}
});
Suppress creating CloudWatch Logs resource policy
When logging is enabled for the domain, the CloudWatch Logs resource policy is created by default.
This resource policy is necessary for logging, but since only a maximum of 10 resource policies can be created per region,
the maximum number of resource policies may be a problem when enabling logging for several domains.
By setting the suppressLogsResourcePolicy
option to true, you can suppress the creation of a CloudWatch Logs resource policy.
If you set the suppressLogsResourcePolicy
option to true, you must create a resource policy before deployment.
Also, to avoid reaching this limit, consider reusing a broader policy that includes multiple log groups.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
EnforceHttps = true,
NodeToNodeEncryption = true,
EncryptionAtRest = new EncryptionAtRestOptions {
Enabled = true
},
FineGrainedAccessControl = new AdvancedSecurityOptions {
MasterUserName = "master-user"
},
Logging = new LoggingOptions {
AuditLogEnabled = true,
SlowSearchLogEnabled = true,
AppLogEnabled = true,
SlowIndexLogEnabled = true
},
SuppressLogsResourcePolicy = true
});
Visit <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/createdomain-configure-slow-logs.html">Monitoring OpenSearch logs with Amazon CloudWatch Logs</a> for more details.
UltraWarm
UltraWarm nodes can be enabled to provide a cost-effective way to store large amounts of read-only data.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
Capacity = new CapacityConfig {
MasterNodes = 2,
WarmNodes = 2,
WarmInstanceType = "ultrawarm1.medium.search"
}
});
Cold storage
Cold storage can be enabled on the domain. You must enable UltraWarm storage to enable cold storage.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
Capacity = new CapacityConfig {
MasterNodes = 2,
WarmNodes = 2,
WarmInstanceType = "ultrawarm1.medium.search"
},
ColdStorageEnabled = true
});
Custom endpoint
Custom endpoints can be configured to reach the domain under a custom domain name.
new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
CustomEndpoint = new CustomEndpointOptions {
DomainName = "search.example.com"
}
});
It is also possible to specify a custom certificate instead of the auto-generated one.
Additionally, an automatic CNAME-Record is created if a hosted zone is provided for the custom endpoint
Advanced options
Advanced options can used to configure additional options.
new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
AdvancedOptions = new Dictionary<string, string> {
{ "rest.action.multi.allow_explicit_index", "false" },
{ "indices.fielddata.cache.size", "25" },
{ "indices.query.bool.max_clause_count", "2048" }
}
});
Amazon Cognito authentication for OpenSearch Dashboards
The domain can be configured to use Amazon Cognito authentication for OpenSearch Dashboards.
Visit <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/cognito-auth.html">Configuring Amazon Cognito authentication for OpenSearch Dashboards</a> for more details.
Role cognitoConfigurationRole;
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_0,
CognitoDashboardsAuth = new CognitoOptions {
Role = cognitoConfigurationRole,
IdentityPoolId = "example-identity-pool-id",
UserPoolId = "example-user-pool-id"
}
});
## Enable support for Multi-AZ with Standby deployment
The domain can be configured to use multi-AZ with standby.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_3,
Ebs = new EbsOptions {
VolumeSize = 10,
VolumeType = EbsDeviceVolumeType.GENERAL_PURPOSE_SSD_GP3
},
ZoneAwareness = new ZoneAwarenessConfig {
Enabled = true,
AvailabilityZoneCount = 3
},
Capacity = new CapacityConfig {
MultiAzWithStandbyEnabled = true,
MasterNodes = 3,
DataNodes = 3
}
});
Define off-peak windows
The domain can be configured to use a daily 10-hour window considered as off-peak hours.
Off-peak windows were introduced on February 16, 2023. All domains created before this date have the off-peak window disabled by default. You must manually enable and configure the off-peak window for these domains. All domains created after this date will have the off-peak window enabled by default. You can't disable the off-peak window for a domain after it's enabled.
Visit <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/off-peak.html">Defining off-peak windows for Amazon OpenSearch Service</a> for more details.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_3,
OffPeakWindowEnabled = true, // can be omitted if offPeakWindowStart is set
OffPeakWindowStart = new WindowStartTime {
Hours = 20,
Minutes = 0
}
});
Configuring service software updates
The domain can be configured to use service software updates.
Visit <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/service-software.html">Service software updates in Amazon OpenSearch Service</a> for more details.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_3,
EnableAutoSoftwareUpdate = true
});
IP address type
You can specify either dual stack or IPv4 as your IP address type.
var domain = new Domain(this, "Domain", new DomainProps {
Version = EngineVersion.OPENSEARCH_1_3,
IpAddressType = IpAddressType.DUAL_STACK
});
Classes
AdvancedSecurityOptions | Specifies options for fine-grained access control. |
CapacityConfig | Configures the capacity of the cluster such as the instance type and the number of instances. |
CfnDomain | The AWS::OpenSearchService::Domain resource creates an Amazon OpenSearch Service domain. |
CfnDomain.AdvancedSecurityOptionsInputProperty | Specifies options for fine-grained access control. |
CfnDomain.ClusterConfigProperty | The cluster configuration for the OpenSearch Service domain. |
CfnDomain.CognitoOptionsProperty | Configures OpenSearch Service to use Amazon Cognito authentication for OpenSearch Dashboards. |
CfnDomain.ColdStorageOptionsProperty | Container for the parameters required to enable cold storage for an OpenSearch Service domain. |
CfnDomain.DomainEndpointOptionsProperty | Specifies additional options for the domain endpoint, such as whether to require HTTPS for all traffic or whether to use a custom endpoint rather than the default endpoint. |
CfnDomain.EBSOptionsProperty | The configurations of Amazon Elastic Block Store (Amazon EBS) volumes that are attached to data nodes in the OpenSearch Service domain. |
CfnDomain.EncryptionAtRestOptionsProperty | Whether the domain should encrypt data at rest, and if so, the AWS Key Management Service key to use. |
CfnDomain.IdpProperty | The SAML Identity Provider's information. |
CfnDomain.LogPublishingOptionProperty | Specifies whether the OpenSearch Service domain publishes application, search slow logs, or index slow logs to Amazon CloudWatch. |
CfnDomain.MasterUserOptionsProperty | Specifies information about the master user. |
CfnDomain.NodeToNodeEncryptionOptionsProperty | Specifies options for node-to-node encryption. |
CfnDomain.OffPeakWindowOptionsProperty | Off-peak window settings for the domain. |
CfnDomain.OffPeakWindowProperty | A custom 10-hour, low-traffic window during which OpenSearch Service can perform mandatory configuration changes on the domain. |
CfnDomain.SAMLOptionsProperty | Container for information about the SAML configuration for OpenSearch Dashboards. |
CfnDomain.ServiceSoftwareOptionsProperty | The current status of the service software for an Amazon OpenSearch Service domain. |
CfnDomain.SnapshotOptionsProperty | DEPRECATED . |
CfnDomain.SoftwareUpdateOptionsProperty | Options for configuring service software updates for a domain. |
CfnDomain.VPCOptionsProperty | The virtual private cloud (VPC) configuration for the OpenSearch Service domain. |
CfnDomain.WindowStartTimeProperty | A custom start time for the off-peak window, in Coordinated Universal Time (UTC). |
CfnDomain.ZoneAwarenessConfigProperty | Specifies zone awareness configuration options. |
CfnDomainProps | Properties for defining a |
CognitoOptions | Configures Amazon OpenSearch Service to use Amazon Cognito authentication for OpenSearch Dashboards. |
CustomEndpointOptions | Configures a custom domain endpoint for the Amazon OpenSearch Service domain. |
Domain | Provides an Amazon OpenSearch Service domain. |
DomainAttributes | Reference to an Amazon OpenSearch Service domain. |
DomainProps | Properties for an Amazon OpenSearch Service domain. |
EbsOptions | The configurations of Amazon Elastic Block Store (Amazon EBS) volumes that are attached to data nodes in the Amazon OpenSearch Service domain. |
EncryptionAtRestOptions | Whether the domain should encrypt data at rest, and if so, the AWS Key Management Service (KMS) key to use. |
EngineVersion | OpenSearch version. |
IpAddressType | The IP address type for the domain. |
LoggingOptions | Configures log settings for the domain. |
SAMLOptionsProperty | Container for information about the SAML configuration for OpenSearch Dashboards. |
TLSSecurityPolicy | The minimum TLS version required for traffic to the domain. |
WindowStartTime | |
ZoneAwarenessConfig | Specifies zone awareness configuration options. |
Interfaces
CfnDomain.IAdvancedSecurityOptionsInputProperty | Specifies options for fine-grained access control. |
CfnDomain.IClusterConfigProperty | The cluster configuration for the OpenSearch Service domain. |
CfnDomain.ICognitoOptionsProperty | Configures OpenSearch Service to use Amazon Cognito authentication for OpenSearch Dashboards. |
CfnDomain.IColdStorageOptionsProperty | Container for the parameters required to enable cold storage for an OpenSearch Service domain. |
CfnDomain.IDomainEndpointOptionsProperty | Specifies additional options for the domain endpoint, such as whether to require HTTPS for all traffic or whether to use a custom endpoint rather than the default endpoint. |
CfnDomain.IEBSOptionsProperty | The configurations of Amazon Elastic Block Store (Amazon EBS) volumes that are attached to data nodes in the OpenSearch Service domain. |
CfnDomain.IEncryptionAtRestOptionsProperty | Whether the domain should encrypt data at rest, and if so, the AWS Key Management Service key to use. |
CfnDomain.IIdpProperty | The SAML Identity Provider's information. |
CfnDomain.ILogPublishingOptionProperty | Specifies whether the OpenSearch Service domain publishes application, search slow logs, or index slow logs to Amazon CloudWatch. |
CfnDomain.IMasterUserOptionsProperty | Specifies information about the master user. |
CfnDomain.INodeToNodeEncryptionOptionsProperty | Specifies options for node-to-node encryption. |
CfnDomain.IOffPeakWindowOptionsProperty | Off-peak window settings for the domain. |
CfnDomain.IOffPeakWindowProperty | A custom 10-hour, low-traffic window during which OpenSearch Service can perform mandatory configuration changes on the domain. |
CfnDomain.ISAMLOptionsProperty | Container for information about the SAML configuration for OpenSearch Dashboards. |
CfnDomain.IServiceSoftwareOptionsProperty | The current status of the service software for an Amazon OpenSearch Service domain. |
CfnDomain.ISnapshotOptionsProperty | DEPRECATED . |
CfnDomain.ISoftwareUpdateOptionsProperty | Options for configuring service software updates for a domain. |
CfnDomain.IVPCOptionsProperty | The virtual private cloud (VPC) configuration for the OpenSearch Service domain. |
CfnDomain.IWindowStartTimeProperty | A custom start time for the off-peak window, in Coordinated Universal Time (UTC). |
CfnDomain.IZoneAwarenessConfigProperty | Specifies zone awareness configuration options. |
IAdvancedSecurityOptions | Specifies options for fine-grained access control. |
ICapacityConfig | Configures the capacity of the cluster such as the instance type and the number of instances. |
ICfnDomainProps | Properties for defining a |
ICognitoOptions | Configures Amazon OpenSearch Service to use Amazon Cognito authentication for OpenSearch Dashboards. |
ICustomEndpointOptions | Configures a custom domain endpoint for the Amazon OpenSearch Service domain. |
IDomain | An interface that represents an Amazon OpenSearch Service domain - either created with the CDK, or an existing one. |
IDomainAttributes | Reference to an Amazon OpenSearch Service domain. |
IDomainProps | Properties for an Amazon OpenSearch Service domain. |
IEbsOptions | The configurations of Amazon Elastic Block Store (Amazon EBS) volumes that are attached to data nodes in the Amazon OpenSearch Service domain. |
IEncryptionAtRestOptions | Whether the domain should encrypt data at rest, and if so, the AWS Key Management Service (KMS) key to use. |
ILoggingOptions | Configures log settings for the domain. |
ISAMLOptionsProperty | Container for information about the SAML configuration for OpenSearch Dashboards. |
IWindowStartTime | |
IZoneAwarenessConfig | Specifies zone awareness configuration options. |