Using an Amazon CloudFormation template to start Neptune-to-OpenSearch replication

VPC –  Provide the name of the VPC where the polling Lambda function will run.

List of Subnet IDs –  The subnets to which a network interface is established. Add subnets corresponding to your Neptune cluster.

List of Security Group Ids –  Provide the IDs of security groups that grant write inbound access to your source Neptune DB cluster.

List of Route Table Ids –  This is needed to create an Amazon DynamoDB endpoint in your Neptune VPC, if you do not already have one. You must provide a comma-separated list of route table IDs associated with the subnets.

Require to create Dynamo DB VPC Endpoint –  A Boolean value that defaults to true. You only need to change it to false if you have already created a DynamoDB endpoint in your VPC.

Require to create Monitoring VPC Endpoint –  A Boolean value that defaults to true. You only need to change it to false if you have already created a monitoring endpoint in your VPC.

Application Name –  You can generally leave this set to the default (NeptuneStream). If you use a different name, it must be unique.

Memory size for Lambda Poller –  Used to set the memory size available to the Lambda poller function. The default value is 2,048 megabytes.

Lambda Runtime –  The language used in the Lambda function that retrieves items from the Neptune stream. You can set this either to python3.9 or to java8.

S3 Bucket having Lambda code artifacts –  Leave this blank unless you are using a custom Lambda polling function that loads from a different S3 bucket.

S3 Key corresponding to Lambda Code artifacts –  Leave this blank unless you are using a custom Lambda polling function.

StartingCheckpoint –  the starting checkpoint for the stream poller. The default is 0:0, which signifies starting from the beginning of the Neptune stream.

StreamPollerInitialState –  The initial state of the poller. The default is ENABLED, which means that the stream replication will start as soon as the entire stack creation is complete.

Logging level for Lambda –  In general, leave this set to the default value, INFO.

Managed Policies for Lambda Execution –  In general, leave this blank unless you are using a custom Lambda polling function.

Stream Records Handler –  In general, leave this blank unless you are using a custom handler for the records in Neptune streams.

Maximum records Fetched from Stream –  You can use this parameter to tune performance. The default (100) is a good place to start. The maximum allowable is 10,000. The higher the number, the fewer network calls are needed to read records from the stream, but the more memory is required to process the records.

Max wait time between two Polls (in Seconds) –  Determines how frequently the Lambda poller is invoked to poll the Neptune streams. Set this value to 0 for continuous polling. The maximum value is 3,600 seconds (1 hour). The default value (60 seconds) is a good place to start, depending on how fast your graph data changes.

Maximum Continuous polling period (in Seconds) –  Used to set a timeout for the Lambda polling function. It should be between 5 seconds and 900 seconds. The default value (600 seconds) is a good place to start.

Step Function Fallback Period –  The number of step-function-fallback-period units to wait for the poller, after which the step function is called through Amazon CloudWatch Events to recover from a failure. The default (5 minutes) is a good place to start.

Step Function Fallback Period Unit –  The time units used to measure the preceding Step Function Fallback Period (minutes, hours, days). The default (minutes) is generally sufficient.

Data replication scope –  Determines whether to replicate both nodes and edges, or only nodes to OpenSearch (this applies to Gremlin engine data only). The default value (All) is generally a good place to start.

Ignore OpenSearch missing document error –  Flag to determine whether a missing document error in OpenSearch can be ignored. Missing document errors occur rarely but need manual intervention if not ignored. The default value (True) is generally a good place to start.

Enable Non-String Indexing –  Flag to enable or disable indexing of fields that do not have string content. If this flag is set to true, non-string fields are indexed in OpenSearch, or if false, only string fields are indexed. The default is true.

Properties to exclude from being inserted into OpenSearch –  A comma-delimited list of property or predicate keys to exclude from OpenSearch indexing. If this CFN parameter value is left blank, all the property keys are indexed.

Datatypes to exclude from being inserted into OpenSearch –  A comma-delimited list of property or predicate datatypes to exclude from OpenSearch indexing. If this CFN parameter value is left blank, all the property values that can safely be converted to OpenSearch datatypes are indexed.

Endpoint of source Neptune Stream –  (Required) This takes one of two forms:

Neptune Query Engine –  Choose Gremlin or SPARQL.

Is IAM Auth Enabled? –  If your Neptune DB cluster is using IAM authentication, set this parameter to true.

Neptune Cluster Resource Id –  If your Neptune DB cluster is using IAM authentication, set this parameter to the cluster resource ID. The resource ID is not the same as the cluster ID. Instead, it takes the form: cluster- followed by 28 alpha-numeric characters. It can be found under Cluster Details in the Neptune console.

Endpoint for OpenSearch service –  (Required) Provide the endpoint for the OpenSearch service in your VPC.

Number of Shards for OpenSearch Index –  The default value (5) is generally a good place to start.

Number of Replicas for OpenSearch Index –  The default value (1) is generally a good place to start.

Geo Location Fields for Mapping –  If you are using geolocation fields, list the property keys here.

Require to create Cloud watch Alarm –  Set this to true if you want to create a CloudWatch alarm for the new stack.

SNS Topic ARN for Cloudwatch Alarm Notifications   –   The SNS topic ARN where CloudWatch alarm notifications should be sent (only needed if alarms are enabled).

Email for Alarm Notifications –  The email address to which alarm notifications should be sent (only needed if alarms are enabled).