Tutorial: How to design your own cluster
Following are the one-time actions you must take in order to design your own ElastiCache cluster.
For further information on setting up ElastiCache see Setting up ElastiCache.
Topics
Step 1: Create a subnet group
Before you create a cluster, you first create a subnet group. A cache subnet group is a collection of subnets that you may want to designate for your cache clusters in a VPC. When launching a cache cluster in a VPC, you need to select a cache subnet group. Then ElastiCache uses that cache subnet group to assign IP addresses within that subnet to each cache node in the cluster.
When you create a new subnet group, note the number of available IP addresses. If the subnet has very few free IP addresses, you might be constrained as to how many more nodes you can add to the cluster. To resolve this issue, you can assign one or more subnets to a subnet group so that you have a sufficient number of IP addresses in your cluster's Availability Zone. After that, you can add more nodes to your cluster.
The following procedures show you how to create a subnet group called
mysubnetgroup
(console) and the Amazon CLI.
The following procedure shows how to create a subnet group (console).
To create a subnet group (Console)
-
Sign in to the Amazon Management Console, and open the ElastiCache console at https://console.amazonaws.cn/elasticache/
. -
In the navigation list, choose Subnet Groups.
-
Choose Create Subnet Group.
-
In the Create Subnet Group wizard, do the following. When all the settings are as you want them, choose Yes, Create.
-
In the Name box, type a name for your subnet group.
-
In the Description box, type a description for your subnet group.
-
In the VPC ID box, choose the Amazon VPC that you created.
-
In the Availability Zone and Subnet ID lists, choose the Availability Zone or Local Zone and ID of your private subnet, and then choose Add.
-
-
In the confirmation message that appears, choose Close.
Your new subnet group appears in the Subnet Groups list of the ElastiCache console. At the bottom of the window you can choose the subnet group to see details, such as all of the subnets associated with this group.
At a command prompt, use the command create-cache-subnet-group
to create a subnet group.
For Linux, OS X, or Unix:
aws elasticache create-cache-subnet-group \ --cache-subnet-group-name
mysubnetgroup
\ --cache-subnet-group-description"Testing"
\ --subnet-idssubnet-53df9c3a
For Windows:
aws elasticache create-cache-subnet-group ^ --cache-subnet-group-name
mysubnetgroup
^ --cache-subnet-group-description"Testing"
^ --subnet-idssubnet-53df9c3a
This command should produce output similar to the following:
{
"CacheSubnetGroup": {
"VpcId": "vpc-37c3cd17",
"CacheSubnetGroupDescription": "Testing",
"Subnets": [
{
"SubnetIdentifier": "subnet-53df9c3a",
"SubnetAvailabilityZone": {
"Name": "us-west-2a"
}
}
],
"CacheSubnetGroupName": "mysubnetgroup"
}
}
For more information, see the Amazon CLI topic create-cache-subnet-group.
Step 2: Create a cluster
Before creating a cluster for production use, you obviously need to consider how you will configure the cluster to meet your business needs. Those issues are addressed in the Preparing a cluster in ElastiCache section. For the purposes of this Getting Started exercise, you will create a cluster with cluster mode disabled and you can accept the default configuration values where they apply.
The cluster you create will be live, and not running in a sandbox. You will
incur the standard ElastiCache usage fees for the instance until you delete it. The total
charges will be minimal (typically less than a dollar) if you complete the exercise
described here in one sitting and delete your cluster when you are finished. For more
information about ElastiCache usage rates, see Amazon ElastiCache
Your cluster is launched in a virtual private cloud (VPC) based on the Amazon VPC service.
Creating a Redis OSS (cluster mode disabled) cluster (Console)
To create a Redis OSS (cluster mode disabled) cluster using the ElastiCache console
-
Sign in to the Amazon Web Services Management Console and open the Amazon ElastiCache console at https://console.amazonaws.cn/elasticache/
. -
From the list in the upper-right corner, choose the Amazon Region that you want to launch this cluster in.
-
Choose Get started from the navigation pane.
Choose Create VPC and follow the steps outlined at Creating a Virtual Private Cloud (VPC).
On the ElastiCache dashboard page, choose Redis OSS cache and then choose Create Redis OSS cache.
-
Under Cluster settings, do the following:
Choose Configure and create a new cluster.
For Cluster mode, choose Disabled.
-
For Cluster info enter a value for Name.
-
(Optional) Enter a value for Description.
Under Location:
As soon as your cluster's status is available, you can grant Amazon EC2 access to it, connect to it, and begin using it. For more information, see Step 3: Authorize access to the cluster and Step 4: Connect to the cluster's node.
Important
Once your cluster is available, you're billed for each hour or partial hour that the cluster is active, even if you're not actively using it. To stop incurring charges for this cluster, you must delete it. See Deleting a cluster in ElastiCache.
Example
The following CLI code creates a Redis OSS (cluster mode disabled) cache cluster with no replicas.
For Linux, OS X, or Unix:
aws elasticache create-cache-cluster \ --cache-cluster-id
my-cluster
\ --cache-node-typecache.r4.large
\ --engineredis
\ --num-cache-nodes1
\ --snapshot-arnsarn:aws:s3:::my_bucket/snapshot.rdb
For Windows:
aws elasticache create-cache-cluster ^ --cache-cluster-id
my-cluster
^ --cache-node-typecache.r4.large
^ --engineredis
^ --num-cache-nodes1
^ --snapshot-arnsarn:aws:s3:::my_bucket/snapshot.rdb
To work with cluster mode enabled, see the following topics:
To use the console, see Creating a Redis OSS (cluster mode enabled) cluster (Console).
To use the Amazon CLI, see Creating a Redis OSS (cluster mode enabled) cluster (Amazon CLI).
Step 3: Authorize access to the cluster
This section assumes that you are familiar with launching and connecting to Amazon EC2 instances. For more information, see the Amazon EC2 Getting Started Guide.
All ElastiCache clusters are designed to be accessed from an Amazon EC2 instance. The most common scenario is to access an ElastiCache cluster from an Amazon EC2 instance in the same Amazon Virtual Private Cloud (Amazon VPC), which will be the case for this exercise.
By default, network access to your cluster is limited to the account that was used to create it. Before you can connect to a cluster from an EC2 instance, you must authorize the EC2 instance to access the cluster. The steps required depend upon whether you launched your cluster into EC2-VPC or EC2-Classic.
The most common use case is when an application deployed on an EC2 instance needs to connect to a cluster in the same VPC. The simplest way to manage access between EC2 instances and clusters in the same VPC is to do the following:
-
Create a VPC security group for your cluster. This security group can be used to restrict access to the cluster instances. For example, you can create a custom rule for this security group that allows TCP access using the port you assigned to the cluster when you created it and an IP address you will use to access the cluster.
The default port for Redis OSS clusters and replication groups is
6379
.Important
Amazon ElastiCache security groups are only applicable to clusters that are not running in an Amazon Virtual Private Cloud environment (VPC). If you are running in an Amazon Virtual Private Cloud, Security Groups is not available in the console navigation pane.
If you are running your ElastiCache nodes in an Amazon VPC, you control access to your clusters with Amazon VPC security groups, which are different from ElastiCache security groups. For more information about using ElastiCache in an Amazon VPC, see Amazon VPCs and ElastiCache security
-
Create a VPC security group for your EC2 instances (web and application servers). This security group can, if needed, allow access to the EC2 instance from the Internet via the VPC's routing table. For example, you can set rules on this security group to allow TCP access to the EC2 instance over port 22.
-
Create custom rules in the security group for your Cluster that allow connections from the security group you created for your EC2 instances. This would allow any member of the security group to access the clusters.
Note
If you are planning to use Local Zones, ensure that you have enabled them. When you create a subnet group in that local zone, your VPC is extended to that Local Zone and your VPC will treat the subnet as any subnet in any other Availability Zone. All relevant gateways and route tables will be automatically adjusted.
To create a rule in a VPC security group that allows connections from another security group
-
Sign in to the Amazon Management Console and open the Amazon VPC console at https://console.aws.amazon.com/vpc
. -
In the navigation pane, choose Security Groups.
-
Select or create a security group that you will use for your Cluster instances. Under Inbound Rules, select Edit Inbound Rules and then select Add Rule. This security group will allow access to members of another security group.
-
From Type choose Custom TCP Rule.
-
For Port Range, specify the port you used when you created your cluster.
The default port for Redis OSS clusters and replication groups is
6379
. -
In the Source box, start typing the ID of the security group. From the list select the security group you will use for your Amazon EC2 instances.
-
-
Choose Save when you finish.
Once you have enabled access, you are now ready to connect to the node, as discussed in the next section.
For information on accessing your ElastiCache cluster from a different Amazon VPC, a different Amazon Region, or even your corporate network, see the following:
Step 4: Connect to the cluster's node
Before you continue, complete Step 3: Authorize access to the cluster.
This section assumes that you've created an Amazon EC2 instance and can connect to it. For instructions on how to do this, see the Amazon EC2 Getting Started Guide.
An Amazon EC2 instance can connect to a cluster node only if you have authorized it to do so.
Find your node endpoints
When your cluster is in the available state and you've authorized access to it, you can log in to an Amazon EC2 instance and connect to the cluster. To do so, you must first determine the endpoint.
Finding a Redis OSS (Cluster Mode Disabled) Cluster's Endpoints (Console)
If a Redis OSS (cluster mode disabled) cluster has only one node, the node's endpoint is used for both reads and writes. If the cluster has multiple nodes, there are three types of endpoints; the primary endpoint, the reader endpoint and the node endpoints.
The primary endpoint is a DNS name that always resolves to the primary node in the cluster. The primary endpoint is immune to changes to your cluster, such as promoting a read replica to the primary role. For write activity, we recommend that your applications connect to the primary endpoint.
A reader endpoint will evenly split incoming connections to the endpoint between all read replicas in a ElastiCache (Redis OSS) cluster. Additional factors such as when the application creates the connections or how the application (re)-uses the connections will determine the traffic distribution. Reader endpoints keep up with cluster changes in real-time as replicas are added or removed. You can place your ElastiCache (Redis OSS) cluster’s multiple read replicas in different Amazon Availability Zones (AZ) to ensure high availability of reader endpoints.
Note
A reader endpoint is not a load balancer. It is a DNS record that will resolve to an IP address of one of the replica nodes in a round robin fashion.
For read activity, applications can also connect to any node in the cluster. Unlike the primary endpoint, node endpoints resolve to specific endpoints. If you make a change in your cluster, such as adding or deleting a replica, you must update the node endpoints in your application.
To find a Redis OSS (cluster mode disabled) cluster's endpoints
-
Sign in to the Amazon Web Services Management Console and open the ElastiCache console at https://console.amazonaws.cn/elasticache/
. -
From the navigation pane, choose Redis OSS caches.
The clusters screen will appear with a list that will include any existing Redis OSS serverless caches, Redis OSS (cluster mode disabled) and Redis OSS (cluster mode enabled) clusters. Choose the cluster you created in the Creating a Redis OSS (cluster mode disabled) cluster (Console) section.
-
To find the cluster's Primary and/or Reader endpoints, choose the cluster's name (not the radio button).
Primary and Reader endpoints for a Redis OSS (cluster mode disabled) cluster
If there is only one node in the cluster, there is no primary endpoint and you can continue at the next step.
-
If the Redis OSS (cluster mode disabled) cluster has replica nodes, you can find the cluster's replica node endpoints by choosing the cluster's name and then choosing the Nodes tab.
The nodes screen appears with each node in the cluster, primary and replicas, listed with its endpoint.
Node endpoints for a Redis OSS (cluster mode disabled) cluster
-
To copy an endpoint to your clipboard:
-
One endpoint at a time, find the endpoint you want to copy.
-
Choose the copy icon directly in front of the endpoint.
The endpoint is now copied to your clipboard. For information on using the endpoint to connect to a node, see Connecting to nodes.
-
A Redis OSS (cluster mode disabled) primary endpoint looks something like the following. There is a difference depending upon whether or not In-Transit encryption is enabled.
In-transit encryption not enabled
clusterName.xxxxxx
.nodeId
.regionAndAz
.cache.amazonaws.com:port
redis-01.7abc2d.0001.usw2.cache.amazonaws.com:6379
In-transit encryption enabled
master.clusterName
.xxxxxx
.regionAndAz
.cache.amazonaws.com:port
master.ncit.ameaqx.use1.cache.amazonaws.com:6379
To further explore how to find your endpoints, see the relevant topics for the engine and cluster type you're running.
Finding Endpoints for a Redis OSS (Cluster Mode Enabled) Cluster (Console)—You need the cluster's Configuration endpoint.
Now that you have the endpoint you need, you can log in to an EC2 instance and connect to the cluster or replication group. In the following example, you use the redis-cli utility to connect to a cluster. The latest version of redis-cli also supports SSL/TLS for connecting encryption/authentication enabled clusters.
The following example uses Amazon EC2 instances running Amazon Linux and Amazon Linux 2. For details on installing and compiling redis-cli with other Linux distributions, see the documentation for your specific operating system..
Note
This process covers testing a connection using redis-cli utility for unplanned use only. For a list of supported Redis OSS clients, see the
Redis OSS documentation
Connecting to a cluster mode disabled unencrypted-cluster
Run the following command to connect to the cluster and replace
primary-endpoint
andport number
with the endpoint of your cluster and your port number. (The default port for Redis OSS is 6379.)src/redis-cli -h
primary-endpoint
-pport number
The result in a Redis OSS command prompt looks similar to the following:
primary-endpoint
:port number
You can now run Redis OSS commands.
set x Hello OK get x "Hello"
Connecting to a cluster mode enabled unencrypted-cluster
Run the following command to connect to the cluster and replace
configuration-endpoint
andport number
with the endpoint of your cluster and your port number. (The default port for Redis OSS is 6379.)src/redis-cli -h
configuration-endpoint
-c -pport number
Note
In the preceding command, option -c enables cluster mode following -ASK and -MOVED redirections
. The result in a Redis OSS command prompt looks similar to the following:
configuration-endpoint
:port number
You can now run Redis OSS commands. Note that redirection occurs because you enabled it using the -c option. If redirection isn't enabled, the command returns the MOVED error. For more information on the MOVED error, see Redis OSS cluster specification
. set x Hi -> Redirected to slot [16287] located at 172.31.28.122:6379 OK set y Hello OK get y "Hello" set z Bye -> Redirected to slot [8157] located at 172.31.9.201:6379 OK get z "Bye" get x -> Redirected to slot [16287] located at 172.31.28.122:6379 "Hi"
Connecting to an Encryption/Authentication enabled cluster
By default, redis-cli uses an unencrypted TCP connection when connecting to Redis OSS. The option BUILD_TLS=yes
enables SSL/TLS at the time of redis-cli compilation
as shown in the preceding Download and set up redis-cli section. Enabling AUTH is optional.
However, you must enable encryption in-transit in order to enable AUTH. For more details on ElastiCache
encryption and authentication, see ElastiCache in-transit encryption (TLS).
Note
You can use the option --tls
with redis-cli to connect to both cluster mode enabled and disabled encrypted clusters. If a cluster has an AUTH token set, then you can use the option -a
to provide an AUTH password.
In the following examples, be sure to replace cluster-endpoint
and port number
with the endpoint of your cluster and your port number. (The default port for Redis OSS is 6379.)
Connect to cluster mode disabled encrypted clusters
The following example connects to an encryption and authentication enabled cluster:
src/redis-cli -h
cluster-endpoint
--tls -ayour-password
-pport number
The following example connects to a cluster that has only encryption enabled:
src/redis-cli -h
cluster-endpoint
--tls -pport number
Connect to cluster mode enabled encrypted clusters
The following example connects to an encryption and authentication enabled cluster:
src/redis-cli -c -h
cluster-endpoint
--tls -ayour-password
-pport number
The following example connects to a cluster that has only encryption enabled:
src/redis-cli -c -h
cluster-endpoint
--tls -pport number
After you connect to the cluster, you can run the Redis OSS commands as shown in the preceding examples for unencrypted clusters.
Redis-cli alternative
If the cluster isn't cluster mode enabled and you need to make a connection to the cluster for a short test
but without going through the redis-cli compilation, you can use telnet or openssl.
In the following example commands, be sure to replace cluster-endpoint
and port number
with the endpoint of your cluster and your port number. (The default port for Redis OSS is 6379.)
The following example connects to an encryption and/or authentication enabled cluster mode disabled cluster:
openssl s_client -connect
cluster-endpoint
:port number
If the cluster has a password set, first connect to the cluster. After connecting, authenticate the cluster using the following command, then press the Enter
key.
In the following example, replace your-password
with the password for your cluster.
Auth
your-password
The following example connects to a cluster mode disabled cluster that doesn't have encryption or authentication enabled:
telnet
cluster-endpoint
port number
In order to connect to the Redis OSS cluster from an EC2 Windows instance using the Redis OSS CLI, you must download the redis-cli package and use redis-cli.exe to connect to the Redis OSS cluster from an EC2 Windows instance.
In the following example, you use the redis-cli utility to connect to
a cluster that is not encryption enabled and running Redis OSS. For more information
about Redis OSS and available Redis OSS commands, see Redis OSS commands
To connect to a Redis OSS cluster that is not encryption-enabled using redis-cli
-
Connect to your Amazon EC2 instance using the connection utility of your choice. For instructions on how to connect to an Amazon EC2 instance, see the Amazon EC2 Getting Started Guide.
Copy and paste the link https://github.com/microsoftarchive/redis/releases/download/win-3.0.504/Redis-x64-3.0.504.zip
in an Internet browser to download the zip file for the Redis OSS client from the available release at GitHub https://github.com/microsoftarchive/redis/releases/tag/win-3.0.504 Extract the zip file to you desired folder/path.
Open the Command Prompt and change to the Redis OSS directory and run the command
c:\Redis>redis-cli -h
.Redis_Cluster_Endpoint
-p 6379For example:
c:\Redis>redis-cli -h cmd.xxxxxxx.ng.0001.usw2.cache.amazonaws.com -p 6379
-
Run Redis OSS commands.
You are now connected to the cluster and can run Redis OSS commands like the following.
set a "hello"
// Set key "a" with a string value and no expiration OKget a
// Get value for key "a" "hello"get b
// Get value for key "b" results in miss (nil)set b "Good-bye" EX 5
// Set key "b" with a string value and a 5 second expiration "Good-bye"get b
// Get value for key "b" "Good-bye" // wait >= 5 secondsget b
(nil) // key has expired, nothing returnedquit
// Exit from redis-cli