Data tiering - Amazon MemoryDB for Redis
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Data tiering

Clusters that use a node type from the r6gd family have their data tiered between memory and local SSD (solid state drives) storage. Data tiering provides a new price-performance option for Redis workloads by utilizing lower-cost solid state drives (SSDs) in each cluster node in addition to storing data in memory. Similar to other node types, the data written to r6gd nodes is durably stored in a multi-AZ transaction log. Data tiering is ideal for workloads that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate additional latency when accessing data on SSD.

On clusters with data tiering, MemoryDB monitors the last access time of every item it stores. When available memory (DRAM) is fully consumed, MemoryDB uses a least-recently used (LRU) algorithm to automatically move infrequently accessed items from memory to SSD. When data on SSD is subsequently accessed, MemoryDB automatically and asynchronously moves it back to memory before processing the request. If you have a workload that accesses only a subset of its data regularly, data tiering is an optimal way to scale your capacity cost-effectively.

Note that when using data tiering, keys themselves always remain in memory, while the LRU governs the placement of values on memory vs. disk. In general, we recommend that your key sizes are smaller than your value sizes when using data tiering.

Data tiering is designed to have minimal performance impact to application workloads. For example, assuming 500-byte String values, you can typically expect an additional 450 microseconds of latency for read requests to data stored on SSD compared to read requests to data in memory.

With the largest data tiering node size (db.r6gd.8xlarge), you can store up to ~500 TBs in a single 500-node cluster (250 TB when using 1 read replica). For Data tiering, MemoryDB reserves 19% of (DRAM) memory per node for non-data use. Data tiering is compatible with all Redis commands and data structures supported in MemoryDB. You don't need any client-side changes to use this feature.

Best practices

We recommend the following best practices:

  • Data tiering is ideal for workloads that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate additional latency when accessing data on SSD.

  • When using SSD capacity available on data-tiered nodes, we recommend that value size be larger than the key size. Value size cannot be greater than 128MB; else it will not be moved to disk. When items are moved between DRAM and SSD, keys will always remain in memory and only the values are moved to the SSD tier.

Limitations

Data tiering has the following limitations:

  • The node type you use must be from the r6gd family, which is available in the following regions: us-east-2, us-east-1, us-west-2, us-west-1, eu-west-1, eu-west-3, eu-central-1, ap-northeast-1, ap-southeast-1, ap-southeast-2, ap-south-1, ca-central-1 and sa-east-1.

  • You cannot restore a snapshot of an r6gd cluster into another cluster unless it also uses r6gd.

  • You cannot export a snapshot to Amazon S3 for data-tiering clusters.

  • Forkless save is not supported.

  • Scaling is not supported from a data tiering cluster (for example, a cluster using an r6gd node type) to a cluster that does not use data tiering (for example, a cluster using an r6g node type).

  • Data tiering only supports volatile-lru, allkeys-lru and noeviction maxmemory policies.

  • Items larger than 128 MiB are not moved to SSD.

Data tiering pricing

R6gd nodes have 5x more total capacity (memory + SSD) and can help you achieve over 60 percent storage cost savings when running at maximum utilization compared to R6g nodes (memory only). For more information, see MemoryDB pricing.

Monitoring

MemoryDB offers metrics designed specifically to monitor the performance clusters that use data tiering. To monitor the ratio of items in DRAM compared to SSD, you can use the CurrItems metric at Metrics for MemoryDB.. You can calculate the percentage as: (CurrItems with Dimension: Tier = Memory * 100) / (CurrItems with no dimension filter). When the percentage of items in memory decreases below 5 percent, we recommend that you consider Scaling MemoryDB clusters.

For more information, see Metrics for MemoryDB clusters that use data tiering at Metrics for MemoryDB.

Using data tiering

When creating a cluster, you use data tiering by selecting a node type from the r6gd family, such as db.r6gd.xlarge. Selecting that node type automatically enables data tiering.

For more information on creating a cluster, see Step 1: Create a cluster.

When creating a cluster using the Amazon CLI, you use data tiering by selecting a node type from the r6gd family, such as db.r6gd.xlarge and setting the --data-tiering parameter.

You cannot opt out of data tiering when selecting a node type from the r6gd family. If you set the --no-data-tiering parameter, the operation will fail.

For Linux, OS X, or Unix:

aws memorydb create-cluster \ --cluster-name my-cluster \ --node-type db.r6gd.xlarge \ --acl-name my-acl \ --subnet-group my-sg \ --data-tiering

For Windows:

aws memorydb create-cluster ^ --cluster-name my-cluster ^ --node-type db.r6gd.xlarge ^ --acl-name my-acl ^ --subnet-group my-sg --data-tiering

After running this operation, you will see a response similar to the following:

{ "Cluster": { "Name": "my-cluster", "Status": "creating", "NumberOfShards": 1, "AvailabilityMode": "MultiAZ", "ClusterEndpoint": { "Port": 6379 }, "NodeType": "db.r6gd.xlarge", "EngineVersion": "6.2", "EnginePatchVersion": "6.2.6", "ParameterGroupName": "default.memorydb-redis6", "ParameterGroupStatus": "in-sync", "SubnetGroupName": "my-sg", "TLSEnabled": true, "ARN": "arn:aws:memorydb:us-east-1:xxxxxxxxxxxxxx:cluster/my-cluster", "SnapshotRetentionLimit": 0, "MaintenanceWindow": "wed:03:00-wed:04:00", "SnapshotWindow": "04:30-05:30", "ACLName": "my-acl", "DataTiering":"true", "AutoMinorVersionUpgrade": true } }

Restoring data from a snapshot into clusters with data tiering enabled

You can restore a snapshot to a new cluster with data tiering enabled using the (Console), (Amazon CLI) or (MemoryDB API). When you create a cluster using node types in the r6gd family, data tiering is enabled.

Restoring data from a snapshot into clusters with data tiering enabled (console)

To restore a snapshot to a new cluster with data tiering enabled (console), follow the steps at Restoring from a snapshot (Console)

Note that to enable data-tiering, you need to select a node type from the r6gd family.

When creating a cluster using the Amazon CLI, data tiering is by default used by selecting a node type from the r6gd family, such as db.r6gd.xlarge and setting the --data-tiering parameter.

You cannot opt out of data tiering when selecting a node type from the r6gd family. If you set the --no-data-tiering parameter, the operation will fail.

For Linux, OS X, or Unix:

aws memorydb create-cluster \ --cluster-name my-cluster \ --node-type db.r6gd.xlarge \ --acl-name my-acl \ --subnet-group my-sg \ --data-tiering \ --snapshot-name my-snapshot

For Linux, OS X, or Unix:

aws memorydb create-cluster ^ --cluster-name my-cluster ^ --node-type db.r6gd.xlarge ^ --acl-name my-acl ^ --subnet-group my-sg ^ --data-tiering ^ --snapshot-name my-snapshot

After running this operation, you will see a response similar to the following:

{ "Cluster": { "Name": "my-cluster", "Status": "creating", "NumberOfShards": 1, "AvailabilityMode": "MultiAZ", "ClusterEndpoint": { "Port": 6379 }, "NodeType": "db.r6gd.xlarge", "EngineVersion": "6.2", "EnginePatchVersion": "6.2.6", "ParameterGroupName": "default.memorydb-redis6", "ParameterGroupStatus": "in-sync", "SubnetGroupName": "my-sg", "TLSEnabled": true, "ARN": "arn:aws:memorydb:us-east-1:xxxxxxxxxxxxxx:cluster/my-cluster", "SnapshotRetentionLimit": 0, "MaintenanceWindow": "wed:03:00-wed:04:00", "SnapshotWindow": "04:30-05:30", "ACLName": "my-acl", "DataTiering": "true" }