Vector search features and limits - Amazon MemoryDB for Redis
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Vector search features and limits

This feature is in preview release for MemoryDB for Redis and is subject to change.

Vector search availability

Vector search-enabled MemoryDB configuration is supported on R6g, R7g, and T4g node types and is available in the following Amazon Regions: US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland).

Parametric restrictions

The following table shows limits for various vector search items in the preview:

Item Maximum value
Number of dimensions in a vector 32768
Number of indexes that can be created 10
Number of fields in an index 50
Number of simultaneous index FT.CREATE backfill operations 1
FT.SEARCH and FT.AGGREGATE TIMEOUT clause (milliSeconds) 60000
Number of pipeline stages in FT.AGGREGATE command 32
Number of fields in FT.AGGREGATE LOAD clause 1024
Number of fields in FT.AGGREGATE GROUPBY clause 16
Number of fields in FT.AGGREGATE SORTBY clause 16
Number of parameters in FT.AGGREGATE PARAM clause 32
HNSW M parameter 512
HNSW EF_CONSTRUCTION parameter 4096
HNSW EF_RUNTIME parameter 4096

Scaling limits

Vector search for MemoryDB is currently limited to a single shard and horizontal scaling is not supported. Vector search supports vertical and replica scaling.

Operational restrictions

Index Persistence and Backfilling

The vector search preview persists the definition of indexes, but not their content. Thus any operational request or event that causes a node to start or restart requires that all indexes be rebuilt from their definition and source key data. The rebuild process is initiated automatically once all data has been restored — no user action is required to initiate this. The rebuild is performed as a backfill operation as soon as data is restored. This is functionally equivalent to the system automatically executing an FT.CREATE command for each defined index. Note that the node becomes available for application operations as soon as the data is restored but likely before index backfill has completed, meaning that backfill(s) will again become visible to applications, for example, search commands using backfilling indexes may be rejected. For more information on backfilling, see Vector search overview.

The completion of index backfill is not synchronized between a primary and a replica. This lack of synchronization can unexpectedly become visible to applications and thus it is recommended that applications verify backfill completion on primaries and all replicas before initiating search operations.

Snapshot import/export and Live Migration

The presence of search indexes in an RDB file limits the compatible transportability of that data. The format of the indexes defined by the preview edition is only understood by another preview edition cluster. Thus RDB files with search indexes can only be transferred between or utilized by preview enabled MemoryDB clusters.

However, RDB files that do not contain indexes are not restricted in this fashion. Thus data within a preview cluster can be exported to non-preview clusters by deleting the indexes prior to the export.

Memory consumption

The current implementation of vector indexes consumes approximately twice the amount of memory as will be consumed by the General Availability implementation.

Out of Memory during backfill

Similar to Redis write operations, an index backfill is subjected to out-of-memory limitations. If Redis memory is filled up while a backfill is in progress, all backfills are paused. If memory becomes available, the backfill process is resumed. It is also possible to delete and index when backfill is paused due to out of memory.

Transactions

The commands FT.CREATE, FT.DROPINDEX, FT.ALIASADD, FT.ALIASDEL, and FT.ALIASUPDATE cannot be executed in a transactional context, i.e., not within a MULTI/EXEC block or within a LUA or FUNCTION script.