Migrating to Amazon DocumentDB

• Using MongoDB as a Source for Amazon DMS

• Using Amazon DocumentDB as a Target for Amazon Database Migration Service

• Walkthrough: Migrating from MongoDB to Amazon DocumentDB

Architecture Details

• Name

  Choose a unique name for tracking this deployment.

   

• Version

  Record the version of MongoDB that your deployment is running. To find the version, connect to a replica set member with the mongo shell and run the db.version() operation.

   

• Type

  Record whether your deployment is a standalone mongo instance, a replica set, or a sharded cluster.

   

• Members

  Record the hostnames, addresses, and ports of each cluster, replica set, or standalone member.

   

  For a clustered deployment, you can find shard members by connecting to a mongo host with the mongo shell and running the sh.status() operation.

   

  For a replica set, you can obtain the members by connecting to a replica set member with the mongo shell and running the rs.status() operation.

   

• Oplog sizes

  For replica sets or sharded clusters, record the size of the oplog for each replica set member. To find a member’s oplog size, connect to the replica set member with the mongo shell and run the ps.printReplicationInfo() operation.

   

• Replica set member priorities

  For replica sets or sharded clusters, record the priority for each replica set member. To find the replica set member priorities, connect to a replica set member with the mongo shell and run the rs.conf() operation. The priority is shown as the value of the priority key.

   

• TLS/SSL usage

  Record whether Transport Layer Security (TLS)/Secure Sockets Layer (SSL) is used on each node for encryption in transit.

Operational Characteristics

• Database statistics

  For each collection, record the following information:

  • Name

  • Data size

  • Collection count

   

  To find the database statistics, connect to your database with the mongo shell and run the command db.runCommand({dbstats: 1}).

   

• Collection statistics

  For each collection, record the following information:

  • Namespace

  • Data size

  • Index count

  • Whether the collection is capped

   

• Index statistics

  For each collection, record the following index information:

  • Namespace

  • ID

  • Size

  • Keys

  • TTL

  • Sparse

  • Background

   

  To find the index information, connect to your database with the mongo shell and run the command db.collection.getIndexes().

   

• Opcounters

  This information helps you understand your current MongoDB workload patterns (read-heavy, write-heavy, or balanced). It also provides guidance on your initial Amazon DocumentDB instance selection.

   

  The following are the key pieces of information to collect over the monitoring period (in counts/sec):

  • Queries

  • Inserts

  • Updates

  • Deletes

   

  You can obtain this information by graphing the output of the db.serverStatus() command over time. You can also use the mongostat tool to obtain instantaneous values for these statistics. However, with this option you run the risk of planning your migration on usage periods other than your peak load.

   

• Network statistics

  This information helps you understand your current MongoDB workload patterns (read-heavy, write-heavy, or balanced). It also provides guidance on your initial Amazon DocumentDB instance selection.

   

  The following are the key pieces of information to collect over the monitoring period (in counts/sec):

  • Connections

  • Network bytes in

  • Network bytes out

   

  You can get this information by graphing the output of the db.serverStatus() command over time. You can also use the mongostat tool to obtain instantaneous values for these statistics. However, with this option you run the risk of planning your migration on usage periods other than your peak load.

• Availability

  Amazon DocumentDB provides high availability through the deployment of replica instances, which can be promoted to a primary instance in a process known as failover. By deploying replica instances to different Availability Zones, you can achieve higher levels of availability.

   

  The following table provides guidelines for Amazon DocumentDB deployment configurations to meet specific availability goals.

   

  Availability Goal	Total Instances	Replicas	Availability Zones
  99%	1	0	1
  99.9%	2	1	2
  99.99%	3	2	3

   

  Overall system reliability must consider all components, not just the database. For best practices and recommendations for meeting overall system reliability needs, see the Amazon Well-Architected Reliability Pillar Whitepaper.

   

• Performance

  Amazon DocumentDB instances allow you to read from and write to your cluster’s storage volume. Cluster instances come in a number of types, with varying amounts of memory and vCPU, which affect your cluster’s read and write performance. Using the information you gathered in the discovery phase, choose an instance type that can support your workload performance requirements. For a list of supported instance types, see Managing Instance Classes.

   

  When choosing an instance type for your Amazon DocumentDB cluster, consider the following aspects of your workload's performance requirements:

  • vCPUs—Architectures that require higher connection counts might benefit from instances with more vCPUS.

     

  • Memory—When possible, keeping your working dataset in memory provides maximum performance. A starting guideline is to reserve a third of your instance’s memory for the Amazon DocumentDB engine, leaving two-thirds for your working dataset.

     

  • Connections—The minimum optimal connection count is eight connections per Amazon DocumentDB instance vCPU. Although the Amazon DocumentDB instance connection limit is much higher, performance benefits of additional connections decline above eight connections per vCPU.

     

  • Network—Workloads with a large number of clients or connections should consider the aggregate network performance required for inserted and retrieved data. Bulk operations can make more efficient use of network resources.

     

  • Insert Performance—Single document inserts are generally the slowest way to insert data into Amazon DocumentDB. Bulk insert operations can be dramatically faster than single inserts.

     

  • Read Performance—Reads from working memory are always faster than reads returned from the storage volume. Therefore, optimizing your instance memory size to retain your working set in memory is ideal.

     

  In addition to serving reads from your primary instance, Amazon DocumentDB clusters are automatically configured as replica sets. You can then route read-only queries to read replicas by setting read preference in your MongoDB driver. You can scale read traffic by adding replicas, reducing the overall load on the primary instance.

   

  It is possible to deploy Amazon DocumentDB replicas of different instance types in the same cluster. An example use case might be to stand up a replica with a larger instance type to serve temporary analytics traffic. If you deploy a mixed set of instance types, be sure to configure the failover priority for each instance. This helps ensure that a failover event always promotes a replica of sufficient size to handle your write load.

   

• Recovery

  Amazon DocumentDB continuously backs up your data as it is written. It provides point-in-time recovery (PITR) capabilities within a configurable period of 1–35 days, known as the backup retention period. The default backup retention period is one day. Amazon DocumentDB also automatically creates daily snapshots of your storage volume, which are also retained for the configured backup retention period.

   

  If you want to retain snapshots beyond the backup retention period, you can also initiate manual snapshots at any time using the Amazon Web Services Management Console and Amazon Command Line Interface (Amazon CLI). For more information, see Backing Up and Restoring in Amazon DocumentDB.

   

  Consider the following as you plan your migration:

  • Choose a backup retention period of 1–35 days that meets your recovery point objective (RPO).

  • Decide if you require manual snapshots, and if so, at what interval.

• Verify that your chosen approach achieves your desired migration outcome.

• Verify that your instance type and read preference choices meet your application performance requirements.

• Verify your application's behavior during failover.

• Before running mongodump or starting an Amazon DMS migration task, you must disable the sharded cluster balancer and wait for any in-process migrations to complete. For more information, see Disable the Balancer in the MongoDB documentation.

• If you are using Amazon DMS to replicate data, run the cleanupOrphaned command on each shard before running the migration tasks. If you don't run this command, the tasks might fail because of duplicate document IDs. Note that this command might affect performance. For more information, see cleanupOrphaned in the MongoDB documentation.

• If you are using the mongodump tool to dump data, you should run one mongodump process per shard. The most time-efficient approach might require multiple migration servers to maximize your dump performance.

• If you are using Amazon Database Migration Service to replicate data, you must create a source endpoint for each shard. Also run at least one migration task for each shard that you are migrating. The most time-efficient approach might require multiple replication instances to maximize your migration performance.

• Network throughput

• Write throughput

• Read throughput

• Replica lag

See the following topics in the Amazon Database Migration Service User Guide:

• Using Amazon DocumentDB as a Target for Amazon Database Migration Service

• Walkthrough: Migrating from MongoDB to Amazon DocumentDB