Amazon DataSync quotas - Amazon DataSync
Storage system, file, and object limitsDataSync task quotasDataSync Discovery quotasRequest a quota increase
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon DataSync quotas

Find out about quotas and limits when working with Amazon DataSync.

Storage system, file, and object limits

The following table describes the limits that DataSync has when working with storage systems, files, and objects.

Description Limit

Maximum total file path length

4,096 bytes

Maximum file path component (file name, directory, or subdirectory) length

255 bytes

Maximum length of Windows domain

253 characters

Maximum length of server hostname

255 characters

Maximum Amazon S3 object name length

1,024 UTF-8 characters

DataSync task quotas

The following table describes the quotas for DataSync tasks in a specific Amazon Web Services account and Amazon Web Services Region.

Resource Quota Can you increase the quota?

Maximum number of tasks you can create

100

Yes

Maximum number of files, objects, or directories that DataSync can work with per task execution between self-managed storage or another cloud and Amazon storage services

50 million

Important

Remember the following about this quota:

  • If you transfer Amazon S3 objects with prefixes, the prefixes are treated as directories and count towards the quota. For example, DataSync would considers3://bucket/foo/bar.txt as two directories (./ and ./foo/) and one object (bar.txt).

  • If your task is working with more than 20 million files, objects, or directories, make sure that you allocate a minimum of 64 GB of RAM to your DataSync agent. For more information, see agent requirements for DataSync transfers.

For more information, see How DataSync transfers files, objects, and directories.

Yes

Tip

Instead of requesting an increase, you can create tasks that focus on specific directories using include and exclude filters. For more information, see filtering the data transferred by DataSync.

Maximum number of files, objects, or directories that DataSync can work with per task execution between Amazon storage services

25 million

Important

If you transfer Amazon S3 objects with prefixes, the prefixes are treated as directories and count towards the quota. For example, DataSync would considers3://bucket/foo/bar.txt as two directories (./ and ./foo/) and one object (bar.txt).

For more information, see How DataSync transfers files, objects, and directories.

Yes

Tip

Instead of requesting an increase, you can create tasks that focus on specific directories using include and exclude filters. For more information, see filtering the data transferred by DataSync.

Maximum number of files, objects, or directories per task execution when running DataSync on an Amazon Snowcone device

200,000

No

Maximum throughput per task

10 Gbps (for transfers that use a DataSync agent)

No

5 Gbps (for transfers that don't use a DataSync agent)

No
Maximum number of characters you can include in a task filter

102,400 characters

Note

If you're using the DataSync console, this limit includes all the characters combined in your include and exclude patterns.

 No
Maximum number of queued executions for a task 50 No
Maximum number of days a task execution's history is retained 30 No

DataSync Discovery quotas

The following table describes the quotas for DataSync Discovery in a specific Amazon Web Services account and Amazon Web Services Region.

Resource Quota Can you increase the quota?

Maximum number of storage systems you can use with DataSync Discovery

10

No

Maximum number of storage systems a DataSync agent can access at a time

4

No

Request a quota increase

You can request an increase for some DataSync quotas. Increases aren't granted right away and might take a couple of days to take effect.

To request a quota increase

  1. Open the Service Quotas console at https://console.amazonaws.cn/servicequotas/.

  2. In the navigation pane, choose Amazon services and then choose Amazon DataSync.

  3. Choose the quota that you want to increase, then choose Request increase at account-level.

  4. Enter the total amount that you want the quota to be, then choose Request.

    If you need to increase a different quota, fill out a separate request.