Do I need an Amazon DataSync agent? - Amazon DataSync
Situations when you need a DataSync agentSituations when you don't need a DataSync agentUsing multiple DataSync agentsNext steps
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Do I need an Amazon DataSync agent?

To use Amazon DataSync, you might need an agent. An agent is a virtual machine (VM) appliance that you deploy in your storage environment for data transfers or storage discovery.

Whether you need an agent depends on several factors, including the type of storage you're transferring to or from, if you're transferring across Amazon Web Services accounts, and which Amazon Web Services Regions you're transferring between. Before reading further, check that DataSync supports the transfer you're interested in.

Once you know that DataSync supports your transfer, review the following information to help you understand if you need an agent.

Situations when you need a DataSync agent

Most situations that require a DataSync agent involve storage that's managed by you or another cloud provider.

  • Transferring to or from on-premises storage

  • Transferring to or from other cloud storage

  • Transferring to or from edge storage

  • Transferring between a commercial Amazon Web Services Region and Amazon GovCloud (US) Region

  • Transferring between Amazon storage services when at least one storage location is in an opt-in Region

Situations when you don't need a DataSync agent

The situations that don't require an agent apply whether you're transferring in the same Amazon Web Services Region or across Regions.

  • Transferring between Amazon storage services in the same Amazon Web Services account

  • Transferring between an S3 bucket and a different Amazon storage service across Amazon Web Services accounts

Using multiple DataSync agents

You can use more than one DataSync agent with your data transfers. While most transfers only need one agent, using multiple agents can speed up transfers of large datasets with millions of files.

In these situations, we recommend running transfer tasks in parallel. This approach spreads out the transfer workload across multiple tasks (each of which uses its own agent). It also helps reduce the time it takes DataSync to prepare and transfer your data.

Another option—especially if you have millions of small files—is using multiple agents with a transfer location. For example, you can connect up to four agents to your on-premises Network File System (NFS) file service. This option can speed up your transfer, though the time it takes DataSync to prepare the transfer doesn’t change.

With either approach, be mindful that these can increase the I/O operations on your storage system and affect your network bandwidth. For more information on using multiple agents for your DataSync transfers, see the Amazon Storage Blog.

If you're thinking of using multiple agents, remember the following:

  • Using multiple agents with a location doesn't provide high availability. All the agents associated with a location must be online before you can start your transfer task. If one of the agents is offline, you can't run your task.

  • If you're using a virtual private cloud (VPC) service endpoint to communicate with the DataSync service, all the agents must use the same endpoint and subnet.

  • With DataSync Discovery, you can only use one agent per storage system.

Next steps

  • If you need an agent, review the agent requirements to understand what makes sense for your storage environment and what you need DataSync for.

  • If you don't need an agent for your transfer, you can start configuring your transfer.