Troubleshooting Amazon DataSync location and task issues - Amazon DataSync
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China.

Troubleshooting Amazon DataSync location and task issues

The following topics describe issues common to Amazon DataSync locations and tasks and how you can resolve them.

How do I configure DataSync to use a specific NFS or SMB version to mount my file share?

For locations that support Network File System (NFS) or Server Message Block (SMB), DataSync by default chooses the protocol version for you. You can also specify the version yourself by using the DataSync console or API.

Action to take (DataSync console)

When creating your NFS or SMB location, configure the protocol version that you want DataSync to use. For more information, see Creating an NFS location for Amazon DataSync or Creating an SMB location for Amazon DataSync).

Action to take (DataSync API)

When creating or updating your NFS or SMB location, specify the Version parameter. For example, see CreateLocationNfs or CreateLocationSmb.

The following example Amazon CLI command creates an NFS location that DataSync mounts using NFS version 4.0.

$ aws datasync create-location-nfs --server-hostname your-server-address \ --on-prem-config AgentArns=your-agent-arns \ --subdirectory nfs-export-path \ --mount-options Version="NFS4_0"

The following example Amazon CLI command creates an SMB location that DataSync mounts using SMB version 3.

$ aws datasync create-location-smb --server-hostname your-server-address \ --on-prem-config AgentArns=your-agent-arns \ --subdirectory smb-export-path \ --mount-options Version="SMB3"

Error: Invalid SyncOption value. Option: TransferMode,PreserveDeletedFiles, Value: ALL,REMOVE.

This error occurs when you're creating or editing your DataSync task and you select the Transfer all data option and deselect the Keep deleted files option. When you transfer all data, DataSync doesn't scan your destination location and doesn't know what to delete.

My task keeps failing with an EniNotFound error

This error occurs if you delete one of your task's network interfaces in your virtual private cloud (VPC). If your task is scheduled or queued, the task will fail if it's missing a network interface required to transfer your data.

Actions to take

You have the following options to work around this issue:

  • Manually restart the task. When you do this, DataSync will create any missing network interfaces it needs to run the task.

  • If you need to clean up resources in your VPC, make sure you don't delete network interfaces related to a DataSync task you're still using.

  • Consider not running your tasks automatically. This could include disabling task queueing or scheduling (through DataSync or custom automation).

My task failed with a DataSync currently does not support server-side NFSv4 ID mapping error

This error can occur if a file system involved in your transfer uses NFS version 4 ID mapping, a feature that DataSync doesn't support.

Action to take

You have a couple options to work around this issue:

  • Create a new DataSync location for the file system that uses NFS version 3.

  • Disable NFS version 4 ID mapping on the file system.

Retry the transfer. Either option should resolve the issue.

My task status is unavailable and indicates a mount error

DataSync will indicate that your task is unavailable if your agent can't mount an NFS location.

Action to take

First, make sure that the NFS server and export that you specified are both valid. If they aren't, delete the task, create a new one using the correct NFS server, and then export. For more information, see Creating an NFS location for Amazon DataSync.

If the NFS server and export are both valid, it generally indicates one of two things. Either a firewall is preventing the agent from mounting the NFS server, or the NFS server isn't configured to allow the agent to mount it.

Make sure that there's no firewall between the agent and the NFS server. Then make sure that the NFS server is configured to allow the agent to mount the export end specified in the task. For information about network and firewall requirements, see Amazon DataSync network requirements.

If you perform these actions and the agent still can't mount the NFS server and export, open a support channel with Amazon Support. For information about how to open a support channel, see Getting help with your agent from Amazon Web Services Support.

My task failed with a Cannot allocate memory error

When your DataSync task fails with a Cannot allocate memory error, it can mean a few different things.

Action to take

Try the following until you no longer see the issue:

My task failed with an input/output error

You can get an input/output error message if your storage system fails I/O requests from the DataSync agent. Common reasons for this include a server disk failure, changes to your firewall configuration, or a network router failure.

If the error involves an NFS server or Hadoop Distributed File System (HDFS) cluster, use the following steps to resolve the error.

Action to take (NFS)

First, check your NFS server's logs and metrics to determine if the problem started on the NFS server. If yes, resolve that issue.

Next, check that your network configuration hasn't changed. To check if the NFS server is configured correctly and that DataSync can access it, do the following:

  1. Set up another NFS client on the same network subnet as the agent.

  2. Mount your share on that client.

  3. Validate that the client can read and write to the share successfully.

Action to take (HDFS)

Make sure that your HDFS cluster allows the agent to communicate with the cluster's NameNode and DataNode ports. In most clusters, you can find the port numbers the cluster uses in the following configuration files.

  1. To find the NameNode port, look in the core-site.xml file under the fs.default or fs.default.name property (depending on the Hadoop distribution).

  2. To find the DataNode port, look in the hdfs-site.xml file under the dfs.datanode.address property.

My task execution has a launching status but nothing seems to be happening

Your task execution can become stuck in the Launching status when DataSync can't instruct the specified source agent to begin a task. This issue usually occurs because the agent either is powered off or has lost network connectivity.

Action to take

Make sure that the agent is connected and the status is ONLINE. If the status is OFFLINE, then the agent is not connected. For information about how to test network connectivity, see Testing your agent's connection to DataSync endpoints.

Next, make sure that your agent is powered on. If it isn't, power it on.

If the agent is powered on and the task is still stuck in Launching status, then a network connectivity problem between the agent and DataSync is the most likely issue. Check your network and firewall settings to make sure that the agent can connect to DataSync.

If you perform these actions and the issue isn't resolved, open a support channel with Amazon Web Services Support. For information about how to open a support channel, see Getting help with your agent from Amazon Web Services Support.

My task execution has been in the preparing status for a long time

The time DataSync spends in the Preparing status depends on the number of files in both the source and destination file systems, and the performance of these file systems. When a task starts, DataSync performs a recursive directory listing to discover all files and file metadata in the source and destination file system. These listings are used to identify differences and determine what to copy. This process usually takes between a few minutes to a few hours. For more information, see Starting your Amazon DataSync task.

Action to take

You shouldn't have to do anything. Continue to wait for the Preparing status to change to Transferring. If the status still doesn't change, contact Amazon Web Services Support.

My task failed with a permissions denied error

You can get a "permissions denied" error message if you configure your NFS server with root_squash or all_squash enabled and your files don't have all read access.

Action to take

To fix this issue, you can configure the NFS export with no_root_squash. Or you can make sure that the permissions for all of the files that you want to transfer allow read access for all users. Doing either enables the agent to read the files. For the agent to access directories, you must additionally enable all-execute access.

To make sure that the directory can be mounted, first connect to any computer that has the same network configuration as your agent. Then run the following CLI command.

mount -t nfs -o nfsvers=<your-nfs-server-version> <your-nfs-server-name>:<nfs-export-path-youspecified> <new-test-folder-on-your-computer>

If you perform these actions and the issue isn't resolved, contact Amazon Web Services Support.

How long does it take to verify a task I've run?

The time DataSync spends in the VERIFYING status depends on a number of factors. These are the number of files, the total size of all files in the source and destination file systems, and the performance of these file systems. By default, Verification mode is enabled in the options setting. The verification DataSync performs includes an SHA256 checksum on all file content and an exact comparison of all file metadata.

Action to take

You shouldn't have to do anything. Continue to wait for the VERIFYING status to complete. If the status still doesn't change, contact Amazon Web Services Support.

My task fails when transferring to an S3 bucket in another Amazon Web Services account

Unlike DataSync transfers between resources in the same Amazon Web Services account, copying data to an S3 bucket in a different Amazon Web Services account requires some extra steps.

  • If your DataSync task fails with an error related to S3 bucket permissions: When creating the task, make sure you're logged in to the Amazon Web Services Management Console using the same IAM user name (or role) which you specified in your destination S3 bucket's policy. (Note: This isn't the IAM role that gives DataSync permission to write to the S3 bucket.)

  • If you're also copying data to a bucket in another Amazon Web Services Region and get an S3 endpoint connection error: Create the DataSync task in the same Region as the destination S3 bucket.

My task execution times don't match up with the logs

The task execution start and end times that you see in the DataSync console may differ between timestamps you see elsewhere related to your transfer. This is because the console doesn’t take into account the time a task execution spends in the launching or queueing states.

For example, your Amazon CloudWatch logs can indicate that your task execution ended later than what's displayed in the DataSync console. You may notice a similar discrepancy in the following areas:

  • Logs for the file system or object storage system involved in your transfer

  • The last modified date on an Amazon S3 object that DataSync wrote to

  • Network traffic coming from the DataSync agent

  • Amazon EventBridge events