Managing how Amazon DataSync transfers files, objects, and metadata
You can configure how Amazon DataSync handles your files, objects, and their associated metadata when copying between locations.
For example, with recurring transfers, you might want to overwrite files in your destination with changes in the source to keep the locations in sync. You can copy properties such as POSIX permissions for files and folders, tags associated with objects, and access control lists (ACLs).
Before you begin: The metadata that DataSync can preserve in a transfer depends on the storage systems involved and whether those systems use a similar metadata structure. Before configuring your task, make sure you understand how DataSync handles metadata and special files.
The following instructions describe how to configure what DataSync transfers when creating a task. You also can configure this when editing or starting a task.
Open the Amazon DataSync console at https://console.amazonaws.cn/datasync/
. In the left navigation pane, expand Data transfer, then choose Tasks, and then choose Create task.
-
Configure your task's source and destination locations.
For more information, see Where can I transfer my data with Amazon DataSync?.
-
For Transfer mode, choose one of the following options:
-
Transfer only data that has changed – DataSync copies only the data and metadata that differs between the source and destination location.
-
Transfer all data – DataSync copies everything in the source to the destination without comparing differences between the locations.
-
-
Select Keep deleted files if you want DataSync to maintain files or objects in the destination location that don't exist in the source.
If you don't choose this option and your task deletes objects from your Amazon S3 bucket, you might incur minimum storage duration charges for certain storage classes. For detailed information, see Storage class considerations with Amazon S3 transfers.
Warning
You can't deselect this option and enable Transfer all data. When you transfer all data, DataSync doesn't scan your destination location and doesn't know what to delete.
-
Select Overwrite files if you want DataSync to modify data in the destination location when the source data or metadata has changed.
If your task overwrites objects, you might incur additional charges for certain storage classes (for example, for retrieval or early deletion). For detailed information, see Storage class considerations with Amazon S3 transfers.
If you don't choose this option, the destination data isn't overwritten even if the source data differs.
-
Under Data transfer configuration, select how you want DataSync to copy metadata.
Important
The settings you see in the console depend on your task's source and destination locations. In some cases, you may have to expand Additional settings to see these options.
Before making your selections, make sure you understand how DataSync will handle metadata and special files for your scenario.
Setting Description Copy ownership
DataSync copies POSIX file and folder ownership, such as the group ID of the file's owners and the user ID of the file's owner.
Copy permissions
DataSync copies POSIX permissions for files and folders from the source to the destination.
Copy timestamps DataSync copies the timestamp metadata from the source to the destination.
Copy object tags DataSync preserves the tags associated with your objects when transferring between object storage systems.
Copy ownership, DACLs, and SACLs DataSync copies the following:
-
The object owner.
-
NTFS discretionary access lists (DACLs), which determine whether to grant access to an object.
-
NTFS system access control lists (SACLs), which are used by administrators to log attempts to access a secured object.
Note: SACLs are not copied if you use SMB version 1.0.
Copying DACLs and SACLs requires granting specific permissions to the Windows user that DataSync uses to access your location using SMB. For more information, see creating a location for SMB, FSx for Windows File Server, or FSx for ONTAP (depending on the type of location in your transfer).
Copy ownership and DACLs DataSync copies the following:
-
The object owner.
-
DACLs, which determine whether to grant access to an object.
DataSync won't copy SACLs when you choose this option.
Do not copy ownership or ACLs DataSync doesn't copy any ownership or permissions data. The objects that DataSync writes to your destination location are owned by the user whose credentials are provided for DataSync to access the destination. Destination object permissions are determined based on the permissions configured on the destination server.
-
You can configure these task settings by using the Options
parameter with any of the following operations: