Configuring how Amazon DataSync handles files, objects, and metadata - Amazon DataSync
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Configuring how Amazon DataSync handles files, objects, and metadata

You can configure how Amazon DataSync handles your files, objects, and their associated metadata when transferring between locations.

For example, with recurring transfers, you might want to overwrite files in your destination with changes in the source to keep the locations in sync. You can copy properties such as POSIX permissions for files and folders, tags associated with objects, and access control lists (ACLs).

Before you begin: The metadata that DataSync can preserve in a transfer depends on the storage systems involved and whether those systems use a similar metadata structure. Before configuring your task, make sure you understand how DataSync handles metadata and special files.

The following instructions describe how to configure what DataSync transfers when creating a task. You also can configure this when editing or starting a task.

  1. Open the Amazon DataSync console at https://console.amazonaws.cn/datasync/.

  2. In the left navigation pane, expand Data transfer, then choose Tasks, and then choose Create task.

  3. Configure your task's source and destination locations.

    For more information, see Where can I transfer my data with Amazon DataSync?

  4. For Transfer mode, choose one of the following options:

    • Transfer only data that has changed – DataSync copies only the data and metadata that differs between the source and destination location.

    • Transfer all data – DataSync copies everything in the source to the destination without comparing differences between the locations.

  5. Select Keep deleted files if you want DataSync to maintain files or objects in the destination location that don't exist in the source.

    If you don't choose this option and your task deletes objects from your Amazon S3 bucket, you might incur minimum storage duration charges for certain storage classes. For detailed information, see Storage class considerations with Amazon S3 transfers.

    Warning

    You can't deselect this option and enable Transfer all data. When you transfer all data, DataSync doesn't scan your destination location and doesn't know what to delete.

  6. Select Overwrite files if you want DataSync to modify data in the destination location when the source data or metadata has changed.

    If your task overwrites objects, you might incur additional charges for certain storage classes (for example, for retrieval or early deletion). For detailed information, see Storage class considerations with Amazon S3 transfers.

    If you don't choose this option, the destination data isn't overwritten even if the source data differs.

  7. Under Transfer options, select how you want DataSync to copy metadata.

    Important

    The settings you see in the console depend on your task's source and destination locations. In some cases, you may have to expand Additional settings to see these options.

    Setting Description

    Copy ownership

    DataSync copies POSIX file and folder ownership, such as the group ID of the file's owners and the user ID of the file's owner.

    Copy permissions

    DataSync copies POSIX permissions for files and folders from the source to the destination.
    Copy timestamps

    DataSync copies the timestamp metadata from the source to the destination.
    Copy object tags

    DataSync preserves the tags associated with your objects when transferring between object storage systems.
    Copy ownership, DACLs, and SACLs

    DataSync copies the following:

    • The object owner.

    • NTFS discretionary access lists (DACLs), which determine whether to grant access to an object.

    • NTFS system access control lists (SACLs), which are used by administrators to log attempts to access a secured object.

      Note: SACLs are not copied if you use SMB version 1.0.

      Copying DACLs and SACLs requires granting specific permissions to the Windows user that DataSync uses to access your location using SMB. For more information, see creating a location for SMB, FSx for Windows File Server, or FSx for ONTAP (depending on the type of location in your transfer).

    Copy ownership and DACLs

    DataSync copies the following:

    • The object owner.

    • DACLs, which determine whether to grant access to an object.

    DataSync won't copy SACLs when you choose this option.
    Do not copy ownership or ACLs

    DataSync doesn't copy any ownership or permissions data. The objects that DataSync writes to your destination location are owned by the user whose credentials are provided for DataSync to access the destination. Destination object permissions are determined based on the permissions configured on the destination server.

You can configure these task settings by using the Options parameter with any of the following operations: