Other use cases for Amazon DataSync - Amazon DataSync
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Other use cases for Amazon DataSync

Learn about about some less-common use cases with Amazon DataSync that are not common to most users.

Transferring data in opposite directions

Transferring data in opposite directions allows for workflows where the active application moves between locations. Amazon DataSync doesn't support workflows where multiple active applications write to both locations at the same time. Use the steps in the following procedure to configure DataSync to transfer data in opposite directions.

To configure DataSync to data transfers in opposite directions
  1. Create a location and name it Location A.

  2. Create a second location and name it Location B.

  3. Create a task, name it Task A-B, and then configure Location A as the source location and Location B as the destination location.

  4. Create a second task, name it Task B-A, and then configure Location B as the source location and Location A as the destination location.

  5. To update Location B with data from Location A, run Task A-B.

    To update Location A with data from Location B, run Task B-A.

    Don't run these two tasks concurrently. DataSync can transfer files in opposite directions periodically. However, it doesn't support workflows where multiple active applications write to both Location A and Location B simultaneously.

Using multiple DataSync tasks to transfer to the same S3 bucket

In certain use cases, you might want different tasks to write to the same Amazon S3 bucket. In this case, you create different folders in the S3 bucket for each of the task. This approach prevents file name conflicts between the tasks, and also means that you can set different permissions for each of folders.

For example, you might have three tasks: task1, task2, and task3 write to an S3 bucket named MyBucket.

You create three folders in the bucket:

s3://MyBucket/task1

s3://MyBucket/task2

s3://MyBucket/task3

For each task, you choose the folder in MyBucket that corresponds to the task as the destination, and set different permissions for each of the three folders.

Allowing DataSync to access a restricted S3 bucket

In some cases, you might want to limit access to your Amazon S3 bucket. You can edit the S3 bucket policy so that DataSync can still access the bucket when you run a task.

To allow DataSync to access a restricted S3 bucket
  1. Copy the following sample policy.

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": [ "arn:aws-cn:s3:::bucket-name", "arn:aws-cn:s3:::bucket-name/*" ], "Condition": { "StringNotLike": { "aws:userid": [ "datasync-role-id:*", "your-role-id" ] } } } ] }
  2. In the sample policy, replace these values:

    • bucket-name: The name of the S3 bucket that you're restricting access to.

    • datasync-role-id: The ID of the IAM role that DataSync accesses the S3 bucket with. Run the following Amazon CLI command to get the IAM role ID:

      aws iam get-role --role-name datasync-iam-role-name

      In the output, look for the RoleId value:

      "RoleId": "ANPAJ2UCCR6DPCEXAMPLE"

    • your-role-id: The ID of the IAM role that you create the DataSync location for the S3 bucket with. Run the following command to get the IAM role ID:

      aws iam get-role --role-name your-iam-role-name

      In the output, look for the UserId value:

      "RoleId": "AIDACKCEVSQ6C2EXAMPLE"

  3. Add this policy to your S3 bucket policy. For more information, see how to edit a bucket policy in the Amazon S3 User Guide.

Once you've updated the S3 bucket policy, you must add additional IAM roles or users to the policy for those who need to access the S3 bucket.