Migrating to Amazon FSx for Lustre using Amazon DataSync - FSx for Lustre
Migrating files with Amazon DataSync
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Migrating to Amazon FSx for Lustre using Amazon DataSync

You can use Amazon DataSync to transfer data between FSx for Lustre file systems. DataSync is a data transfer service that simplifies, automates, and accelerates moving and replicating data between self-managed storage systems and Amazon storage services over the internet or Amazon Direct Connect. DataSync can transfer your file system data and metadata, such as ownership, timestamps, and access permissions.

How to migrate existing files to FSx for Lustre using Amazon DataSync

You can use DataSync with FSx for Lustre file systems to perform one-time data migrations, periodically ingest data for distributed workloads, and schedule replication for data protection and recovery. For information about specific transfer scenarios, see Where can I transfer my data with Amazon DataSync? in the Amazon DataSync User Guide.

Prerequisites

To migrate data into your FSx for Lustre setup, you need a server and network that meet the DataSync requirements. To learn more, see Setting up with Amazon DataSync in the Amazon DataSync User Guide.

  • You have created a destination FSx for Lustre file system. For more information, see Step 1: Create your FSx for Lustre file system.

  • The source and destination file systems are connected in the same virtual private cloud (VPC). The source file system can be located on-premises or in another Amazon VPC, Amazon Web Services account, or Amazon Web Services Region, but it must be in a network peered with that of the destination file system using Amazon VPC Peering, Transit Gateway, Amazon Direct Connect, or Amazon VPN. For more information, see What is VPC peering? in the Amazon VPC Peering Guide.

    Note

    DataSync can only transfer across Amazon Web Services accounts to or from FSx for Lustre if the other transfer location is Amazon S3.

Basic steps for migrating files using DataSync

Transferring files from a source to a destination using DataSync involves the following basic steps:

  1. Download and deploy an agent in your environment and activate it (not required if transferring between Amazon Web Services services).

  2. Create a source and destination location.

  3. Create a task.

  4. Run the task to transfer files from the source to the destination.

For more information, see the following topics in the Amazon DataSync User Guide: