Working with shadow copies - Amazon FSx for Windows File Server
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Working with shadow copies

A Microsoft Windows shadow copy is a snapshot of a Windows file system at a point in time. With shadow copies enabled, your users can easily view and restore individual files or folders from an earlier snapshot in Windows File Explorer. Doing this enables users to easily undo changes and compare file versions. Storage administrators using Amazon FSx can easily schedule shadow copies to be taken periodically using Windows PowerShell commands.

Shadow copies are stored alongside your file system's data, and therefore consume the file system's storage capacity. However, shadow copies consume storage capacity only for the changed portions of files. All shadow copies stored in your file system are included in backups of your file system.

Note

Shadow copies are not enabled on FSx for Windows File Server by default. To have shadow copies running on your file system, you must enable shadow copies and set up a shadow copy schedule on your file system. For more information, see Setting up shadow copies using default settings.

Warning

Shadow copies are not a substitute for backups. If you enable shadow copies, make sure that you continue performing regular backups.

For information about managing shadow copies, see Shadow copies.

Shadow copies configuration overview

You enable and schedule periodic shadow copies on your file system using Windows PowerShell commands defined by Amazon FSx. Shadow copy configuration contains three settings:

  • The maximum amount of storage that shadow copies can consume on your file system

  • (Optional) The maximum number of shadow copies that can be stored on your file system. Default value is 20.

  • (Optional) A schedule to take shadow copies at defined times and intervals, such as daily, weekly, and monthly

You can store up to 500 shadow copies per file system at any point in time; however, we recommend maintaining fewer than 64 shadow copies at any time to ensure availability and performance. When you reach this limit, the next shadow copy that you take replaces the oldest shadow copy. Similarly, when the maximum shadow copy storage amount is reached, one or more of the oldest shadow copies are deleted to make sufficient storage space for the next shadow copy.

For information about how to quickly enable and schedule periodic shadow copies by using default Amazon FSx settings, see Setting up shadow copies using default settings. For information about how to customize your shadow copy configuration, see Shadow copies.

Considerations for allocating shadow copy storage

A shadow copy is a block-level copy of file changes that were made since the last shadow copy. The entire file is not copied, only the changes. Therefore, previous versions of files typically don't take up as much storage space as the current file. The amount of volume space used for changes can vary according to your workload. When a file is modified, the storage space used by shadow copies depends on your workload. When you determine how much storage space to allocate for shadow copies, you should account for your workload's file system usage patterns.

When you enable shadow copies, you can specify the maximum amount of storage that shadow copies can consume on the file system. The default limit is 10 percent of your file system. We recommend that you increase the limit if your users frequently add or modify files. Setting the limit too small can result in the oldest shadow copies being deleted more often than users might expect.

You can set the shadow copy storage as unbounded (Set-FsxShadowStorage -Maxsize "UNBOUNDED"). However, an unbounded configuration can result in a large number of shadow copies consuming your file system storage. This could result in not having enough storage capacity for your workloads. If you set an unbounded storage, be sure to scale your storage capacity as the shadow copy limits are reached. For information about configuring your shadow copy storage to a specific size or as unbounded, see Setting shadow copy storage.

After you enable shadow copies, you can monitor the amount of storage space consumed by the shadow copies. For more information, see Viewing your shadow copy storage.

Considerations for configuring maximum shadow copy number

When you enable shadow copies, you can specify the maximum number of shadow copies stored on the file system. The default limit is 20, as recommended by Microsoft. Because Windows requires a high level of I/O performance to maintain shadow copies, we recommend using SSD storage and increasing throughput capacity up to a value as high as three times that of your expected workload. This helps to ensure that your file system has enough resources to avoid issues like the unwanted deletion of shadow copies.

You can set the maximum number of shadow copies up to 500. However, if you have a large number of shadow copies or shadow copies that occupy a large amount of storage (TB-scale) on a single file system, processes such as failover and failback may take longer than expect. This is due to the need for Windows to run consistency checks on the shadow copy storage. You may also experience higher latency of I/O operations due to the need for Windows to perform copy-on-write activity while maintaining the shadow copies. To minimize availability and performance impact from shadow copies, Microsoft recommends configuring the maximum number of shadow copies below 64.

File system recommendations for shadow copies

Following are file system recommendations for using shadow copies.

  • Make sure you provision sufficient performance capacity for your workload needs on your file system. Amazon FSx delivers the Shadow Copies feature as provided by Microsoft Windows Server. By design, Microsoft Windows uses a copy-on-write method for recording the changes since the most recent shadow copy point, and this copy-on-write activity can result in up to three I/O operations for every file write operation. If Windows is unable to keep up with the incoming rate of I/O operations per second, it can cause all shadow copies to be deleted because it can no longer maintain the shadow copies via copy-on-write. Therefore, it is important that you provision sufficient I/O performance capacity for your workload needs on your file system (both the throughput capacity dimension that determines the file server I/O performance, and the storage type and capacity that determine the storage I/O performance).

  • We generally recommend that you use file systems configured with SSD storage rather than HDD storage when you enable shadow copies, given that Windows consumes a higher I/O performance to maintain shadow copies, and given that HDD storage provides lower performance capacity for I/O operations.

  • Your file system should have at least 320 MB of free space, in addition to the maximum shadow copy storage amount configured (MaxSpace). For example, if you allocated 5 GB MaxSpace to shadow copies, your file system should always have at least 320 MB free space in addition to the 5 GB MaxSpace.

Warning

When configuring your shadow copy schedule, make sure that you don't schedule shadow copies when migrating data or when data deduplication jobs are scheduled to run. You should schedule shadow copies when you expect your file system to be idle. For information about configuring a custom shadow copy schedule, see Creating a custom shadow copy schedule.

Setting up shadow copies using default settings

You can quickly set up shadow copies on your file system by using the default settings available for shadow copy storage and schedule. The default shadow copy storage setting lets shadow copies consume a maximum of 10 percent of your file system. If you increase your file system's storage capacity (either as a percentage or an absolute value), the amount of the currently allocated shadow copy storage is not similarly increased.

The default schedule automatically takes shadow copies every Monday, Tuesday, Wednesday, Thursday, and Friday, at 7:00 AM and 12:00 PM UTC.

To set up the default level of shadow copy storage
  1. Connect to a Windows compute instance that has network connectivity with your file system.

  2. Log in to the Windows compute instance as a member of the file system administrators group. In Amazon Managed Microsoft AD, that group is Amazon Delegated FSx Administrators. In your self-managed Microsoft AD, that group is Domain Admins or the custom group that you specified for administration when you created your file system. For more information, see Connecting to Your Windows Instance in the Amazon EC2 User Guide for Windows Instances.

  3. Set the default amount of shadow storage using the following command. Replace FSxFileSystem-Remote-PowerShell-Endpoint with the Windows Remote PowerShell endpoint of file system that you want to administer. You can find the Windows Remote PowerShell endpoint in the Amazon FSx console, in the Network & Security section of the file system details screen, or in the response of the DescribeFileSystem API operation.

    PS C:\Users\delegateadmin> Invoke-Command -ComputerName FSxFileSystem-Remote-PowerShell-Endpoint -ConfigurationName FSxRemoteAdmin -scriptblock {Set-FsxShadowStorage -Default}

    The response looks like the following.

    FSx Shadow Storage Configuration AllocatedSpace UsedSpace MaxSpace MaxShadowCopyNumber -------------- --------- -------- ------------------- 0 0 10737418240 20
To create the default shadow copy schedule
  • Set the default shadow copy schedule by entering the following command.

    PS C:\Users\delegateadmin> Invoke-Command -ComputerName FSxFileSystem-Remote-PowerShell-Endpoint -ConfigurationName FSxRemoteAdmin -scriptblock {Set-FsxShadowCopySchedule -Default}

    The response displays the default schedule that is now set.

    FSx Shadow Copy Schedule Start Time Days of week WeeksInterval ---------- ------------ ------------- 2019-07-16T07:00:00+00:00 Monday,Tuesday,Wednesday,Thursday,Friday 1 2019-07-16T12:00:00+00:00 Monday,Tuesday,Wednesday,Thursday,Friday 1

To learn about additional options and creating a custom shadow copy schedule, see Creating a custom shadow copy schedule.

Restoring individual files and folders

After you configure shadow copies on your Amazon FSx file system, your users can quickly restore previous versions of individual files or folders. Doing this enables them to recover deleted or changed files stored on the shared file system. They do this in a self-service manner directly on their desktop without administrator assistance. This self-service approach increases productivity and reduces administrative workload.

Users restore files to previous versions using the familiar Windows File Explorer interface. To restore a file, you choose the file to restore, then choose Restore previous versions from the context (right-click) menu.


     
      Restore previous versions in Windows File Explore

Users can then view and restore a previous version from the Previous Versions list.


     Restore previous versions in Windows File Explorer

To learn about the complete set of custom PowerShell commands available for managing shadow copies on your FSx for Windows File Server shares, see Shadow copies.