Troubleshooting: File Gateway issues

Verify that the gateway is not attempting to use a read-only domain controller (RODC) to join the domain.

Verify that the gateway is configured to use the correct DNS servers.

For example, if you are trying to join an Amazon EC2 gateway instance to an Amazon-managed Active Directory, verify that the DHCP option set for your EC2 VPC specifies the Amazon-managed Active Directory DNS servers.

DNS servers that you configure through the VPC DHCP options set are provided to the all EC2 instances in the VPC. If you want to specify a DNS server for an individual gateway, you can do so using that gateway's EC2 local console.

For on-premises gateways, you specify a DNS server using the VM local console.

Verify gateway network connectivity by running the following commands from the command prompt in the gateway's local console. Replace the highlighted variables with the actual domain name and IP addresses from your deployment.

dig -d ExampleDomainName
ncport -d ExampleDomainControllerIPAddress -p 445
ncport -d ExampleDomainControllerIPAddress -p 389

Verify that your Active Directory service account has the requisite permissions. For more information, see Active Directory service account permission requirements.

Verify that the gateway joins the correct Organizational Unit (OU).

Joining a domain creates an Active Directory computer account in the default computers container (which is not an OU), using the gateway's Gateway ID as the account name (for example, SGW-1234ADE).

If your Active Directory environment has a designated OU for new computer objects, you must specify that OU when joining the domain.

If you encounter access denied errors when attempting to join the designated OU, check with your Active Directory domain administrator. The administrator may need to pre-stage the gateway's computer account before it can join the domain. For more information, see How can I troubleshoot issues with joining my Storage Gateway file gateway to a domain for Microsoft Active Directory authentication?.

Verify that your gateway's hostname is resolvable in DNS by running the following command from the command prompt in the gateway's local console. Replace the highlighted variable with the actual hostname for your gateway.

dig -d ExampleHostName -r A

If you configured a custom hostname for your gateway, you must manually add a DNS A-record that points to its IP address.

Verify that network latency between the gateway and the domain controller is reasonably low. The query to join a domain can time out if the gateway does not receive a response from the domain controller within 20 seconds.

If you join the gateway to the domain using the JoinDomain CLI command, you can can add the --timeout-in-seconds flag to increase the timeout to a maximum of 3,600 seconds.

Verify that the Active Directory user you are using to join the gateway to the domain has the privileges required to do so.

If the IndexFetch metric is greater than 0 when you run an ls command or browse directories, your File Gateway started without information on the contents of the directory affected and had to access FSx for Windows File Server. Subsequent efforts to list the contents of that directory should go faster.

If the IndexEviction metric is greater than 0, it means that your File Gateway has reached the limit of what it can manage in its cache at that time. In this case, your File Gateway has to free some storage space from the least recently accessed directory to list a new directory. If this occurs frequently and there is a performance impact, contact Amazon Web Services Support.

Discuss with Amazon Web Services Support the contents of the related Amazon FSx file system and recommendations to improve performance based on your use case.

If there was a recent reboot or software update, then check the IOWaitPercent metric. This metric shows the percentage of time that the CPU is idle when there is an outstanding disk I/O request. In some cases, this might be high (10 or greater) and might have risen after the server was rebooted or updated. In these cases, then your File Gateway might be bottlenecked by a slow root disk as it rebuilds the index cache to RAM. You can address this issue by using a faster physical disk for the root disk.

If the MemUsedBytes metric is at or nearly the same as the MemTotalBytes metric, then your File Gateway is running out of available RAM. Make sure that your File Gateway has at least the minimum required RAM. If it already does, consider adding more RAM to your File Gateway based on your workload and use case.

If the file share is SMB, the issue might also be due to the number of SMB clients connected to the file share. To see the number of clients connected at any given time, check the SMBV(1/2/3)Sessions metric. If there are many clients connected, you might need to add more RAM to your File Gateway.

If the CachePercentDirty metric is 80 or greater, your File Gateway is writing data faster to disk than it can upload the data to Amazon FSx for Windows File Server. Consider increasing the bandwidth for upload from your File Gateway, adding one or more cache disks, or slowing down client writes, or increase the throughput capacity for associated Amazon FSx for Windows File Server.

If the CachePercentDirty metric is low, check the IoWaitPercent metric. If IoWaitPercent is greater than 10, your File Gateway might be bottlenecked by the speed of the local cache disk. We recommend local solid state drive (SSD) disks for your cache, preferably NVM Express (NVMe). If such disks aren't available, try using multiple cache disks from separate physical disks for a performance improvement.

If the CachePercentDirty metric is 90 percent or greater, your File Gateway can't accept new writes to disk because there is not enough available space on the cache disk. To see how fast your File Gateway is uploading to FSx for Windows File Server, view the CloudBytesUploaded metric. Compare that metric with the WriteBytes metric, which shows how fast the client is writing files to your File Gateway. If the SMB client is writing to your File Gateway faster than it can upload to FSx for Windows File Server, add more cache disks to cover the size of the backup job at a minimum. Or, increase the upload bandwidth.

If a large file copy such as backup job fails but the CachePercentDirty metric is less than 80 percent, your File Gateway might be hitting a client-side session timeout. For SMB, you can increase this timeout using the PowerShell command Set-SmbClientConfiguration -SessionTimeout 300. Running this command sets the timeout to 300 seconds.