Troubleshooting Amazon Transfer Family
Use the following information to help you diagnose and fix common issues that you might encounter when working with Amazon Transfer Family.
For issues with IAM in Transfer Family, see Troubleshooting Amazon Transfer Family identity and access.
Topics
- Troubleshoot service-managed users
- Troubleshoot Amazon API Gateway issues
- Troubleshoot policies for encrypted Amazon S3 buckets
- Troubleshoot authentication issues
- Troubleshoot managed workflows issues
- Troubleshoot workflow decryption issues
- Troubleshoot Amazon EFS issues
- Troubleshoot testing your identity provider
- Troubleshoot adding trusted host keys for your SFTP connector
- Troubleshoot file upload issues
- Troubleshoot ResourceNotFound exception
- Troubleshoot SFTP connector issues
- Troubleshoot AS2 issues
Troubleshoot service-managed users
This section describes possible solutions for the following issues.
Topics
Troubleshoot Amazon EFS service-managed users
Description
You run the sftp
command and the prompt doesn't appear, and instead you
see the following message:
Couldn't canonicalize: Permission denied Need cwd
Cause
Your Amazon Identity and Access Management (IAM) user's role does not have permission to access Amazon Elastic File System (Amazon EFS).
Solution
Increase the policy permissions for your user's role. You can add an Amazon managed
policy, such as AmazonElasticFileSystemClientFullAccess
.
Troubleshoot public key body too long
Description
When you try to create a service-managed user, you receive the following error:
Failed to create user (1 validation error detected: 'sshPublicKeyBody' failed to satisfy constraint: Member must have length less than or equal to 2048)
Cause
You might be entering a PGP key for the public key body, and Amazon Transfer Family does not support PGP keys for service-managed users.
Solution
If the PGP key is RSA-based, you can convert it to PEM format. For example, Ubuntu provides a conversion tool here:
https://manpages.ubuntu.com/manpages/xenial/man1/openpgp2ssh.1.html
Troubleshoot failed to add SSH public key
Description
When you try to add a public key for a service-managed user, you receive the following error:
Failed to add SSH public key (Unsupported or invalid SSH public key format)
Cause
You might be attempting to import an SSH2-formatted public key, and Amazon Transfer Family does not support SSH2-formatted public keys for service-managed users.
Solution
You need to convert the key into OpenSSH format. This process is described in Converting an SSH2 public key to PEM format.
Troubleshoot Amazon API Gateway issues
This section describes possible solutions for the following API Gateway issues.
Too many authentication failures
Description
When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you get the following error:
Received disconnect from 3.15.127.197 port 22:2: Too many authentication failures Authentication failed. Couldn't read packet: Connection reset by peer
Cause
You might have entered an incorrect password for your user. Try again to enter the correct password.
If the password is correct, the issue might be caused by a role Amazon Resource Name (ARN) that is not valid. To confirm that this is the issue, test the identity provider for your server. If you see a response similar to the following, the role ARN is a placeholder only, as indicated by the role ID value of all zeros:
{ "Response": "{\"Role\": \"arn:aws-cn:iam::000000000000:role/MyUserS3AccessRole\",\"HomeDirectory\": \"/\"}", "StatusCode": 200, "Message": "", "Url": "https://
api-gateway-ID
.execute-api.us-east-1.amazonaws.com/prod/servers/transfer-server-ID
/users/myuser/config" }
Solution
Replace the placeholder role ARN with an actual role that has permission to access the server.
To update the role
-
Open the Amazon CloudFormation console at https://console.amazonaws.cn/cloudformation
. -
In the left navigation pane, choose Stacks.
-
In the Stacks list, choose your stack, and then choose the Parameters tab.
-
Choose Update. On the Update stack page, choose Use current template, and then choose Next.
-
Replace UserRoleArn with a role ARN that has sufficient permissions for accessing your Transfer Family server.
Note
To grant the necessary permissions, you can add the
AmazonAPIGatewayAdministrator
and theAmazonS3FullAccess
managed policies to your role. -
Choose Next, and then choose Next again. On the Review
stack
page, select I acknowledge that Amazon CloudFormation might create IAM resources, and then choose Update stack.
Connection closed
Description
When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you get the following error:
Connection closed
Cause
One possible cause for this issue is that your Amazon CloudWatch logging role does not have a trust relationship with Transfer Family.
Solution
Make sure that the logging role for the server has a trust relationship with Transfer Family. For more information, see To establish a trust relationship.
Troubleshoot policies for encrypted Amazon S3 buckets
Description
You have an encrypted Amazon S3 bucket that you are using as storage for your Transfer Family server.
If you try to upload a file to the server, you receive the error Couldn't close
file: Permission denied
.
And if you view the server logs, you see the following errors:
ERROR Message="Access denied" Operation=CLOSE Path=/bucket/user/test.txt BytesIn=13 ERROR Message="Access denied"
Cause
The policy for your IAM user does not have permission to access the encrypted bucket.
Solution
You must specify additional permissions in your policy to grant the required Amazon Key Management Service (Amazon KMS) permissions. For details, see Data protection and encryption.
Troubleshoot authentication issues
This section describes possible solutions for the following authentication issues.
Topics
Authentication failures—SSH/SFTP
Description
When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you receive a message similar to the following:
Received disconnect from 3.130.115.105 port 22:2: Too many authentication failures Authentication failed.
Note
If you are using an API Gateway and receive this error, see Too many authentication failures.
Cause
You have not added an RSA key pair for your user, so you must authenticate using a password instead.
Solution
When you run the sftp
command, specify the -o
PubkeyAuthentication=no
option. This option forces the system to request your
password. For example:
sftp -o PubkeyAuthentication=no
sftp-user
@server-id
.server.transfer.region-id
.amazonaws.com
Managed AD mismatched realms issue
Description
A user's realm and their group realm must match. They must both be in the default realm, or they must both be in the trusted realm.
Cause
If a user and their group do not match, the user cannot be authenticated by Transfer Family.
If you test the identity provider for the user, you receive the error No
associated access found for user's groups
.
Solution
Reference a group in the user's realm that matches the group realm (either default or trusted).
Miscellaneous authentication issues
Description
You receive an authentication error and none of the other troubleshooting works
Cause
You might have specified a target for a logical directory that contains a leading or trailing slash (/).
Solution
Update your logical directory target, to make sure it begins with a slash, and
does not contain a trailing slash. For example, /amzn-s3-demo-bucket/images
is
acceptable, but amzn-s3-demo-bucket/images
and /amzn-s3-demo-bucket/images/
are not.
Troubleshoot managed workflows issues
This section describes possible solutions for the following workflow issues.
Topics
Troubleshoot workflow-related errors using Amazon CloudWatch
Description
If you are having issues with your workflows, you can use Amazon CloudWatch to investigate the cause.
Cause
There can be several causes. Use Amazon CloudWatch Logs to investigate.
Solution
Transfer Family emits workflow execution status into CloudWatch Logs. The following types of workflow errors can appear in CloudWatch Logs:
-
"type": "StepErrored"
-
"type": "ExecutionErrored"
-
"type": "ExecutionThrottled"
-
"Service failure on starting workflow"
You can filter your workflow's execution logs using different filter and pattern syntax. For example, you can create a log filter in your CloudWatch logs to capture workflow execution logs that contain the ExecutionErrored message. For details, see Real-time processing of log data with subscriptions and Filter and pattern syntax in the Amazon CloudWatch Logs User Guide.
StepErrored
2021-10-29T12:57:26.272-05:00 {"type":"StepErrored","details":{"errorType":"BAD_REQUEST","errorMessage":"Cannot tag Efs file","stepType":"TAG","stepName":"successful_tag_step"}, "workflowId":"w-abcdef01234567890","executionId":"1234abcd-56ef-78gh-90ij-1234klmno567", "transferDetails":{"serverId":"s-1234567890abcdef0","username":"lhr","sessionId":"1234567890abcdef0"}
Here, StepErrored
indicates that a step within the workflow has generated
an error. In a single workflow, you can have multiple steps configured. This error tells
you in which step the error occurred and provides an error message. In this particular
example, the step was configured to tag a file; however, tagging a file in an Amazon EFS file
system is not supported, so the step generated an error.
ExecutionErrored
2021-10-29T12:57:26.618-05:00 {"type":"ExecutionErrored","details":{},"workflowId":"w-w-abcdef01234567890", "executionId":"1234abcd-56ef-78gh-90ij-1234klmno567","transferDetails":{"serverId":"s-1234567890abcdef0", "username":"lhr","sessionId":"1234567890abcdef0"}}
When a workflow is not able to execute any step, it generates an ExecutionErrored
message.
For example, if you have configured a single step in a given workflow, and if the step is not able to execute, the overall workflow fails.
Executionthrottled
Execution is throttled if a workflow is getting triggered at a rate that is faster
than the system can support. This log message indicates that you must slow down your
execution rate for workflows. If you are not able to scale down your workflow-execution
rate, contact Amazon Web Services Support at Contact
Amazon
Service failure on starting workflow
Anytime you remove a workflow from a server and replace it with a new one, or update server configuration (which impacts a workflow's execution role), you must wait approximately 10 minutes before executing the new workflow. The Transfer Family server caches the workflow details, and it takes 10 minutes for the server to refresh its cache.
Additionally, you must log out of any active SFTP sessions, and then log back in after the 10-minute waiting period to see the changes.
Troubleshoot workflow copy errors
Description
If you're executing a workflow that contains a step to copy the uploaded file, you could encounter the following error:
{ "type": "StepErrored", "details": { "errorType": "BAD_REQUEST", "errorMessage": "Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID:
request-ID
; S3 Extended Request ID:request-ID
Proxy: null)", "stepType": "COPY", "stepName": "copy-step-name
" }, "workflowId": "workflow-ID
", "executionId": "execution-ID
", "transferDetails": { "serverId": "server-ID
", "username": "user-name
", "sessionId": "session-ID
" } }
Cause
The source file is in an Amazon S3 bucket that is in a different Amazon Web Services Region than the destination bucket.
Solution
If you're executing a workflow that includes a copy step, make sure that the source and destination buckets are in the same Amazon Web Services Region.
Troubleshoot workflow decryption issues
This section describes possible solutions for the following issues with encrypted workflows.
Troubleshoot error for signed encryption file
Description
Your decrypt workflow fails and you receive the following error:
"Encrypted file with signed message unsupported"
Cause
Transfer Family does not currently support signing for encrypted files.
Solution
In your PGP client, if there is an option to sign the encrypted file, make sure to clear the selection, as Transfer Family does not currently support signing for encrypted files.
Troubleshoot error for a FIPS algorithm
Description
Your decrypt workflow fails, and the log message resembles the following:
{ "type": "StepErrored", "details": { "errorType": "BAD_REQUEST", "errorMessage": "File encryption algorithm not supported with FIPS mode enabled.", "stepType": "DECRYPT", "stepName": "
step-name
" }, "workflowId": "workflow-ID
", "executionId": "execution-ID
", "transferDetails": { "serverId": "server-ID
", "username": "user-name
", "sessionId": "session-ID
" } }
Cause
Your Transfer Family server has FIPS mode enabled and an associated Decrypt workflow step. When encrypting the files before uploading to your Transfer Family server, the encryption client might generate encrypted files that use non-FIPS approved symmetric encryption algorithms. In such a scenario, the workflow is unable to decrypt files. In the following example, GnuPG version 2.4.0 is using OCB (a non-FIPS block cipher mode) to encrypt files: this causes the workflow to fail.
Solution
You must edit the GPG key that you used to encrypt your files, and then re-encrypt them. The following procedure describes the steps you must take.
To edit your PGP keys
-
Identify the key that you must edit by running
gpg ‐‐list-keys
This returns a list of keys. Each key has details similar to the following:
pub ed25519 2022-07-07 [SC] wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY uid [ultimate] Mary Major <marymajor@example.com> sub cv25519 2022-07-07 [E]
-
Identify the key that you want to edit. In the example shown in the previous step, the ID is
wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
. -
Run
gpg ‐‐edit-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
.The system responds with details about the GnuPG program and the specified key.
-
At the
gpg>
prompt, entershowpref
. The following details are returned:[ultimate] (1). Mary Major <marymajor@example.com> Cipher: AES256, AES192, AES, 3DES AEAD: OCB Digest: SHA512, SHA384, SHA256, SHA224, SHA1 Compression: ZLIB, BZIP2, ZIP, Uncompressed Features: MDC, AEAD, Keyserver no-modify
Note that the preferred algorithms that are stored on the key are listed.
-
We want to edit the key to retain all algorithms except for OCB. Run the
setpref
command, specifying all the algorithms to retain:gpg> setpref AES256, AES192,AES,3DES,SHA512, SHA384, SHA256, SHA224, SHA1,ZLIB, BZIP2, ZIP, Uncompressed
This returns the following details:
Set preference list to: Cipher: AES256, AES192, AES, 3DES AEAD: Digest: SHA512, SHA384, SHA256, SHA224, SHA1 Compression: ZLIB, BZIP2, ZIP, Uncompressed Features: MDC, Keyserver no-modify Really update the preferences? (y/N)
-
Enter
y
to update, then enter your password when prompted to confirm the change. -
Save the changes.
gpg> save
Before re-running your decrypt workflow, you must re-encrypt your files, using the edited key.
Troubleshoot Amazon EFS issues
This section describes possible solutions for the following Amazon EFS issues.
Troubleshoot missing POSIX profile
Description
If you're using Amazon EFS storage for your server and you're using a custom identity provider, you must provide your Amazon Lambda function with a POSIX profile.
Cause
One possible cause is that the templates that we provide for creating an Amazon Lambda-backed Amazon API Gateway method do not currently contain POSIX information.
If you did provide POSIX information, the format that you used for providing the POSIX information might not be getting parsed correctly by Transfer Family.
Solution
Make sure that you are providing a JSON element to Transfer Family for the
PosixProfile
parameter.
For example, if you're using Python, you could add the following line where you parse
the PosixProfile
parameter:
if PosixProfile: response_data["PosixProfile"] = json.loads(PosixProfile)
Or, in JavaScript, you could add the following line, where the
and
uid-value
are integers, 0 or greater,
that represent the User ID (UID) and Group ID (GID) respectively:gid-value
PosixProfile: {"Uid":
uid-value
, "Gid":gid-value
},
These code examples send the PosixProfile
parameter to Transfer Family as a JSON
object, rather than as a string.
Also, within Amazon Secrets Manager, you must store the PosixProfile
parameter as
follows. Replace
and
your-uid
with your actual values for the
GID and UID.your-gid
{"Uid":
your-uid
, "Gid":your-gid
, "SecondaryGids": []}
Troubleshoot logical directories with Amazon EFS
Description
If the user's home directory does not exist, and they run an ls
command, the system
responds as follows:
sftp> ls remote readdir ("/"): No such file or directory
Cause
If your Transfer Family server uses Amazon EFS, the home directory for the user must be created with
read and write access before the user can work in their logical home directory. The user
cannot create this directory themselves, as they would lack permissions for
mkdir
on their logical home directory.
Solution
A user with administrative access to the parent directory needs to create the user's logical home directory.
Troubleshoot testing your identity provider
Description
If you test your identity provider using the console or the
TestIdentityProvider
API operation, the Response
field is
empty. For example:
{ "Response": "{}", "StatusCode": 200, "Message": "" }
Cause
The most likely cause is that the authentication failed because of an incorrect user name or password.
Solution
Make sure that you are using the correct credentials for your user, and make updates to the username or password, if necessary.
Troubleshoot adding trusted host keys for your SFTP connector
Description
When you are creating or editing an SFTP connector, and you are adding a trusted host key, you receive the following error:
Failed to edit connector details (Invalid host key format.)
Cause
If you paste in a correct public key, the issue might be that you included the
comment
portion of the key. Amazon Transfer Family does not currently accept the
comment portion of the key.
Solution
Delete the comment portion of the key, when you paste it into the text field. For example, assume your key looks similar to the following:
ssh-rsa AAAA...== marymajor@dev-dsk-marymajor-1d-c1234567.us-east-1.amazon.com
Remove the text that follows the ==
characters and only paste in the
portion of the key up to and including the ==
.
ssh-rsa AAAA...==
Troubleshoot file upload issues
This section describes possible solutions for the following file upload issues.
Troubleshoot Amazon S3 file upload errors
Description
When you are attempting to upload a file to Amazon S3 storage using Transfer Family, you receive the
following error message: Amazon Transfer does not support random access writes
to S3 objects.
Cause
When you're using Amazon S3 for your server's storage, Transfer Family does not support multiple connections for a single transfer.
Solution
If your Transfer Family server is using Amazon S3 for its storage, disable any options for your client software that mention using multiple connections for a single transfer.
Troubleshoot unreadable file names
Description
You see corrupted file names in some of your uploaded files. Users sometimes encounter problems with FTP and SFTP transfers that garble certain characters in file names, such as umlauts, accented letters, or certain scripts, such as Chinese or Arabic.
Cause
Although the FTP and SFTP protocols can allow for character encoding of files names to be negotiated by clients, Amazon S3 and Amazon EFS do not. Instead, they require UTF-8 character encoding. As a result, certain characters are not rendered correctly.
Solution
To solve this problem, review your client application for file name character encoding and make sure it is set to UTF-8.
Troubleshoot ResourceNotFound
exception
Description
You receive an error where the resource cannot be found. For example, if you run UpdateServer
, you might get the following error:
An error occurred (ResourceNotFoundException) when calling the UpdateServer operation: Unknown server
Cause
There are several reasons for receiving a ResourceNotFoundException
message. In most cases, the resource that
you specified in your API command does not exist. If you did specify an existing resource, then the most probable cause is that
your default region is different than the region for your resource. For example, if your default region is us-east-1, and your Transfer Family
server is in us-east-2, you will receive an Unknown resource
exception.
For details about setting a default region, see Quick configuration with aws configure
.
Solution
Add a region parameter to your API command to explicitly specify where to find a particular resource.
aws transfer -describe-server --server-id
server-id
--region us-east-2
Troubleshoot SFTP connector issues
This section describes possible solutions for the following SFTP connector issues.
Key negotiation fails
Description
You receive an error where the key exchange negotiation fails. For example:
Key exchange negotiation failed due to incompatible host key algorithms. Client offered: [ecdsa-sha2-nistp256, ecdsa-sha2-nistp384, ecdsa-sha2-nistp521, rsa-sha2-512, rsa-sha2-256] Server offered: [ssh-rsa]
Cause
This error is because there's no overlap between the host key algorithms supported by the server and those supported by the connector.
Solution
Ensure that the remote server supports at least one of the Client host key algorithms listed in the error message. For the list of supported algorithms, see Security policies for Amazon Transfer Family SFTP connectors.
Miscellaneous SFTP connector issues
Description
You receive an error after you run StartFileTransfer
, but do not know
the cause of the issue, and only the connector ID is returned after the API
call.
Cause
This error can have several causes. To troubleshoot, we recommend that you test your connector and search your CloudWatch logs.
Solution
-
Test your connector: See Test an SFTP connector. If the test fails, the system provides an error message based on the reason the test failed. That section describes how to test your connector from either the console or by using the TestConnection API command.
-
View CloudWatch logs for your connector: See Example log entries for SFTP connectors. This topic provides examples for SFTP connector log entries, and the naming convention to help you find the appropriate logs.
Troubleshoot AS2 issues
Error messages and troubleshooting tips for Applicability Statement 2 (AS2)-enabled servers are described here: AS2 error codes.