Configure SFTP connectors - Amazon Transfer Family
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Configure SFTP connectors

This topic describes how to create SFTP connectors, the security algorithms associated with them, how to store a secret to hold credentials, details about formatting the private key, and instructions for testing your connectors.

Create an SFTP connector

This procedure explains how to create SFTP connectors by using the Amazon Transfer Family console or Amazon CLI.

Console
To create an SFTP connector
  1. Open the Amazon Transfer Family console at https://console.amazonaws.cn/transfer/.

  2. In the left navigation pane, choose Connectors, then choose Create connector.

  3. Choose SFTP for the connector type to create an SFTP connector, and then choose Next.

    
                                        The Transfer Family console, showing the Create
                                            connector page, where you choose the
                                            connector type. SFTP is
                                            selected.
  4. In the Connector configuration section, provide the following information:

    • For the URL, enter the URL for a remote SFTP server. This URL must be formatted as sftp://partner-SFTP-server-url, for example sftp://AnyCompany.com.

      Note

      Optionally, you can provide a port number in your URL. The format is sftp://partner-SFTP-server-url:port-number. The default port number (when no port is specified) is port 22.

    • For the Access role, choose the Amazon Resource Name (ARN) of the Amazon Identity and Access Management (IAM) role to use.

      • Make sure that this role provides read and write access to the parent directory of the file location that's used in the StartFileTransfer request.

      • Make sure that this role provides permission for secretsmanager:GetSecretValue to access the secret.

        Note

        In the policy, you must specify the ARN for the secret. The ARN contains the secret name, but appends the name with six, random, alphanumeric characters. An ARN for a secret has the following format.

        arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/SecretName-6RandomCharacters
      • Make sure this role contains a trust relationship that allows the connector to access your resources when servicing your users' transfer requests. For details on establishing a trust relationship, see To establish a trust relationship.

      The following example grants the necessary permissions to access the DOC-EXAMPLE-BUCKET in Amazon S3, and the specified secret stored in Secrets Manager.

      { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowListingOfUserFolder", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Effect": "Allow", "Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET" ] }, { "Sid": "HomeDirObjectAccess", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:GetObjectVersion", "s3:GetObjectACL", "s3:PutObjectACL" ], "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*" }, { "Sid": "GetConnectorSecretValue", "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": "arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/SecretName-6RandomCharacters" } ] }
      Note

      For the access role, the example grants access to a single secret. However, you can use a wildcard character, which can save work if you want to reuse the same IAM role for multiple users and secrets. For example, the following resource statement grants permissions for all secrets that have names beginning with aws/transfer.

      "Resource": "arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/*"

      You can also store secrets containing your SFTP credentials in another Amazon Web Services account. For details on enabling cross-account secret access, see Permissions to Amazon Secrets Manager secrets for users in a different account.

    • (Optional) For the Logging role, choose the IAM role for the connector to use to push events to your CloudWatch logs. The following example policy lists the necessary permissions to log events for SFTP connectors.

      { "Version": "2012-10-17", "Statement": [{ "Sid": "SFTPConnectorPermissions", "Effect": "Allow", "Action": [ "logs:CreateLogStream", "logs:DescribeLogStreams", "logs:CreateLogGroup", "logs:PutLogEvents" ], "Resource": [ "arn:aws:logs:*:*:log-group:/aws/transfer/*" ] }] }
  5. In the SFTP Configuration section, provide the following information:

    • For Connector credentials, from the dropdown list, choose the name of a secret in Amazon Secrets Manager that contains the SFTP user's private key or password. You must create a secret and store it in a specific manner. For details, see Store a secret for use with an SFTP connector.

    • For Trusted host keys, paste in the public portion of the host key that is used to identify the external server. You can add more than one key, by choosing Add trusted host key to add an additional key. You can use the ssh-keyscan command against the SFTP server to retrieve the necessary key. For details about the format and type of trusted host keys that Transfer Family supports, see SFTPConnectorConfig.

  6. In the Cryptographic algorithm options section, choose a Security policy from the dropdown list in the Security Policy field. The security policy enables you to select the cryptographic algorithms that your connector supports. For details on the available security policies and algorithms, see Security policies for Amazon Transfer Family SFTP connectors.

  7. (Optional) In the Tags section, for Key and Value, enter one or more tags as key-value pairs.

  8. After you have confirmed all of your settings, choose Create connector to create the SFTP connector. If the connector is created successfully, a screen appears with a list of the assigned static IP addresses and a Test connection button. Use the button to test the configuration for your new connector.

    
                                        The connector creation screen that appears when an
                                            SFTP connector has been successfully created. It
                                            contains a button for testing the connection and a list
                                            of the service-managed static IP addresses of this
                                            connector.

The Connectors page appears, with the ID of your new SFTP connector added to the list. To view the details for your connectors, see View SFTP connector details.

CLI

You use the create-connector command to create a connector. To use this command to create an SFTP connector, you must provide the following information.

  • The URL for a remote SFTP server. This URL must be formatted as sftp://partner-SFTP-server-url, for example sftp://AnyCompany.com.

  • The access role. Choose the Amazon Resource Name (ARN) of the Amazon Identity and Access Management (IAM) role to use.

    • Make sure that this role provides read and write access to the parent directory of the file location that's used in the StartFileTransfer request.

    • Make sure that this role provides permission for secretsmanager:GetSecretValue to access the secret.

      Note

      In the policy, you must specify the ARN for the secret. The ARN contains the secret name, but appends the name with six, random, alphanumeric characters. An ARN for a secret has the following format.

      arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/SecretName-6RandomCharacters
    • Make sure this role contains a trust relationship that allows the connector to access your resources when servicing your users' transfer requests. For details on establishing a trust relationship, see To establish a trust relationship.

    The following example grants the necessary permissions to access the DOC-EXAMPLE-BUCKET in Amazon S3, and the specified secret stored in Secrets Manager.

    { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowListingOfUserFolder", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Effect": "Allow", "Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET" ] }, { "Sid": "HomeDirObjectAccess", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:GetObjectVersion", "s3:GetObjectACL", "s3:PutObjectACL" ], "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*" }, { "Sid": "GetConnectorSecretValue", "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": "arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/SecretName-6RandomCharacters" } ] }
    Note

    For the access role, the example grants access to a single secret. However, you can use a wildcard character, which can save work if you want to reuse the same IAM role for multiple users and secrets. For example, the following resource statement grants permissions for all secrets that have names beginning with aws/transfer.

    "Resource": "arn:aws-cn:secretsmanager:region:account-id:secret:aws/transfer/*"

    You can also store secrets containing your SFTP credentials in another Amazon Web Services account. For details on enabling cross-account secret access, see Permissions to Amazon Secrets Manager secrets for users in a different account.

  • (Optional) Choose the IAM role for the connector to use to push events to your CloudWatch logs. The following example policy lists the necessary permissions to log events for SFTP connectors.

    { "Version": "2012-10-17", "Statement": [{ "Sid": "SFTPConnectorPermissions", "Effect": "Allow", "Action": [ "logs:CreateLogStream", "logs:DescribeLogStreams", "logs:CreateLogGroup", "logs:PutLogEvents" ], "Resource": [ "arn:aws:logs:*:*:log-group:/aws/transfer/*" ] }] }
  • Provide the following SFTP configuration information.

    • The ARN of a secret in Amazon Secrets Manager that contains the SFTP user's private key or password.

    • The public portion of the host key that is used to identify the external server. You can provide multiple trusted host keys if you like.

    The easiest way to provide the SFTP information is to save it to a file. For example, copy the following example text to a file named testSFTPConfig.json.

    // Listing for testSFTPConfig.json { "UserSecretId": "arn:aws-cn::secretsmanager:us-east-2:123456789012:secret:aws/transfer/example-username-key", "TrustedHostKeys": [ "sftp.example.com ssh-rsa AAAAbbbb...EEEE=" ] }
  • Specify a security policy for your connector, entering the security policy name.

Note

The SecretId can be either the entire ARN or the name of the secret (example-username-key in the previous listing).

Then run the following command to create the connector.

aws transfer create-connector --url "sftp://partner-SFTP-server-url" \ --access-role your-IAM-role-for-bucket-access \ --logging-role arn:aws-cn:iam::your-account-id:role/service-role/AWSTransferLoggingAccess \ --sftp-config file:///path/to/testSFTPConfig.json --security-policy-name security-policy-name

Store a secret for use with an SFTP connector

You can use Secrets Manager to store user credentials for your SFTP connectors. When you create your secret, you must provide a username. Additionally, you can provide either a password, a private key, or both. For details, see Quotas for SFTP connectors.

Note

When you store secrets in Secrets Manager, your Amazon Web Services account incurs charges. For information about pricing, see Amazon Secrets Manager Pricing.

To store user credentials in Secrets Manager for an SFTP connector
  1. Sign in to the Amazon Web Services Management Console and open the Amazon Secrets Manager console at https://console.amazonaws.cn/secretsmanager/.

  2. In the left navigation pane, choose Secrets.

  3. On the Secrets page, choose Store a new secret.

  4. On the Choose secret type page, for Secret type, choose Other type of secret.

  5. In the Key/value pairs section, choose the Key/value tab.

    • Key – Enter Username.

    • value – Enter the name of the user that is authorized to connect to the partner' server.

  6. If you want to provide a password, choose Add row, and in the Key/value pairs section, choose the Key/value tab.

    Choose Add row, and in the Key/value pairs section, choose the Key/value tab.

    • Key – Enter Password.

    • value – Enter the password for the user.

  7. If you want to provide a private key, see Generate and format the SFTP connector private key, which describes how to enter private key data.

    Note

    The private key data that you enter must correspond to the public key that is stored for this user in the remote SFTP server.

  8. Choose Next.

  9. On the Configure secret page, enter a name and description for your secret. We recommend that you use a prefix of aws/transfer/ for the name. For example, you could name your secret aws/transfer/connector-1.

  10. Choose Next, and then accept the defaults on the Configure rotation page. Then choose Next.

  11. On the Review page, choose Store to create and store the secret.

Generate and format the SFTP connector private key

Complete details for generating a public/private key pair are described in Creating SSH keys on macOS, Linux, or Unix.

As an example, to generate a private key for use with SFTP connectors, the following sample command produces the correct type of key (replace key_name with the actual file name for your key pair):

ssh-keygen -t rsa -b 4096 -m PEM -f key_name -N ""
Note

When you create your key pair for use with SFTP connectors, do not use a passphrase. An empty passphrase is necessary for the SFTP configuration to function correctly.

This command creates an RSA key pair, with a key size of 4096 bits. The key is generated in the legacy PEM format, which is required by Transfer Family for use with the SFTP connector secret. The keys are saved in key_name (private key) and key_name.pub (public key) in the current directory: that is, the directory where you run the ssh-keygen command.

Note

Transfer Family does not support the OpenSSH format (-----BEGIN OPENSSH PRIVATE KEY-----) for the keys used for your SFTP connector. The key must be in legacy PEM format (-----BEGIN RSA PRIVATE KEY----- or -----BEGIN EC PRIVATE KEY----- ). You can use the ssh-keygen tool to convert your key, by supplying the -m PEM option when you run the command.

After you generate the key, you must make sure that the private key is formatted with embedded newline characters ("\n") in JSON format.

Use a command to convert your existing private key into the correct format—JSON format with embedded newline characters. Here we provide examples for jq and Powershell. You can use any tool or command that you'd like to convert the private key into JSON format with embedded newline characters.

jq command

This example uses the jq command, which is available for download from Download jq.

jq -sR . path-to-private-key-file

For example, if your private key file is located in ~/.ssh/my_private_key, the command is as follows.

jq -sR . ~/.ssh/my_private_key

This outputs the key in the correct format (with embedded newline characters) to standard output.

PowerShell

If you are using Windows, you can use PowerShell to convert the key to the correct format. The following Powershell command converts the private key to the correct format.

Get-Content -Raw path-to-private-key-file | ConvertTo-Json
To add private key data to the secret for use with SFTP connectors
  1. In the Secrets Manager console, when storing Other type of Secret, choose the Plaintext tab. The text should be empty, with only an opening and closing brace, {}.

  2. Paste in your username, private key data, and/or password using the following format. For your private key data, paste the output from the command that you ran in step 1.

    {"Username":"SFTP-USER","Password":"SFTP-USER-PASSWORD","PrivateKey":"PASTE-PRIVATE-KEY-DATA-HERE"}
    
                            The secret in plain text, showing the sample text.

    If you paste the private key data correctly, you should see the following upon selecting the Key/value tab. Notice that the private key data is displayed line-by-line, rather than as a continuous string of text.

    
                            The secret showing the details in the Key/value tab.
  3. Continue the procedure in Store a secret for use with an SFTP connector at step 8, and follow that procedure until the end.

Test an SFTP connector

After you create an SFTP connector, we recommend that you test it before you attempt to transfer any files using your new connector.

To test an SFTP connector
  1. Open the Amazon Transfer Family console at https://console.amazonaws.cn/transfer/.

  2. In the left navigation pane, choose Connectors, and select a connector.

  3. From the Actions menu, choose Test connection.

    
                            The Transfer Family console, showing an SFTP connector selected, and the
                                    Test connectionTest connection action
                                highlighted.

The system returns a message, indicating whether the test passes or fails. If the test fails, the system provides an error message based on the reason the test failed.


                    The  SFTP connector test connection panel, showing a successful
                        test.

                    The  SFTP connector test connection panel, showing a failed test: the
                        error message indicates that the access role for the connector is
                        incorrect.
Note

To use the API to test your connector, see the TestConnection API documentation.