Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Create an Amazon S3 bucket

Amazon Transfer Family accesses your Amazon S3 bucket to service your users' transfer requests, so you need to provide an Amazon S3 bucket as part of setting up your file transfer protocol-enabled server. You can use an existing bucket, or you can create a new one.

When you set up your users, you assign them each an IAM role. This role determines the level of access that they have to your Amazon S3 bucket.

For information on creating a new bucket, see How do I create an S3 bucket? in the Amazon Simple Storage Service User Guide.

Amazon S3 access points

Amazon Transfer Family supports Amazon S3 Access Points, a feature of Amazon S3 that allows you to easily manage granular access to shared data sets. You can use S3 Access Point aliases anywhere you use an S3 bucket name. You can create hundreds of access points in Amazon S3 for users who have different permissions to access shared data in an Amazon S3 bucket.

For example, you can use access points to allow three different teams to have access to the same shared dataset where one team can read data from S3, a second team can write data to S3, and the third team can read, write, and delete data from S3. To implement a granular access control as mentioned above, you can create an S3 access point that contains a policy that gives asymmetrical access to different teams. You can use S3 access points with your Transfer Family server to achieve a fine-grained access control, without creating a complex S3 bucket policy that spans hundreds of use cases. To learn more about how to use S3 access points with a Transfer Family server, refer to the Enhance data access control with Amazon Transfer Family and Amazon S3 blog post.

Amazon S3 HeadObject behavior

In Amazon S3, buckets and objects are the primary resources, and objects are stored in buckets. Amazon S3 can mimic a hierarchical file system, but can sometimes behave differently than a typical file system. For example, directories are not a first-class concept in Amazon S3 but instead are based on object keys. Amazon Transfer Family infers a directory path by splitting an object's key by the forward slash character (/), treating the last element as the file name, then grouping file names which have the same prefix together under the same path. Zero-byte objects are created to represent a folder's path when you create an empty directory using mkdir or by using the Amazon S3 console. The key for these objects ends in a trailing forward slash. These zero-byte objects are described in Organizing objects in the Amazon S3 console using folders in the Amazon S3 User Guide.

When you run an ls command, and some results are Amazon S3 zero-byte objects (these objects have keys that end in the forward slash character), Transfer Family issues a HeadObject request for each of these objects (see HeadObject in the Amazon Simple Storage Service API Reference for details). This can result in the following problems when using Amazon S3 as your storage with Transfer Family.

Grant ability to only write and list files

In some cases, customers want to only offer write access to their Amazon S3 objects. They want to provide access to write/upload and list objects in a bucket, but not read/download. This translates to the Amazon S3 permissions ListObjects and PutObject to perform ls and mkdir commands using file transfer clients. However, when Transfer Family needs to make a HeadObject call to either write or list files, it fails with an error of Access denied, because this call requires the GetObject permission.

In this case, you can grant access by adding a policy condition that adds the GetObject permission for any objects that end in a /. This prevents GetObject on files so they cannot be read, while allowing the user to list and traverse folders. The following example policy offers only write and list access to their Amazon S3 buckets (replace DOC-EXAMPLE-BUCKET with the actual name of your bucket).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListing",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET"
        },
        {
            "Sid": "AllowReadWrite",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": [
                "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*"
            ]
         },
      {
            "Sid": "DenyIfNotFolder",
            "Effect": "Deny",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "NotResource": [
                "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/"
            ]
         }
      ]
}

Large number of zero-byte objects causing latency issues

If your Amazon S3 buckets contain a large number of these zero-byte objects, Transfer Family issues a lot of HeadObject calls, which can result processing delays.

One possible solution to this problem is to delete all of your zero-byte objects. Note the following:

Another possible solution is to limit the number of objects visible through a policy condition to reduce the number of HeadObject calls. For this to be a workable solution, you need to accept that you might only be able to view a limited set of all of your sub-directories.