Package software.amazon.awssdk.services.s3.internal.multipart

Class MultipartS3AsyncClient

java.lang.Object
software.amazon.awssdk.services.s3.DelegatingS3AsyncClient
software.amazon.awssdk.services.s3.internal.multipart.MultipartS3AsyncClient
All Implemented Interfaces:
AutoCloseable, AwsClient, SdkClient, S3AsyncClient, SdkAutoCloseable
public final class MultipartS3AsyncClient extends DelegatingS3AsyncClient
An S3AsyncClient that automatically converts PUT, COPY requests to their respective multipart call. CRC32 will be enabled for the PUT and COPY requests, unless the the checksum is specified or checksum validation is disabled. Note: GET is not yet supported.
See Also:

  • Method Details

    • putObject

      public CompletableFuture<PutObjectResponse> putObject(PutObjectRequest putObjectRequest, AsyncRequestBody requestBody)
      Description copied from class: DelegatingS3AsyncClient

      Adds an object to a bucket.

      • Amazon S3 never adds partial objects; if you receive a success response, Amazon S3 added the entire object to the bucket. You cannot use PutObject to only update a single piece of metadata for an existing object. You must put the entire object with updated metadata if you want to update some values.

      • If your bucket uses the bucket owner enforced setting for Object Ownership, ACLs are disabled and no longer affect permissions. All objects written to the bucket by any account will be owned by the bucket owner.

      • Directory buckets - For directory buckets, you must make requests for this API operation to the Zonal endpoint. These endpoints support virtual-hosted-style requests in the format https://bucket_name.s3express-az_id.region.amazonaws.com/key-name . Path-style requests are not supported. For more information, see Regional and Zonal endpoints in the Amazon S3 User Guide.

      Amazon S3 is a distributed system. If it receives multiple write requests for the same object simultaneously, it overwrites all but the last object written. However, Amazon S3 provides features that can modify this behavior:

      • S3 Object Lock - To prevent objects from being deleted or overwritten, you can use Amazon S3 Object Lock in the Amazon S3 User Guide.

        This functionality is not supported for directory buckets.

      • S3 Versioning - When you enable versioning for a bucket, if Amazon S3 receives multiple write requests for the same object simultaneously, it stores all versions of the objects. For each write request that is made to the same object, Amazon S3 automatically generates a unique version ID of that object being stored in Amazon S3. You can retrieve, replace, or delete any version of the object. For more information about versioning, see Adding Objects to Versioning-Enabled Buckets in the Amazon S3 User Guide. For information about returning the versioning state of a bucket, see GetBucketVersioning.

        This functionality is not supported for directory buckets.

      Permissions

      • General purpose bucket permissions - The following permissions are required in your policies when your PutObject request includes specific headers.

        • s3:PutObject - To successfully complete the PutObject request, you must always have the s3:PutObject permission on a bucket to add an object to it.

        • s3:PutObjectAcl - To successfully change the objects ACL of your PutObject request, you must have the s3:PutObjectAcl.

        • s3:PutObjectTagging - To successfully set the tag-set with your PutObject request, you must have the s3:PutObjectTagging.

      • Directory bucket permissions - To grant access to this API operation on a directory bucket, we recommend that you use the CreateSession API operation for session-based authorization. Specifically, you grant the s3express:CreateSession permission to the directory bucket in a bucket policy or an IAM identity-based policy. Then, you make the CreateSession API call on the bucket to obtain a session token. With the session token in your request header, you can make API requests to this operation. After the session token expires, you make another CreateSession API call to generate a new session token for use. Amazon Web Services CLI or SDKs create session and refresh the session token automatically to avoid service interruptions when a session expires. For more information about authorization, see CreateSession .

      Data integrity with Content-MD5

      • General purpose bucket - To ensure that data is not corrupted traversing the network, use the Content-MD5 header. When you use this header, Amazon S3 checks the object against the provided MD5 value and, if they do not match, Amazon S3 returns an error. Alternatively, when the object's ETag is its MD5 digest, you can calculate the MD5 while putting the object to Amazon S3 and compare the returned ETag to the calculated MD5 value.

      • Directory bucket - This functionality is not supported for directory buckets.

      HTTP Host header syntax

      Directory buckets - The HTTP Host header syntax is Bucket_name.s3express-az_id.region.amazonaws.com.

      For more information about related Amazon S3 APIs, see the following:

      Specified by:
      putObject in interface S3AsyncClient
      Overrides:
      putObject in class DelegatingS3AsyncClient
      Parameters:
      putObjectRequest -
      requestBody - Functional interface that can be implemented to produce the request content in a non-blocking manner. The size of the content is expected to be known up front. See AsyncRequestBody for specific details on implementing this interface as well as links to precanned implementations for common scenarios like uploading from a file. The service documentation for the request content is as follows '

      Object data.

      '
      Returns:
      A Java Future containing the result of the PutObject operation returned by the service.
      The CompletableFuture returned by this method can be completed exceptionally with the following exceptions. The exception returned is wrapped with CompletionException, so you need to invoke Throwable.getCause() to retrieve the underlying exception.
      • SdkException Base class for all exceptions that can be thrown by the SDK (both service and client). Can be used for catch all scenarios.
      • SdkClientException If any client side error occurs such as an IO related failure, failure to get credentials, etc.
      • S3Exception Base class for all service exceptions. Unknown exceptions will be thrown as an instance of this type.

    • copyObject

      public CompletableFuture<CopyObjectResponse> copyObject(CopyObjectRequest copyObjectRequest)
      Description copied from class: DelegatingS3AsyncClient

      Creates a copy of an object that is already stored in Amazon S3.

      You can store individual objects of up to 5 TB in Amazon S3. You create a copy of your object up to 5 GB in size in a single atomic action using this API. However, to copy an object greater than 5 GB, you must use the multipart upload Upload Part - Copy (UploadPartCopy) API. For more information, see Copy Object Using the REST Multipart Upload API.

      You can copy individual objects between general purpose buckets, between directory buckets, and between general purpose buckets and directory buckets.

      Directory buckets - For directory buckets, you must make requests for this API operation to the Zonal endpoint. These endpoints support virtual-hosted-style requests in the format https://bucket_name.s3express-az_id.region.amazonaws.com/key-name . Path-style requests are not supported. For more information, see Regional and Zonal endpoints in the Amazon S3 User Guide.

      Both the Region that you want to copy the object from and the Region that you want to copy the object to must be enabled for your account. For more information about how to enable a Region for your account, see Enable or disable a Region for standalone accounts in the Amazon Web Services Account Management Guide.

      Amazon S3 transfer acceleration does not support cross-Region copies. If you request a cross-Region copy using a transfer acceleration endpoint, you get a 400 Bad Request error. For more information, see Transfer Acceleration.

      Authentication and authorization

      All CopyObject requests must be authenticated and signed by using IAM credentials (access key ID and secret access key for the IAM identities). All headers with the x-amz- prefix, including x-amz-copy-source, must be signed. For more information, see REST Authentication.

      Directory buckets - You must use the IAM credentials to authenticate and authorize your access to the CopyObject API operation, instead of using the temporary security credentials through the CreateSession API operation.

      Amazon Web Services CLI or SDKs handles authentication and authorization on your behalf.

      Permissions

      You must have read access to the source object and write access to the destination bucket.

      • General purpose bucket permissions - You must have permissions in an IAM policy based on the source and destination bucket types in a CopyObject operation.

        • If the source object is in a general purpose bucket, you must have s3:GetObject permission to read the source object that is being copied.

        • If the destination bucket is a general purpose bucket, you must have s3:PutObject permission to write the object copy to the destination bucket.

      • Directory bucket permissions - You must have permissions in a bucket policy or an IAM identity-based policy based on the source and destination bucket types in a CopyObject operation.

        • If the source object that you want to copy is in a directory bucket, you must have the s3express:CreateSession permission in the Action element of a policy to read the object. By default, the session is in the ReadWrite mode. If you want to restrict the access, you can explicitly set the s3express:SessionMode condition key to ReadOnly on the copy source bucket.

        • If the copy destination is a directory bucket, you must have the s3express:CreateSession permission in the Action element of a policy to write the object to the destination. The s3express:SessionMode condition key can't be set to ReadOnly on the copy destination bucket.

        For example policies, see Example bucket policies for S3 Express One Zone and Amazon Web Services Identity and Access Management (IAM) identity-based policies for S3 Express One Zone in the Amazon S3 User Guide.

      Response and special errors

      When the request is an HTTP 1.1 request, the response is chunk encoded. When the request is not an HTTP 1.1 request, the response would not contain the Content-Length. You always need to read the entire response body to check if the copy succeeds. to keep the connection alive while we copy the data.

      • If the copy is successful, you receive a response with information about the copied object.

      • A copy request might return an error when Amazon S3 receives the copy request or while Amazon S3 is copying the files. A 200 OK response can contain either a success or an error.

        • If the error occurs before the copy action starts, you receive a standard Amazon S3 error.

        • If the error occurs during the copy operation, the error response is embedded in the 200 OK response. For example, in a cross-region copy, you may encounter throttling and receive a 200 OK response. For more information, see Resolve the Error 200 response when copying objects to Amazon S3. The 200 OK status code means the copy was accepted, but it doesn't mean the copy is complete. Another example is when you disconnect from Amazon S3 before the copy is complete, Amazon S3 might cancel the copy and you may receive a 200 OK response. You must stay connected to Amazon S3 until the entire response is successfully received and processed.

          If you call this API operation directly, make sure to design your application to parse the content of the response and handle it appropriately. If you use Amazon Web Services SDKs, SDKs handle this condition. The SDKs detect the embedded error and apply error handling per your configuration settings (including automatically retrying the request as appropriate). If the condition persists, the SDKs throw an exception (or, for the SDKs that don't use exceptions, they return an error).

      Charge

      The copy request charge is based on the storage class and Region that you specify for the destination object. The request can also result in a data retrieval charge for the source if the source storage class bills for data retrieval. If the copy source is in a different region, the data transfer is billed to the copy source account. For pricing information, see Amazon S3 pricing.

      HTTP Host header syntax

      Directory buckets - The HTTP Host header syntax is Bucket_name.s3express-az_id.region.amazonaws.com.

      The following operations are related to CopyObject:

      Specified by:
      copyObject in interface S3AsyncClient
      Overrides:
      copyObject in class DelegatingS3AsyncClient
      Parameters:
      copyObjectRequest -
      Returns:
      A Java Future containing the result of the CopyObject operation returned by the service.
      The CompletableFuture returned by this method can be completed exceptionally with the following exceptions. The exception returned is wrapped with CompletionException, so you need to invoke Throwable.getCause() to retrieve the underlying exception.
      • ObjectNotInActiveTierErrorException The source object of the COPY action is not in the active tier and is only stored in Amazon S3 Glacier.
      • SdkException Base class for all exceptions that can be thrown by the SDK (both service and client). Can be used for catch all scenarios.
      • SdkClientException If any client side error occurs such as an IO related failure, failure to get credentials, etc.
      • S3Exception Base class for all service exceptions. Unknown exceptions will be thrown as an instance of this type.

    • getObject

      public <ReturnT> CompletableFuture<ReturnT> getObject(GetObjectRequest getObjectRequest, AsyncResponseTransformer<GetObjectResponse,ReturnT> asyncResponseTransformer)
      Description copied from class: DelegatingS3AsyncClient

      Retrieves an object from Amazon S3.

      In the GetObject request, specify the full key name for the object.

      General purpose buckets - Both the virtual-hosted-style requests and the path-style requests are supported. For a virtual hosted-style request example, if you have the object photos/2006/February/sample.jpg, specify the object key name as /photos/2006/February/sample.jpg. For a path-style request example, if you have the object photos/2006/February/sample.jpg in the bucket named examplebucket, specify the object key name as /examplebucket/photos/2006/February/sample.jpg. For more information about request types, see HTTP Host Header Bucket Specification in the Amazon S3 User Guide.

      Directory buckets - Only virtual-hosted-style requests are supported. For a virtual hosted-style request example, if you have the object photos/2006/February/sample.jpg in the bucket named examplebucket--use1-az5--x-s3, specify the object key name as /photos/2006/February/sample.jpg. Also, when you make requests to this API operation, your requests are sent to the Zonal endpoint. These endpoints support virtual-hosted-style requests in the format https://bucket_name.s3express-az_id.region.amazonaws.com/key-name . Path-style requests are not supported. For more information, see Regional and Zonal endpoints in the Amazon S3 User Guide.

      Permissions

      • General purpose bucket permissions - You must have the required permissions in a policy. To use GetObject, you must have the READ access to the object (or version). If you grant READ access to the anonymous user, the GetObject operation returns the object without using an authorization header. For more information, see Specifying permissions in a policy in the Amazon S3 User Guide.

        If you include a versionId in your request header, you must have the s3:GetObjectVersion permission to access a specific version of an object. The s3:GetObject permission is not required in this scenario.

        If you request the current version of an object without a specific versionId in the request header, only the s3:GetObject permission is required. The s3:GetObjectVersion permission is not required in this scenario.

        If the object that you request doesn’t exist, the error that Amazon S3 returns depends on whether you also have the s3:ListBucket permission.

        • If you have the s3:ListBucket permission on the bucket, Amazon S3 returns an HTTP status code 404 Not Found error.

        • If you don’t have the s3:ListBucket permission, Amazon S3 returns an HTTP status code 403 Access Denied error.

      • Directory bucket permissions - To grant access to this API operation on a directory bucket, we recommend that you use the CreateSession API operation for session-based authorization. Specifically, you grant the s3express:CreateSession permission to the directory bucket in a bucket policy or an IAM identity-based policy. Then, you make the CreateSession API call on the bucket to obtain a session token. With the session token in your request header, you can make API requests to this operation. After the session token expires, you make another CreateSession API call to generate a new session token for use. Amazon Web Services CLI or SDKs create session and refresh the session token automatically to avoid service interruptions when a session expires. For more information about authorization, see CreateSession .

      Storage classes

      If the object you are retrieving is stored in the S3 Glacier Flexible Retrieval storage class, the S3 Glacier Deep Archive storage class, the S3 Intelligent-Tiering Archive Access tier, or the S3 Intelligent-Tiering Deep Archive Access tier, before you can retrieve the object you must first restore a copy using RestoreObject. Otherwise, this operation returns an InvalidObjectState error. For information about restoring archived objects, see Restoring Archived Objects in the Amazon S3 User Guide.

      Directory buckets - For directory buckets, only the S3 Express One Zone storage class is supported to store newly created objects. Unsupported storage class values won't write a destination object and will respond with the HTTP status code 400 Bad Request.

      Encryption

      Encryption request headers, like x-amz-server-side-encryption, should not be sent for the GetObject requests, if your object uses server-side encryption with Amazon S3 managed encryption keys (SSE-S3), server-side encryption with Key Management Service (KMS) keys (SSE-KMS), or dual-layer server-side encryption with Amazon Web Services KMS keys (DSSE-KMS). If you include the header in your GetObject requests for the object that uses these types of keys, you’ll get an HTTP 400 Bad Request error.

      Overriding response header values through the request

      There are times when you want to override certain response header values of a GetObject response. For example, you might override the Content-Disposition response header value through your GetObject request.

      You can override values for a set of response headers. These modified response header values are included only in a successful response, that is, when the HTTP status code 200 OK is returned. The headers you can override using the following query parameters in the request are a subset of the headers that Amazon S3 accepts when you create an object.

      The response headers that you can override for the GetObject response are Cache-Control, Content-Disposition, Content-Encoding, Content-Language, Content-Type, and Expires.

      To override values for a set of response headers in the GetObject response, you can use the following query parameters in the request.

      • response-cache-control

      • response-content-disposition

      • response-content-encoding

      • response-content-language

      • response-content-type

      • response-expires

      When you use these parameters, you must sign the request by using either an Authorization header or a presigned URL. These parameters cannot be used with an unsigned (anonymous) request.

      HTTP Host header syntax

      Directory buckets - The HTTP Host header syntax is Bucket_name.s3express-az_id.region.amazonaws.com.

      The following operations are related to GetObject:

      Specified by:
      getObject in interface S3AsyncClient
      Overrides:
      getObject in class DelegatingS3AsyncClient
      Parameters:
      getObjectRequest -
      asyncResponseTransformer - The response transformer for processing the streaming response in a non-blocking manner. See AsyncResponseTransformer for details on how this callback should be implemented and for links to precanned implementations for common scenarios like downloading to a file. The service documentation for the response content is as follows '

      Object data.

      '.
      Returns:
      A future to the transformed result of the AsyncResponseTransformer.
      The CompletableFuture returned by this method can be completed exceptionally with the following exceptions. The exception returned is wrapped with CompletionException, so you need to invoke Throwable.getCause() to retrieve the underlying exception.
      • NoSuchKeyException The specified key does not exist.
      • InvalidObjectStateException Object is archived and inaccessible until restored.

        If the object you are retrieving is stored in the S3 Glacier Flexible Retrieval storage class, the S3 Glacier Deep Archive storage class, the S3 Intelligent-Tiering Archive Access tier, or the S3 Intelligent-Tiering Deep Archive Access tier, before you can retrieve the object you must first restore a copy using RestoreObject. Otherwise, this operation returns an InvalidObjectState error. For information about restoring archived objects, see Restoring Archived Objects in the Amazon S3 User Guide.

      • SdkException Base class for all exceptions that can be thrown by the SDK (both service and client). Can be used for catch all scenarios.
      • SdkClientException If any client side error occurs such as an IO related failure, failure to get credentials, etc.
      • S3Exception Base class for all service exceptions. Unknown exceptions will be thrown as an instance of this type.

    • close

      public void close()
      Description copied from interface: SdkAutoCloseable
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface SdkAutoCloseable
      Overrides:
      close in class DelegatingS3AsyncClient

    • create

      public static MultipartS3AsyncClient create(S3AsyncClient client, MultipartConfiguration multipartConfiguration)