使用适用于 Java 的 SDK 执行和管理分段上传 - Amazon Simple Storage Service
Amazon Web Services 文档中描述的 Amazon Web Services 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅中国的 Amazon Web Services 服务入门

使用适用于 Java 的 SDK 执行和管理分段上传

使用 Amazon S3 on Outposts,您可以在 Amazon Outposts 资源上创建 S3 存储桶,并在本地为需要本地数据访问、本地数据处理和数据驻留的应用程序存储和检索对象。您可以通过 Amazon Web Services Management Console、Amazon Command Line Interface (Amazon CLI)、Amazon SDK 或 REST API 使用 S3 on Outposts。有关更多信息,请参阅 什么是 Amazon S3 on Outposts?

以下示例显示如何结合使用 S3 on Outposts 和 Amazon SDK for Java 来执行和管理分段上传。

在 S3 on Outposts 存储桶中执行对象的分段上传

以下 S3 on Outposts 示例使用适用于 Java 的 SDK 启动、上传和完成对象到存储桶的分段上传。要使用此示例,请将每个 user input placeholder 替换为您自己的信息。有关更多信息,请参阅使用分段上传上传对象

import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import java.util.ArrayList; import java.util.List; public class MultipartUploadCopy { public static void main(String[] args) { String accessPointArn = "*** Source access point ARN ***"; String sourceObjectKey = "*** Source object key ***"; String destObjectKey = "*** Target object key ***"; try { // This code expects that you have Amazon credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); // Initiate the multipart upload. InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(accessPointArn, destObjectKey); InitiateMultipartUploadResult initResult = s3Client.initiateMultipartUpload(initRequest); // Get the object size to track the end of the copy operation. GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest(accessPointArn, sourceObjectKey); ObjectMetadata metadataResult = s3Client.getObjectMetadata(metadataRequest); long objectSize = metadataResult.getContentLength(); // Copy the object using 5 MB parts. long partSize = 5 * 1024 * 1024; long bytePosition = 0; int partNum = 1; List<CopyPartResult> copyResponses = new ArrayList<CopyPartResult>(); while (bytePosition < objectSize) { // The last part might be smaller than partSize, so check to make sure // that lastByte isn't beyond the end of the object. long lastByte = Math.min(bytePosition + partSize - 1, objectSize - 1); // Copy this part. CopyPartRequest copyRequest = new CopyPartRequest() .withSourceBucketName(accessPointArn) .withSourceKey(sourceObjectKey) .withDestinationBucketName(accessPointArn) .withDestinationKey(destObjectKey) .withUploadId(initResult.getUploadId()) .withFirstByte(bytePosition) .withLastByte(lastByte) .withPartNumber(partNum++); copyResponses.add(s3Client.copyPart(copyRequest)); bytePosition += partSize; } // Complete the upload request to concatenate all uploaded parts and make the copied object available. CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest( accessPointArn, destObjectKey, initResult.getUploadId(), getETags(copyResponses)); s3Client.completeMultipartUpload(completeRequest); System.out.println("Multipart copy complete."); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } // This is a helper function to construct a list of ETags. private static List<PartETag> getETags(List<CopyPartResult> responses) { List<PartETag> etags = new ArrayList<PartETag>(); for (CopyPartResult response : responses) { etags.add(new PartETag(response.getPartNumber(), response.getETag())); } return etags; }

通过分段上传,在 S3 on Outposts 存储桶中复制大型对象

下面的 S3 on Outposts 示例使用适用于 Java 的 SDK 复制存储桶中的对象。要使用此示例,请将每个 user input placeholder 替换为您自己的信息。这个例子是通过 使用分段上传复制对象 改写的。

import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import java.util.ArrayList; import java.util.List; public class MultipartUploadCopy { public static void main(String[] args) { String accessPointArn = "*** Source access point ARN ***"; String sourceObjectKey = "*** Source object key ***"; String destObjectKey = "*** Target object key ***"; try { // This code expects that you have Amazon credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); // Initiate the multipart upload. InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(accessPointArn, destObjectKey); InitiateMultipartUploadResult initResult = s3Client.initiateMultipartUpload(initRequest); // Get the object size to track the end of the copy operation. GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest(accessPointArn, sourceObjectKey); ObjectMetadata metadataResult = s3Client.getObjectMetadata(metadataRequest); long objectSize = metadataResult.getContentLength(); // Copy the object using 5 MB parts. long partSize = 5 * 1024 * 1024; long bytePosition = 0; int partNum = 1; List<CopyPartResult> copyResponses = new ArrayList<CopyPartResult>(); while (bytePosition < objectSize) { // The last part might be smaller than partSize, so check to make sure // that lastByte isn't beyond the end of the object. long lastByte = Math.min(bytePosition + partSize - 1, objectSize - 1); // Copy this part. CopyPartRequest copyRequest = new CopyPartRequest() .withSourceBucketName(accessPointArn) .withSourceKey(sourceObjectKey) .withDestinationBucketName(accessPointArn) .withDestinationKey(destObjectKey) .withUploadId(initResult.getUploadId()) .withFirstByte(bytePosition) .withLastByte(lastByte) .withPartNumber(partNum++); copyResponses.add(s3Client.copyPart(copyRequest)); bytePosition += partSize; } // Complete the upload request to concatenate all uploaded parts and make the copied object available. CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest( accessPointArn, destObjectKey, initResult.getUploadId(), getETags(copyResponses)); s3Client.completeMultipartUpload(completeRequest); System.out.println("Multipart copy complete."); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } // This is a helper function to construct a list of ETags. private static List<PartETag> getETags(List<CopyPartResult> responses) { List<PartETag> etags = new ArrayList<PartETag>(); for (CopyPartResult response : responses) { etags.add(new PartETag(response.getPartNumber(), response.getETag())); } return etags; } }

在 S3 on Outposts 存储桶中列出对象的分段

以下 S3 on Outposts 示例使用适用于 Java 的 SDK 列出存储桶中对象的分段。要使用此示例,请将每个 user input placeholder 替换为您自己的信息。

import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import java.util.List; public class ListParts { public static void main(String[] args) { String accessPointArn = "*** access point ARN ***"; String keyName = "*** Key name ***"; String uploadId = "*** Upload ID ***"; try { // This code expects that you have Amazon credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); ListPartsRequest listPartsRequest = new ListPartsRequest(accessPointArn, keyName, uploadId); PartListing partListing = s3Client.listParts(listPartsRequest); List<PartSummary> partSummaries = partListing.getParts(); System.out.println(partSummaries.size() + " multipart upload parts"); for (PartSummary p : partSummaries) { System.out.println("Upload part: Part number = \"" + p.getPartNumber() + "\", ETag = " + p.getETag()); } } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } }

检索 S3 on Outposts 存储桶中正在进行的分段上传的列表

以下 S3 on Outposts 示例说明如何使用适用于 Java 的 SDK 从 Outposts 存储桶检索正在进行的分段上传的列表。要使用此示例,请将每个 user input placeholder 替换为您自己的信息。这是根据 Amazon S3 列出分段上传 示例改编的一个示例。

import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.ListMultipartUploadsRequest; import com.amazonaws.services.s3.model.MultipartUpload; import com.amazonaws.services.s3.model.MultipartUploadListing; import java.util.List; public class ListMultipartUploads { public static void main(String[] args) { String accessPointArn = "*** access point ARN ***"; try { // This code expects that you have Amazon credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); // Retrieve a list of all in-progress multipart uploads. ListMultipartUploadsRequest allMultipartUploadsRequest = new ListMultipartUploadsRequest(accessPointArn); MultipartUploadListing multipartUploadListing = s3Client.listMultipartUploads(allMultipartUploadsRequest); List<MultipartUpload> uploads = multipartUploadListing.getMultipartUploads(); // Display information about all in-progress multipart uploads. System.out.println(uploads.size() + " multipart upload(s) in progress."); for (MultipartUpload u : uploads) { System.out.println("Upload in progress: Key = \"" + u.getKey() + "\", id = " + u.getUploadId()); } } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } }