现在,您可以使用Amazon S3转移经理(开发人员预览版)
本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
使用Amazon S3对象
Amazon S3 对象表示一个文件或数据集合。每个对象都必须包含在一个存储桶中。
最佳实践
建议您对 存储桶启用 AbortIncompleteMultipartUploadAmazon S3 生命周期规则。
该规则指示 Amazon S3 中止在启动后没有在指定天数内完成的分段上传。当超过设置的时间限制时,Amazon S3 将中止上传,然后删除未完成的上传数据。
有关更多信息,请参阅 。使用版本控制的存储桶生命周期配置中的Amazon Simple Storage Service用户指南。
这些代码片段假设您了解基本资料,并且已配置默认值Amazon使用中的信息的凭据设置默认凭据和区域.
上传对象
构建 PutObjectRequest 并提供存储桶名称和密钥名称。然后使用 S3ClientputObject
使用的方法RequestBody其中包含对象内容和PutObjectRequest
对象。存储桶必须存在,否则服务将返回错误。
导入
import java.io.IOException; import java.nio.ByteBuffer; import java.util.Random; import software.amazon.awssdk.core.waiters.WaiterResponse; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.waiters.S3Waiter; import software.amazon.awssdk.services.s3.model.HeadBucketRequest; import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
代码
Region region = Region.US_WEST_2; s3 = S3Client.builder() .region(region) .build(); createBucket(s3, bucketName, region); PutObjectRequest objectRequest = PutObjectRequest.builder() .bucket(bucketName) .key(key) .build(); s3.putObject(objectRequest, RequestBody.fromByteBuffer(getRandomByteBuffer(10_000)));
请参阅 GitHub 上的完整示例
分段上传对象
使用 S3ClientcreateMultipartUpload
方法获取上传 ID。然后,使用 uploadPart
方法上传每个段。最后,使用 S3ClientcompleteMultipartUpload
要告诉的方法Amazon S3合并所有已上传的段并完成上传操作。
导入
import java.io.IOException; import java.nio.ByteBuffer; import java.util.Random; import software.amazon.awssdk.core.waiters.WaiterResponse; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.waiters.S3Waiter; import software.amazon.awssdk.services.s3.model.HeadBucketRequest; import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
代码
// First create a multipart upload and get the upload id CreateMultipartUploadRequest createMultipartUploadRequest = CreateMultipartUploadRequest.builder() .bucket(bucketName) .key(key) .build(); CreateMultipartUploadResponse response = s3.createMultipartUpload(createMultipartUploadRequest); String uploadId = response.uploadId(); System.out.println(uploadId); // Upload all the different parts of the object UploadPartRequest uploadPartRequest1 = UploadPartRequest.builder() .bucket(bucketName) .key(key) .uploadId(uploadId) .partNumber(1).build(); String etag1 = s3.uploadPart(uploadPartRequest1, RequestBody.fromByteBuffer(getRandomByteBuffer(5 * mB))).eTag(); CompletedPart part1 = CompletedPart.builder().partNumber(1).eTag(etag1).build(); UploadPartRequest uploadPartRequest2 = UploadPartRequest.builder().bucket(bucketName).key(key) .uploadId(uploadId) .partNumber(2).build(); String etag2 = s3.uploadPart(uploadPartRequest2, RequestBody.fromByteBuffer(getRandomByteBuffer(3 * mB))).eTag(); CompletedPart part2 = CompletedPart.builder().partNumber(2).eTag(etag2).build(); // Finally call completeMultipartUpload operation to tell S3 to merge all uploaded // parts and finish the multipart operation. CompletedMultipartUpload completedMultipartUpload = CompletedMultipartUpload.builder() .parts(part1, part2) .build(); CompleteMultipartUploadRequest completeMultipartUploadRequest = CompleteMultipartUploadRequest.builder() .bucket(bucketName) .key(key) .uploadId(uploadId) .multipartUpload(completedMultipartUpload) .build(); s3.completeMultipartUpload(completeMultipartUploadRequest);
请参阅 GitHub 上的完整示例
下载对象
构建 GetObjectRequest 并提供存储桶名称和密钥名称。使用 S3ClientgetObject
方法,将它传递GetObjectRequest
对象和ResponseTransformer
对象。ResponseTransformer
将创建一个将响应内容写入到指定的文件或流的响应处理程序。
以下示例指定要将对象内容写入到的文件名。
导入
import java.io.IOException; import java.nio.ByteBuffer; import java.util.Random; import software.amazon.awssdk.core.waiters.WaiterResponse; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.waiters.S3Waiter; import software.amazon.awssdk.services.s3.model.HeadBucketRequest; import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
代码
GetObjectRequest getObjectRequest = GetObjectRequest.builder() .bucket(bucketName) .key(key) .build(); s3.getObject(getObjectRequest);
请参阅 GitHub 上的完整示例
删除对象
构建 DeleteObjectRequest 并提供存储桶名称和密钥名称。使用 S3ClientdeleteObject
方法,并向其传递要删除的存储桶和对象的名称。指定的存储桶和对象键必须存在,否则服务将返回错误。
导入
import java.io.IOException; import java.nio.ByteBuffer; import java.util.Random; import software.amazon.awssdk.core.waiters.WaiterResponse; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; import software.amazon.awssdk.core.sync.RequestBody; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.PutObjectRequest; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.ListObjectsV2Response; import software.amazon.awssdk.services.s3.model.S3Object; import software.amazon.awssdk.services.s3.model.GetObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteObjectRequest; import software.amazon.awssdk.services.s3.model.DeleteBucketRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest; import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse; import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload; import software.amazon.awssdk.services.s3.model.CreateBucketRequest; import software.amazon.awssdk.services.s3.model.CompletedPart; import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration; import software.amazon.awssdk.services.s3.model.UploadPartRequest; import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest; import software.amazon.awssdk.services.s3.waiters.S3Waiter; import software.amazon.awssdk.services.s3.model.HeadBucketRequest; import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
代码
DeleteObjectRequest deleteObjectRequest = DeleteObjectRequest.builder() .bucket(bucketName) .key(key) .build(); s3.deleteObject(deleteObjectRequest);
请参阅 GitHub 上的完整示例
复制对象
构建一个 CopyObjectRequest,并提供一个要将此对象负责复制到的存储桶名称、一个 URL 编码的字符串值(请参阅 URlenCoder.code 方法)以及该对象的密钥名称。使用 S3ClientcopyObject
方法,然后通过CopyObjectRequest对象。指定的存储桶和对象键必须存在,否则服务将返回错误。
导入
import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.CopyObjectRequest; import software.amazon.awssdk.services.s3.model.CopyObjectResponse; import software.amazon.awssdk.services.s3.model.S3Exception; import java.io.UnsupportedEncodingException; import java.net.URLEncoder; import java.nio.charset.StandardCharsets;
代码
public static String copyBucketObject (S3Client s3, String fromBucket, String objectKey, String toBucket) { String encodedUrl = null; try { encodedUrl = URLEncoder.encode(fromBucket + "/" + objectKey, StandardCharsets.UTF_8.toString()); } catch (UnsupportedEncodingException e) { System.out.println("URL could not be encoded: " + e.getMessage()); } CopyObjectRequest copyReq = CopyObjectRequest.builder() .copySource(encodedUrl) .destinationBucket(toBucket) .destinationKey(objectKey) .build(); try { CopyObjectResponse copyRes = s3.copyObject(copyReq); return copyRes.copyObjectResult().toString(); } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); } return ""; }
请参阅 GitHub 上的完整示例
列出对象
构建 ListObjectsRequest 并提供存储桶名称。然后调用 S3ClientlistObjects
方法然后通过ListObjectsRequest对象。此方法返回 ListObjectsResponse,其中包含存储桶中的所有对象。您可以调用此对象的 contents 方法以获取对象的列表。您可以遍历这个列表来显示对象,如以下代码所示。
导入
import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.ListObjectsRequest; import software.amazon.awssdk.services.s3.model.ListObjectsResponse; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.S3Object; import java.util.List; import java.util.ListIterator;
代码
public static void listBucketObjects(S3Client s3, String bucketName ) { try { ListObjectsRequest listObjects = ListObjectsRequest .builder() .bucket(bucketName) .build(); ListObjectsResponse res = s3.listObjects(listObjects); List<S3Object> objects = res.contents(); for (ListIterator iterVals = objects.listIterator(); iterVals.hasNext(); ) { S3Object myValue = (S3Object) iterVals.next(); System.out.print("\n The name of the key is " + myValue.key()); System.out.print("\n The object is " + calKb(myValue.size()) + " KBs"); System.out.print("\n The owner is " + myValue.owner()); } } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); } } //convert bytes to kbs private static long calKb(Long val) { return val/1024; }
请参阅 GitHub 上的完整示例