使用 Amazon SDK for Java 在 Amazon S3 Glacier 中下载文件库清单 - Amazon S3 Glacier
Amazon Web Services 文档中描述的 Amazon Web Services 服务或功能可能因区域而异。要查看适用于中国区域的差异,请参阅 中国的 Amazon Web Services 服务入门 (PDF)

如果您不熟悉 Amazon Simple Storage Service (Amazon S3) 中的归档存储功能,建议您先详细了解 Amazon S3 中的 S3 Glacier 存储类、S3 Glacier 即时检索S3 Glacier 灵活检索S3 Glacier 深度归档。有关更多信息,请参阅 Amazon S3 用户指南中的 S3 Glacier 存储类和用于存档对象的存储类。

本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。

使用 Amazon SDK for Java 在 Amazon S3 Glacier 中下载文件库清单

以下是使用Amazon SDK for Java低级 API 检索文件库清单的步骤。该高级 API 不支持检索文件库库存。

  1. 创建 AmazonGlacierClient 类(客户端)的实例。

    您需要指定文件库所在的 Amazon 区域。您使用此客户端执行的所有操作都会应用到该 Amazon 区域。

  2. 通过执行 initiateJob 方法启动清单检索任务。

    通过在 initiateJob 对象中提供任务信息来运行 InitiateJobRequest

    注意

    请注意,如果文件库的清单操作尚未完成,则会返回错误。Amazon S3 Glacier (S3 Glacier) 每 24 小时会定期为每个文件库准备一份库存。

    作为响应,S3 Glacier 返回任务 ID。该响应位于一个 InitiateJobResult 类的实例中。

    InitiateJobRequest initJobRequest = new InitiateJobRequest() .withVaultName("*** provide vault name ***") .withJobParameters( new JobParameters() .withType("inventory-retrieval") .withSNSTopic("*** provide SNS topic ARN ****") ); InitiateJobResult initJobResult = client.initiateJob(initJobRequest); String jobId = initJobResult.getJobId();
  3. 等待任务完成。

    您必须等到任务输出已作好供您下载的准备。如果您在文件库中设置了通知配置,或者在启动任务时指定了 Amazon Simple Notification Service (Amazon SNS) 主题,则 S3 Glacier 会在完成任务后向该主题发送消息。

    此外,您还可以通过调用 describeJob 方法轮询 S3 Glacier 来确定任务完成状态。但是,使用 Amazon SNS 主题进行通知才是推荐的方法。以下部分给出的代码示例使用适用于 S3 Glacier 的 Amazon SNS 来发布消息。

  4. 通过执行 getJobOutput 方法下载任务输出(文件库清单数据)。

    您可以通过创建一个 GetJobOutputRequest 类的实例来提供您的账户 ID、任务 ID 和文件库名称。如果您不提供账户 ID,则系统会使用与您提供来对请求签名的证书相关联的账户 ID。有关更多信息,请参阅将 Amazon SDK for Java 与 Amazon S3 Glacier 结合使用

    S3 Glacier 返回的输出位于 GetJobOutputResult 对象中。

    GetJobOutputRequest jobOutputRequest = new GetJobOutputRequest() .withVaultName("*** provide vault name ***") .withJobId("*** provide job ID ***"); GetJobOutputResult jobOutputResult = client.getJobOutput(jobOutputRequest); // jobOutputResult.getBody(); provides the output stream.

注意

有关任务相关的底层 REST API 的信息,请参阅任务操作

示例:使用适用于 Java 的 Amazon SDK 检索文件库清单

以下 Java 代码示例会检索指定文件库的文件库清单。

该示例执行以下任务:

  • 创建 Amazon Simple Notification Service (Amazon SNS) 主题。

    完成任务后,S3 Glacier 会向此主题发送通知。

  • 创建 Amazon Simple Queue Service (Amazon SQS) 队列。

    该示例会向该队列附加策略,以使 Amazon SNS 主题能够向该队列发布消息。

  • 启动任务以下载指定的档案。

    在任务请求中,指定了创建的 Amazon SNS 主题,以便 S3 Glacier 可以在完成任务后向该主题发布通知。

  • 检查 Amazon SQS 队列是否有包含该任务 ID 的消息。

    如果有消息,则分析 JSON,并检查任务是否已成功完成。如果已成功完成,则下载档案。

  • 通过删除它创建的 Amazon SNS 主题和 Amazon SQS 队列清除相关数据。

import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.FileWriter; import java.io.IOException; import java.io.InputStreamReader; import java.util.HashMap; import java.util.List; import java.util.Map; import com.fasterxml.jackson.core.JsonFactory; import com.fasterxml.jackson.core.JsonParseException; import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.amazonaws.AmazonClientException; import com.amazonaws.auth.policy.Policy; import com.amazonaws.auth.policy.Principal; import com.amazonaws.auth.policy.Resource; import com.amazonaws.auth.policy.Statement; import com.amazonaws.auth.policy.Statement.Effect; import com.amazonaws.auth.policy.actions.SQSActions; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.model.GetJobOutputRequest; import com.amazonaws.services.glacier.model.GetJobOutputResult; import com.amazonaws.services.glacier.model.InitiateJobRequest; import com.amazonaws.services.glacier.model.InitiateJobResult; import com.amazonaws.services.glacier.model.JobParameters; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sns.model.CreateTopicRequest; import com.amazonaws.services.sns.model.CreateTopicResult; import com.amazonaws.services.sns.model.DeleteTopicRequest; import com.amazonaws.services.sns.model.SubscribeRequest; import com.amazonaws.services.sns.model.SubscribeResult; import com.amazonaws.services.sns.model.UnsubscribeRequest; import com.amazonaws.services.sqs.AmazonSQSClient; import com.amazonaws.services.sqs.model.CreateQueueRequest; import com.amazonaws.services.sqs.model.CreateQueueResult; import com.amazonaws.services.sqs.model.DeleteQueueRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesResult; import com.amazonaws.services.sqs.model.Message; import com.amazonaws.services.sqs.model.ReceiveMessageRequest; import com.amazonaws.services.sqs.model.SetQueueAttributesRequest; public class AmazonGlacierDownloadInventoryWithSQSPolling { public static String vaultName = "*** provide vault name ***"; public static String snsTopicName = "*** provide topic name ***"; public static String sqsQueueName = "*** provide queue name ***"; public static String sqsQueueARN; public static String sqsQueueURL; public static String snsTopicARN; public static String snsSubscriptionARN; public static String fileName = "*** provide file name ***"; public static String region = "*** region ***"; public static long sleepTime = 600; public static AmazonGlacierClient client; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); client = new AmazonGlacierClient(credentials); client.setEndpoint("https://glacier." + region + ".amazonaws.com"); sqsClient = new AmazonSQSClient(credentials); sqsClient.setEndpoint("https://sqs." + region + ".amazonaws.com"); snsClient = new AmazonSNSClient(credentials); snsClient.setEndpoint("https://sns." + region + ".amazonaws.com"); try { setupSQS(); setupSNS(); String jobId = initiateJobRequest(); System.out.println("Jobid = " + jobId); Boolean success = waitForJobToComplete(jobId, sqsQueueURL); if (!success) { throw new Exception("Job did not complete successfully."); } downloadJobOutput(jobId); cleanUp(); } catch (Exception e) { System.err.println("Inventory retrieval failed."); System.err.println(e); } } private static void setupSQS() { CreateQueueRequest request = new CreateQueueRequest() .withQueueName(sqsQueueName); CreateQueueResult result = sqsClient.createQueue(request); sqsQueueURL = result.getQueueUrl(); GetQueueAttributesRequest qRequest = new GetQueueAttributesRequest() .withQueueUrl(sqsQueueURL) .withAttributeNames("QueueArn"); GetQueueAttributesResult qResult = sqsClient.getQueueAttributes(qRequest); sqsQueueARN = qResult.getAttributes().get("QueueArn"); Policy sqsPolicy = new Policy().withStatements( new Statement(Effect.Allow) .withPrincipals(Principal.AllUsers) .withActions(SQSActions.SendMessage) .withResources(new Resource(sqsQueueARN))); Map<String, String> queueAttributes = new HashMap<String, String>(); queueAttributes.put("Policy", sqsPolicy.toJson()); sqsClient.setQueueAttributes(new SetQueueAttributesRequest(sqsQueueURL, queueAttributes)); } private static void setupSNS() { CreateTopicRequest request = new CreateTopicRequest() .withName(snsTopicName); CreateTopicResult result = snsClient.createTopic(request); snsTopicARN = result.getTopicArn(); SubscribeRequest request2 = new SubscribeRequest() .withTopicArn(snsTopicARN) .withEndpoint(sqsQueueARN) .withProtocol("sqs"); SubscribeResult result2 = snsClient.subscribe(request2); snsSubscriptionARN = result2.getSubscriptionArn(); } private static String initiateJobRequest() { JobParameters jobParameters = new JobParameters() .withType("inventory-retrieval") .withSNSTopic(snsTopicARN); InitiateJobRequest request = new InitiateJobRequest() .withVaultName(vaultName) .withJobParameters(jobParameters); InitiateJobResult response = client.initiateJob(request); return response.getJobId(); } private static Boolean waitForJobToComplete(String jobId, String sqsQueueUrl) throws InterruptedException, JsonParseException, IOException { Boolean messageFound = false; Boolean jobSuccessful = false; ObjectMapper mapper = new ObjectMapper(); JsonFactory factory = mapper.getFactory(); while (!messageFound) { List<Message> msgs = sqsClient.receiveMessage( new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(10)).getMessages(); if (msgs.size() > 0) { for (Message m : msgs) { JsonParser jpMessage = factory.createJsonParser(m.getBody()); JsonNode jobMessageNode = mapper.readTree(jpMessage); String jobMessage = jobMessageNode.get("Message").textValue(); JsonParser jpDesc = factory.createJsonParser(jobMessage); JsonNode jobDescNode = mapper.readTree(jpDesc); String retrievedJobId = jobDescNode.get("JobId").textValue(); String statusCode = jobDescNode.get("StatusCode").textValue(); if (retrievedJobId.equals(jobId)) { messageFound = true; if (statusCode.equals("Succeeded")) { jobSuccessful = true; } } } } else { Thread.sleep(sleepTime * 1000); } } return (messageFound && jobSuccessful); } private static void downloadJobOutput(String jobId) throws IOException { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() .withVaultName(vaultName) .withJobId(jobId); GetJobOutputResult getJobOutputResult = client.getJobOutput(getJobOutputRequest); FileWriter fstream = new FileWriter(fileName); BufferedWriter out = new BufferedWriter(fstream); BufferedReader in = new BufferedReader(new InputStreamReader(getJobOutputResult.getBody())); String inputLine; try { while ((inputLine = in.readLine()) != null) { out.write(inputLine); } }catch(IOException e) { throw new AmazonClientException("Unable to save archive", e); }finally{ try {in.close();} catch (Exception e) {} try {out.close();} catch (Exception e) {} } System.out.println("Retrieved inventory to " + fileName); } private static void cleanUp() { snsClient.unsubscribe(new UnsubscribeRequest(snsSubscriptionARN)); snsClient.deleteTopic(new DeleteTopicRequest(snsTopicARN)); sqsClient.deleteQueue(new DeleteQueueRequest(sqsQueueURL)); } }