此页面仅适用于使用保管库的 S3 Glacier 服务的现有客户以及 2012 年以RESTAPI来的原始客户。
如果您正在寻找档案存储解决方案,我们建议您在亚马逊 S3、S3 Glacier 即时检索、S3 Glacier 灵活检索和 S3 Glacier Deep Archive Dee p Archive 中使用 S3 Glacier 存储类。要了解有关这些存储选项的更多信息,请参阅 Amazon S3 用户指南中的 S3 Glacier 存储类
本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
使用 Amazon SDK for Java 在 Amazon S3 Glacier 中下载档案
适用于 Java 的 Amazon SDK 提供的高级和低级 API 都提供了下载档案的方法。
使用Amazon SDK for Java高级 API 下载档案
该高级 API 的 ArchiveTransferManager
类提供了您可以用来下载档案的 download
方法。
重要
ArchiveTransferManager
类将创建 Amazon Simple Notification Service (Amazon SNS) 主题,以及该主题订阅的 Amazon Simple Queue Service (Amazon SQS) 队列。然后,它启动了档案检索任务,并对队列进行轮询以便找到可用档案。如果存档可用,则开始下载。有关检索时间的详细信息,请参阅 档案检索选项。
示例:使用Amazon SDK for Java的高级 API 下载档案
以下 Java 代码示例将从美国西部(俄勒冈州)区域 (us-west-2
) 中的文件库 (examplevault
) 中下载档案。
有关运行此示例的分步说明,请参阅“使用 Eclipse 运行 Amazon S3 Glacier 的 Java 示例”。您需要更新现有档案 ID 和已下载档案的本地文件保存路径旁显示的代码
import java.io.File; import java.io.IOException; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.transfer.ArchiveTransferManager; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sqs.AmazonSQSClient; public class ArchiveDownloadHighLevel { public static String vaultName = "examplevault"; public static String archiveId = "*** provide archive ID ***"; public static String downloadFilePath = "*** provide location to download archive ***"; public static AmazonGlacierClient glacierClient; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); glacierClient = new AmazonGlacierClient(credentials); sqsClient = new AmazonSQSClient(credentials); snsClient = new AmazonSNSClient(credentials); glacierClient.setEndpoint("glacier.us-west-2.amazonaws.com"); sqsClient.setEndpoint("sqs.us-west-2.amazonaws.com"); snsClient.setEndpoint("sns.us-west-2.amazonaws.com"); try { ArchiveTransferManager atm = new ArchiveTransferManager(glacierClient, sqsClient, snsClient); atm.download(vaultName, archiveId, new File(downloadFilePath)); System.out.println("Downloaded file to " + downloadFilePath); } catch (Exception e) { System.err.println(e); } } }
使用Amazon SDK for Java低级 API 下载档案
以下是使用Amazon SDK for Java低级 API 检索文件库清单的步骤。
-
创建
AmazonGlacierClient
类(客户端)的实例。您需要指定要从中下载档案的 Amazon 区域。您使用此客户端执行的所有操作都会应用到该 Amazon 区域。
-
通过执行
archive-retrieval
方法启动initiateJob
任务。您可以通过创建一个
InitiateJobRequest
类的实例提供任务信息,例如,您要下载的档案的档案 ID,以及您希望 Amazon S3 Glacier (S3 Glacier) 向其发布任务完成消息的可选 Amazon SNS 主题。作为响应,S3 Glacier 返回任务 ID。该响应位于一个InitiateJobResult
类的实例中。JobParameters jobParameters = new JobParameters() .withArchiveId("*** provide an archive id ***") .withDescription("archive retrieval") .withRetrievalByteRange("*** provide a retrieval range***") // optional .withType("archive-retrieval"); InitiateJobResult initiateJobResult = client.initiateJob(new InitiateJobRequest() .withJobParameters(jobParameters) .withVaultName(vaultName)); String jobId = initiateJobResult.getJobId();
您可以选择指定字节范围,以请求 S3 Glacier 只准备档案的一部分。例如,您可以通过添加以下语句更新前面的请求,以请求 S3 Glacier 只准备档案的 1MB 到 2MB 部分。
int ONE_MEG = 1048576; String retrievalByteRange = String.format("%s-%s", ONE_MEG, 2*ONE_MEG -1); JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withRetrievalByteRange(retrievalByteRange) .withSNSTopic(snsTopicARN); InitiateJobResult initiateJobResult = client.initiateJob(new InitiateJobRequest() .withJobParameters(jobParameters) .withVaultName(vaultName)); String jobId = initiateJobResult.getJobId();
-
等待任务完成。
您必须等到任务输出已作好供您下载的准备。如果您在文件库中设置了标识 Amazon Simple Notification Service (Amazon SNS) 主题的通知配置,或者在启动任务时指定了Amazon SNS 主题,则 S3 Glacier 会在完成任务后向该主题发送消息。
此外,您还可以通过调用
describeJob
方法轮询 S3 Glacier 来确定任务完成状态。尽管如此,使用 Amazon SNS 主题进行通知才是推荐的方法。 -
通过执行
getJobOutput
方法下载任务输出(档案数据)。您可以通过创建一个
GetJobOutputRequest
类的实例来提供请求信息,例如,任务 ID 和文件库名称。S3 Glacier 返回的输出位于GetJobOutputResult
对象中。GetJobOutputRequest jobOutputRequest = new GetJobOutputRequest() .withJobId("*** provide a job ID ***") .withVaultName("*** provide a vault name ****"); GetJobOutputResult jobOutputResult = client.getJobOutput(jobOutputRequest); // jobOutputResult.getBody() // Provides the input stream.
前面的代码段会下载整个任务输出。您可以选择性地只检索输出的一部分,或者通过在您的
GetJobOutputRequest
中指定字节范围以较小的区块下载整个输出。GetJobOutputRequest jobOutputRequest = new GetJobOutputRequest() .withJobId("*** provide a job ID ***") .withRange("bytes=0-1048575") // Download only the first 1 MB of the output. .withVaultName("*** provide a vault name ****");
在响应您的
GetJobOutput
调用时,S3 Glacier 返回您下载的数据部分的校验和(如果满足特定条件)。有关更多信息,请参阅下载数据时接收校验和。要确认下载中没有错误,您随后可以在客户端计算校验和,并将它与 S3 Glacier 在响应中发送的校验和相比较。
对于指定了可选范围的档案检索任务,您在获取任务描述时,它会包括您要检索的范围的校验和 (SHA256TreeHash)。您可以使用此值进一步确认您稍后下载的整个字节范围的准确性。例如,如果您启动任务以检索以树形哈希对齐的档案范围,然后以区块下载输出,使得您的每个
GetJobOutput
请求均返回一个校验和,则您可以在客户端计算您下载的每个部分的校验和,然后计算树形哈希。您可以将它与 S3 Glacier 为响应您的描述任务请求而返回的校验和相比较,以确认您下载的整个字节范围与 S3 Glacier 中存储的字节范围相同。要了解可工作的示例,请参阅 示例 2:使用 Amazon SDK for Java低级 API 检索档案— 按区块下载输出 。
示例 1:使用Amazon SDK for Java低级 API 检索档案
以下 Java 代码示例会从指定的文件库下载档案。任务完成后,该示例会在单一 getJobOutput
调用中下载整个输出。有关以区块下载输出的示例,请参阅“示例 2:使用 Amazon SDK for Java低级 API 检索档案— 按区块下载输出 ”。
该示例执行以下任务:
-
创建 Amazon Simple Notification Service (Amazon SNS) 主题。
完成任务后,S3 Glacier 会向此主题发送通知。
-
创建 Amazon Simple Queue Service (Amazon SQS) 队列。
该示例会向该队列附加策略,以使 Amazon SNS 主题能够向该队列发布消息。
-
启动任务以下载指定的档案。
在任务请求中,指定了创建的 Amazon SNS 主题,以便 S3 Glacier 可以在完成任务后向该主题发布通知。
-
定期检查 Amazon SQS 队列是否有包含该任务 ID 的消息。
如果有消息,则分析 JSON,并检查任务是否已成功完成。如果已成功完成,则下载档案。
-
通过删除它创建的 Amazon SNS 主题和 Amazon SQS 队列清除相关数据。
import java.io.BufferedInputStream; import java.io.BufferedOutputStream; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.FileOutputStream; import java.io.FileWriter; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.util.HashMap; import java.util.List; import java.util.Map; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonNode; import org.codehaus.jackson.JsonParseException; import org.codehaus.jackson.JsonParser; import org.codehaus.jackson.map.ObjectMapper; import com.amazonaws.AmazonClientException; import com.amazonaws.auth.policy.Policy; import com.amazonaws.auth.policy.Principal; import com.amazonaws.auth.policy.Resource; import com.amazonaws.auth.policy.Statement; import com.amazonaws.auth.policy.Statement.Effect; import com.amazonaws.auth.policy.actions.SQSActions; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.model.GetJobOutputRequest; import com.amazonaws.services.glacier.model.GetJobOutputResult; import com.amazonaws.services.glacier.model.InitiateJobRequest; import com.amazonaws.services.glacier.model.InitiateJobResult; import com.amazonaws.services.glacier.model.JobParameters; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sns.model.CreateTopicRequest; import com.amazonaws.services.sns.model.CreateTopicResult; import com.amazonaws.services.sns.model.DeleteTopicRequest; import com.amazonaws.services.sns.model.SubscribeRequest; import com.amazonaws.services.sns.model.SubscribeResult; import com.amazonaws.services.sns.model.UnsubscribeRequest; import com.amazonaws.services.sqs.AmazonSQSClient; import com.amazonaws.services.sqs.model.CreateQueueRequest; import com.amazonaws.services.sqs.model.CreateQueueResult; import com.amazonaws.services.sqs.model.DeleteQueueRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesResult; import com.amazonaws.services.sqs.model.Message; import com.amazonaws.services.sqs.model.ReceiveMessageRequest; import com.amazonaws.services.sqs.model.SetQueueAttributesRequest; public class AmazonGlacierDownloadArchiveWithSQSPolling { public static String archiveId = "*** provide archive ID ****"; public static String vaultName = "*** provide vault name ***"; public static String snsTopicName = "*** provide topic name ***"; public static String sqsQueueName = "*** provide queue name ***"; public static String sqsQueueARN; public static String sqsQueueURL; public static String snsTopicARN; public static String snsSubscriptionARN; public static String fileName = "*** provide file name ***"; public static String region = "*** region ***"; public static long sleepTime = 600; public static AmazonGlacierClient client; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); client = new AmazonGlacierClient(credentials); client.setEndpoint("https://glacier." + region + ".amazonaws.com"); sqsClient = new AmazonSQSClient(credentials); sqsClient.setEndpoint("https://sqs." + region + ".amazonaws.com"); snsClient = new AmazonSNSClient(credentials); snsClient.setEndpoint("https://sns." + region + ".amazonaws.com"); try { setupSQS(); setupSNS(); String jobId = initiateJobRequest(); System.out.println("Jobid = " + jobId); Boolean success = waitForJobToComplete(jobId, sqsQueueURL); if (!success) { throw new Exception("Job did not complete successfully."); } downloadJobOutput(jobId); cleanUp(); } catch (Exception e) { System.err.println("Archive retrieval failed."); System.err.println(e); } } private static void setupSQS() { CreateQueueRequest request = new CreateQueueRequest() .withQueueName(sqsQueueName); CreateQueueResult result = sqsClient.createQueue(request); sqsQueueURL = result.getQueueUrl(); GetQueueAttributesRequest qRequest = new GetQueueAttributesRequest() .withQueueUrl(sqsQueueURL) .withAttributeNames("QueueArn"); GetQueueAttributesResult qResult = sqsClient.getQueueAttributes(qRequest); sqsQueueARN = qResult.getAttributes().get("QueueArn"); Policy sqsPolicy = new Policy().withStatements( new Statement(Effect.Allow) .withPrincipals(Principal.AllUsers) .withActions(SQSActions.SendMessage) .withResources(new Resource(sqsQueueARN))); Map<String, String> queueAttributes = new HashMap<String, String>(); queueAttributes.put("Policy", sqsPolicy.toJson()); sqsClient.setQueueAttributes(new SetQueueAttributesRequest(sqsQueueURL, queueAttributes)); } private static void setupSNS() { CreateTopicRequest request = new CreateTopicRequest() .withName(snsTopicName); CreateTopicResult result = snsClient.createTopic(request); snsTopicARN = result.getTopicArn(); SubscribeRequest request2 = new SubscribeRequest() .withTopicArn(snsTopicARN) .withEndpoint(sqsQueueARN) .withProtocol("sqs"); SubscribeResult result2 = snsClient.subscribe(request2); snsSubscriptionARN = result2.getSubscriptionArn(); } private static String initiateJobRequest() { JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withSNSTopic(snsTopicARN); InitiateJobRequest request = new InitiateJobRequest() .withVaultName(vaultName) .withJobParameters(jobParameters); InitiateJobResult response = client.initiateJob(request); return response.getJobId(); } private static Boolean waitForJobToComplete(String jobId, String sqsQueueUrl) throws InterruptedException, JsonParseException, IOException { Boolean messageFound = false; Boolean jobSuccessful = false; ObjectMapper mapper = new ObjectMapper(); JsonFactory factory = mapper.getJsonFactory(); while (!messageFound) { List<Message> msgs = sqsClient.receiveMessage( new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(10)).getMessages(); if (msgs.size() > 0) { for (Message m : msgs) { JsonParser jpMessage = factory.createJsonParser(m.getBody()); JsonNode jobMessageNode = mapper.readTree(jpMessage); String jobMessage = jobMessageNode.get("Message").getTextValue(); JsonParser jpDesc = factory.createJsonParser(jobMessage); JsonNode jobDescNode = mapper.readTree(jpDesc); String retrievedJobId = jobDescNode.get("JobId").getTextValue(); String statusCode = jobDescNode.get("StatusCode").getTextValue(); if (retrievedJobId.equals(jobId)) { messageFound = true; if (statusCode.equals("Succeeded")) { jobSuccessful = true; } } } } else { Thread.sleep(sleepTime * 1000); } } return (messageFound && jobSuccessful); } private static void downloadJobOutput(String jobId) throws IOException { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() .withVaultName(vaultName) .withJobId(jobId); GetJobOutputResult getJobOutputResult = client.getJobOutput(getJobOutputRequest); InputStream input = new BufferedInputStream(getJobOutputResult.getBody()); OutputStream output = null; try { output = new BufferedOutputStream(new FileOutputStream(fileName)); byte[] buffer = new byte[1024 * 1024]; int bytesRead = 0; do { bytesRead = input.read(buffer); if (bytesRead <= 0) break; output.write(buffer, 0, bytesRead); } while (bytesRead > 0); } catch (IOException e) { throw new AmazonClientException("Unable to save archive", e); } finally { try {input.close();} catch (Exception e) {} try {output.close();} catch (Exception e) {} } System.out.println("Retrieved archive to " + fileName); } private static void cleanUp() { snsClient.unsubscribe(new UnsubscribeRequest(snsSubscriptionARN)); snsClient.deleteTopic(new DeleteTopicRequest(snsTopicARN)); sqsClient.deleteQueue(new DeleteQueueRequest(sqsQueueURL)); } }
示例 2:使用 Amazon SDK for Java低级 API 检索档案— 按区块下载输出
以下 Java 代码示例从 S3 Glacier 检索档案。该代码示例会通过在 GetJobOutputRequest
数据元中指定字节范围来以区块下载任务输出。
import java.io.BufferedInputStream; import java.io.ByteArrayInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.util.HashMap; import java.util.List; import java.util.Map; import com.fasterxml.jackson.core.JsonFactory; import com.fasterxml.jackson.core.JsonParseException; import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.amazonaws.auth.policy.Policy; import com.amazonaws.auth.policy.Principal; import com.amazonaws.auth.policy.Resource; import com.amazonaws.auth.policy.Statement; import com.amazonaws.auth.policy.Statement.Effect; import com.amazonaws.auth.policy.actions.SQSActions; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.TreeHashGenerator; import com.amazonaws.services.glacier.model.GetJobOutputRequest; import com.amazonaws.services.glacier.model.GetJobOutputResult; import com.amazonaws.services.glacier.model.InitiateJobRequest; import com.amazonaws.services.glacier.model.InitiateJobResult; import com.amazonaws.services.glacier.model.JobParameters; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sns.model.CreateTopicRequest; import com.amazonaws.services.sns.model.CreateTopicResult; import com.amazonaws.services.sns.model.DeleteTopicRequest; import com.amazonaws.services.sns.model.SubscribeRequest; import com.amazonaws.services.sns.model.SubscribeResult; import com.amazonaws.services.sns.model.UnsubscribeRequest; import com.amazonaws.services.sqs.AmazonSQSClient; import com.amazonaws.services.sqs.model.CreateQueueRequest; import com.amazonaws.services.sqs.model.CreateQueueResult; import com.amazonaws.services.sqs.model.DeleteQueueRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesResult; import com.amazonaws.services.sqs.model.Message; import com.amazonaws.services.sqs.model.ReceiveMessageRequest; import com.amazonaws.services.sqs.model.SetQueueAttributesRequest; public class ArchiveDownloadLowLevelWithRange { public static String vaultName = "*** provide vault name ***"; public static String archiveId = "*** provide archive id ***"; public static String snsTopicName = "glacier-temp-sns-topic"; public static String sqsQueueName = "glacier-temp-sqs-queue"; public static long downloadChunkSize = 4194304; // 4 MB public static String sqsQueueARN; public static String sqsQueueURL; public static String snsTopicARN; public static String snsSubscriptionARN; public static String fileName = "*** provide file name to save archive to ***"; public static String region = "*** region ***"; public static long sleepTime = 600; public static AmazonGlacierClient client; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); client = new AmazonGlacierClient(credentials); client.setEndpoint("https://glacier." + region + ".amazonaws.com"); sqsClient = new AmazonSQSClient(credentials); sqsClient.setEndpoint("https://sqs." + region + ".amazonaws.com"); snsClient = new AmazonSNSClient(credentials); snsClient.setEndpoint("https://sns." + region + ".amazonaws.com"); try { setupSQS(); setupSNS(); String jobId = initiateJobRequest(); System.out.println("Jobid = " + jobId); long archiveSizeInBytes = waitForJobToComplete(jobId, sqsQueueURL); if (archiveSizeInBytes==-1) { throw new Exception("Job did not complete successfully."); } downloadJobOutput(jobId, archiveSizeInBytes); cleanUp(); } catch (Exception e) { System.err.println("Archive retrieval failed."); System.err.println(e); } } private static void setupSQS() { CreateQueueRequest request = new CreateQueueRequest() .withQueueName(sqsQueueName); CreateQueueResult result = sqsClient.createQueue(request); sqsQueueURL = result.getQueueUrl(); GetQueueAttributesRequest qRequest = new GetQueueAttributesRequest() .withQueueUrl(sqsQueueURL) .withAttributeNames("QueueArn"); GetQueueAttributesResult qResult = sqsClient.getQueueAttributes(qRequest); sqsQueueARN = qResult.getAttributes().get("QueueArn"); Policy sqsPolicy = new Policy().withStatements( new Statement(Effect.Allow) .withPrincipals(Principal.AllUsers) .withActions(SQSActions.SendMessage) .withResources(new Resource(sqsQueueARN))); Map<String, String> queueAttributes = new HashMap<String, String>(); queueAttributes.put("Policy", sqsPolicy.toJson()); sqsClient.setQueueAttributes(new SetQueueAttributesRequest(sqsQueueURL, queueAttributes)); } private static void setupSNS() { CreateTopicRequest request = new CreateTopicRequest() .withName(snsTopicName); CreateTopicResult result = snsClient.createTopic(request); snsTopicARN = result.getTopicArn(); SubscribeRequest request2 = new SubscribeRequest() .withTopicArn(snsTopicARN) .withEndpoint(sqsQueueARN) .withProtocol("sqs"); SubscribeResult result2 = snsClient.subscribe(request2); snsSubscriptionARN = result2.getSubscriptionArn(); } private static String initiateJobRequest() { JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withSNSTopic(snsTopicARN); InitiateJobRequest request = new InitiateJobRequest() .withVaultName(vaultName) .withJobParameters(jobParameters); InitiateJobResult response = client.initiateJob(request); return response.getJobId(); } private static long waitForJobToComplete(String jobId, String sqsQueueUrl) throws InterruptedException, JsonParseException, IOException { Boolean messageFound = false; Boolean jobSuccessful = false; long archiveSizeInBytes = -1; ObjectMapper mapper = new ObjectMapper(); JsonFactory factory = mapper.getFactory(); while (!messageFound) { List<Message> msgs = sqsClient.receiveMessage( new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(10)).getMessages(); if (msgs.size() > 0) { for (Message m : msgs) { JsonParser jpMessage = factory.createJsonParser(m.getBody()); JsonNode jobMessageNode = mapper.readTree(jpMessage); String jobMessage = jobMessageNode.get("Message").textValue(); JsonParser jpDesc = factory.createJsonParser(jobMessage); JsonNode jobDescNode = mapper.readTree(jpDesc); String retrievedJobId = jobDescNode.get("JobId").textValue(); String statusCode = jobDescNode.get("StatusCode").textValue(); archiveSizeInBytes = jobDescNode.get("ArchiveSizeInBytes").longValue(); if (retrievedJobId.equals(jobId)) { messageFound = true; if (statusCode.equals("Succeeded")) { jobSuccessful = true; } } } } else { Thread.sleep(sleepTime * 1000); } } return (messageFound && jobSuccessful) ? archiveSizeInBytes : -1; } private static void downloadJobOutput(String jobId, long archiveSizeInBytes) throws IOException { if (archiveSizeInBytes < 0) { System.err.println("Nothing to download."); return; } System.out.println("archiveSizeInBytes: " + archiveSizeInBytes); FileOutputStream fstream = new FileOutputStream(fileName); long startRange = 0; long endRange = (downloadChunkSize > archiveSizeInBytes) ? archiveSizeInBytes -1 : downloadChunkSize - 1; do { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() .withVaultName(vaultName) .withRange("bytes=" + startRange + "-" + endRange) .withJobId(jobId); GetJobOutputResult getJobOutputResult = client.getJobOutput(getJobOutputRequest); BufferedInputStream is = new BufferedInputStream(getJobOutputResult.getBody()); byte[] buffer = new byte[(int)(endRange - startRange + 1)]; System.out.println("Checksum received: " + getJobOutputResult.getChecksum()); System.out.println("Content range " + getJobOutputResult.getContentRange()); int totalRead = 0; while (totalRead < buffer.length) { int bytesRemaining = buffer.length - totalRead; int read = is.read(buffer, totalRead, bytesRemaining); if (read > 0) { totalRead = totalRead + read; } else { break; } } System.out.println("Calculated checksum: " + TreeHashGenerator.calculateTreeHash(new ByteArrayInputStream(buffer))); System.out.println("read = " + totalRead); fstream.write(buffer); startRange = startRange + (long)totalRead; endRange = ((endRange + downloadChunkSize) > archiveSizeInBytes) ? archiveSizeInBytes : (endRange + downloadChunkSize); is.close(); } while (endRange <= archiveSizeInBytes && startRange < archiveSizeInBytes); fstream.close(); System.out.println("Retrieved file to " + fileName); } private static void cleanUp() { snsClient.unsubscribe(new UnsubscribeRequest(snsSubscriptionARN)); snsClient.deleteTopic(new DeleteTopicRequest(snsTopicARN)); sqsClient.deleteQueue(new DeleteQueueRequest(sqsQueueURL)); } }