**This page is only for existing customers of the Amazon Glacier service using Vaults and the original REST API from 2012.** If you're looking for archival storage solutions, we recommend using the Amazon Glacier storage classes in Amazon S3, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive. To learn more about these storage options, see [Amazon Glacier storage classes](https://www.amazonaws.cn/s3/storage-classes/glacier/). Amazon Glacier (original standalone vault-based service) is no longer accepting new customers. Amazon Glacier is a standalone service with its own APIs that stores data in vaults and is distinct from Amazon S3 and the Amazon S3 Glacier storage classes. Your existing data will remain secure and accessible in Amazon Glacier indefinitely. No migration is required. For low-cost, long-term archival storage, Amazon recommends the [Amazon S3 Glacier storage classes](https://www.amazonaws.cn/s3/storage-classes/glacier/), which deliver a superior customer experience with S3 bucket-based APIs, full Amazon Web Services Region availability, lower costs, and Amazon service integration. If you want enhanced capabilities, consider migrating to Amazon S3 Glacier storage classes by using our [Amazon Solutions Guidance for transferring data from Amazon Glacier vaults to Amazon S3 Glacier storage classes](https://www.amazonaws.cn/solutions/guidance/data-transfer-from-amazon-s3-glacier-vaults-to-amazon-s3/). # Downloading an Archive in Amazon Glacier Amazon Glacier provides a management console, which you can use to create and delete vaults. However, you cannot download archives from Amazon Glacier by using the management console. To download data, such as photos, videos, and other documents, you must either use the Amazon Command Line Interface (Amazon CLI) or write code to make requests, by using either the REST API directly or by using the Amazon SDKs. For information about using Amazon Glacier with the Amazon CLI, see the [Amazon CLI Reference for Amazon Glacier](http://docs.amazonaws.cn/cli/latest/reference/glacier/index.html). To install the Amazon CLI, see [Amazon Command Line Interface](http://www.amazonaws.cn/cli/). The following topics describe how to download archives to Amazon Glacier by using the Amazon SDK for Java, the Amazon SDK for .NET, and the Amazon Glacier REST API. **Topics** + [ # Retrieving Amazon Glacier Archives ](downloading-an-archive-two-steps.md) + [ # Downloading an Archive in Amazon Glacier Using the Amazon SDK for Java ](downloading-an-archive-using-java.md) + [ # Downloading an Archive in Amazon Glacier Using the Amazon SDK for .NET ](downloading-an-archive-using-dotnet.md) + [ # Downloading a Large Archive Using Parallel Processing with Python ](downloading-large-archive-parallel-python.md) + [ # Downloading an Archive by Using the REST API ](downloading-an-archive-using-rest.md) + [ # Downloading an Archive in Amazon Glacier Using the Amazon CLI ](downloading-an-archive-using-cli.md) # Retrieving Amazon Glacier Archives Retrieving an archive from Amazon Glacier is an asynchronous operation in which you first initiate a job, and then download the output after the job is completed. To initiate an archive retrieval job, you use the [Initiate Job (POST jobs)](api-initiate-job-post.md) REST API operation or the equivalent in the Amazon CLI, or the Amazon SDKs. **Topics** + [ ## Archive Retrieval Options ](#api-downloading-an-archive-two-steps-retrieval-options) + [ ## Ranged Archive Retrievals ](#downloading-an-archive-range) Retrieving an archive from Amazon Glacier is a two-step process. The following is an overview of this process. **To retrieve an archive** 1. Initiate an archive retrieval job. 1. Get the ID of the archive that you want to retrieve. You can get the archive ID from an inventory of the vault. You can get the archive ID with the REST API, Amazon CLI, or Amazon SDKs. For more information, see [Downloading a Vault Inventory in Amazon Glacier](vault-inventory.md). 1. Initiate a job that requests Amazon Glacier to prepare an entire archive or a portion of the archive for subsequent download by using the [Initiate Job (POST jobs)](api-initiate-job-post.md) operation. When you initiate a job, Amazon Glacier returns a job ID in the response and runs the job asynchronously. (You cannot download the job output until after the job is completed, as described in Step 2.) **Important** For Standard retrievals only, a data retrieval policy can cause your `Initiate Job` request to fail with a `PolicyEnforcedException` exception. For more information about data retrieval policies, see [Amazon Glacier Data Retrieval Policies](data-retrieval-policy.md). For more information about the `PolicyEnforcedException` exception, see [Error Responses](api-error-responses.md). When required, you can restore large segments of the data stored in Amazon Glacier. For more information about restoring data from the Amazon Glacier storage classes, see [ Storage Classes for Archiving Objects]( https://docs.amazonaws.cn/AmazonS3/latest/userguide/storage-class-intro.html#sc-glacier) in the *Amazon Simple Storage Service User Guide*. 1. After the job is completed, download the bytes by using the [Get Job Output (GET output)](api-job-output-get.md) operation. You can download all bytes or specify a byte range to download only a portion of the job output. For larger output, downloading the output in chunks helps in the event of a download failure, such as a network failure. If you get job output in a single request and there is a network failure, you have to restart downloading the output from the beginning. However, if you download the output in chunks, in the event of any failure, you need only restart the download of the smaller portion and not the entire output. Amazon Glacier must complete a job before you can get its output. After completion, a job does not expire for at least 24 hours, which means that you can download the output within the 24-hour period after the job is completed. The restore can expire any time after 24 hours from the completion of the job. To determine if your job is complete, check its status by using one of the following options: + **Wait for a job-completion notification** – You can specify an Amazon Simple Notification Service (Amazon SNS) topic to which Amazon Glacier can post a notification after the job is completed. Amazon Glacier sends a notification only after it completes the job. You can specify an Amazon SNS topic for a job when you initiate the job. In addition to specifying an Amazon SNS topic in your job request, if your vault has notifications set for archive retrieval events, then Amazon Glacier also publishes a notification to that SNS topic. For more information, see [Configuring Vault Notifications in Amazon Glacier](configuring-notifications.md). + **Request job information explicitly** – You can also use the Amazon Glacier `Describe Job` API operation ([Describe Job (GET JobID)](api-describe-job-get.md)) to periodically poll for job information. However, we recommend using Amazon SNS notifications. **Note** The information that you get by using an Amazon SNS notification is the same as what you get by calling the `Describe Job` API operation. ## Archive Retrieval Options When initiating a job to retrieve an archive, you can specify one of the following retrieval options, based on your access time and cost requirements. For information about retrieval pricing, see [Amazon Glacier Pricing](https://aws.amazon.com/s3/glacier/pricing/). + **Expedited** – Expedited retrievals allow you to quickly access your data that's stored in the S3 Glacier Flexible Retrieval storage class or the S3 Intelligent-Tiering Archive Access tier when occasional urgent requests for restoring archives are required. For all but the largest archives (more than 250 MB), data accessed by using Expedited retrievals is typically made available within 1–5 minutes. Provisioned capacity ensures that retrieval capacity for Expedited retrievals is available when you need it. For more information, see [Provisioned Capacity](#api-downloading-an-archive-two-steps-retrieval-expedited-capacity). + **Standard** – Standard retrievals allow you to access any of your archives within several hours. Standard retrievals are typically completed within 3–5 hours. Standard is the default option for retrieval requests that do not specify the retrieval option. + **Bulk** – Bulk retrievals are the lowest-cost Amazon Glacier retrieval option, which you can use to retrieve large amounts, even petabytes, of data inexpensively in a day. Bulk retrievals are typically completed within 5–12 hours. The following table summarizes the archive retrieval options. For information about pricing, see [Amazon Glacier pricing](http://www.amazonaws.cn/s3/glacier/pricing/). | Service | Expedited | Standard | Bulk | | --- | --- | --- | --- | | Amazon Glacier | 1–5 minutes | 3–5 hours | 5–12 hours | To make an `Expedited`, `Standard`, or `Bulk` retrieval, set the `Tier` request element in the [https://docs.amazonaws.cn/AmazonS3/latest/API/RESTObjectPOSTrestore.html](https://docs.amazonaws.cn/AmazonS3/latest/API/RESTObjectPOSTrestore.html) REST API operation request to the option that you want, or the equivalent in the Amazon Command Line Interface (Amazon CLI) or Amazon SDKs. If you purchased provisioned capacity, all Expedited retrievals are automatically served through your provisioned capacity. ### Provisioned Capacity Provisioned capacity helps ensure that your retrieval capacity for Expedited retrievals is available when you need it. Each unit of capacity provides that at least three Expedited retrievals can be performed every 5 minutes and provides up to 150 megabytes per second (MBps) of retrieval throughput. If your workload requires highly reliable and predictable access to a subset of your data in minutes, we recommend that you purchase provisioned retrieval capacity. Without provisioned capacity, Expedited retrievals are typically accepted, except for rare situations of unusually high demand. However, if you require access to Expedited retrievals under all circumstances, you must purchase provisioned retrieval capacity. #### Purchasing Provisioned Capacity You can purchase provisioned capacity units by using the Amazon Glacier console, the [Purchase Provisioned Capacity (POST provisioned-capacity)](api-PurchaseProvisionedCapacity.md) REST API operation, the Amazon SDKs, or the Amazon CLI. For provisioned capacity pricing information, see [Amazon Glacier Pricing](https://aws.amazon.com/s3/glacier/pricing/). A provisioned capacity unit lasts for one month, starting at the date and time of purchase. If the start date is on the 31st day of a month, the expiration date is the last day of the next month. For example, if the start date is August 31, the expiration date is September 30. If the start date is January 31, the expiration date is February 28. **To purchase provisioned capacity by using the Amazon Glacier console** 1. Sign in to the Amazon Web Services Management Console and open the Amazon Glacier console at [https://console.amazonaws.cn/glacier/home](https://console.amazonaws.cn/glacier/home). 1. In the left navigation pane, choose **Data retrieval settings**. 1. Under **Provisioned capacity units (PCUs)**, choose **Purchase PCU**. The **Purchase PCU** dialog box appears. 1. If you want to purchase provisioned capacity, enter **confirm** in the **To confirm purchase** box. 1. Choose **Purchase PCU**. ## Ranged Archive Retrievals When you retrieve an archive from Amazon Glacier, you can optionally specify a range, or portion, of the archive to retrieve. The default is to retrieve the whole archive. Specifying a range of bytes can be helpful when you want to do the following: + **Manage your data downloads** – Amazon Glacier allows retrieved data to be downloaded for 24 hours after the retrieval request is completed. Therefore, you might want to retrieve only portions of the archive so that you can manage the schedule of downloads within the given download window. + **Retrieve a targeted part of a large archive** – For example, suppose you have previously aggregated many files and uploaded them as a single archive, and now you want to retrieve a few of the files. In this case, you can specify a range of the archive that contains the files that you are interested in by using one retrieval request. Or, you can initiate multiple retrieval requests, each with a range for one or more files. When initiating a retrieval job using range retrievals, you must provide a range that is megabyte aligned. In other words, the byte range can start at zero (the beginning of your archive), or at any 1-MB interval thereafter (1 MB, 2 MB, 3 MB, and so on). The end of the range can either be the end of your archive or any 1 MB interval greater than the beginning of your range. Furthermore, if you want to get checksum values when you download the data (after the retrieval job is completed), the range that you request in the job initiation must also be tree-hash aligned. You can use checksums to help ensure that your data was not corrupted during transmission. For more information about megabyte alignment and tree-hash alignment, see [Receiving Checksums When Downloading Data](checksum-calculations-range.md). # Downloading an Archive in Amazon Glacier Using the Amazon SDK for Java Both the [high-level and low-level APIs](using-aws-sdk.md) provided by the Amazon SDK for Java provide a method to download an archive. **Topics** + [ ## Downloading an Archive Using the High-Level API of the Amazon SDK for Java ](#downloading-an-archive-using-java-highlevel-api) + [ ## Downloading an Archive Using the Low-Level API of the Amazon SDK for Java ](#downloading-an-archive-using-java-lowlevel-api) ## Downloading an Archive Using the High-Level API of the Amazon SDK for Java The `ArchiveTransferManager` class of the high-level API provides the `download` method you can use to download an archive. **Important** The `ArchiveTransferManager` class creates an Amazon Simple Notification Service (Amazon SNS) topic, and an Amazon Simple Queue Service (Amazon SQS) queue that is subscribed to that topic. It then initiates the archive retrieval job and polls the queue for the archive to be available. When the archive is available, download begins. For information about retrieval times, see [Archive Retrieval Options](downloading-an-archive-two-steps.md#api-downloading-an-archive-two-steps-retrieval-options). ### Example: Downloading an Archive Using the High-Level API of the Amazon SDK for Java The following Java code example downloads an archive from a vault (`examplevault`) in the US West (Oregon) Region (`us-west-2`). For step-by-step instructions to run this sample, see [Running Java Examples for Amazon Glacier Using Eclipse](using-aws-sdk-for-java.md#setting-up-and-testing-sdk-java). You need to update the code as shown with an existing archive ID and the local file path where you want to save the downloaded archive. **Example** ``` import java.io.File; import java.io.IOException; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.transfer.ArchiveTransferManager; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sqs.AmazonSQSClient; public class ArchiveDownloadHighLevel { public static String vaultName = "examplevault"; public static String archiveId = "*** provide archive ID ***"; public static String downloadFilePath = "*** provide location to download archive ***"; public static AmazonGlacierClient glacierClient; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); glacierClient = new AmazonGlacierClient(credentials); sqsClient = new AmazonSQSClient(credentials); snsClient = new AmazonSNSClient(credentials); glacierClient.setEndpoint("glacier.us-west-2.amazonaws.com"); sqsClient.setEndpoint("sqs.us-west-2.amazonaws.com"); snsClient.setEndpoint("sns.us-west-2.amazonaws.com"); try { ArchiveTransferManager atm = new ArchiveTransferManager(glacierClient, sqsClient, snsClient); atm.download(vaultName, archiveId, new File(downloadFilePath)); System.out.println("Downloaded file to " + downloadFilePath); } catch (Exception e) { System.err.println(e); } } } ``` ## Downloading an Archive Using the Low-Level API of the Amazon SDK for Java The following are the steps to retrieve a vault inventory using the Amazon SDK for Java low-level API. 1. Create an instance of the `AmazonGlacierClient` class (the client). You need to specify an Amazon Region from where you want to download the archive. All operations you perform using this client apply to that Amazon Region. 1. Initiate an `archive-retrieval` job by executing the `initiateJob` method. You provide job information, such as the archive ID of the archive you want to download and the optional Amazon SNS topic to which you want Amazon Glacier (Amazon Glacier) to post a job completion message, by creating an instance of the `InitiateJobRequest` class. Amazon Glacier returns a job ID in response. The response is available in an instance of the `InitiateJobResult` class. ``` JobParameters jobParameters = new JobParameters() .withArchiveId("*** provide an archive id ***") .withDescription("archive retrieval") .withRetrievalByteRange("*** provide a retrieval range***") // optional .withType("archive-retrieval"); InitiateJobResult initiateJobResult = client.initiateJob(new InitiateJobRequest() .withJobParameters(jobParameters) .withVaultName(vaultName)); String jobId = initiateJobResult.getJobId(); ``` You can optionally specify a byte range to request Amazon Glacier to prepare only a portion of the archive. For example, you can update the preceding request by adding the following statement to request Amazon Glacier to prepare only the 1 MB to 2 MB portion of the archive. ``` int ONE_MEG = 1048576; String retrievalByteRange = String.format("%s-%s", ONE_MEG, 2*ONE_MEG -1); JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withRetrievalByteRange(retrievalByteRange) .withSNSTopic(snsTopicARN); InitiateJobResult initiateJobResult = client.initiateJob(new InitiateJobRequest() .withJobParameters(jobParameters) .withVaultName(vaultName)); String jobId = initiateJobResult.getJobId(); ``` 1. Wait for the job to complete. You must wait until the job output is ready for you to download. If you have either set a notification configuration on the vault identifying an Amazon Simple Notification Service (Amazon SNS) topic or specified an Amazon SNS topic when you initiated a job, Amazon Glacier sends a message to that topic after it completes the job. You can also poll Amazon Glacier by calling the `describeJob` method to determine the job completion status. Although, using an Amazon SNS topic for notification is the recommended approach. 1. Download the job output (archive data) by executing the `getJobOutput` method. You provide the request information such as the job ID and vault name by creating an instance of the `GetJobOutputRequest` class. The output that Amazon Glacier returns is available in the `GetJobOutputResult` object. ``` GetJobOutputRequest jobOutputRequest = new GetJobOutputRequest() .withJobId("*** provide a job ID ***") .withVaultName("*** provide a vault name ****"); GetJobOutputResult jobOutputResult = client.getJobOutput(jobOutputRequest); // jobOutputResult.getBody() // Provides the input stream. ``` The preceding code snippet downloads the entire job output. You can optionally retrieve only a portion of the output, or download the entire output in smaller chunks by specifying the byte range in your `GetJobOutputRequest`. ``` GetJobOutputRequest jobOutputRequest = new GetJobOutputRequest() .withJobId("*** provide a job ID ***") .withRange("bytes=0-1048575") // Download only the first 1 MB of the output. .withVaultName("*** provide a vault name ****"); ``` In response to your `GetJobOutput` call, Amazon Glacier returns the checksum of the portion of the data you downloaded, if certain conditions are met. For more information, see [Receiving Checksums When Downloading Data](checksum-calculations-range.md). To verify there are no errors in the download, you can then compute the checksum on the client-side and compare it with the checksum Amazon Glacier sent in the response. For an archive retrieval job with the optional range specified, when you get the job description, it includes the checksum of the range you are retrieving (SHA256TreeHash). You can use this value to further verify the accuracy of the entire byte range that you later download. For example, if you initiate a job to retrieve a tree-hash aligned archive range and then download output in chunks such that each of your `GetJobOutput` requests return a checksum, then you can compute checksum of each portion you download on the client-side and then compute the tree hash. You can compare it with the checksum Amazon Glacier returns in response to your describe job request to verify that the entire byte range you have downloaded is the same as the byte range that is stored in Amazon Glacier. For a working example, see [Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for Java—Download Output in Chunks](#downloading-an-archive-with-range-using-java-example). ### Example 1: Retrieving an Archive Using the Low-Level API of the Amazon SDK for Java The following Java code example downloads an archive from the specified vault. After the job completes, the example downloads the entire output in a single `getJobOutput` call. For an example of downloading output in chunks, see [Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for Java—Download Output in Chunks](#downloading-an-archive-with-range-using-java-example). The example performs the following tasks: + Creates an Amazon Simple Notification Service (Amazon SNS) topic. Amazon Glacier sends a notification to this topic after it completes the job. + Creates an Amazon Simple Queue Service (Amazon SQS) queue. The example attaches a policy to the queue to enable the Amazon SNS topic to post messages to the queue. + Initiates a job to download the specified archive. In the job request, the Amazon SNS topic that was created is specified so that Amazon Glacier can publish a notification to the topic after it completes the job. + Periodically checks the Amazon SQS queue for a message that contains the job ID. If there is a message, parse the JSON and check if the job completed successfully. If it did, download the archive. + Cleans up by deleting the Amazon SNS topic and the Amazon SQS queue that it created. ``` import java.io.BufferedInputStream; import java.io.BufferedOutputStream; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.FileOutputStream; import java.io.FileWriter; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.util.HashMap; import java.util.List; import java.util.Map; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonNode; import org.codehaus.jackson.JsonParseException; import org.codehaus.jackson.JsonParser; import org.codehaus.jackson.map.ObjectMapper; import com.amazonaws.AmazonClientException; import com.amazonaws.auth.policy.Policy; import com.amazonaws.auth.policy.Principal; import com.amazonaws.auth.policy.Resource; import com.amazonaws.auth.policy.Statement; import com.amazonaws.auth.policy.Statement.Effect; import com.amazonaws.auth.policy.actions.SQSActions; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.model.GetJobOutputRequest; import com.amazonaws.services.glacier.model.GetJobOutputResult; import com.amazonaws.services.glacier.model.InitiateJobRequest; import com.amazonaws.services.glacier.model.InitiateJobResult; import com.amazonaws.services.glacier.model.JobParameters; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sns.model.CreateTopicRequest; import com.amazonaws.services.sns.model.CreateTopicResult; import com.amazonaws.services.sns.model.DeleteTopicRequest; import com.amazonaws.services.sns.model.SubscribeRequest; import com.amazonaws.services.sns.model.SubscribeResult; import com.amazonaws.services.sns.model.UnsubscribeRequest; import com.amazonaws.services.sqs.AmazonSQSClient; import com.amazonaws.services.sqs.model.CreateQueueRequest; import com.amazonaws.services.sqs.model.CreateQueueResult; import com.amazonaws.services.sqs.model.DeleteQueueRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesResult; import com.amazonaws.services.sqs.model.Message; import com.amazonaws.services.sqs.model.ReceiveMessageRequest; import com.amazonaws.services.sqs.model.SetQueueAttributesRequest; public class AmazonGlacierDownloadArchiveWithSQSPolling { public static String archiveId = "*** provide archive ID ****"; public static String vaultName = "*** provide vault name ***"; public static String snsTopicName = "*** provide topic name ***"; public static String sqsQueueName = "*** provide queue name ***"; public static String sqsQueueARN; public static String sqsQueueURL; public static String snsTopicARN; public static String snsSubscriptionARN; public static String fileName = "*** provide file name ***"; public static String region = "*** region ***"; public static long sleepTime = 600; public static AmazonGlacierClient client; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); client = new AmazonGlacierClient(credentials); client.setEndpoint("https://glacier." + region + ".amazonaws.com"); sqsClient = new AmazonSQSClient(credentials); sqsClient.setEndpoint("https://sqs." + region + ".amazonaws.com"); snsClient = new AmazonSNSClient(credentials); snsClient.setEndpoint("https://sns." + region + ".amazonaws.com"); try { setupSQS(); setupSNS(); String jobId = initiateJobRequest(); System.out.println("Jobid = " + jobId); Boolean success = waitForJobToComplete(jobId, sqsQueueURL); if (!success) { throw new Exception("Job did not complete successfully."); } downloadJobOutput(jobId); cleanUp(); } catch (Exception e) { System.err.println("Archive retrieval failed."); System.err.println(e); } } private static void setupSQS() { CreateQueueRequest request = new CreateQueueRequest() .withQueueName(sqsQueueName); CreateQueueResult result = sqsClient.createQueue(request); sqsQueueURL = result.getQueueUrl(); GetQueueAttributesRequest qRequest = new GetQueueAttributesRequest() .withQueueUrl(sqsQueueURL) .withAttributeNames("QueueArn"); GetQueueAttributesResult qResult = sqsClient.getQueueAttributes(qRequest); sqsQueueARN = qResult.getAttributes().get("QueueArn"); Policy sqsPolicy = new Policy().withStatements( new Statement(Effect.Allow) .withPrincipals(Principal.AllUsers) .withActions(SQSActions.SendMessage) .withResources(new Resource(sqsQueueARN))); Map queueAttributes = new HashMap(); queueAttributes.put("Policy", sqsPolicy.toJson()); sqsClient.setQueueAttributes(new SetQueueAttributesRequest(sqsQueueURL, queueAttributes)); } private static void setupSNS() { CreateTopicRequest request = new CreateTopicRequest() .withName(snsTopicName); CreateTopicResult result = snsClient.createTopic(request); snsTopicARN = result.getTopicArn(); SubscribeRequest request2 = new SubscribeRequest() .withTopicArn(snsTopicARN) .withEndpoint(sqsQueueARN) .withProtocol("sqs"); SubscribeResult result2 = snsClient.subscribe(request2); snsSubscriptionARN = result2.getSubscriptionArn(); } private static String initiateJobRequest() { JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withSNSTopic(snsTopicARN); InitiateJobRequest request = new InitiateJobRequest() .withVaultName(vaultName) .withJobParameters(jobParameters); InitiateJobResult response = client.initiateJob(request); return response.getJobId(); } private static Boolean waitForJobToComplete(String jobId, String sqsQueueUrl) throws InterruptedException, JsonParseException, IOException { Boolean messageFound = false; Boolean jobSuccessful = false; ObjectMapper mapper = new ObjectMapper(); JsonFactory factory = mapper.getJsonFactory(); while (!messageFound) { List msgs = sqsClient.receiveMessage( new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(10)).getMessages(); if (msgs.size() > 0) { for (Message m : msgs) { JsonParser jpMessage = factory.createJsonParser(m.getBody()); JsonNode jobMessageNode = mapper.readTree(jpMessage); String jobMessage = jobMessageNode.get("Message").getTextValue(); JsonParser jpDesc = factory.createJsonParser(jobMessage); JsonNode jobDescNode = mapper.readTree(jpDesc); String retrievedJobId = jobDescNode.get("JobId").getTextValue(); String statusCode = jobDescNode.get("StatusCode").getTextValue(); if (retrievedJobId.equals(jobId)) { messageFound = true; if (statusCode.equals("Succeeded")) { jobSuccessful = true; } } } } else { Thread.sleep(sleepTime * 1000); } } return (messageFound && jobSuccessful); } private static void downloadJobOutput(String jobId) throws IOException { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() .withVaultName(vaultName) .withJobId(jobId); GetJobOutputResult getJobOutputResult = client.getJobOutput(getJobOutputRequest); InputStream input = new BufferedInputStream(getJobOutputResult.getBody()); OutputStream output = null; try { output = new BufferedOutputStream(new FileOutputStream(fileName)); byte[] buffer = new byte[1024 * 1024]; int bytesRead = 0; do { bytesRead = input.read(buffer); if (bytesRead <= 0) break; output.write(buffer, 0, bytesRead); } while (bytesRead > 0); } catch (IOException e) { throw new AmazonClientException("Unable to save archive", e); } finally { try {input.close();} catch (Exception e) {} try {output.close();} catch (Exception e) {} } System.out.println("Retrieved archive to " + fileName); } private static void cleanUp() { snsClient.unsubscribe(new UnsubscribeRequest(snsSubscriptionARN)); snsClient.deleteTopic(new DeleteTopicRequest(snsTopicARN)); sqsClient.deleteQueue(new DeleteQueueRequest(sqsQueueURL)); } } ``` ### Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for Java—Download Output in Chunks The following Java code example retrieves an archive from Amazon Glacier. The code example downloads the job output in chunks by specifying byte range in a `GetJobOutputRequest` object. ``` import java.io.BufferedInputStream; import java.io.ByteArrayInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.util.HashMap; import java.util.List; import java.util.Map; import com.fasterxml.jackson.core.JsonFactory; import com.fasterxml.jackson.core.JsonParseException; import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.amazonaws.auth.policy.Policy; import com.amazonaws.auth.policy.Principal; import com.amazonaws.auth.policy.Resource; import com.amazonaws.auth.policy.Statement; import com.amazonaws.auth.policy.Statement.Effect; import com.amazonaws.auth.policy.actions.SQSActions; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.services.glacier.AmazonGlacierClient; import com.amazonaws.services.glacier.TreeHashGenerator; import com.amazonaws.services.glacier.model.GetJobOutputRequest; import com.amazonaws.services.glacier.model.GetJobOutputResult; import com.amazonaws.services.glacier.model.InitiateJobRequest; import com.amazonaws.services.glacier.model.InitiateJobResult; import com.amazonaws.services.glacier.model.JobParameters; import com.amazonaws.services.sns.AmazonSNSClient; import com.amazonaws.services.sns.model.CreateTopicRequest; import com.amazonaws.services.sns.model.CreateTopicResult; import com.amazonaws.services.sns.model.DeleteTopicRequest; import com.amazonaws.services.sns.model.SubscribeRequest; import com.amazonaws.services.sns.model.SubscribeResult; import com.amazonaws.services.sns.model.UnsubscribeRequest; import com.amazonaws.services.sqs.AmazonSQSClient; import com.amazonaws.services.sqs.model.CreateQueueRequest; import com.amazonaws.services.sqs.model.CreateQueueResult; import com.amazonaws.services.sqs.model.DeleteQueueRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesRequest; import com.amazonaws.services.sqs.model.GetQueueAttributesResult; import com.amazonaws.services.sqs.model.Message; import com.amazonaws.services.sqs.model.ReceiveMessageRequest; import com.amazonaws.services.sqs.model.SetQueueAttributesRequest; public class ArchiveDownloadLowLevelWithRange { public static String vaultName = "*** provide vault name ***"; public static String archiveId = "*** provide archive id ***"; public static String snsTopicName = "glacier-temp-sns-topic"; public static String sqsQueueName = "glacier-temp-sqs-queue"; public static long downloadChunkSize = 4194304; // 4 MB public static String sqsQueueARN; public static String sqsQueueURL; public static String snsTopicARN; public static String snsSubscriptionARN; public static String fileName = "*** provide file name to save archive to ***"; public static String region = "*** region ***"; public static long sleepTime = 600; public static AmazonGlacierClient client; public static AmazonSQSClient sqsClient; public static AmazonSNSClient snsClient; public static void main(String[] args) throws IOException { ProfileCredentialsProvider credentials = new ProfileCredentialsProvider(); client = new AmazonGlacierClient(credentials); client.setEndpoint("https://glacier." + region + ".amazonaws.com"); sqsClient = new AmazonSQSClient(credentials); sqsClient.setEndpoint("https://sqs." + region + ".amazonaws.com"); snsClient = new AmazonSNSClient(credentials); snsClient.setEndpoint("https://sns." + region + ".amazonaws.com"); try { setupSQS(); setupSNS(); String jobId = initiateJobRequest(); System.out.println("Jobid = " + jobId); long archiveSizeInBytes = waitForJobToComplete(jobId, sqsQueueURL); if (archiveSizeInBytes==-1) { throw new Exception("Job did not complete successfully."); } downloadJobOutput(jobId, archiveSizeInBytes); cleanUp(); } catch (Exception e) { System.err.println("Archive retrieval failed."); System.err.println(e); } } private static void setupSQS() { CreateQueueRequest request = new CreateQueueRequest() .withQueueName(sqsQueueName); CreateQueueResult result = sqsClient.createQueue(request); sqsQueueURL = result.getQueueUrl(); GetQueueAttributesRequest qRequest = new GetQueueAttributesRequest() .withQueueUrl(sqsQueueURL) .withAttributeNames("QueueArn"); GetQueueAttributesResult qResult = sqsClient.getQueueAttributes(qRequest); sqsQueueARN = qResult.getAttributes().get("QueueArn"); Policy sqsPolicy = new Policy().withStatements( new Statement(Effect.Allow) .withPrincipals(Principal.AllUsers) .withActions(SQSActions.SendMessage) .withResources(new Resource(sqsQueueARN))); Map queueAttributes = new HashMap(); queueAttributes.put("Policy", sqsPolicy.toJson()); sqsClient.setQueueAttributes(new SetQueueAttributesRequest(sqsQueueURL, queueAttributes)); } private static void setupSNS() { CreateTopicRequest request = new CreateTopicRequest() .withName(snsTopicName); CreateTopicResult result = snsClient.createTopic(request); snsTopicARN = result.getTopicArn(); SubscribeRequest request2 = new SubscribeRequest() .withTopicArn(snsTopicARN) .withEndpoint(sqsQueueARN) .withProtocol("sqs"); SubscribeResult result2 = snsClient.subscribe(request2); snsSubscriptionARN = result2.getSubscriptionArn(); } private static String initiateJobRequest() { JobParameters jobParameters = new JobParameters() .withType("archive-retrieval") .withArchiveId(archiveId) .withSNSTopic(snsTopicARN); InitiateJobRequest request = new InitiateJobRequest() .withVaultName(vaultName) .withJobParameters(jobParameters); InitiateJobResult response = client.initiateJob(request); return response.getJobId(); } private static long waitForJobToComplete(String jobId, String sqsQueueUrl) throws InterruptedException, JsonParseException, IOException { Boolean messageFound = false; Boolean jobSuccessful = false; long archiveSizeInBytes = -1; ObjectMapper mapper = new ObjectMapper(); JsonFactory factory = mapper.getFactory(); while (!messageFound) { List msgs = sqsClient.receiveMessage( new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(10)).getMessages(); if (msgs.size() > 0) { for (Message m : msgs) { JsonParser jpMessage = factory.createJsonParser(m.getBody()); JsonNode jobMessageNode = mapper.readTree(jpMessage); String jobMessage = jobMessageNode.get("Message").textValue(); JsonParser jpDesc = factory.createJsonParser(jobMessage); JsonNode jobDescNode = mapper.readTree(jpDesc); String retrievedJobId = jobDescNode.get("JobId").textValue(); String statusCode = jobDescNode.get("StatusCode").textValue(); archiveSizeInBytes = jobDescNode.get("ArchiveSizeInBytes").longValue(); if (retrievedJobId.equals(jobId)) { messageFound = true; if (statusCode.equals("Succeeded")) { jobSuccessful = true; } } } } else { Thread.sleep(sleepTime * 1000); } } return (messageFound && jobSuccessful) ? archiveSizeInBytes : -1; } private static void downloadJobOutput(String jobId, long archiveSizeInBytes) throws IOException { if (archiveSizeInBytes < 0) { System.err.println("Nothing to download."); return; } System.out.println("archiveSizeInBytes: " + archiveSizeInBytes); FileOutputStream fstream = new FileOutputStream(fileName); long startRange = 0; long endRange = (downloadChunkSize > archiveSizeInBytes) ? archiveSizeInBytes -1 : downloadChunkSize - 1; do { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() .withVaultName(vaultName) .withRange("bytes=" + startRange + "-" + endRange) .withJobId(jobId); GetJobOutputResult getJobOutputResult = client.getJobOutput(getJobOutputRequest); BufferedInputStream is = new BufferedInputStream(getJobOutputResult.getBody()); byte[] buffer = new byte[(int)(endRange - startRange + 1)]; System.out.println("Checksum received: " + getJobOutputResult.getChecksum()); System.out.println("Content range " + getJobOutputResult.getContentRange()); int totalRead = 0; while (totalRead < buffer.length) { int bytesRemaining = buffer.length - totalRead; int read = is.read(buffer, totalRead, bytesRemaining); if (read > 0) { totalRead = totalRead + read; } else { break; } } System.out.println("Calculated checksum: " + TreeHashGenerator.calculateTreeHash(new ByteArrayInputStream(buffer))); System.out.println("read = " + totalRead); fstream.write(buffer); startRange = startRange + (long)totalRead; endRange = ((endRange + downloadChunkSize) > archiveSizeInBytes) ? archiveSizeInBytes : (endRange + downloadChunkSize); is.close(); } while (endRange <= archiveSizeInBytes && startRange < archiveSizeInBytes); fstream.close(); System.out.println("Retrieved file to " + fileName); } private static void cleanUp() { snsClient.unsubscribe(new UnsubscribeRequest(snsSubscriptionARN)); snsClient.deleteTopic(new DeleteTopicRequest(snsTopicARN)); sqsClient.deleteQueue(new DeleteQueueRequest(sqsQueueURL)); } } ``` # Downloading an Archive in Amazon Glacier Using the Amazon SDK for .NET Both the [high-level and low-level APIs](using-aws-sdk.md) provided by the Amazon SDK for .NET provide a method to download an archive. **Topics** + [ ## Downloading an Archive Using the High-Level API of the Amazon SDK for .NET ](#downloading-an-archive-using-dotnet-highlevel-api) + [ ## Downloading an Archive Using the Low-Level API of the Amazon SDK for .NET ](#downloading-an-archive-using-dotnet-lowlevel-api) ## Downloading an Archive Using the High-Level API of the Amazon SDK for .NET The `ArchiveTransferManager` class of the high-level API provides the `Download` method you can use to download an archive. **Important** The `ArchiveTransferManager` class creates an Amazon Simple Notification Service (Amazon SNS) topic, and an Amazon Simple Queue Service (Amazon SQS) queue that is subscribed to that topic. It then initiates the archive retrieval job and polls the queue for the archive to be available. When the archive is available, download begins. For information about retrieval times, see [Archive Retrieval Options](downloading-an-archive-two-steps.md#api-downloading-an-archive-two-steps-retrieval-options) ### Example: Downloading an Archive Using the High-Level API of the Amazon SDK for .NET The following C\$1 code example downloads an archive from a vault (`examplevault`) in the US West (Oregon) Region. For step-by-step instructions on how to run this example, see [Running Code Examples](using-aws-sdk-for-dot-net.md#setting-up-and-testing-sdk-dotnet). You need to update the code as shown with an existing archive ID and the local file path where you want to save the downloaded archive. ``` using System; using Amazon.Glacier; using Amazon.Glacier.Transfer; using Amazon.Runtime; namespace glacier.amazon.com.docsamples { class ArchiveDownloadHighLevel { static string vaultName = "examplevault"; static string archiveId = "*** Provide archive ID ***"; static string downloadFilePath = "*** Provide the file name and path to where to store the download ***"; public static void Main(string[] args) { try { var manager = new ArchiveTransferManager(Amazon.RegionEndpoint.USWest2); var options = new DownloadOptions(); options.StreamTransferProgress += ArchiveDownloadHighLevel.progress; // Download an archive. Console.WriteLine("Intiating the archive retrieval job and then polling SQS queue for the archive to be available."); Console.WriteLine("Once the archive is available, downloading will begin."); manager.Download(vaultName, archiveId, downloadFilePath, options); Console.WriteLine("To continue, press Enter"); Console.ReadKey(); } catch (AmazonGlacierException e) { Console.WriteLine(e.Message); } catch (AmazonServiceException e) { Console.WriteLine(e.Message); } catch (Exception e) { Console.WriteLine(e.Message); } Console.WriteLine("To continue, press Enter"); Console.ReadKey(); } static int currentPercentage = -1; static void progress(object sender, StreamTransferProgressArgs args) { if (args.PercentDone != currentPercentage) { currentPercentage = args.PercentDone; Console.WriteLine("Downloaded {0}%", args.PercentDone); } } } } ``` ## Downloading an Archive Using the Low-Level API of the Amazon SDK for .NET The following are the steps for downloading an Amazon Glacier (Amazon Glacier) archive using the low-level API of the Amazon SDK for .NET. 1. Create an instance of the `AmazonGlacierClient` class (the client). You need to specify an Amazon Region from where you want to download the archive. All operations you perform using this client apply to that Amazon Region. 1. Initiate an `archive-retrieval` job by executing the `InitiateJob` method. You provide job information, such as the archive ID of the archive you want to download and the optional Amazon SNS topic to which you want Amazon Glacier to post a job completion message, by creating an instance of the `InitiateJobRequest` class. Amazon Glacier returns a job ID in response. The response is available in an instance of the `InitiateJobResponse` class. ``` AmazonGlacierClient client; client = new AmazonGlacierClient(Amazon.RegionEndpoint.USWest2); InitiateJobRequest initJobRequest = new InitiateJobRequest() { VaultName = vaultName, JobParameters = new JobParameters() { Type = "archive-retrieval", ArchiveId = "*** Provide archive id ***", SNSTopic = "*** Provide Amazon SNS topic ARN ***", } }; InitiateJobResponse initJobResponse = client.InitiateJob(initJobRequest); string jobId = initJobResponse.JobId; ``` You can optionally specify a byte range to request Amazon Glacier to prepare only a portion of the archive as shown in the following request. The request specifies Amazon Glacier to prepare only the 1 MB to 2 MB portion of the archive. ``` AmazonGlacierClient client; client = new AmazonGlacierClient(Amazon.RegionEndpoint.USWest2); InitiateJobRequest initJobRequest = new InitiateJobRequest() { VaultName = vaultName, JobParameters = new JobParameters() { Type = "archive-retrieval", ArchiveId = "*** Provide archive id ***", SNSTopic = "*** Provide Amazon SNS topic ARN ***", } }; // Specify byte range. int ONE_MEG = 1048576; initJobRequest.JobParameters.RetrievalByteRange = string.Format("{0}-{1}", ONE_MEG, 2 * ONE_MEG -1); InitiateJobResponse initJobResponse = client.InitiateJob(initJobRequest); string jobId = initJobResponse.JobId; ``` 1. Wait for the job to complete. You must wait until the job output is ready for you to download. If you have either set a notification configuration on the vault identifying an Amazon Simple Notification Service (Amazon SNS) topic or specified an Amazon SNS topic when you initiated a job, Amazon Glacier sends a message to that topic after it completes the job. The code example given in the following section uses Amazon SNS for Amazon Glacier to publish a message. You can also poll Amazon Glacier by calling the `DescribeJob` method to determine the job completion status. Although, using an Amazon SNS topic for notification is the recommended approach . 1. Download the job output (archive data) by executing the `GetJobOutput` method. You provide the request information such as the job ID and vault name by creating an instance of the `GetJobOutputRequest` class. The output that Amazon Glacier returns is available in the `GetJobOutputResponse` object. ``` GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() { JobId = jobId, VaultName = vaultName }; GetJobOutputResponse getJobOutputResponse = client.GetJobOutput(getJobOutputRequest); using (Stream webStream = getJobOutputResponse.Body) { using (Stream fileToSave = File.OpenWrite(fileName)) { CopyStream(webStream, fileToSave); } } ``` The preceding code snippet downloads the entire job output. You can optionally retrieve only a portion of the output, or download the entire output in smaller chunks by specifying the byte range in your `GetJobOutputRequest`. ``` GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() { JobId = jobId, VaultName = vaultName }; getJobOutputRequest.SetRange(0, 1048575); // Download only the first 1 MB chunk of the output. ``` In response to your `GetJobOutput` call, Amazon Glacier returns the checksum of the portion of the data you downloaded, if certain conditions are met. For more information, see [Receiving Checksums When Downloading Data](checksum-calculations-range.md). To verify there are no errors in the download, you can then compute the checksum on the client-side and compare it with the checksum Amazon Glacier sent in the response. For an archive retrieval job with the optional range specified, when you get the job description, it includes the checksum of the range you are retrieving (SHA256TreeHash).You can use this value to further verify the accuracy of the entire byte range that you later download. For example, if you initiate a job to retrieve a tree-hash aligned archive range and then download output in chunks such that each of your `GetJobOutput` requests return a checksum, then you can compute checksum of each portion you download on the client-side and then compute the tree hash. You can compare it with the checksum Amazon Glacier returns in response to your describe job request to verify that the entire byte range you have downloaded is the same as the byte range that is stored in Amazon Glacier. For a working example, see [Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for .NET—Download Output in Chunks](#creating-vaults-sdk-dotnet-example2). ### Example 1: Retrieving an Archive Using the Low-Level API of the Amazon SDK for .NET The following C\$1 code example downloads an archive from the specified vault. After the job completes, the example downloads the entire output in a single `GetJobOutput` call. For an example of downloading output in chunks, see [Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for .NET—Download Output in Chunks](#creating-vaults-sdk-dotnet-example2). The example performs the following tasks: + Sets up an Amazon Simple Notification Service (Amazon SNS) topic Amazon Glacier sends a notification to this topic after it completes the job. + Sets up an Amazon Simple Queue Service (Amazon SQS) queue. The example attaches a policy to the queue to enable the Amazon SNS topic to post messages. + Initiates a job to download the specified archive. In the job request, the example specifies the Amazon SNS topic so that Amazon Glacier can send a message after it completes the job. + Periodically checks the Amazon SQS queue for a message. If there is a message, parse the JSON and check if the job completed successfully. If it did, download the archive. The code example uses the JSON.NET library (see [JSON.NET](http://json.codeplex.com/)) to parse the JSON. + Cleans up by deleting the Amazon SNS topic and the Amazon SQS queue it created. ``` using System; using System.Collections.Generic; using System.IO; using System.Threading; using Amazon.Glacier; using Amazon.Glacier.Model; using Amazon.Runtime; using Amazon.SimpleNotificationService; using Amazon.SimpleNotificationService.Model; using Amazon.SQS; using Amazon.SQS.Model; using Newtonsoft.Json; namespace glacier.amazon.com.docsamples { class ArchiveDownloadLowLevelUsingSNSSQS { static string topicArn; static string queueUrl; static string queueArn; static string vaultName = "*** Provide vault name ***"; static string archiveID = "*** Provide archive ID ***"; static string fileName = "*** Provide the file name and path to where to store downloaded archive ***"; static AmazonSimpleNotificationServiceClient snsClient; static AmazonSQSClient sqsClient; const string SQS_POLICY = "{" + " \"Version\" : \"2012-10-17\",&TCX5-2025-waiver;" + " \"Statement\" : [" + " {" + " \"Sid\" : \"sns-rule\"," + " \"Effect\" : \"Allow\"," + " \"Principal\" : {\"Service\" : \"sns.amazonaws.com\" }," + " \"Action\" : \"sqs:SendMessage\"," + " \"Resource\" : \"{QueueArn}\"," + " \"Condition\" : {" + " \"ArnLike\" : {" + " \"aws:SourceArn\" : \"{TopicArn}\"" + " }" + " }" + " }" + " ]" + "}"; public static void Main(string[] args) { AmazonGlacierClient client; try { using (client = new AmazonGlacierClient(Amazon.RegionEndpoint.USWest2)) { Console.WriteLine("Setup SNS topic and SQS queue."); SetupTopicAndQueue(); Console.WriteLine("To continue, press Enter"); Console.ReadKey(); Console.WriteLine("Retrieving..."); RetrieveArchive(client); } Console.WriteLine("Operations successful. To continue, press Enter"); Console.ReadKey(); } catch (AmazonGlacierException e) { Console.WriteLine(e.Message); } catch (AmazonServiceException e) { Console.WriteLine(e.Message); } catch (Exception e) { Console.WriteLine(e.Message); } finally { // Delete SNS topic and SQS queue. snsClient.DeleteTopic(new DeleteTopicRequest() { TopicArn = topicArn }); sqsClient.DeleteQueue(new DeleteQueueRequest() { QueueUrl = queueUrl }); } } static void SetupTopicAndQueue() { snsClient = new AmazonSimpleNotificationServiceClient(Amazon.RegionEndpoint.USWest2); sqsClient = new AmazonSQSClient(Amazon.RegionEndpoint.USWest2); long ticks = DateTime.Now.Ticks; topicArn = snsClient.CreateTopic(new CreateTopicRequest { Name = "GlacierDownload-" + ticks }).TopicArn; Console.Write("topicArn: "); Console.WriteLine(topicArn); CreateQueueRequest createQueueRequest = new CreateQueueRequest(); createQueueRequest.QueueName = "GlacierDownload-" + ticks; CreateQueueResponse createQueueResponse = sqsClient.CreateQueue(createQueueRequest); queueUrl = createQueueResponse.QueueUrl; Console.Write("QueueURL: "); Console.WriteLine(queueUrl); GetQueueAttributesRequest getQueueAttributesRequest = new GetQueueAttributesRequest(); getQueueAttributesRequest.AttributeNames = new List { "QueueArn" }; getQueueAttributesRequest.QueueUrl = queueUrl; GetQueueAttributesResponse response = sqsClient.GetQueueAttributes(getQueueAttributesRequest); queueArn = response.QueueARN; Console.Write("QueueArn: "); Console.WriteLine(queueArn); // Setup the Amazon SNS topic to publish to the SQS queue. snsClient.Subscribe(new SubscribeRequest() { Protocol = "sqs", Endpoint = queueArn, TopicArn = topicArn }); // Add policy to the queue so SNS can send messages to the queue. var policy = SQS_POLICY.Replace("{TopicArn}", topicArn).Replace("{QueueArn}", queueArn); sqsClient.SetQueueAttributes(new SetQueueAttributesRequest() { QueueUrl = queueUrl, Attributes = new Dictionary { { QueueAttributeName.Policy, policy } } }); } static void RetrieveArchive(AmazonGlacierClient client) { // Initiate job. InitiateJobRequest initJobRequest = new InitiateJobRequest() { VaultName = vaultName, JobParameters = new JobParameters() { Type = "archive-retrieval", ArchiveId = archiveID, Description = "This job is to download archive.", SNSTopic = topicArn, } }; InitiateJobResponse initJobResponse = client.InitiateJob(initJobRequest); string jobId = initJobResponse.JobId; // Check queue for a message and if job completed successfully, download archive. ProcessQueue(jobId, client); } private static void ProcessQueue(string jobId, AmazonGlacierClient client) { ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest() { QueueUrl = queueUrl, MaxNumberOfMessages = 1 }; bool jobDone = false; while (!jobDone) { Console.WriteLine("Poll SQS queue"); ReceiveMessageResponse receiveMessageResponse = sqsClient.ReceiveMessage(receiveMessageRequest); if (receiveMessageResponse.Messages.Count == 0) { Thread.Sleep(10000 * 60); continue; } Console.WriteLine("Got message"); Message message = receiveMessageResponse.Messages[0]; Dictionary outerLayer = JsonConvert.DeserializeObject>(message.Body); Dictionary fields = JsonConvert.DeserializeObject>(outerLayer["Message"]); string statusCode = fields["StatusCode"] as string; if (string.Equals(statusCode, GlacierUtils.JOB_STATUS_SUCCEEDED, StringComparison.InvariantCultureIgnoreCase)) { Console.WriteLine("Downloading job output"); DownloadOutput(jobId, client); // Save job output to the specified file location. } else if (string.Equals(statusCode, GlacierUtils.JOB_STATUS_FAILED, StringComparison.InvariantCultureIgnoreCase)) Console.WriteLine("Job failed... cannot download the archive."); jobDone = true; sqsClient.DeleteMessage(new DeleteMessageRequest() { QueueUrl = queueUrl, ReceiptHandle = message.ReceiptHandle }); } } private static void DownloadOutput(string jobId, AmazonGlacierClient client) { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() { JobId = jobId, VaultName = vaultName }; GetJobOutputResponse getJobOutputResponse = client.GetJobOutput(getJobOutputRequest); using (Stream webStream = getJobOutputResponse.Body) { using (Stream fileToSave = File.OpenWrite(fileName)) { CopyStream(webStream, fileToSave); } } } public static void CopyStream(Stream input, Stream output) { byte[] buffer = new byte[65536]; int length; while ((length = input.Read(buffer, 0, buffer.Length)) > 0) { output.Write(buffer, 0, length); } } } } ``` ### Example 2: Retrieving an Archive Using the Low-Level API of the Amazon SDK for .NET—Download Output in Chunks The following C\$1 code example retrieves an archive from Amazon Glacier. The code example downloads the job output in chunks by specifying the byte range in a `GetJobOutputRequest` object. ``` using System; using System.Collections.Generic; using System.IO; using System.Threading; using Amazon.Glacier; using Amazon.Glacier.Model; using Amazon.Glacier.Transfer; using Amazon.Runtime; using Amazon.SimpleNotificationService; using Amazon.SimpleNotificationService.Model; using Amazon.SQS; using Amazon.SQS.Model; using Newtonsoft.Json; using System.Collections.Specialized; namespace glacier.amazon.com.docsamples { class ArchiveDownloadLowLevelUsingSQLSNSOutputUsingRange { static string topicArn; static string queueUrl; static string queueArn; static string vaultName = "*** Provide vault name ***"; static string archiveId = "*** Provide archive ID ***"; static string fileName = "*** Provide the file name and path to where to store downloaded archive ***"; static AmazonSimpleNotificationServiceClient snsClient; static AmazonSQSClient sqsClient; const string SQS_POLICY = "{" + " \"Version\" : \"2012-10-17\",&TCX5-2025-waiver;" + " \"Statement\" : [" + " {" + " \"Sid\" : \"sns-rule\"," + " \"Effect\" : \"Allow\"," + " \"Principal\" : {\"AWS\" : \"arn:aws:iam::123456789012:root\" }," + " \"Action\" : \"sqs:SendMessage\"," + " \"Resource\" : \"{QuernArn}\"," + " \"Condition\" : {" + " \"ArnLike\" : {" + " \"aws:SourceArn\" : \"{TopicArn}\"" + " }" + " }" + " }" + " ]" + "}"; public static void Main(string[] args) { AmazonGlacierClient client; try { using (client = new AmazonGlacierClient(Amazon.RegionEndpoint.USWest2)) { Console.WriteLine("Setup SNS topic and SQS queue."); SetupTopicAndQueue(); Console.WriteLine("To continue, press Enter"); Console.ReadKey(); Console.WriteLine("Download archive"); DownloadAnArchive(archiveId, client); } Console.WriteLine("Operations successful. To continue, press Enter"); Console.ReadKey(); } catch (AmazonGlacierException e) { Console.WriteLine(e.Message); } catch (AmazonServiceException e) { Console.WriteLine(e.Message); } catch (Exception e) { Console.WriteLine(e.Message); } finally { // Delete SNS topic and SQS queue. snsClient.DeleteTopic(new DeleteTopicRequest() { TopicArn = topicArn }); sqsClient.DeleteQueue(new DeleteQueueRequest() { QueueUrl = queueUrl }); } } static void SetupTopicAndQueue() { long ticks = DateTime.Now.Ticks; // Setup SNS topic. snsClient = new AmazonSimpleNotificationServiceClient(Amazon.RegionEndpoint.USWest2); sqsClient = new AmazonSQSClient(Amazon.RegionEndpoint.USWest2); topicArn = snsClient.CreateTopic(new CreateTopicRequest { Name = "GlacierDownload-" + ticks }).TopicArn; Console.Write("topicArn: "); Console.WriteLine(topicArn); CreateQueueRequest createQueueRequest = new CreateQueueRequest(); createQueueRequest.QueueName = "GlacierDownload-" + ticks; CreateQueueResponse createQueueResponse = sqsClient.CreateQueue(createQueueRequest); queueUrl = createQueueResponse.QueueUrl; Console.Write("QueueURL: "); Console.WriteLine(queueUrl); GetQueueAttributesRequest getQueueAttributesRequest = new GetQueueAttributesRequest(); getQueueAttributesRequest.AttributeNames = new List { "QueueArn" }; getQueueAttributesRequest.QueueUrl = queueUrl; GetQueueAttributesResponse response = sqsClient.GetQueueAttributes(getQueueAttributesRequest); queueArn = response.QueueARN; Console.Write("QueueArn: "); Console.WriteLine(queueArn); // Setup the Amazon SNS topic to publish to the SQS queue. snsClient.Subscribe(new SubscribeRequest() { Protocol = "sqs", Endpoint = queueArn, TopicArn = topicArn }); // Add the policy to the queue so SNS can send messages to the queue. var policy = SQS_POLICY.Replace("{TopicArn}", topicArn).Replace("{QuernArn}", queueArn); sqsClient.SetQueueAttributes(new SetQueueAttributesRequest() { QueueUrl = queueUrl, Attributes = new Dictionary { { QueueAttributeName.Policy, policy } } }); } static void DownloadAnArchive(string archiveId, AmazonGlacierClient client) { // Initiate job. InitiateJobRequest initJobRequest = new InitiateJobRequest() { VaultName = vaultName, JobParameters = new JobParameters() { Type = "archive-retrieval", ArchiveId = archiveId, Description = "This job is to download the archive.", SNSTopic = topicArn, } }; InitiateJobResponse initJobResponse = client.InitiateJob(initJobRequest); string jobId = initJobResponse.JobId; // Check queue for a message and if job completed successfully, download archive. ProcessQueue(jobId, client); } private static void ProcessQueue(string jobId, AmazonGlacierClient client) { var receiveMessageRequest = new ReceiveMessageRequest() { QueueUrl = queueUrl, MaxNumberOfMessages = 1 }; bool jobDone = false; while (!jobDone) { Console.WriteLine("Poll SQS queue"); ReceiveMessageResponse receiveMessageResponse = sqsClient.ReceiveMessage(receiveMessageRequest); if (receiveMessageResponse.Messages.Count == 0) { Thread.Sleep(10000 * 60); continue; } Console.WriteLine("Got message"); Message message = receiveMessageResponse.Messages[0]; Dictionary outerLayer = JsonConvert.DeserializeObject>(message.Body); Dictionary fields = JsonConvert.DeserializeObject>(outerLayer["Message"]); string statusCode = fields["StatusCode"] as string; if (string.Equals(statusCode, GlacierUtils.JOB_STATUS_SUCCEEDED, StringComparison.InvariantCultureIgnoreCase)) { long archiveSize = Convert.ToInt64(fields["ArchiveSizeInBytes"]); Console.WriteLine("Downloading job output"); DownloadOutput(jobId, archiveSize, client); // This where we save job output to the specified file location. } else if (string.Equals(statusCode, GlacierUtils.JOB_STATUS_FAILED, StringComparison.InvariantCultureIgnoreCase)) Console.WriteLine("Job failed... cannot download the archive."); jobDone = true; sqsClient.DeleteMessage(new DeleteMessageRequest() { QueueUrl = queueUrl, ReceiptHandle = message.ReceiptHandle }); } } private static void DownloadOutput(string jobId, long archiveSize, AmazonGlacierClient client) { long partSize = 4 * (long)Math.Pow(2, 20); // 4 MB. using (Stream fileToSave = new FileStream(fileName, FileMode.Create, FileAccess.Write)) { long currentPosition = 0; do { GetJobOutputRequest getJobOutputRequest = new GetJobOutputRequest() { JobId = jobId, VaultName = vaultName }; long endPosition = currentPosition + partSize - 1; if (endPosition > archiveSize) endPosition = archiveSize; getJobOutputRequest.SetRange(currentPosition, endPosition); GetJobOutputResponse getJobOutputResponse = client.GetJobOutput(getJobOutputRequest); using (Stream webStream = getJobOutputResponse.Body) { CopyStream(webStream, fileToSave); } currentPosition += partSize; } while (currentPosition < archiveSize); } } public static void CopyStream(Stream input, Stream output) { byte[] buffer = new byte[65536]; int length; while ((length = input.Read(buffer, 0, buffer.Length)) > 0) { output.Write(buffer, 0, length); } } } } ``` # Downloading a Large Archive Using Parallel Processing with Python This topic describes how to download a large archive from Amazon S3 Glacier (S3 Glacier) using parallel processing with Python. This approach allows you to reliably download archives of any size by breaking them into smaller pieces that can be processed independently. ## Overview The Python script provided in this example performs the following tasks: 1. Sets up the necessary Amazon resources (Amazon SNS topic and Amazon SQS queues) for notifications 1. Initiates an archive retrieval job with Amazon Glacier 1. Monitors an Amazon SQS queue for job completion notifications 1. Splits the large archive into manageable chunks 1. Downloads chunks in parallel using multiple worker threads 1. Saves each chunk to disk for later reassembly ## Prerequisites Before you begin, make sure you have: + Python 3.6 or later installed + Amazon SDK for Python (Boto3) installed + Amazon credentials configured with appropriate permissions for Amazon Glacier, Amazon SNS, and Amazon SQS + Sufficient disk space to store the downloaded archive chunks ## Example: Downloading an Archive Using Parallel Processing with Python The following Python script demonstrates how to download a large archive from Amazon Glacier using parallel processing: ``` import boto3 import time import json import jmespath import re import concurrent.futures import os output_file_path = "output_directory_path" vault_name = "vault_name" chunk_size = 1000000000 #1gb - size of chunks for parallel download. notify_queue_name = 'GlacierJobCompleteNotifyQueue' # SQS queue for Glacier recall notification chunk_download_queue_name='GlacierChunkReadyNotifyQueue' # SQS queue for chunks sns_topic_name = 'GlacierRecallJobCompleted' # the SNS topic to be notified when Glacier archive is restored. chunk_queue_visibility_timeout = 7200 # 2 hours - this may need to be adjusted. region = 'us-east-1' archive_id = "archive_id_to_restore" retrieve_archive = True # set to false if you do not want to restore from Glacier - useful for restarting or parallel processing of the chunk queue. workers = 12 # the number of parallel worker threads for downloading chunks. def setup_queues_and_topic(): sqs = boto3.client('sqs') sns = boto3.client('sns') # Create the SNS topic topic_response = sns.create_topic( Name=sns_topic_name ) topic_arn = topic_response['TopicArn'] print("Creating the SNS topic " + topic_arn) # Create the notification queue notify_queue_response = sqs.create_queue( QueueName=notify_queue_name, Attributes={ 'VisibilityTimeout': '300', # 5 minutes 'ReceiveMessageWaitTimeSeconds': '20' # Enable long polling } ) notify_queue_url = notify_queue_response['QueueUrl'] print("Creating the archive-retrieval notification queue " + notify_queue_url) # Create the chunk download queue chunk_queue_response = sqs.create_queue( QueueName=chunk_download_queue_name, Attributes={ 'VisibilityTimeout': str(chunk_queue_visibility_timeout), # 5 minutes 'ReceiveMessageWaitTimeSeconds': '0' } ) chunk_queue_url = chunk_queue_response['QueueUrl'] print("Creating the chunk ready notification queue " + chunk_queue_url) # Get the ARN for the notification queue notify_queue_attributes = sqs.get_queue_attributes( QueueUrl=notify_queue_url, AttributeNames=['QueueArn'] ) notify_queue_arn = notify_queue_attributes['Attributes']['QueueArn'] # Set up the SNS topic policy on the notification queue queue_policy = { "Version": "2012-10-17", "Statement": [{ "Sid": "allow-sns-messages", "Effect": "Allow", "Principal": {"AWS": "*"}, "Action": "SQS:SendMessage", "Resource": notify_queue_arn, "Condition": { "ArnEquals": { "aws:SourceArn": topic_arn } } }] } # Set the queue policy sqs.set_queue_attributes( QueueUrl=notify_queue_url, Attributes={ 'Policy': json.dumps(queue_policy) } ) # Subscribe the notification queue to the SNS topic sns.subscribe( TopicArn=topic_arn, Protocol='sqs', Endpoint=notify_queue_arn ) return { 'topic_arn': topic_arn, 'notify_queue_url': notify_queue_url, 'chunk_queue_url': chunk_queue_url } def split_and_send_chunks(archive_size, job_id,chunk_queue_url): ranges = [] current = 0 chunk_number = 0 while current < archive_size: chunk_number += 1 next_range = min(current + chunk_size - 1, archive_size - 1) ranges.append((current, next_range, chunk_number)) current = next_range + 1 # Send messages to SQS queue for start, end, chunk_number in ranges: body = {"start": start, "end": end, "job_id": job_id, "chunk_number": chunk_number} body = json.dumps(body) print("Sending SQS message for range:" + str(body)) response = sqs.send_message( QueueUrl=chunk_queue_url, MessageBody=str(body) ) def GetJobOutputChunks(job_id, byterange, chunk_number): glacier = boto3.client('glacier') response = glacier.get_job_output( vaultName=vault_name, jobId=job_id, range=byterange, ) with open(os.path.join(output_file_path,str(chunk_number)+".chunk"), 'wb') as output_file: output_file.write(response['body'].read()) return response def ReceiveArchiveReadyMessages(notify_queue_url,chunk_queue_url): response = sqs.receive_message( QueueUrl=notify_queue_url, AttributeNames=['All'], MaxNumberOfMessages=1, WaitTimeSeconds=20, MessageAttributeNames=['Message'] ) print("Polling archive retrieval job ready queue...") # Checking that there is a Messages key before proceeding. No 'Messages' key likely means the queue is empty if 'Messages' in response: print("Received a message from the archive retrieval job queue") jsonresponse = response # Loading the string into JSON and checking that ArchiveSizeInBytes key is present before continuing. jsonresponse=json.loads(jsonresponse['Messages'][0]['Body']) jsonresponse=json.loads(jsonresponse['Message']) if 'ArchiveSizeInBytes' in jsonresponse: receipt_handle = response['Messages'][0]['ReceiptHandle'] if jsonresponse['ArchiveSizeInBytes']: archive_size = jsonresponse['ArchiveSizeInBytes'] print(f'Received message: {response}') if archive_size > chunk_size: split_and_send_chunks(archive_size, jsonresponse['JobId'],chunk_queue_url) sqs.delete_message( QueueUrl=notify_queue_url, ReceiptHandle=receipt_handle) else: print("No ArchiveSizeInBytes value found in message") print(response) else: print('No messages available in the queue at this time.') time.sleep(1) def ReceiveArchiveChunkMessages(chunk_queue_url): response = sqs.receive_message( QueueUrl=chunk_queue_url, AttributeNames=['All'], MaxNumberOfMessages=1, WaitTimeSeconds=0, MessageAttributeNames=['Message'] ) print("Polling archive chunk queue...") print(response) # Checking that there is a Messages key before proceeding. No 'Messages' key likely means the queue is empty if 'Messages' in response: jsonresponse = response # Loading the string into JSON and checking that ArchiveSizeInBytes key is present before continuing. jsonresponse=json.loads(jsonresponse['Messages'][0]['Body']) if 'job_id' in jsonresponse: #checking that there is a job id before continuing job_id = jsonresponse['job_id'] byterange = "bytes="+str(jsonresponse['start']) + '-' + str(jsonresponse['end']) chunk_number = jsonresponse['chunk_number'] receipt_handle = response['Messages'][0]['ReceiptHandle'] if jsonresponse['job_id']: print(f'Received message: {response}') GetJobOutputChunks(job_id,byterange,chunk_number) sqs.delete_message( QueueUrl=chunk_queue_url, ReceiptHandle=receipt_handle) else: print('No messages available in the chunk queue at this time.') def initiate_archive_retrieval(archive_id, topic_arn): glacier = boto3.client('glacier') job_parameters = { "Type": "archive-retrieval", "ArchiveId": archive_id, "Description": "Archive retrieval job", "SNSTopic": topic_arn, "Tier": "Bulk" # You can change this to "Standard" or "Expedited" based on your needs } try: response = glacier.initiate_job( vaultName=vault_name, jobParameters=job_parameters ) print("Archive retrieval job initiated:") print(f"Job ID: {response['jobId']}") print(f"Job parameters: {job_parameters}") print(f"Complete response: {json.dumps(response, indent=2)}") return response['jobId'] except Exception as e: print(f"Error initiating archive retrieval job: {str(e)}") raise def run_async_tasks(chunk_queue_url, workers): max_workers = workers # Set the desired maximum number of concurrent tasks with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor: for _ in range(max_workers): executor.submit(ReceiveArchiveChunkMessages, chunk_queue_url) # One time setup of the necessary queues and topics. queue_and_topic_atts = setup_queues_and_topic() topic_arn = queue_and_topic_atts['topic_arn'] notify_queue_url = queue_and_topic_atts['notify_queue_url'] chunk_queue_url = queue_and_topic_atts['chunk_queue_url'] if retrieve_archive: print("Retrieving the defined archive... The topic arn we will notify when recalling the archive is: "+topic_arn) job_id = initiate_archive_retrieval(archive_id, topic_arn) else: print("Retrieve archive is false, polling queues and downloading only.") while True: ReceiveArchiveReadyMessages(notify_queue_url,chunk_queue_url) run_async_tasks(chunk_queue_url,workers) ``` ## Using the Script To use this script, follow these steps: 1. Replace the placeholder values in the script with your specific information: + *output\$1file\$1path*: Directory where chunk files will be saved + *vault\$1name*: Name of your S3 Glacier vault + *notify\$1queue\$1name*: Name for the job notification queue + *chunk\$1download\$1queue\$1name*: Name for the chunk download queue + *sns\$1topic\$1name*: Name for the SNS topic + *region*: Amazon region where your vault is located + *archive\$1id*: ID of the archive to retrieve 1. Run the script: ``` python download_large_archive.py ``` 1. After all chunks are downloaded, you can combine them into a single file using a command like: ``` cat /path/to/chunks/*.chunk > complete_archive.file ``` ## Important Considerations When using this script, keep the following in mind: + Archive retrieval from S3 Glacier can take several hours to complete, depending on the retrieval tier selected. + The script runs indefinitely, continuously polling the queues. You may want to add a termination condition based on your specific requirements. + Ensure you have sufficient disk space to store all the chunks of your archive. + If the script is interrupted, you can restart it with `retrieve_archive=False` to continue downloading chunks without initiating a new retrieval job. + Adjust the *chunk\$1size* and *workers* parameters based on your network bandwidth and system resources. + Standard Amazon charges apply for Amazon S3 retrievals, Amazon SNS, and Amazon SQS usage. # Downloading an Archive by Using the REST API **To download an archive by using the REST API** Downloading an archive is a two-step process. 1. Initiate a job of the `archive-retrieval` type. For more information, see [Initiate Job (POST jobs)](api-initiate-job-post.md). 1. After the job is completed, download the archive data. For more information, see [Get Job Output (GET output)](api-job-output-get.md). # Downloading an Archive in Amazon Glacier Using the Amazon CLI You can download archives in Amazon Glacier (Amazon Glacier) using the Amazon Command Line Interface (Amazon CLI). **Topics** + [ ## (Prerequisite) Setting Up the Amazon CLI ](#Creating-Vaults-CLI-Setup) + [ ## Example: Download an Archive Using the Amazon CLI ](#Downloading-Archives-CLI-Implementation) ## (Prerequisite) Setting Up the Amazon CLI 1. Download and configure the Amazon CLI. For instructions, see the following topics in the *Amazon Command Line Interface User Guide*: [Installing the Amazon Command Line Interface](https://docs.amazonaws.cn/cli/latest/userguide/installing.html) [Configuring the Amazon Command Line Interface](https://docs.amazonaws.cn/cli/latest/userguide/cli-chap-getting-started.html) 1. Verify your Amazon CLI setup by entering the following commands at the command prompt. These commands don't provide credentials explicitly, so the credentials of the default profile are used. + Try using the help command. ``` aws help ``` + To get a list of Amazon Glacier vaults on the configured account, use the `list-vaults` command. Replace *123456789012* with your Amazon Web Services account ID. ``` aws glacier list-vaults --account-id 123456789012 ``` + To see the current configuration data for the Amazon CLI, use the `aws configure list` command. ``` aws configure list ``` ## Example: Download an Archive Using the Amazon CLI **Note** In order to download your archives you must know your archive ids. Steps 1-4 will retrieve your archive ids. If you already know the archive ids you wish to download skip to step 5. 1. Use the `initiate-job` command to start an inventory-retrieval job. The inventory report will list your archive ids. ``` aws glacier initiate-job --vault-name awsexamplevault --account-id 111122223333 --job-parameters="{\"Type\":\"inventory-retrieval\"}" ``` Expected output: ``` { "location": "/111122223333/vaults/awsexamplevault/jobs/*** jobid ***", "jobId": "*** jobid ***" } ``` 1. Use the `describe-job` command to check status of the previous `` job command. ``` aws glacier describe-job --vault-name awsexamplevault --account-id 111122223333 --job-id *** jobid *** ``` Expected output: ``` { "InventoryRetrievalParameters": { "Format": "JSON" }, "VaultARN": "*** vault arn ***", "Completed": false, "JobId": "*** jobid ***", "Action": "InventoryRetrieval", "CreationDate": "*** job creation date ***", "StatusCode": "InProgress" } ``` 1. Wait for the job to complete. You must wait until the job output is ready for you to download. If you set a notification configuration on the vault or specified an Amazon Simple Notification Service (Amazon SNS) topic when you initiated the job, Amazon Glacier sends a message to the topic after it completes the job. You can set notification configuration for specific events on the vault. For more information, see [Configuring Vault Notifications in Amazon Glacier](configuring-notifications.md). Amazon Glacier sends a message to the specified SNS topic anytime the specific event occurs. 1. When it's complete, use the `get-job-output` command to download the retrieval job to the file `output.json`. This file will contain your archive ids. ``` aws glacier get-job-output --vault-name awsexamplevault --account-id 111122223333 --job-id *** jobid *** output.json ``` This command produces a file with the following fields. ``` { "VaultARN":"arn:aws:glacier:region:111122223333:vaults/awsexamplevault", "InventoryDate":"*** job completion date ***", "ArchiveList":[ {"ArchiveId":"*** archiveid ***", "ArchiveDescription":*** archive description (if set) ***, "CreationDate":"*** archive creation date ***", "Size":"*** archive size (in bytes) ***", "SHA256TreeHash":"*** archive hash ***" } {"ArchiveId": ... ]} ``` 1. Use the `initiate-job` command to start the retrieval process each archive from a vault. You will need to specify the job parameter as `archive-retrieval` as seen below. ``` aws glacier initiate-job --vault-name awsexamplevault --account-id 111122223333 --job-parameters="{\"Type\":\"archive-retrieval\",\"ArchiveId\":\"*** archiveId ***\"}" ``` 1. Wait for the `archive-retrieval` job to complete. Use the `describe-job` command to check status of the previous command. ``` aws glacier describe-job --vault-name awsexamplevault --account-id 111122223333 --job-id *** jobid *** ``` 1. When the above job is complete use the `get-job-output` command to download your archive. ``` aws glacier get-job-output --vault-name awsexamplevault --account-id 111122223333 --job-id *** jobid *** output_file_name ```