Prerequisites for using Snow Family devices - Amazon Snowball Edge Developer Guide
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Prerequisites for using Snow Family devices

Before you get started with a Snow Family device, you need to sign up for an Amazon account if you don't have one. We also recommend learning how to configure your data and compute instances for use with Snow Family devices.

Amazon Snowball Edge is a region-specific service. So before you plan your job, be sure that the service is available in your Amazon Web Services Region. Ensure that your location and Amazon S3 bucket are within the same Amazon Web Services Region or the same country because it will impact your ability to order the device.

To use Amazon S3 compatible storage on Snow Family devices with compute optimized devices for local edge compute and storage jobs, you need to provision S3 capacity on the device or devices when you order. Amazon S3 compatible storage on Snow Family devices supports local bucket management, so you can create S3 buckets on the device or cluster after you receive the device or devices.

As part of the order process, you create an Amazon Identity and Access Management (IAM) role and an Amazon Key Management Service (Amazon KMS) key. The KMS key is used to encrypt the unlock code for your job. For more information about creating IAM roles and KMS keys, see Creating a job to order a Snow Family device.

Note

In the Asia Pacific (Mumbai) Amazon Web Services Region service is provided by Amazon on Internet Services Private Limited (AISPL). For information on signing up for Amazon Web Services in the Asia Pacific (Mumbai) Amazon Web Services Region, see Signing Up for AISPL.

Sign up for an Amazon Web Services account

If you do not have an Amazon Web Services account, use the following procedure to create one.

To sign up for Amazon Web Services
  1. Open http://www.amazonaws.cn/ and choose Sign Up.

  2. Follow the on-screen instructions.

Amazon sends you a confirmation email after the sign-up process is complete. At any time, you can view your current account activity and manage your account by going to http://www.amazonaws.cn/ and choosing My Account.

Secure IAM users

After you sign up for an Amazon Web Services account, safeguard your administrative user by turning on multi-factor authentication (MFA). For instructions, see Enable a virtual MFA device for an IAM user (console) in the IAM User Guide.

To give other users access to your Amazon Web Services account resources, create IAM users. To secure your IAM users, turn on MFA and only give the IAM users the permissions needed to perform their tasks.

For more information about creating and securing IAM users, see the following topics in the IAM User Guide:

Questions about the local environment

Understanding your dataset and how the local environment is set up will help you complete your data transfer. Consider the following before placing your order.

What data are you transferring?

Transferring a large number of small files does not work well with Amazon Snowball Edge. This is because Snowball Edge encrypts each individual object. Small files include files under 1 MB in size. We recommend that you zip them up before transferring them onto the Amazon Snowball Edge device. We also recommend that you have no more than 500,000 files or directories within each directory.

Will the data be accessed during the transfer?

It is important to have a static dataset, (that is, no users or systems are accessing the data during transfer). If not, the file transfer can fail due to a checksum mismatch. The files won't be transferred and the files will be marked as Failed.

To prevent corrupting your data, don't disconnect an Amazon Snowball Edge device or change its network settings while transferring data. Files should be in a static state while being written to the device. Files that are modified while they are being written to the device can result in read/write conflicts.

Will the network support Amazon Snowball data transfer?

Snowball Edge supports the RJ45, SFP+, or QSFP+ networking adapters. Verify that your switch is a gigabit switch. Depending on the brand of switch, it might say gigabit or 10/100/1000. Snowball Edge devices do not support a megabit switch, or 10/100 switch.

Working with filenames that contain special characters

It's important to note that if the names of your objects contain special characters, you might encounter errors. Although Amazon S3 allows special characters, we highly recommend that you avoid the following characters:

  • Backslash ("\")

  • Left curly brace ("{")

  • Right curly brace ("}")

  • Left square bracket ("[")

  • Right square bracket ("]")

  • 'Less Than' symbol ("<")

  • 'Greater Than' symbol (">")

  • Non-printable ASCII characters (128–255 decimal characters)

  • Caret ("^")

  • Percent character ("%")

  • Grave accent / back tick ("`")

  • Quotation marks

  • Tilde ("~")

  • 'Pound' character ("#")

  • Vertical bar / pipe ("|")

If your files have one or more of these characters in object names, rename the objects before you copy them to the Amazon Snowball Edge device. Windows users who have spaces in their file names should be careful when copying individual objects or running a recursive command. In commands, surround the names of objects that include spaces in the names with quotation marks. The following are examples of such files.

Operating system File name: test file.txt

Windows

“C:\Users\<username>\desktop\test file.txt”

iOS

/Users/<username>/test\ file.txt

Linux

/home/<username>/test\ file.txt

Note

The only object metadata that is transferred is the object name and size.

Amazon S3 encryption with Amazon KMS

You can use the default Amazon managed or customer managed encryption keys to protect your data when importing or exporting data.

Using Amazon S3 default bucket encryption with Amazon KMS managed keys

To enable Amazon managed encryption with Amazon KMS
  1. Open the Amazon S3 console at https://console.amazonaws.cn/s3/.

  2. Choose the Amazon S3 bucket that you want to encrypt.

  3. In the wizard that appears on the right side, choose Properties.

  4. In the Default encryption box, choose Disabled (this option is grayed out) to enable default encryption.

  5. Choose Amazon-KMS as the encryption method, and then choose the KMS key that you want to use. This key is used to encrypt objects that are PUT into the bucket.

  6. Choose Save.

After the Snowball Edge job is created, and before the data is imported, add a statement to the existing IAM role policy. This is the role you created during the ordering process. Depending on the job type, the default role name looks similar to Snowball-import-s3-only-role or Snowball-export-s3-only-role.

The following are examples of such a statement.

For importing data

If you use server-side encryption with Amazon KMS managed keys (SSE-KMS) to encrypt the Amazon S3 buckets associated with your import job, you also need to add the following statement to your IAM role.

Example Snowball import IAM role
{ "Effect": "Allow", "Action": [ "kms: GenerateDataKey", "kms: Decrypt" ], "Resource":"arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234-efgh-111111111111" }

For exporting data

If you use server-side encryption with Amazon KMS managed keys to encrypt the Amazon S3 buckets associated with your export job, you also must add the following statement to your IAM role.

Example Snowball export IAM role
{ "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource":"arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234-efgh-111111111111" }

Using S3 default bucket encryption with Amazon KMS customer keys

You can use the default Amazon S3 bucket encryption with your own KMS keys to protect data you are importing and exporting.

For importing data

To enable customer managed encryption with Amazon KMS
  1. Sign in to the Amazon Web Services Management Console and open the Amazon Key Management Service (Amazon KMS) console at https://console.amazonaws.cn/kms.

  2. To change the Amazon Web Services Region, use the Region selector in the upper-right corner of the page.

  3. In the left navigation pane, choose Customer managed keys, and then choose the KMS key associated with the buckets that you want to use.

  4. Expand Key Policy if it is not already expanded.

  5. In the Key Users section, choose Add and search for the IAM role. Choose the IAM role, and then choose Add.

  6. Alternatively, you can choose Switch to Policy view to display the key policy document and add a statement to the key policy. The following is an example of the policy.

Example of a policy for the Amazon KMS customer managed key
{ "Sid": "Allow use of the key", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::111122223333:role/snowball-import-s3-only-role" ] }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" }

After this policy has been added to the Amazon KMS customer managed key, it is also needed to update the IAM role associated with the Snowball job. By default, the role is snowball-import-s3-only-role.

Example of the Snowball import IAM role
{ "Effect": "Allow", "Action": [ "kms: GenerateDataKey", "kms: Decrypt" ], "Resource": "arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234-efgh-111111111111" }

For more information, see Using Identity-Based Policies (IAM Policies) for Amazon Snowball.

The KMS key that is being used looks like the following:

“Resource”:“arn:aws:kms:region:AccoundID:key/*”

For exporting data

Example of a policy for the Amazon KMS customer managed key
{ "Sid": "Allow use of the key", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::111122223333:role/snowball-import-s3-only-role" ] }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" }

After this policy has been added to the Amazon KMS customer managed key, it is also needed to update the IAM role associated with the Snowball job. By default, the role looks like the following:

snowball-export-s3-only-role

Example of the Snowball export IAM role
{ "Effect": "Allow", "Action": [ "kms: GenerateDataKey", "kms: Decrypt" ], "Resource": "arn:aws:kms:us-west-2:123456789012:key/abc123a1-abcd-1234-efgh-111111111111" }

After this policy has been added to the Amazon KMS customer managed key, it is also needed to update the IAM role associated with the Snowball job. By default, the role is snowball-export-s3-only-role.

Prerequisites for using Amazon S3 adapter on Snow Family devices for import and export jobs

You can use S3 adapter on Snow Family devices when you are using the devices to move data from on-premises data sources to the cloud or from the cloud to on-premises data storage. For more information, see Transferring files using the Amazon S3 adapter for data migration.

The Amazon S3 bucket associated with the job must use the Amazon S3 standard storage class. Before creating your first job, keep the following in mind.

For jobs that import data into Amazon S3, follow these steps:

  • Confirm that the files and folders to transfer are named according to the object key naming guidelines for Amazon S3. Any files or folders with names that don't meet these guidelines aren't imported into Amazon S3.

  • Plan what data you want to import into Amazon S3. For more information, see Planning your large transfer.

Before exporting data from Amazon S3, follow these steps:

  • Understand what data is exported when you create your job. For more information, see Using Export Ranges.

  • For any files with a colon (:) in the file name, change the file names in Amazon S3 before you create the export job to get these files. Files with a colon in the file name fail export to Microsoft Windows Server.

Prerequisites for using Amazon S3 compatible storage on Snow Family devices

You use Amazon S3 compatible storage on Snow Family devices when you are storing data on the device at your edge location and using the data for local compute operations. Data used for local compute operations will not be imported to Amazon S3 when the device is returned.

When ordering a Snow device for local compute and storage with Amazon S3 compatible storage, keep the following in mind.

  • You will provision Amazon S3 storage capacity when you order the device. So consider your storage need before ordering a device.

  • You can create Amazon S3 buckets on the device after you receive it rather than while ordering a Snow Family device.

  • You will need to download the latest version of the Amazon CLI (v2.11.15 or higher), Snowball Edge client, or Amazon OpsHub and install it on your computer to use Amazon S3 compatible storage on Snow Family devices.

  • After receiving your device, configure, start, and use Amazon S3 compatible storage on Snow Family devices according to Using Amazon S3 compatible storage on Snow Family devices in this guide.

Prerequisites for using compute instances on Snow Family devices

You can run Amazon EC2-compatible compute instances hosted on an Amazon Snowball Edge with the sbe1, sbe-c, and sbe-g instance types:

  • The sbe1 instance type works on devices with the Snowball Edge Storage Optimized option.

  • The sbe-c instance type works on devices with the Snowball Edge Compute Optimized option.

  • Both the sbe-c and sbe-g instance types work on devices with the Snowball Edge Compute Optimized with GPU option.

All the compute instance types supported on Snowball Edge device options are unique to Amazon Snowball Edge devices. Like their cloud-based counterparts, these instances require Amazon Machine Images (AMIs) to launch. You choose the AMI for an instance before you create your Snowball Edge job.

To use a compute instance on a Snowball Edge, create a job to order a Snow Family device and specify your AMIs. You can do this using the Amazon Snowball Management Console, the Amazon Command Line Interface (Amazon CLI), or one of the Amazon SDKs. Typically, to use your instances, there are some housekeeping prerequisites that you must perform before creating your job.

Note

Ubuntu 16.04 LTS - Xenial (HVM) images are no longer supported in the Amazon Web Services Marketplace, but still supported for use on Snowball Edge devices through Amazon EC2 VM Import/Export and running locally in AMIs.

You can get these images from Amazon Web Services Marketplace.

If you're using SSH to connect to the instances running on a Snowball Edge, you can use your own key pair or you can create one on the Snowball Edge. To use Amazon OpsHub to create a key pair on the device, see Working with key pairs. To use the Amazon CLI to create a key pair on the device, see create-key-pair in List of Supported Amazon EC2-compatible Amazon CLI Commands on a Snowball Edge. For more information on key pairs and Amazon Linux 2, see Amazon EC2 key pairs and Linux instances in the Amazon EC2 User Guide.

For information specific to using compute instances on a device, see Using Amazon EC2-compatible compute instances.