Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon services used by Amazon ParallelCluster

The following Amazon Web Services (Amazon) services are used by Amazon ParallelCluster.

Amazon API Gateway

Amazon API Gateway is an Amazon service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs at any scale

Amazon ParallelCluster uses API Gateway to host the Amazon ParallelCluster API.

For more information about Amazon Batch, see http://www.amazonaws.cn/api-gateway/ and https://docs.amazonaws.cn/apigateway/.

Amazon Batch

Amazon Batch is an Amazon managed job scheduler service. It dynamically provisions the optimal quantity and type of compute resources (for example, CPU or memory-optimized instances) in Amazon Batch clusters. These resources are provisioned based on the specific requirements of your batch jobs, including volume requirements. With Amazon Batch, you don't need to install or manage additional batch computing software or server clusters to run your jobs effectively.

Amazon Batch is used only with Amazon Batch clusters.

For more information about Amazon Batch, see http://www.amazonaws.cn/batch/ and https://docs.amazonaws.cn/batch/.

Amazon CloudFormation

Amazon CloudFormation is an infrastructure-as-code service that provides a common language to model and provision Amazon and third-party application resources in your cloud environment. It is the main service used by Amazon ParallelCluster. Each cluster in Amazon ParallelCluster is represented as a stack, and all resources required by each cluster are defined within the Amazon ParallelCluster Amazon CloudFormation template. In most cases, Amazon ParallelCluster CLI commands directly correspond to Amazon CloudFormation stack commands, such as create, update, and delete commands. Instances that are launched within a cluster make HTTPS calls to the Amazon CloudFormation endpoint in the Amazon Web Services Region where the cluster is launched.

For more information about Amazon CloudFormation, see http://www.amazonaws.cn/cloudformation/ and https://docs.amazonaws.cn/cloudformation/.

Amazon CloudWatch

Amazon CloudWatch (CloudWatch) is a monitoring and observability service that provides you with data and actionable insights. These insights can be used to monitor your applications, respond to performance changes and service exceptions, and optimize resource utilization. In Amazon ParallelCluster, CloudWatch is used for a dashboard, to monitor and log Docker image build steps and the output of the Amazon Batch jobs.

Before Amazon ParallelCluster version 2.10.0, CloudWatch was used only with Amazon Batch clusters.

For more information about CloudWatch, see http://www.amazonaws.cn/cloudwatch/ and https://docs.amazonaws.cn/cloudwatch/.

Amazon CloudWatch Events

Amazon CloudWatch Events (CloudWatch Events) delivers a near real-time stream of system events that describe changes in Amazon Web Services (Amazon) resources. Using simple rules that you can quickly set up, you can match events and route them to one or more target functions or streams. In Amazon ParallelCluster, CloudWatch Events is used for Amazon Batch jobs.

For more information about CloudWatch Events, see https://docs.amazonaws.cn//eventbridge/latest/userguide/eb-cwe-now-eb.

Amazon CloudWatch Logs

Amazon CloudWatch Logs (CloudWatch Logs) is one of the core features of Amazon CloudWatch. You can use it to monitor, store, view, and search the log files for many of the components used by Amazon ParallelCluster.

Before Amazon ParallelCluster version 2.6.0, CloudWatch Logs was only used with Amazon Batch clusters.

For more information, see Integration with Amazon CloudWatch Logs.

Amazon CodeBuild

Amazon CodeBuild (CodeBuild) is an Amazon managed continuous integration service that complies source code, runs tests, and produces software packages that are ready to deploy. In Amazon ParallelCluster, CodeBuild is used to automatically and transparently build Docker images when clusters are created.

CodeBuild is used only with Amazon Batch clusters.

For more information about CodeBuild, see http://www.amazonaws.cn/codebuild/ and https://docs.amazonaws.cn/codebuild/.

Amazon DynamoDB

Amazon DynamoDB (DynamoDB) is a fast and flexible NoSQL database service. It is used to store the minimal state information of the cluster. The head node tracks provisioned instances in a DynamoDB table.

DynamoDB is not used with Amazon Batch clusters.

For more information about DynamoDB, see http://www.amazonaws.cn/dynamodb/ and https://docs.amazonaws.cn/dynamodb/.

Amazon Elastic Block Store

Amazon Elastic Block Store (Amazon EBS) is a high-performance block storage service that provides persistent storage for shared volumes. All Amazon EBS settings can be passed through the configuration. Amazon EBS volumes can either be initialized empty or from an existing Amazon EBS snapshot.

For more information about Amazon EBS, see http://www.amazonaws.cn/ebs/ and https://docs.amazonaws.cn/ebs/.

Amazon Elastic Compute Cloud

Amazon Elastic Compute Cloud (Amazon EC2) provides the computing capacity for Amazon ParallelCluster. The head and compute nodes are Amazon EC2 instances. Any instance type that support HVM can be selected. The head and compute nodes can be different instance types. Moreover, if multiple queues are used, some or all of compute nodes can also be launched as a Spot Instance. Instance store volumes found on the instances are mounted as striped LVM volumes.

For more information about Amazon EC2, see http://www.amazonaws.cn/ec2/ and https://docs.amazonaws.cn/ec2/.

Amazon Elastic Container Registry

Amazon Elastic Container Registry (Amazon ECR) is a fully managed Docker container registry that makes it easy to store, manage, and deploy Docker container images. In Amazon ParallelCluster, Amazon ECR stores the Docker images that are built when clusters are created. The Docker images are then used by Amazon Batch to run the containers for the submitted jobs.

Amazon ECR is used only with Amazon Batch clusters.

For more information, see http://www.amazonaws.cn/ecr/ and https://docs.amazonaws.cn/ecr/.

Amazon EFS

Amazon Elastic File System (Amazon EFS) provides a simple, scalable, and fully managed elastic NFS file system for use with Amazon Web Services Cloud services and on-premises resources. Amazon EFS is used when the EfsSettings are specified. Support for Amazon EFS was added in Amazon ParallelCluster version 2.1.0.

For more information about Amazon EFS, see http://www.amazonaws.cn/efs/ and https://docs.amazonaws.cn/efs/.

Amazon FSx for Lustre

FSx for Lustre provides a high-performance file system that uses the open-source Lustre file system. FSx for Lustre is used when the FsxLustreSettings properties are specified. Support for FSx for Lustre was added in Amazon ParallelCluster version 2.2.1.

For more information about FSx for Lustre, see http://www.amazonaws.cn/fsx/lustre/ and https://docs.amazonaws.cn/fsx/.

Amazon FSx for NetApp ONTAP

FSx for ONTAP provides a fully managed shared storage system built on NetApp's popular ONTAP file system. FSx for ONTAP is used when FsxOntapSettings properties are specified. Support for FSx for ONTAP was added in Amazon ParallelCluster version 3.2.0.

For more information about FSx for ONTAP, see http://www.amazonaws.cn/fsx/netapp-ontap/ and https://docs.amazonaws.cn/fsx/.

Amazon FSx for OpenZFS

FSx for OpenZFS provides a fully managed shared storage system built on the popular OpenZFS file system. FSx for OpenZFS is used when the FsxOpenZfsSettings properties are specified. Support for FSx for OpenZFS was added in Amazon ParallelCluster version 3.2.0.

For more information about FSx for OpenZFS, see http://www.amazonaws.cn/fsx/openzfs/ and https://docs.amazonaws.cn/fsx/.

Amazon Identity and Access Management

Amazon Identity and Access Management (IAM) is used within Amazon ParallelCluster to provide a least privileged IAM role for Amazon EC2 for the instance that is specific to each individual cluster. Amazon ParallelCluster instances are given access only to the specific API calls that are required to deploy and manage the cluster.

With Amazon Batch clusters, IAM roles are also created for the components that are involved with the Docker image building process when clusters are created. These components include the Lambda functions that are allowed to add and delete Docker images to and from the Amazon ECR repository. They also include the functions allowed to delete the Amazon S3 bucket that is created for the cluster and CodeBuild project. There are also roles for Amazon Batch resources, instances, and jobs.

For more information about IAM, see http://www.amazonaws.cn/iam/ and https://docs.amazonaws.cn/iam/.

Amazon Lambda

Amazon Lambda (Lambda) runs the functions that orchestrate the creation of Docker images. Lambda also manages the cleanup of custom cluster resources, such as Docker images stored in the Amazon ECR repository and on Amazon S3.

For more information about Lambda, see http://www.amazonaws.cn/lambda/ and https://docs.amazonaws.cn/lambda/.

Amazon RDS

Amazon Relational Database Service (Amazon RDS) is a web service that makes it easier to set up, operate, and scale a relational database in the Amazon Cloud.

Amazon ParallelCluster uses Amazon RDS for Amazon Batch and Slurm.

For more information about Amazon RDS, see http://www.amazonaws.cn/rds/ and https://docs.amazonaws.cn/rds/.

Amazon Route 53

Amazon Route 53 (Route 53) is used to create hosted zones with hostnames and fully qualified domain names for each of the compute nodes.

For more information about Route 53, see http://www.amazonaws.cn/route53/ and https://docs.amazonaws.cn/route53/.

Amazon Simple Notification Service

(Amazon SNS) is a managed service that provides message delivery from publishers to subscribers (also known as producers and consumers).

Amazon ParallelCluster uses Amazon SNS for API hosting.

For more information about Amazon SNS, see http://www.amazonaws.cn/sns/ and https://docs.amazonaws.cn/sns/.

Amazon Simple Storage Service

Amazon Simple Storage Service (Amazon S3) stores Amazon ParallelCluster templates located in each Amazon Web Services Region. Amazon ParallelCluster can be configured to allow CLI/SDK tools to use Amazon S3.

Amazon ParallelCluster also creates an Amazon S3 bucket in your Amazon Web Services account to store resources that are used by your clusters, such as the cluster configuration file. Amazon ParallelCluster maintains one Amazon S3 bucket in each Amazon Web Services Region that you create clusters in.

When you use Amazon Batch cluster, an Amazon S3 bucket in your account is used for storing related data. For example, the bucket stores artifacts created when a Docker image and scripts are created from submitted jobs.

For more information, see http://www.amazonaws.cn/s3/ and https://docs.amazonaws.cn/s3/.

Amazon VPC

An Amazon VPC defines a network used by the nodes in your cluster.

For more information about Amazon VPC, see http://www.amazonaws.cn/vpc/ and https://docs.amazonaws.cn/vpc/.

Elastic Fabric Adapter

Elastic Fabric Adapter (EFA) is a network interface for Amazon EC2 instances that customers can use to run applications requiring high levels of inter-node communications at scale on Amazon.

For more information about EC2 Image Builder, see http://www.amazonaws.cn/hpc/efa/.

EC2 Image Builder

EC2 Image Builder is a fully managed Amazon service that helps you to automate the creation, management, and deployment of customized, secure, and up-to-date server images.

Amazon ParallelCluster uses Image Builder to create and manage Amazon ParallelCluster images.

For more information about EC2 Image Builder, see http://www.amazonaws.cn/image-builder/ and https://docs.amazonaws.cn/imagebuilder/.

NICE DCV

NICE DCV is a high-performance remote display protocol that provides a secure way to deliver remote desktops and application streaming to any device over varying network conditions. NICE DCV is used when the HeadNode section / Dcv settings are specified. Support for NICE DCV was added in Amazon ParallelCluster version 2.5.0.

For more information about NICE DCV, see http://www.amazonaws.cn/hpc/dcv/ and https://docs.amazonaws.cn/dcv/.