Amazon ParallelCluster shared storage considerations - Amazon ParallelCluster
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon ParallelCluster shared storage considerations

Consider the following when working with shared storage in Amazon ParallelCluster.

  • Back up your file system data with Amazon Backup or another method to manage backups for all of your storage systems.

  • To add shared storage, you add a shared storage section to your configuration file and create or update the cluster.

  • To remove shared storage, you remove the shared storage section from your configuration file and update the cluster.

  • To replace existing Amazon ParallelCluster managed shared storage with new managed storage, change the value for SharedStorage / Name and update the cluster.

    Warning

    By default, the existing Amazon ParallelCluster managed storage and data is deleted when you perform the cluster update with a new Name parameter. If you need to change Name and retain the existing managed shared storage data, make sure you either set the DeletionPolicy to Retain or back up the data before you update the cluster.

  • If you don't back up Amazon ParallelCluster managed storage data and DeletionPolicy is Delete, your data is deleted when either your cluster is deleted or when your managed storage is removed from the cluster configuration and the cluster is updated.

  • If you don't back up Amazon ParallelCluster managed storage data and DeletionPolicy is Retain, your file system is detached before the cluster is deleted and can be re-attached to another cluster as an external file system. Your data is preserved.

  • If Amazon ParallelCluster managed storage is removed from the cluster configuration and DeletionPolicy is Retain, it can be re-attached to the cluster as an external file system with your cluster data preserved.

  • Starting with Amazon ParallelCluster version 3.4.0, you can enhance security for Amazon EFS file system mounts by configuring SharedStorage / EfsSettings / EncryptionInTransit and IamAuthorization settings.

  • When mounting an external filesystem to the /home directory, Amazon ParallelCluster copies the contents of the head node's /home directory to the external filesystem. It copies existing data in the /home directory without overwriting existing files or directories on the external storage. This includes the cluster's SSH key for the default user in case it does not already exist on the external filesystem. Consequently all other clusters that mount the same external filesystem to their respective /home directory will also have the same SSH key for their default user of the cluster.

  • In a multi-cluster environment that mounts the same external filesystem to the /home directories of clusters, SSH keys that grant access to the compute nodes, created on the head node by Amazon ParallelCluster, are generated only once when the first cluster mounts the external filesystem to /home. All other clusters use the same SSH key. As a result, anyone possessing the SSH key for the default user of these shared clusters can access any cluster. All compute nodes allow connections using the initially generated key.