Enabling Apache Spark encryption - Amazon Athena
Athena consoleAmazon CLIAthena API
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Enabling Apache Spark encryption

You can enable Apache Spark encryption on Athena. Doing so encrypts data in transit between Spark nodes and also encrypts data at rest stored locally by Spark. To enhance security for this data, Athena uses the following encryption configuration:

spark.io.encryption.keySizeBits="256" 
spark.io.encryption.keygen.algorithm="HmacSHA384"

To enable Spark encryption, you can use the Athena console, the Amazon CLI, or the Athena API.

Using the Athena console to enable Spark encryption

To create a new notebook that has Spark encryption enabled

  1. Open the Athena console at https://console.amazonaws.cn/athena/.

  2. If the console navigation pane is not visible, choose the expansion menu on the left.

  3. Do one of the following:

    • In Notebook explorer, choose Create notebook.

    • In Notebook editor, choose Create notebook, or choose the plus icon (+) to add a notebook.

  4. For Notebook name, enter a name for the notebook.

  5. Expand the Spark properties option.

  6. Select Turn on Spark encryption.

  7. Choose Create.

The notebook session that you create is encrypted. Use the new notebook as you normally would. When you later launch new sessions that use the notebook, the new sessions will also be encrypted.

You can also use the Athena console to enable Spark encryption for an existing notebook.

To enable encryption for an existing notebook

  1. Open a new session for a previously created notebook.

  2. In the notebook editor, from the Session menu on the upper right, choose Edit session.

  3. In the Edit session details dialog box, expand Spark properties.

  4. Select Turn on Spark encryption.

  5. Choose Save.

The console launches a new session that has encryption enabled. Later sessions that you create for this notebook will also have encryption enabled.

Using the Amazon CLI to enable Spark encryption

You can use the Amazon CLI to enable encryption when you launch a session by specifying the appropriate Spark properties.

To use the Amazon CLI to enable Spark encryption

  1. Use a command like the following to create an engine configuration JSON object that specifies Spark encryption properties.

    ENGINE_CONFIGURATION_JSON=$( 
  cat <<EOF 
{ 
    "CoordinatorDpuSize": 1, 
    "MaxConcurrentDpus": 20, 
    "DefaultExecutorDpuSize": 1, 
    "SparkProperties": { 
      "spark.authenticate": "true", 
      "spark.io.encryption.enabled": "true", 
      "spark.network.crypto.enabled": "true" 
    } 
} 
EOF 
)

  2. In the Amazon CLI, use the athena start-session command and pass in the JSON object that you created to the --engine-configuration argument, as in the following example:

    aws athena start-session \ 
   --region "region" \ 
   --work-group "your-work-group" \ 
   --engine-configuration "$ENGINE_CONFIGURATION_JSON"

Using the Athena API to enable Spark encryption

To enable Spark encryption with the Athena API, use the StartSession action and its EngineConfiguration SparkProperties parameter to specify the encryption configuration in your StartSession request.