Troubleshooting - Amazon EMR
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Troubleshooting

Inputting your own Livy and Spark configurations while installing Livy

You can configure any Apache Livy or Apache Spark environment variable with the env.* Helm property. Follow the steps below to convert the example configuration example.config.with-dash.withUppercase to a supported environment variable format.

  1. Replace uppercase letters with a 1 and a lowercase of the letter. For example, example.config.with-dash.withUppercase becomes example.config.with-dash.with1uppercase.

  2. Replace dashes (-) with 0. For example, example.config.with-dash.with1uppercase becomes example.config.with0dash.with1uppercase

  3. Replace dots (.) with underscores (_). For example, example.config.with0dash.with1uppercase becomes example_config_with0dash_with1uppercase.

  4. Replace all lowercase letters with uppercase letters.

  5. Add the prefix LIVY_ to the variable name.

  6. Use the variable while installing Livy through the helm chart using the format --set env.YOUR_VARIABLE_NAME.value=yourvalue

For example, to set the Livy and Spark configurations livy.server.recovery.state-store = filesystem and spark.kubernetes.executor.podNamePrefix = my-prefix, use these Helm properties:

—set env.LIVY_LIVY_SERVER_RECOVERY_STATE0STORE.value=filesystem
—set env.LIVY_SPARK_KUBERNETES_EXECUTOR_POD0NAME0PREFIX.value=myprefix