Enabling the Apache Spark web UI for development endpoints - Amazon Glue
Using the Amazon Web Services Management ConsoleUsing the Amazon CLI
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Enabling the Apache Spark web UI for development endpoints

Use the Apache Spark web UI to monitor and debug Spark applications running on Amazon Glue development endpoints. You can configure the development endpoints with the Spark Web UI using the Amazon Glue console or the Amazon Command Line Interface (Amazon CLI).

Enabling the Spark UI (console)

Follow these steps to configure the Spark UI using the Amazon Web Services Management Console.

To create a development endpoint with the Spark UI enabled

  1. Sign in to the Amazon Web Services Management Console and open the Amazon Glue console at https://console.amazonaws.cn/glue/.

  2. In the navigation pane, choose Dev endpoints.

  3. Choose Add endpoint.

  4. In Configuration, open the Spark UI options.

  5. In the Spark UI tab, choose Enable.

  6. Specify an Amazon S3 path for storing the Spark event logs.

Enabling the Spark UI (Amazon CLI)

To create a development endpoint with the Spark UI enabled using the Amazon CLI, pass in the following arguments in JSON syntax.

{
    "EndpointName": "Name",
    "RoleArn": "role_ARN",
    "PublicKey": "public_key_contents",
    "NumberOfNodes": 2,
    "Arguments": {
        "--enable-spark-ui": "true",
        "--spark-event-logs-path": "s3://s3-event-log-path"
    }
}