Runtime Properties in Kinesis Data Analytics for Apache Flink - Amazon Kinesis Data Analytics
Working with Runtime Properties in the ConsoleWorking with Runtime Properties in the CLIAccessing Runtime Properties in a Kinesis Data Analytics Application
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Runtime Properties in Kinesis Data Analytics for Apache Flink

You can use runtime properties to configure your application without recompiling your application code.

Working with Runtime Properties in the Console

You can add, update, or remove runtime properties from your Kinesis Data Analytics application using the console.

Note

You can't add runtime properties when you create an application in the Kinesis Data Analytics console.

Update Runtime Properties for a Kinesis Data Analytics application

  1. Open the Kinesis Data Analytics console at https://console.amazonaws.cn/kinesisanalytics.

  2. Choose your Kinesis Data Analytics application. Choose Application details.

  3. On the page for your application, choose Configure.

  4. Expand the Properties section.

  5. Use the controls in the Properties section to define a property group with key-value pairs. Use these controls to add, update, or remove property groups and runtime properties.

  6. Choose Update.

Working with Runtime Properties in the CLI

You can add, update, or remove runtime properties using the Amazon CLI.

This section includes example requests for API actions for configuring runtime properties for an application. For information about how to use a JSON file for input for an API action, see Kinesis Data Analytics API Example Code.

Note

Replace the sample account ID (012345678901) in the examples following with your account ID.

Adding Runtime Properties when Creating an Application

The following example request for the CreateApplication action adds two runtime property groups (ProducerConfigProperties and ConsumerConfigProperties) when you create an application:

{
    "ApplicationName": "MyApplication",
    "ApplicationDescription": "my java test app",
    "RuntimeEnvironment": "FLINK-1_15",
    "ServiceExecutionRole": "arn:aws:iam::012345678901:role/KA-stream-rw-role",
    "ApplicationConfiguration": {
        "ApplicationCodeConfiguration": {
            "CodeContent": {
                "S3ContentLocation": {
                    "BucketARN": "arn:aws:s3:::ka-app-code-username",
                    "FileKey": "java-getting-started-1.0.jar"
                }
            },
            "CodeContentType": "ZIPFILE"
        },
        "EnvironmentProperties":  { 
         "PropertyGroups": [ 
            { 
               "PropertyGroupId": "ProducerConfigProperties",
               "PropertyMap" : {
                    "flink.stream.initpos" : "LATEST",
                    "aws.region" : "us-west-2",
                    "AggregationEnabled" : "false"
               }
            },
            { 
               "PropertyGroupId": "ConsumerConfigProperties",
               "PropertyMap" : {
                    "aws.region" : "us-west-2"
               }
            }
         ]
      }
    }
}

Adding and Updating Runtime Properties in an Existing Application

The following example request for the UpdateApplication action adds or updates runtime properties for an existing application:

{
  "ApplicationName": "MyApplication",
  "CurrentApplicationVersionId": 2,
  "ApplicationConfigurationUpdate": {
    "EnvironmentPropertyUpdates": {
      "PropertyGroups": [ 
        { 
          "PropertyGroupId": "ProducerConfigProperties",
          "PropertyMap" : {
            "flink.stream.initpos" : "LATEST",
            "aws.region" : "us-west-2",
            "AggregationEnabled" : "false"
          }
        },
        { 
          "PropertyGroupId": "ConsumerConfigProperties",
          "PropertyMap" : {
            "aws.region" : "us-west-2"
          }
        }
      ]
    }
  }
}
Note

If you use a key that has no corresponding runtime property in a property group, Kinesis Data Analytics adds the key-value pair as a new property. If you use a key for an existing runtime property in a property group, Kinesis Data Analytics updates the property value.

Removing Runtime Properties

The following example request for the UpdateApplication action removes all runtime properties and property groups from an existing application:

{
  "ApplicationName": "MyApplication",
  "CurrentApplicationVersionId": 3,
  "ApplicationConfigurationUpdate": {
    "EnvironmentPropertyUpdates": {
      "PropertyGroups": []
    }
  }
}
Important

If you omit an existing property group or an existing property key in a property group, that property group or property is removed.

Accessing Runtime Properties in a Kinesis Data Analytics Application

You retrieve runtime properties in your Java application code using the static KinesisAnalyticsRuntime.getApplicationProperties() method, which returns a Map<String, Properties> object.

The following Java code example retrieves runtime properties for your application:

 Map<String, Properties> applicationProperties = KinesisAnalyticsRuntime.getApplicationProperties();

You retrieve a property group (as a Java.Util.Properties object) as follows:

Properties consumerProperties = applicationProperties.get("ConsumerConfigProperties");

You typically configure an Apache Flink source or sink by passing in the Properties object without needing to retrieve the individual properties. The following code example demonstrates how to create an Flink source by passing in a Properties object retrieved from runtime properties:

private static FlinkKinesisProducer<String> createSinkFromApplicationProperties() throws IOException {
  Map<String, Properties> applicationProperties = KinesisAnalyticsRuntime.getApplicationProperties();
  FlinkKinesisProducer<String> sink = new FlinkKinesisProducer<String>(new SimpleStringSchema(),
    applicationProperties.get("ProducerConfigProperties"));

  sink.setDefaultStream(outputStreamName);
  sink.setDefaultPartition("0");
  return sink;
}

For a complete code example that uses runtime properties, see Getting Started (DataStream API). Source code for the Getting Started application is available at Getting Started in the Kinesis Data Analytics Java Examples GitHub repository.