Creating Kinesis Data Analytics applications with Apache Beam - Amazon Kinesis Data Analytics
Using Apache Beam with Kinesis Data AnalyticsBeam Capabilities
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Creating Kinesis Data Analytics applications with Apache Beam

You can use the Apache Beam framework with your Kinesis Data Analytics application to process streaming data. Kinesis Data Analytics applications that use Apache Beam use Apache Flink runner to execute Beam pipelines.

For a tutorial about how to use Apache Beam in a Kinesis Data Analytics application, see Using CloudFormation with Kinesis Data Analytics Creating an application using Apache Beam.

This topic contains the following sections:

Using Apache Beam with Kinesis Data Analytics

Note the following about using the Apache Flink runner with Kinesis Data Analytics:

  • Apache Beam metrics are not viewable in the Kinesis Data Analytics console.

  • Apache Beam is only supported with Kinesis Data Analytics applications that use Apache Flink version 1.8 and above. Apache Beam is not supported with Kinesis Data Analytics applications that use Apache Flink version 1.6.

Beam Capabilities

Kinesis Data Analytics supports the same Apache Beam capabilties as the Apache Flink runner. For information about what features are supported with the Apache Flink runner, see the Beam Compatibility Matrix.

We recommend that you test your Apache Flink application in the Kinesis Data Analytics service to verify that we support all the features that your application needs.