Amazon Managed Service for Apache Flink 1.15 - Managed Service for Apache Flink
Changes in Amazon Managed Service for Apache Flink with Apache Flink 1.15 Components
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Managed Service for Apache Flink was previously known as Amazon Kinesis Data Analytics for Apache Flink.

Amazon Managed Service for Apache Flink 1.15

Managed Service for Apache Flink supports the following new features in Apache 1.15.2:

Feature Description Apache FLIP reference
Async Sink An Amazon contributed framework for building async destinations that allows developers to build custom Amazon connectors with less than half the previous effort. For more information, see The Generic Asynchronous Base Sink. FLIP-171: Async Sink.
Kinesis Data Firehose Sink Amazon has contributed a new Amazon Kinesis Firehose Sink using the Async framework. Amazon Kinesis Data Firehose Sink.
Stop with Savepoint Stop with Savepoint ensures a clean stop operation, most importantly supporting exactly-once semantics for customers that rely on them. FLIP-34: Terminate/Suspend Job with Savepoint.
Scala Decoupling Users can now leverage the Java API from any Scala version, including Scala 3. Customers will need to bundle the Scala standard library of their choice in their Scala applications. FLIP-28: Long-term goal of making flink-table Scala-free.
Scala See Scala decoupling above FLIP-28: Long-term goal of making flink-table Scala-free.
Unified Connector Metrics Flink has defined standard metrics for jobs, tasks and operators. Managed Service for Apache Flink will continue to support sink and source metrics and in 1.15 introduce numRestarts in parallel with fullRestarts for Availability Metrics. FLIP-33: Standardize Connector Metrics and FLIP-179: Expose Standardized Operator Metrics.
Checkpointing finished tasks This feature is enabled by default in Flink 1.15 and makes it possible to continue performing checkpoints even if parts of the job graph have finished processing all data, which might happen if it contains bounded (batch) sources. FLIP-147: Support Checkpoints After Tasks Finished.

Studio notebooks

Managed Service for Apache Flink Studio now supports Apache Flink 1.15. Managed Service for Apache Flink Studio utilizes Apache Zeppelin notebooks to provide a single-interface development experience for developing, debugging code, and running Apache Flink stream processing applications. You can learn more about Managed Service for Apache Flink Studio and how to get started at Using a Studio notebook with Managed Service for Apache Flink.

EFO connector

When upgrading to Managed Service for Apache Flink version 1.15, ensure that you are using the most recent EFO Connector, that is any version 1.15.3 or newer. For more information as to why, see FLINK-29324.

Scala Decoupling

Starting with Flink 1.15.2, you will need to bundle the Scala standard library of your choice in your Scala applications.

Kinesis Data Firehose Sink

When upgrading to Managed Service for Apache Flink version 1.15, ensure that you are using the most recent Amazon Kinesis Data Firehose Sink.

Kafka Connectors

When upgrading to Amazon Managed Service for Apache Flink for Apache Flink version 1.15, ensure that you are using the most recent Kafka connector APIs. Apache Flink has deprecated FlinkKafkaConsumer and FlinkKafkaProducer These APIs for the Kafka sink cannot commit to Kafka for Flink 1.15. Ensure you are using KafkaSource and KafkaSink.

Component Version
Java 11 (recommended)
Scala 2.12
Managed Service for Apache Flink Flink Runtime (aws-kinesisanalytics-runtime) 1.2.0
Amazon Kinesis Connector (flink-connector-kinesis) 1.15.4
Apache Beam (Beam applications only) 2.33.0, with Jackson version 2.12.2