Update Java applications - Managed Service for Apache Flink
flink-connector-kinesisOther affected connectors
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Managed Service for Apache Flink was previously known as Amazon Kinesis Data Analytics for Apache Flink.

Update Java applications

Follow the procedures below to update Java applications:

If the application uses flink-connector-kinesis:

Kinesis connector uses shading to package some dependencies, including the Amazon SDK, into the connector jar. To update the Amazon SDK version, use the following procedure to replace these shaded classes:

Maven

  1. Add Kinesis connector and required Amazon SDK modules as project dependencies.

  2. Configure maven-shade-plugin:

    1. Add filter to exclude shaded Amazon SDK classes when copying content of the Kinesis connector jar.

    2. Add relocation rule to move updated Amazon SDK classes to package, expected by Kinesis connector.

    pom.xml 

    <project>
    ...    
    <dependencies>
        ...
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kinesis</artifactId>
            <version>1.15.4</version>
        </dependency>
        
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>kinesis</artifactId>
            <version>2.20.144</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>netty-nio-client</artifactId>
            <version>2.20.144</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>sts</artifactId>
            <version>2.20.144</version>
        </dependency>
        ...
    </dependencies>
    ...
    <build>
        ...
        <plugins>
            ...
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            ...
                            <filters>
                                ...
                                <filter>
                                    <artifact>org.apache.flink:flink-connector-kinesis</artifact>
                                    <excludes>
                                        <exclude>org/apache/flink/kinesis/shaded/software/amazon/awssdk/**</exclude>
                                        <exclude>org/apache/flink/kinesis/shaded/org/reactivestreams/**</exclude>
                                        <exclude>org/apache/flink/kinesis/shaded/io/netty/**</exclude>
                                        <exclude>org/apache/flink/kinesis/shaded/com/typesafe/netty/**</exclude>
                                    </excludes>
                                </filter>
                                ...
                            </filters>
                            <relocations>
                                ...
                                <relocation>
                                    <pattern>software.amazon.awssdk</pattern>
                                    <shadedPattern>org.apache.flink.kinesis.shaded.software.amazon.awssdk</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.reactivestreams</pattern>
                                    <shadedPattern>org.apache.flink.kinesis.shaded.org.reactivestreams</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>io.netty</pattern>
                                    <shadedPattern>org.apache.flink.kinesis.shaded.io.netty</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.typesafe.netty</pattern>
                                    <shadedPattern>org.apache.flink.kinesis.shaded.com.typesafe.netty</shadedPattern>
                                </relocation>
                                ...
                            </relocations>
                           ...
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            ...
        </plugins>
        ... 
    </build>
</project>
Gradle

  1. Add Kinesis connector and required Amazon SDK modules as project dependencies.

  2. Adjust shadowJar configuration:

    1. Exclude shaded Amazon SDK classes when copying content of the Kinesis connector jar.

    2. Relocate updated Amazon SDK classes to a package expected by Kinesis connector.

    build.gradle

    ...
dependencies {
    ...
    flinkShadowJar("org.apache.flink:flink-connector-kinesis:1.15.4")
    
    flinkShadowJar("software.amazon.awssdk:kinesis:2.20.144")
    flinkShadowJar("software.amazon.awssdk:sts:2.20.144")
    flinkShadowJar("software.amazon.awssdk:netty-nio-client:2.20.144")
    ...
}
...
shadowJar {
    configurations = [project.configurations.flinkShadowJar]

    exclude("software/amazon/kinesis/shaded/software/amazon/awssdk/**/*")
    exclude("org/apache/flink/kinesis/shaded/org/reactivestreams/**/*.class")
    exclude("org/apache/flink/kinesis/shaded/io/netty/**/*.class")
    exclude("org/apache/flink/kinesis/shaded/com/typesafe/netty/**/*.class")
    
    relocate("software.amazon.awssdk", "org.apache.flink.kinesis.shaded.software.amazon.awssdk")
    relocate("org.reactivestreams", "org.apache.flink.kinesis.shaded.org.reactivestreams")
    relocate("io.netty", "org.apache.flink.kinesis.shaded.io.netty")
    relocate("com.typesafe.netty", "org.apache.flink.kinesis.shaded.com.typesafe.netty")
}
...

If the application uses another affected connector:

In order to update the Amazon SDK version, the SDK version should be enforced in the project build configuration.

Maven

Add Amazon SDK bill of materials (BOM) to the dependency management section of the pom.xml file to enforce SDK version for the project.

pom.xml

<project>
    ...    
    <dependencyManagement>
        <dependencies>
            ...
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>2.20.144</version>
                <scope>import</scope>
                <type>pom</type>
            </dependency>
            ...
        </dependencies>
    </dependencyManagement>
    ...
</project>
Gradle

Add platform dependency on the Amazon SDK bill of materials (BOM) to enforce SDK version for the project. This requires Gradle 5.0 or newer:

build.gradle

...
dependencies {
    ...
    flinkShadowJar(platform("software.amazon.awssdk:bom:2.20.144"))
    ...
}
...