Amazon Managed Service for Apache Flink was previously known as Amazon Kinesis Data Analytics for Apache Flink.
Get started with Amazon Managed Service for Apache Flink (Table API)
This section introduces you to the fundamental concepts of Managed Service for Apache Flink and implementing an application in Java using the Table API and SQL. It demonstrates how to switch between different APIs within the same application, and it describes the available options for creating and testing your applications. It also provides instructions for installing the necessary tools to complete the tutorials in this guide and to create your first application.
Topics
Review the components of the Managed Service for Apache Flink application
Note
Managed Service for Apache Flink supports all Apache Flink APIs
To process data, your Managed Service for Apache Flink application uses a Java application that processes input and produces output using the Apache Flink runtime.
A typical Apache Flink application has the following components:
-
Runtime properties: You can use runtime properties to pass configuration parameters to your application without modifying and republishing the code.
-
Sources: The application consumes data from one or more sources. A source uses a connector
to read data from and external system, such as a Kinesis data stream or an Amazon MSK topic. For development or testing, you can also have sources random[ly generate test data. For more information, see Add streaming data sources to Managed Service for Apache Flink. With SQL or Table API, sources are defined as source tables. -
Transformations: The application processes data through one or more transformations that can filter, enrich, or aggregate data. When using SQL or Table API, transformations are defined as queries over tables or views.
-
Sinks: The application sends data to external systems through sinks. A sink uses a connector
to send data to an external system, such as a Kinesis data stream, an Amazon MSK topic, an Amazon S3 bucket, or a relational database. You can also use a special connector to print the output for development purposes only. When using SQL or Table API, sinks are defined as sink tables where you will insert results. For more information, see Write data using sinks in Managed Service for Apache Flink.
Your application requires some external dependencies, such as Flink connectors your application uses, or potentially a Java library. To run in Amazon Managed Service for Apache Flink, you must package the application along with dependencies in a fat-JAR and upload it to an Amazon S3 bucket. You then create a Managed Service for Apache Flink application. You pass the code package location, along with other runtime configuration parameters. This tutorial demonstrates how to use Apache Maven to package the application and how to run the application locally in the IDE of your choice.
Complete the required prerequisites
Before starting this tutorial, complete the first two steps of the Get started with Amazon Managed Service for Apache Flink (DataStream API):
To get started, see Create an application.