Use Kinesis Client Library - Amazon Kinesis Data Streams
What is Kinesis Client Library? KCL key features and benefits
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Use Kinesis Client Library

What is Kinesis Client Library?

Kinesis Client Library (KCL) is a standalone Java software library designed to simplify the process of consuming and processing data from Amazon Kinesis Data Streams. KCL handles many of the complex tasks associated with distributed computing, letting developers to focus on implementing their business logic for processing data. It manages activities such as load balancing across multiple workers, responding to worker failures, checkpointing processed records, and responding to changes in the number of shards in the stream.

KCL is frequently updated to incorporate newer versions of underlying libraries, security improvements, and bug fixes. We recommend that you to use the latest version of KCL. To find the latest KCL version, see KCL Github.

Important

KCL is a Java library. Support for languages other than Java is provided using a Java-based daemon called MultiLangDaemon. MultiLangDaemon interacts with the KCL application over STDIN and STDOUT. For more information about the MultiLangDaemon on GitHub, see Develop consumers with KCL in non-Java languages.

KCL key features and benefits

Following are the key features and related benefits of the KCL:

  • Scalability: KCL enables applications to scale dynamically by distributing the processing load across multiple workers. You can scale your application in or out, manually or with auto-scaling, without worrying about load redistribution.

  • Load balancing: KCL automatically balances the processing load across available workers, resulting in an even distribution of work across workers.

  • Checkpointing: KCL manages checkpointing of processed records, enabling applications to resume processing from their last sucessfully processed position.

  • Fault tolerance: KCL provides built-in fault tolerance mechanisms, making sure that data processing continues even if individual workers fail. KCL also provides at-least-once delivery.

  • Handling stream-level changes: KCL adapts to shard splits and merges that might occur due to changes in data volume. It maintains ordering by making sure that child shards are processed only after their parent shard is completed and checkpointed.

  • Monitoring: KCL integrates with Amazon CloudWatch for consumer-level monitoring.

  • Multi-language support: KCL natively supports Java and enables multiple non-Java programming languages through MultiLangDaemon.