Supported Apache Kafka versions
Amazon Managed Streaming for Apache Kafka (Amazon MSK) supports the following Apache Kafka and Amazon MSK versions.
Topics
- Apache Kafka version 3.3.2
- Apache Kafka version 3.3.1
- Apache Kafka version 3.2.0
- Apache Kafka version 3.1.1
- Apache Kafka version 2.8.1 [recommended]
- Apache Kafka version 2.8.0
- Apache Kafka version 2.7.2
- Apache Kafka version 2.7.1
- Apache Kafka version 2.6.3
- Apache Kafka version 2.6.2
- Apache Kafka version 2.7.0
- Apache Kafka version 2.6.1
- Apache Kafka version 2.6.0
- Apache Kafka version 2.5.1
- Amazon MSK bug-fix version 2.4.1.1
- Apache Kafka version 2.4.1 (use 2.4.1.1 instead)
- Apache Kafka version 2.3.1
- Apache Kafka version 2.2.1
- Apache Kafka version 1.1.1 (for existing clusters only)
Apache Kafka version 3.3.2
For information about Apache Kafka version 3.3.2, see its release
notes
Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability.
Apache Kafka version 3.3.1
For information about Apache Kafka version 3.3.1, see its release
notes
Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability.
Apache Kafka version 3.2.0
For information about Apache Kafka version 3.2.0, see its release
notes
Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability.
Apache Kafka version 3.1.1
For information about Apache Kafka version 3.1.1, see its release
notes
Amazon MSK will continue to use and manage Zookeeper for quorum management in this release for stability.
Apache Kafka version 2.8.1 [recommended]
For information about Apache Kafka version 2.8.1, see its release
notes
Apache Kafka version 2.8.0
For information about Apache Kafka version 2.8.0, see its release
notes
Apache Kafka version 2.7.2
For information about Apache Kafka version 2.7.2, see its release
notes
Apache Kafka version 2.7.1
For information about Apache Kafka version 2.7.1, see its release
notes
Apache Kafka version 2.6.3
For information about Apache Kafka version 2.6.3, see its release
notes
Apache Kafka version 2.6.2
For information about Apache Kafka version 2.6.2, see its release
notes
Apache Kafka version 2.7.0
For information about Apache Kafka version 2.7.0, see its release
notes
Apache Kafka version 2.6.1
For information about Apache Kafka version 2.6.1, see its release
notes
Apache Kafka version 2.6.0
For information about Apache Kafka version 2.6.0, see its release
notes
Apache Kafka version 2.5.1
Apache Kafka version 2.5.1 includes several bug fixes and new features, including encryption in-transit for Apache ZooKeeper and administration clients. Amazon MSK provides TLS ZooKeeper endpoints, which you can query with the DescribeCluster operation.
The output of the
DescribeCluster operation includes the ZookeeperConnectStringTls
node, which lists
the TLS zookeeper endpoints.
The following example shows the ZookeeperConnectStringTls
node of the response for the
DescribeCluster
operation:
"ZookeeperConnectStringTls": "z-3.awskafkatutorialc.abcd123.c3.kafka.us-east-1.amazonaws.com:2182,z-2.awskafkatutorialc.abcd123.c3.kafka.us-east-1.amazonaws.com:2182,z-1.awskafkatutorialc.abcd123.c3.kafka.us-east-1.amazonaws.com:2182"
For information about using TLS encryption with zookeeper, see Using TLS security with Apache ZooKeeper.
For more information about Apache Kafka version 2.5.1, see its release
notes
Amazon MSK bug-fix version 2.4.1.1
This release is an Amazon MSK-only bug-fix version of Apache Kafka version 2.4.1.
This bug-fix release contains
a fix for KAFKA-9752PreparingRebalance
state. This issue affects clusters running Apache
Kafka versions 2.3.1 and 2.4.1. This release contains a community-produced
fix that is available in Apache Kafka version 2.5.0.
Amazon MSK clusters running version 2.4.1.1 are compatible with any Apache Kafka client that is compatible with Apache Kafka version 2.4.1.
We recommend that you use MSK bug-fix version 2.4.1.1 for new Amazon MSK clusters if you prefer to use Apache Kafka 2.4.1. You can update existing clusters running Apache Kafka version 2.4.1 to this version to incorporate this fix. For information about upgrading an existing cluster, see Updating the Apache Kafka version.
To work around this issue without upgrading the cluster to version 2.4.1.1, see the Consumer group stuck in PreparingRebalance state section of the Troubleshooting your Amazon MSK cluster guide.
Apache Kafka version 2.4.1 (use 2.4.1.1 instead)
You can no longer create an MSK cluster with Apache Kafka version 2.4.1. Instead, you can use Amazon MSK bug-fix version 2.4.1.1 with clients compatible with Apache Kafka version 2.4.1. And if you already have an MSK cluster with Apache Kafka version 2.4.1, we recommend you update it to use Apache Kafka version 2.4.1.1 instead.
KIP-392 is one of the key Kafka Improvement Proposals that are included in the 2.4.1
release of Apache Kafka. This improvement allows consumers to fetch from the closest
replica. To use this feature, set client.rack
in the consumer properties to
the ID of the consumer's Availability Zone. An example AZ ID is use1-az1
.
Amazon MSK sets broker.rack
to the IDs of the Availability Zones of the brokers.
You must also set the replica.selector.class
configuration property to
org.apache.kafka.common.replica.RackAwareReplicaSelector
, which is an
implementation of rack awareness provided by Apache Kafka.
When you use this version of Apache Kafka, the metrics in the
PER_TOPIC_PER_BROKER
monitoring level appear only after their values
become nonzero for the first time. For more information about this, see PER_TOPIC_PER_BROKER Level
monitoring.
For information about how to find Availability Zone IDs, see AZ IDs for Your Resource
For information about setting configuration properties, see Amazon MSK configuration.
For more information about KIP-392, see Allow Consumers to Fetch from Closest Replica
For more information about Apache Kafka version 2.4.1, see its release
notes
Apache Kafka version 2.3.1
For information about Apache Kafka version 2.3.1, see its release
notes
Apache Kafka version 2.2.1
For information about Apache Kafka version 2.2.1, see its release
notes
Apache Kafka version 1.1.1 (for existing clusters only)
You can no longer create a new MSK cluster with Apache Kafka version 1.1.1.
You can continue to use existing clusters that are configured with Apache Kafka
version 1.1.1. For information about Apache Kafka version 1.1.1, see its release
notes