To create a cluster that supports client authentication To set up a client to use authentication To produce and consume messages using authentication

Mutual TLS authentication

You can enable client authentication with TLS for connections from your applications to your Amazon MSK brokers and ZooKeeper nodes. To use client authentication, you need an Amazon Private CA. The Amazon Private CA can be either in the same Amazon Web Services account as your cluster, or in a different account. For information about Amazon Private CAs, see Creating and Managing a Amazon Private CA.

Note

TLS authentication is not currently available in the Beijing and Ningxia Regions.

Amazon MSK doesn't support certificate revocation lists (CRLs). To control access to your cluster topics or block compromised certificates, use Apache Kafka ACLs and Amazon security groups. For information about using Apache Kafka ACLs, see Apache Kafka ACLs.

This topic contains the following sections:

To create a cluster that supports client authentication
To set up a client to use authentication
To produce and consume messages using authentication

To create a cluster that supports client authentication

This procedure shows you how to enable client authentication using a Amazon Private CA.

Note

We highly recommend using independent Amazon Private CA for each MSK cluster when you use mutual TLS to control access. Doing so will ensure that TLS certificates signed by PCAs only authenticate with a single MSK cluster.

Create a file named clientauthinfo.json with the following contents. Replace Private-CA-ARN with the ARN of your PCA.
```
{
   "Tls": {
       "CertificateAuthorityArnList": ["Private-CA-ARN"]
    }
}
```
Create a file named brokernodegroupinfo.json as described in Creating a cluster using the Amazon CLI.
Client authentication requires that you also enable encryption in transit between clients and brokers. Create a file named encryptioninfo.json with the following contents. Replace KMS-Key-ARN with the ARN of your KMS key. You can set ClientBroker to TLS or TLS_PLAINTEXT.
```
{
   "EncryptionAtRest": {
       "DataVolumeKMSKeyId": "KMS-Key-ARN"
    },
   "EncryptionInTransit": {
        "InCluster": true,
        "ClientBroker": "TLS"
    }
}
```
For more information about encryption, see Amazon MSK encryption.

On a machine where you have the Amazon CLI installed, run the following command to create a cluster with authentication and in-transit encryption enabled. Save the cluster ARN provided in the response.


aws kafka create-cluster --cluster-name "AuthenticationTest" --broker-node-group-info file://brokernodegroupinfo.json --encryption-info file://encryptioninfo.json --client-authentication file://clientauthinfo.json --kafka-version "{YOUR KAFKA VERSION}" --number-of-broker-nodes 3

To set up a client to use authentication

Create an Amazon EC2 instance to use as a client machine. For simplicity, create this instance in the same VPC you used for the cluster. See Step 3: Create a client machine for an example of how to create such a client machine.
Create a topic. For an example, see the instructions under Step 4: Create a topic.
On a machine where you have the Amazon CLI installed, run the following command to get the bootstrap brokers of the cluster. Replace Cluster-ARN with the ARN of your cluster.
```
aws kafka get-bootstrap-brokers --cluster-arn Cluster-ARN
```
Save the string associated with BootstrapBrokerStringTls in the response.
On your client machine, run the following command to use the JVM trust store to create your client trust store. If your JVM path is different, adjust the command accordingly.
```
cp /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64/jre/lib/security/cacerts kafka.client.truststore.jks
```
On your client machine, run the following command to create a private key for your client. Replace Distinguished-Name, Example-Alias, Your-Store-Pass, and Your-Key-Pass with strings of your choice.
```
keytool -genkey -keystore kafka.client.keystore.jks -validity 300 -storepass Your-Store-Pass -keypass Your-Key-Pass -dname "CN=Distinguished-Name" -alias Example-Alias -storetype pkcs12
```

On your client machine, run the following command to create a certificate request with the private key you created in the previous step.


keytool -keystore kafka.client.keystore.jks -certreq -file client-cert-sign-request -alias Example-Alias -storepass Your-Store-Pass -keypass Your-Key-Pass

Open the client-cert-sign-request file and ensure that it starts with -----BEGIN CERTIFICATE REQUEST----- and ends with -----END CERTIFICATE REQUEST-----. If it starts with -----BEGIN NEW CERTIFICATE REQUEST-----, delete the word NEW (and the single space that follows it) from the beginning and the end of the file.
On a machine where you have the Amazon CLI installed, run the following command to sign your certificate request. Replace Private-CA-ARN with the ARN of your PCA. You can change the validity value if you want. Here we use 300 as an example.
```
aws acm-pca issue-certificate --certificate-authority-arn Private-CA-ARN --csr fileb://client-cert-sign-request --signing-algorithm "SHA256WITHRSA" --validity Value=300,Type="DAYS"
```
Save the certificate ARN provided in the response.

Note
To retrieve your client certificate, use the acm-pca get-certificate command and specify your certificate ARN. For more information, see get-certificate in the Amazon CLI Command Reference.
Run the following command to get the certificate that Amazon Private CA signed for you. Replace Certificate-ARN with the ARN you obtained from the response to the previous command.
```
aws acm-pca get-certificate --certificate-authority-arn Private-CA-ARN --certificate-arn Certificate-ARN
```
From the JSON result of running the previous command, copy the strings associated with Certificate and CertificateChain. Paste these two strings in a new file named signed-certificate-from-acm. Paste the string associated with Certificate first, followed by the string associated with CertificateChain. Replace the \n characters with new lines. The following is the structure of the file after you paste the certificate and certificate chain in it.
```
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----            
```

Run the following command on the client machine to add this certificate to your keystore so you can present it when you talk to the MSK brokers.


keytool -keystore kafka.client.keystore.jks -import -file signed-certificate-from-acm -alias Example-Alias -storepass Your-Store-Pass -keypass Your-Key-Pass

Create a file named client.properties with the following contents. Adjust the truststore and keystore locations to the paths where you saved kafka.client.truststore.jks. Substitute your Kafka client version for the {YOUR KAFKA VERSION} placeholders.
```
security.protocol=SSL
ssl.truststore.location=/tmp/kafka_2.12-{YOUR KAFKA VERSION}/kafka.client.truststore.jks
ssl.keystore.location=/tmp/kafka_2.12-{YOUR KAFKA VERSION}/kafka.client.keystore.jks
ssl.keystore.password=Your-Store-Pass
ssl.key.password=Your-Key-Pass
```

To produce and consume messages using authentication

Run the following command to create a topic.


<path-to-your-kafka-installation>/bin/kafka-topics.sh --create --zookeeper ZooKeeper-Connection-String --replication-factor 3 --partitions 1 --topic ExampleTopic

Run the following command to start a console producer. The file named client.properties is the one you created in the previous procedure.


<path-to-your-kafka-installation>/bin/kafka-console-producer.sh --broker-list BootstrapBroker-String --topic ExampleTopic --producer.config client.properties

In a new command window on your client machine, run the following command to start a console consumer.


<path-to-your-kafka-installation>/bin/kafka-console-consumer.sh --bootstrap-server BootstrapBroker-String --topic ExampleTopic --consumer.config client.properties

Type messages in the producer window and watch them appear in the consumer window.

Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

IAM access control

SASL/SCRAM authentication