Public access - Amazon Managed Streaming for Apache Kafka
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Public access

Amazon MSK gives you the option to turn on public access to the brokers of MSK clusters running Apache Kafka 2.6.0 or later versions. For security reasons, you can't turn on public access while creating an MSK cluster. However, you can update an existing cluster to make it publicly accessible. You can also create a new cluster and then update it to make it publicly accessible.

You can turn on public access to an MSK cluster at no additional cost, but standard Amazon data transfer costs apply for data transfer in and out of the cluster. For information about pricing, see Amazon EC2 On-Demand Pricing.

To turn on public access to a cluster, first ensure that the cluster meets all of the following conditions:

  • The subnets that are associated with the cluster must be public. This means that the subnets must have an associated route table with an internet gateway attached. For information about how to create and attach an internet gateway, see Internet gateways in the Amazon VPC user guide.

  • Unauthenticated access control must be off and at least one of the following access-control methods must be on: SASL/IAM, SASL/SCRAM, mTLS. For information about how to update the access-control method of a cluster, see Updating security settings of a cluster.

  • Encryption within the cluster must be turned on. The on setting is the default when creating a cluster. It's not possible to turn on encryption within the cluster for a cluster that was created with it turned off. It is therefore not possible to turn on public access for a cluster that was created with encryption within the cluster turned off.

  • Plaintext traffic between brokers and clients must be off. For information about how to turn it off if it's on, see Updating security settings of a cluster.

  • If you are using the SASL/SCRAM or mTLS access-control methods, you must set Apache Kafka ACLs for your cluster. After you set the Apache Kafka ACLs for your cluster, update the cluster's configuration to have the property allow.everyone.if.no.acl.found to false for the cluster. For information about how to update the configuration of a cluster, see Amazon MSK configuration operations. If you are using IAM access control and want to apply authorization policies or update your authorization policies, see IAM access control. For information about Apache Kafka ACLs, see Apache Kafka ACLs.

After you ensure that an MSK cluster meets the conditions listed above, you can use the Amazon Web Services Management Console, the Amazon CLI, or the Amazon MSK API to turn on public access. After you turn on public access to a cluster, you can get a public bootstrap-brokers string for it. For information about getting the bootstrap brokers for a cluster, see Getting the bootstrap brokers for an Amazon MSK cluster.

Important

In addition to turning on public access, ensure that the cluster's security groups have inbound TCP rules that allow public access from your IP address. We recommend that you make these rules as restrictive as possible. For information about security groups and inbound rules, see Security groups for your VPC in the Amazon VPC User Guide. For port numbers, see Port information. For instructions on how to change a cluster's security group, see Changing an Amazon MSK cluster's security group.

Note

If you use the following instructions to turn on public access and then still cannot access the cluster, see Unable to access cluster that has public access turned on.

Turning on public access using the console

  1. Sign in to the Amazon Web Services Management Console, and open the Amazon MSK console at https://console.amazonaws.cn/msk/home?region=us-east-1#/home/.

  2. In the list of clusters, choose the cluster to which you want to turn on public access.

  3. Choose the Properties tab, then find the Network settings section.

  4. Choose Edit public access.

Turning on public access using the Amazon CLI

  1. Run the following Amazon CLI command, replacing ClusterArn and Current-Cluster-Version with the ARN and current version of the cluster. To find the current version of the cluster, use the DescribeCluster operation or the describe-cluster Amazon CLI command. An example version is KTVPDKIKX0DER.

    aws kafka update-connectivity --cluster-arn ClusterArn --current-version Current-Cluster-Version --connectivity-info '{"PublicAccess": {"Type": "SERVICE_PROVIDED_EIPS"}}'

    The output of this update-connectivity command looks like the following JSON example.

    {
    "ClusterArn": "arn:aws:kafka:us-east-1:012345678012:cluster/exampleClusterName/abcdefab-1234-abcd-5678-cdef0123ab01-2",
    "ClusterOperationArn": "arn:aws:kafka:us-east-1:012345678012:cluster-operation/exampleClusterName/abcdefab-1234-abcd-5678-cdef0123ab01-2/0123abcd-abcd-4f7f-1234-9876543210ef"
}
    Note

    To turn off public access, use a similar Amazon CLI command, but with the following connectivity info instead:

    '{"PublicAccess": {"Type": "DISABLED"}}'

  2. To get the result of the update-connectivity operation, run the following command, replacing ClusterOperationArn with the ARN that you obtained in the output of the update-connectivity command.

    aws kafka describe-cluster-operation --cluster-operation-arn ClusterOperationArn

    The output of this describe-cluster-operation command looks like the following JSON example.

    {
    "ClusterOperationInfo": {
        "ClientRequestId": "982168a3-939f-11e9-8a62-538df00285db",
        "ClusterArn": "arn:aws:kafka:us-east-1:012345678012:cluster/exampleClusterName/abcdefab-1234-abcd-5678-cdef0123ab01-2",
        "CreationTime": "2019-06-20T21:08:57.735Z",
        "OperationArn": "arn:aws:kafka:us-east-1:012345678012:cluster-operation/exampleClusterName/abcdefab-1234-abcd-5678-cdef0123ab01-2/0123abcd-abcd-4f7f-1234-9876543210ef",
        "OperationState": "UPDATE_COMPLETE",
        "OperationType": "UPDATE_CONNECTIVITY",
        "SourceClusterInfo": {
            "ConnectivityInfo": {
                "PublicAccess": {
                    "Type": "DISABLED"
                }
            }
        },
        "TargetClusterInfo": {
            "ConnectivityInfo": {
                "PublicAccess": {
                    "Type": "SERVICE_PROVIDED_EIPS"
                }
            }
        }
    }
}

    If OperationState has the value UPDATE_IN_PROGRESS, wait a while, then run the describe-cluster-operation command again.

Turning on public access using the Amazon MSK API
Note

For security reasons, Amazon MSK doesn't allow public access to Apache ZooKeeper nodes. For information about how to control access to the Apache ZooKeeper nodes of your MSK cluster from within Amazon, see Controlling access to Apache ZooKeeper.