Ensuring effective Amazon MQ performance
The following design patterns can improve the effectiveness and performance of your Amazon MQ broker.
Topics
Disable Concurrent Store and Dispatch for Queues with Slow Consumers
By default, Amazon MQ optimizes for queues with fast consumers:
-
Consumers are considered fast if they are able to keep up with the rate of messages generated by producers.
-
Consumers are considered slow if a queue builds up a backlog of unacknowledged messages, potentially causing a decrease in producer throughput.
To instruct Amazon MQ to optimize for queues with slow consumers, set the
concurrentStoreAndDispatchQueues attribute to false.
For an example configuration, see concurrentStoreAndDispatchQueues.
Choose the Correct Broker Instance Type for the Best Throughput
The message throughput of a broker instance type depends on your application's use case and the following factors:
-
Use of ActiveMQ in persistent mode
-
Message size
-
The number of producers and consumers
-
The number of destinations
Understanding the relationship between message size, latency, and throughput
Depending on your use case, a larger broker instance type might not necessarily improve system throughput. When ActiveMQ writes messages to durable storage, the size of your messages determines your system's limiting factor:
-
If your messages are smaller than 100 KB, persistent storage latency is the limiting factor.
-
If your messages are larger than 100 KB, persistent storage throughput is the limiting factor.
When you use ActiveMQ in persistent mode, writing to storage normally occurs when there are either few consumers or when the consumers are slow. In non-persistent mode, writing to storage also occurs with slow consumers if the heap memory of the broker instance is full.
To determine the best broker instance type for your application, we recommend
testing different broker instance types. For more information, see Broker instance types and
also Measuring the Throughput for Amazon MQ using the JMS Benchmark
Use cases for larger broker instance types
There are three common use cases when larger broker instance types improve throughput:
-
Non-persistent mode – When your application is less sensitive to losing messages during broker instance failover (for example, when broadcasting sports scores), you can often use ActiveMQ's non-persistent mode. In this mode, ActiveMQ writes messages to persistent storage only if the heap memory of the broker instance is full. Systems that use non-persistent mode can benefit from the higher amount of memory, faster CPU, and faster network available on larger broker instance types.
-
Fast consumers – When active consumers are available and the concurrentStoreAndDispatchQueues flag is enabled, ActiveMQ allows messages to flow directly from producer to consumer without sending messages to storage (even in persistent mode). If your application can consume messages quickly (or if you can design your consumers to do this), your application can benefit from a larger broker instance type. To let your application consume messages more quickly, add consumer threads to your application instances or scale up your application instances vertically or horizontally.
-
Batched transactions – When you use persistent mode and send multiple messages per transaction, you can achieve an overall higher message throughput by using larger broker instance types. For more information, see Should I Use Transactions?
in the ActiveMQ documentation.
Choose the correct broker storage type for the best throughput
To take advantage of high durability and replication across multiple Availability Zones, use Amazon EFS. To take advantage of low latency and high throughput, use Amazon EBS. For more information, see Storage.
Configure Your Network of Brokers Correctly
When you create a network of brokers, configure it correctly for your application:
-
Enable persistent mode – Because (relative to its peers) each broker instance acts like a producer or a consumer, networks of brokers don't provide distributed replication of messages. The first broker that acts as a consumer receives a message and persists it to storage. This broker sends an acknowledgement to the producer and forwards the message to the next broker. When the second broker acknowledges the persistence of the message, the first broker deletes the message.
If persistent mode is disabled, the first broker acknowledges the producer without persisting the message to storage. For more information, see Replicated Message Store
and What is the difference between persistent and non-persistent delivery? in the Apache ActiveMQ documentation. -
Don't disable advisory messages for broker instances – For more information, see Advisory Message
in the Apache ActiveMQ documentation. -
Don't use multicast broker discovery – Amazon MQ doesn't support broker discovery using multicast. For more information, see What is the difference between discovery, multicast, and zeroconf?
in the Apache ActiveMQ documentation.