Use automatic request batching for Amazon SQS with the Amazon SDK for Java 2.x
The Automatic Request Batching API for Amazon SQS is a high-level library that provides an efficient way to batch and buffer requests for SQS operations. By using the batching API, you reduce the number of requests to SQS, which improves throughput and minimizes costs.
Because the batch API methods match the SqsAsyncClient
methods—sendMessage
,
changeMessageVisibility
, deleteMessage
,
receiveMessage
—you can use the batch API as a drop-in replacement with
minimal changes.
This topic gives you an overview of how to configure and work with the Automatic Request Batching API for Amazon SQS.
Check prerequisites
You need to use version 2.28.0 or later of the
SDK for Java 2.x to have access to the batching API. Your Maven pom.xml
should
at least contain the following elements.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.28.231
</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sqs</artifactId>
</dependency>
</dependencies>
Create a batch manager
The automatic request batching API is implemented by the SqsAsyncBatchManager
1. Default configuration by using
SqsAsyncClient
The simplest way you can create a batch manager is to call the
batchManager
factory method on an existing SqsAsyncClient
SqsAsyncClient asyncClient = SqsAsyncClient.create(); SqsAsyncBatchManager sqsAsyncBatchManager = asyncClient.batchManager();
When you use this approach, the SqsAsyncBatchManager
instance uses the
default values that are shown in the table in the Override configuration settings for
SqsAsyncBatchManager section. Additionally, the
SqsAsyncBatchManager
instance uses the ExecutorService
of
the SqsAsyncClient
instance that it was created from.
1. Custom configuration by using
SqsAsyncBatchManager.Builder
For more advanced use cases, you can customize the batch manager using the SqsAsyncBatchManager.Builder
SqsAsyncBatchManager
instance, you can fine tune the batching
behavior. The following snippet shows an example of how to use the builder to customize
batching behavior.
SqsAsyncBatchManager batchManager = SqsAsyncBatchManager.builder() .client(SqsAsyncClient.create()) .scheduledExecutor(Executors.newScheduledThreadPool(5)) .overrideConfiguration(b -> b .receiveMessageMinWaitDuration(Duration.ofSeconds(10)) .receiveMessageVisibilityTimeout(Duration.ofSeconds(1)) .receiveMessageAttributeNames(Collections.singletonList("*")) .receiveMessageSystemAttributeNames(Collections.singletonList(MessageSystemAttributeName.ALL))) .build();
When you use this approach, you can adjust the settings on the
BatchOverrideConfiguration
object that are shown in the table in the
Override configuration settings for
SqsAsyncBatchManager section. You can also provide a
custom ScheduledExecutorService
Send messages
To send messages with the batch manager, use the SqsAsyncBatchManager#sendMessage
method. The SDK buffers requests and
sends them as a batch when the maxBatchSize
or
sendRequestFrequency
values are reached.
The following example show a sendMessage
request immediately following by
another request. In this case, the SDK sends both messages in a single batch.
// Sending the first message CompletableFuture<SendMessageResponse> futureOne = sqsAsyncBatchManager.sendMessage(r -> r.messageBody("One").queueUrl("queue")); // Sending the second message CompletableFuture<SendMessageResponse> futureTwo = sqsAsyncBatchManager.sendMessage(r -> r.messageBody("Two").queueUrl("queue")); // Waiting for both futures to complete and retrieving the responses SendMessageResponse messageOne = futureOne.join(); SendMessageResponse messageTwo = futureTwo.join();
Change the message visibility timeout
You can change the visibility timeout of messages in a batch by using the SqsAsyncBatchManager#changeMessageVisibility
maxBatchSize
or
sendRequestFrequency
values are reached.
The following example shows how to call the changeMessageVisibility
method.
CompletableFuture<ChangeMessageVisibilityResponse> futureOne = sqsAsyncBatchManager.changeMessageVisibility(r -> r.receiptHandle("receiptHandle") .queueUrl("queue")); ChangeMessageVisibilityResponse response = futureOne.join();
Delete messages
You can delete messages in a batch using the SqsAsyncBatchManager#deleteMessage
maxBatchSize
or
sendRequestFrequency
values are reached.
The following example shows how you can call the deleteMessage
method.
CompletableFuture<DeleteMessageResponse> futureOne = sqsAsyncBatchManager.deleteMessage(r -> r.receiptHandle("receiptHandle") .queueUrl("queue")); DeleteMessageResponse response = futureOne.join();
Receive messages
Use default settings
When you poll the SqsAsyncBatchManager#receiveMessage
method in your application,
the batch manager fetches messages from its internal buffer, which the SDK automatically
updates in the background.
The following example shows how to call the receiveMessage
method.
CompletableFuture<ReceiveMessageResponse> responseFuture = sqsAsyncBatchManager.receiveMessage(r -> r.queueUrl("queueUrl"));
Use custom settings
If you want to customize the request further, for example by setting custom wait times and specifying the number of messages to retrieve, you can customize the request as shown in the following example.
CompletableFuture<ReceiveMessageResponse> response = sqsAsyncBatchManager.receiveMessage(r -> r.queueUrl("queueUrl") .waitTimeSeconds(5) .visibilityTimeout(20));
Note
If you call receiveMessage
with a ReceiveMessageRequest
that includes any of the following
parameters, the SDK bypasses the batch manager and sends a regular asynchronous
receiveMessage
request:
-
messageAttributeNames
-
messageSystemAttributeNames
-
messageSystemAttributeNamesWithStrings
-
overrideConfiguration
Override configuration settings for
SqsAsyncBatchManager
You can adjust the following settings when you create an
SqsAsyncBatchManager
instance. The following list of settings are available
on the BatchOverrideConfiguration.Builder
Setting | Description | Default value |
---|---|---|
maxBatchSize |
Maximum number of request per batch for each
SendMessageBatchRequest ,
ChangeMessageVisibilityBatchRequest , or
DeleteMessageBatchRequest . The maximum value is 10. |
10 |
sendRequestFrequency |
Time before sending a batch, unless |
200ms |
receiveMessageVisibilityTimeout |
Visibility timeout for messages. If unset, the queue's default is used. | Queue's default |
receiveMessageMinWaitDuration |
Minimum wait time for receiveMessage requests. Avoid setting
to 0 to prevent CPU waste. |
50ms |
receiveMessageSystemAttributeNames |
List of system attribute namesreceiveMessage
calls. |
None |
receiveMessageAttributeNames |
List of attribute names to request for
receiveMessage calls. |
None |