Learn how to configure a dead-letter queue redrive in Amazon SQS - Amazon Simple Queue Service
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Learn how to configure a dead-letter queue redrive in Amazon SQS

You can use dead-letter queue redrive to move unconsumed messages out of an existing dead-letter queue. By default, dead-letter queue redrive moves messages from a dead-letter queue to a source queue. However, you can also configure any other queue as the redrive destination if both queues are the same type. For example, if the dead-letter queue is a FIFO queue, the redrive destination queue must be a FIFO queue as well. Additionally, you can configure the redrive velocity to set the rate at which Amazon SQS moves messages.

Note

When a message is moved from a FIFO queue to a FIFO DLQ, the original message's deduplication ID will be replaced with the original message's ID. This is to make sure that the DLQ deduplication will not prevent storing of two independent messages that happen to share a deduplication ID.

Dead-letter queues redrive messages in the order they are received, starting with the oldest message. However, the destination queue ingests the redriven messages, as well as new messages from other producers, according to the order in which it receives them. For example, if a producer is sending messages to a source FIFO queue when simultaneously receiving redriven messages from a dead letter queue, the redriven messages will interweave with the new messages from the producer.

Note

The redrive task resets the retention period. All redriven messages are considered new messages with a new messageID and enqueueTime are assigned to redriven messages.

Configuring a dead-letter queue redrive for an existing standard queue using the Amazon SQS API

You can configure a dead-letter queue redrive using the SendMessageBatch, ReceiveMessage, and DeleteMessageBatch API actions:

API action Description

StartMessageMoveTask

Starts an asynchronous task to move messages from a specified source queue to a specified destination queue.

ListMessageMoveTasks

Gets the most recent message movement tasks (up to 10) under a specific source queue.

CancelMessageMoveTask

Cancels a specified message movement task. A message movement can only be cancelled when the current status is RUNNING.

Configuring a dead-letter queue redrive for an existing standard queue using the Amazon SQS console

  1. Open the Amazon SQS console at https://console.amazonaws.cn/sqs/.

  2. In the navigation pane, choose Queues.

  3. Choose the name of queue that you have configured as a dead-letter queue.

  4. Choose Start DLQ redrive.

  5. Under Redrive configuration, for Message destination, do either of the following:

    • To redrive messages to their source queue, choose Redrive to source queue(s).

    • To redrive messages to another queue, choose Redrive to custom destination. Then, enter the Amazon Resource Name (ARN) of an existing destination queue.

  6. Under Velocity control settings, choose one of the following:

    • System optimized - Redrive dead-letter queue messages at the maximum number of messages per second.

    • Custom max velocity - Redrive dead-letter queue messages with a custom maximum rate of messages per second. The maximum allowed rate is 500 messages per second.

      • It is recommended to start with a small value for Custom max velocity and verify that the source queue doesn't get overwhelmed with messages. From there, gradually ramp-up the Custom max velocity value, continuing to monitor the state of the source queue.

  7. When you finish configuring the dead-letter queue redrive, choose Redrive messages.

    Important

    Amazon SQS doesn't support filtering and modifying messages while redriving them from the dead-letter queue.

    A dead-letter queue redrive task can run a maximum of 36 hours. Amazon SQS supports a maximum of 100 active redrive tasks per account.

  8. If you want to cancel the message redrive task, on the Details page for your queue, choose Cancel DLQ redrive. When canceling an in progress message redrive, any messages that have already been successfully moved to their move destination queue will remain in the destination queue.

Configuring queue permissions for dead-letter queue redrive

You can give user access to specific dead-letter queue actions by adding permissions to your policy. The minimum required permissions for a dead-letter queue redrive are as follows:

Minimum Permissions Required API methods
To start a message redrive
  • Add the sqs:StartMessageMoveTask, sqs:ReceiveMessage, sqs:DeleteMessage, and sqs:GetQueueAttributes of the dead-letter queue. If either the dead-letter queue or the original source queue are encrypted (also known as an SSE queue), kms:Decrypt for any KMS key that has been used to encrypt the messages is also required.

  • Add the sqs:SendMessage of the destination queue. If the destination queue is encrypted, kms:GenerateDataKey and kms:Decrypt are also required.

To cancel an in-progress message redrive
  • Add the sqs:CancelMessageMoveTask, sqs:ReceiveMessage, sqs:DeleteMessage, and sqs:GetQueueAttributes of the dead-letter queue. If the dead-letter queue is encrypted (also known as an SSE queue), kms:Decrypt is also required.

To show a message move status
  • Add the sqs:ListMessageMoveTasks and sqs:GetQueueAttributes of the dead-letter queue.

To configure permissions for an encrypted queue pair (a source queue with a dead-letter queue)

Use the following steps to configure minimum permissions for a dead-letter queue redrive:

  1. Sign in to the Amazon Web Services Management Console and open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Policies.

  3. Create a policy with the following permissions and attach it to your login IAM user or role:

    • sqs:StartMessageMoveTask

    • sqs:CancelMessageMoveTask

    • sqs:ListMessageMoveTasks

    • sqs:ListDeadLetterSourceQueues

    • sqs:ReceiveMessage

    • sqs:DeleteMessage

    • sqs:GetQueueAttributes

    • The Resource ARN of the dead-letter queue (for example, "arn:aws-cn:sqs:<DLQ_region>:<DLQ_accountId>:<DLQ_name>").

    • sqs:SendMessage

    • The Resource ARN of the destination queue (for example, "arn:aws-cn:sqs:<DestQueue_region>:<DestQueue_accountId>:<DestQueue_name>").

    • kms:Decrypt – Allows decryption action.

    • kms:GenerateDataKey

    • The Resource ARN(s) of any KMS encryption key that has been used to encrypt the messages in the original source queue (for example, "arn:aws-cn:kms:<region>:<accountId>:key/<keyId_used to encrypt the message body>").

    • The Resource ARN of the KMS encryption key that is used for the redrive destination queue (for example, "arn:aws-cn:kms:<region>:<accountId>:key/<keyId_used for the destination queue>").

    Your access policy should resemble the following:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sqs:StartMessageMoveTask", "sqs:CancelMessageMoveTask", "sqs:ListMessageMoveTasks", "sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes", "sqs:ListDeadLetterSourceQueues" ], "Resource": "arn:aws-cn:sqs:<DLQ_region>:<DLQ_accountId>:<DLQ_name>" }, { "Effect": "Allow", "Action": "sqs:SendMessage", "Resource": "arn:aws-cn:sqs:<DestQueue_region>:<DestQueue_accountId>:<DestQueue_name>" }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "arn:aws-cn:kms:<region>:<accountId>:key/<keyId>" } ] }
To configure permissions using a non-encrypted queue pair (a source queue with a dead-letter queue)

Use the following steps to configure minimum permissions for a standard unencrypted dead-letter queue. Required minimum permissions are to receive, delete and get attributes from the dead-letter queue, and send attributes to the source queue.

  1. Sign in to the Amazon Web Services Management Console and open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Policies.

  3. Create a policy with the following permissions and attach it to your login IAM user or role:

    • sqs:StartMessageMoveTask

    • sqs:CancelMessageMoveTask

    • sqs:ListMessageMoveTasks

    • sqs:ListDeadLetterSourceQueues

    • sqs:ReceiveMessage

    • sqs:DeleteMessage

    • sqs:GetQueueAttributes

    • The Resource ARN of the dead-letter queue (for example, "arn:aws-cn:sqs:<DLQ_region>:<DLQ_accountId>:<DLQ_name>") .

    • sqs:SendMessage

    • The Resource ARN of the destination queue (for example, "arn:aws-cn:sqs:<DestQueue_region>:<DestQueue_accountId>:<DestQueue_name>").

    Your access policy should resemble the following:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sqs:StartMessageMoveTask", "sqs:CancelMessageMoveTask", "sqs:ListMessageMoveTasks", "sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes", "sqs:ListDeadLetterSourceQueues" ], "Resource": "arn:aws-cn:sqs:<DLQ_region>:<DLQ_accountId>:<DLQ_name>" }, { "Effect": "Allow", "Action": "sqs:SendMessage", "Resource": "arn:aws-cn:sqs:<DestQueue_region>:<DestQueue_accountId>:<DestQueue_name>" } ] }