Using dead-letter queues in Amazon SQS
Amazon SQS supports dead-letter queues (DLQs), which source queues can target for messages that are not processed successfully. DLQs are useful for debugging your application because you can isolate unconsumed messages to determine why processing did not succeed. For optimal performance, it is a best practice to keep the source queue and DLQ within the same Amazon Web Services account and Region. Once messages are in a dead-letter queue, you can:
-
Examine logs for exceptions that might have caused messages to be moved to a dead-letter queue.
-
Analyze the contents of messages moved to the dead-letter queue to diagnose application issues.
-
Determine whether you have given your consumer sufficient time to process messages.
-
Move messages out of the dead-letter queue using dead-letter queue redrive.
You must first create a new queue before configuring it as a dead-letter queue. For information about configuring a dead-letter queue using the Amazon SQS console, see Configure a dead-letter queue using the Amazon SQS console. For help with dead-letter queues, such as how to configure an alarm for any messages moved to a dead-letter queue, see Creating alarms for dead-letter queues using Amazon CloudWatch.
Note
Don't use a dead-letter queue with a FIFO queue if you don't want to break the exact order of messages or operations. For example, don't use a dead-letter queue with instructions in an Edit Decision List (EDL) for a video editing suite, where changing the order of edits changes the context of subsequent edits.
Using policies for dead-letter queues
Use a redrive policy to specify the
maxReceiveCount. The maxReceiveCount is the number of
times a consumer can receive a message from a source queue before it is moved to a
dead-letter queue. For example, if the maxReceiveCount is set to a low
value such as 1, one failure to receive a message would cause the message to move to the
dead-letter queue. To ensure that your system is resilient against errors, set the
maxReceiveCount high enough to allow for sufficient retries.
The redrive allow policy specifies which source
queues can access the dead-letter queue. You can choose whether to allow all source
queues, allow specific source queues, or deny all source queues use of the dead-letter
queue. The default allows all source queues to use the dead-letter queue. If you choose
to allow specific queues using the byQueue option, you can specify up to 10
source queues using the source queue Amazon Resource Name (ARN). If you specify
denyAll, the queue cannot be used as a dead-letter queue.
Understanding message retention periods for dead-letter queues
For standard queues, the expiration of a message is always based on its original
enqueue timestamp. When a message is moved to a dead-letter queue, the enqueue timestamp
is unchanged. The ApproximateAgeOfOldestMessage metric indicates when the
message moved to the dead-letter queue, not when the message was originally sent. For
example, assume that a message spends 1 day in the original queue before it's moved to a
dead-letter queue. If the dead-letter queue's retention period is 4 days, the message is
deleted from the dead-letter queue after 3 days and the
ApproximateAgeOfOldestMessage is 3 days. Thus, it is a best practice to
always set the retention period of a dead-letter queue to be longer than the retention
period of the original queue.
For FIFO queues, the enqueue timestamp resets when the message is moved to a
dead-letter queue. The ApproximateAgeOfOldestMessage metric indicates when
the message moved to the dead-letter queue. In the same example above, the message is
deleted from the dead-letter queue after four days and the
ApproximateAgeOfOldestMessage is four days.