Amazon SNS message delivery retries
Amazon SNS defines a delivery policy for each delivery protocol. The delivery policy defines how Amazon SNS retries the delivery of messages when server-side errors occur (when the system that hosts the subscribed endpoint becomes unavailable). When the delivery policy is exhausted, Amazon SNS stops retrying the delivery and discards the message—unless a dead-letter queue is attached to the subscription. For more information, see Amazon SNS dead-letter queues.
Delivery protocols and policies
Note
-
With the exception of HTTP/S, you can't change Amazon SNS-defined delivery policies. Only HTTP/S supports custom policies. See Creating an HTTP/S delivery policy.
-
Amazon SNS applies jittering to delivery retries. For more information, see the Exponential Backoff and Jitter
post on the Amazon Architecture Blog. -
The total policy retry time for an HTTP/S endpoint cannot be greater than 3,600 seconds. This is a hard limit and cannot be increased.
Endpoint type | Delivery protocols | Immediate retry (no delay) phase | Pre-backoff phase | Backoff phase | Post-backoff phase | Total attempts |
---|---|---|---|---|---|---|
Amazon managed endpoints | Amazon Data Firehose¹ | 3 times, without delay | 2 times, 1 second apart | 10 times, with exponential backoff, from 1 second to 20 seconds | 100,000 times, 20 seconds apart | 100,015 times, over 23 days |
Amazon Lambda | ||||||
Amazon SQS | ||||||
Customer managed endpoints | SMTP | 0 times, without delay | 2 times, 10 seconds apart | 10 times, with exponential backoff, from 10 seconds to 600 seconds (10 minutes) | 38 times, 600 seconds (10 minutes) apart | 50 attempts, over 6 hours |
SMS | ||||||
Mobile push |
¹ For throttling errors with the Firehose protocol, Amazon SNS uses the same delivery policy as for customer managed endpoints.
Delivery policy stages
The following diagram shows the phases of a delivery policy.
Each delivery policy is comprised of four phases.
-
Immediate Retry Phase (No Delay) – This phase occurs immediately after the initial delivery attempt. There is no delay between retries in this phase.
-
Pre-Backoff Phase – This phase follows the Immediate Retry Phase. Amazon SNS uses this phase to attempt a set of retries before applying a backoff function. This phase specifies the number of retries and the amount of delay between them.
-
Backoff Phase – This phase controls the delay between retries by using the retry-backoff function. This phase sets a minimum delay, a maximum delay, and a retry-backoff function that defines how quickly the delay increases from the minimum to the maximum delay. The backoff function can be arithmetic, exponential, geometric, or linear.
-
Post-Backoff Phase – This phase follows the backoff phase. It specifies a number of retries and the amount of delay between them. This is the final phase.
Creating an HTTP/S delivery policy
You can use a delivery policy and its four phases to define how Amazon SNS retries the delivery of messages to HTTP/S endpoints. Amazon SNS lets you override the default retry policy for HTTP endpoints when you might, for example, want to customize the policy based on your HTTP server's capacity.
You can set your HTTP/S delivery policy as a JSON object at the subscription or topic
level. When you define the policy at the topic level, it applies to all HTTP/S
subscriptions associated with the topic. To set the delivery policy at the subscription
level, you can use either the Subscribe
or SetSubscriptionAttributes
API action. To set the delivery
policy at the topic level, you can use either the CreateTopic
or SetTopicAttributes
API action. Alternatively, you can also
use the AWS::SNS::Subscription resource in your Amazon CloudFormation templates.
You should customize your delivery policy according to your HTTP/S server's capacity. You can set the policy as a topic attribute or a subscription attribute. If all HTTP/S subscriptions in your topic target the same HTTP/S server, we recommend that you set the delivery policy as a topic attribute, so that it remains valid for all HTTP/S subscriptions in the topic. Otherwise, you must compose a delivery policy for each HTTP/S subscription in your topic, according the capacity of the HTTP/S server that the policy targets.
You can also set the Content-Type header in the request policy to specify the media
type of the notification. By default, Amazon SNS sends all the notification to HTTP/S
endpoints with content type set to text/plain; charset=UTF-8
. Amazon SNS lets
you override the default request policy. See the table below for supported headerContentType and
restraints.
The following JSON object represents a delivery policy that instructs Amazon SNS to retry a failed HTTP/S delivery attempt, as follows:
-
3 times immediately in the no-delay phase
-
2 times (1 second apart) in the pre-backoff phase
-
10 times (with exponential backoff from 1 second to 60 seconds)
-
35 times (60 seconds apart) in the post-backoff phase
In this sample delivery policy, Amazon SNS makes a total of 50 attempts before discarding the message. To keep the message after the retries specified in the delivery policy are exhausted, configure your subscription to move undeliverables messages to a dead-letter queue (DLQ). For more information, see Amazon SNS dead-letter queues.
Note
This delivery policy also instructs Amazon SNS to throttle deliveries to no more than
10 per second, using the maxReceivesPerSecond
property. This
self-throttling rate could result in more messages published (inbound traffic) than
delivered (outbound traffic). When there's more inbound than outbound traffic, your
subscription can accumulate a large message backlog, which might cause high message
delivery latency. In your delivery policies, be sure to specify a value for
maxReceivesPerSecond
that doesn't adversely impact your workload.
Note
This delivery policy overrides the default content type for HTTP/S notification to
application/json
.
{ "healthyRetryPolicy": { "minDelayTarget": 1, "maxDelayTarget": 60, "numRetries": 50, "numNoDelayRetries": 3, "numMinDelayRetries": 2, "numMaxDelayRetries": 35, "backoffFunction": "exponential" }, "throttlePolicy": { "maxReceivesPerSecond": 10 }, "requestPolicy": { "headerContentType": "application/json" } }
The delivery policy is composed of a retry policy, throttle policy and a request policy. In total, there are 9 attributes in a delivery policy.
Policy | Description | Constraint |
---|---|---|
minDelayTarget |
The minimum delay for a retry. Unit: Seconds |
1 to maximum delay Default: 20 |
maxDelayTarget |
The maximum delay for a retry. Unit: Seconds |
Minimum delay to 3,600 Default: 20 |
numRetries |
The total number of retries, including immediate, pre-backoff, backoff, and post-backoff retries. | 0 to 100 Default: 3 |
numNoDelayRetries |
The number of retries to be done immediately, with no delay between them. | 0 or greater Default: 0 |
numMinDelayRetries |
The number of retries in the pre-backoff phase, with the specified minimum delay between them. | 0 or greater Default: 0 |
numMaxDelayRetries |
The number of retries in the post-backoff phase, with the maximum delay between them. | 0 or greater Default: 0 |
backoffFunction |
The model for backoff between retries. |
One of four options:
Default: linear |
maxReceivesPerSecond
|
The maximum number of deliveries per second, per subscription. | 1 or greater Default: No throttling |
headerContentType
|
The content type of the notification being sent to HTTP/S endpoints. |
If the request policy is not defined, the content type defaults to
When the raw message delivery is disabled for a subscription
(default), or when the delivery policy is defined on the topic-level,
the supported header content types are When the raw message delivery is enabled for a subscription, the following content types are supported:
|
Amazon SNS uses the following formula to calculate the number of retries in the backoff phase:
numRetries - numNoDelayRetries - numMinDelayRetries - numMaxDelayRetries
You can use three parameters to control the frequency of retries in the backoff phase.
-
minDelayTarget
– Defines the delay associated with the first retry attempt in the backoff phase. -
maxDelayTarget
– Defines the delay associated with the final retry attempt in the backoff phase. -
backoffFunction
– Defines the algorithm that Amazon SNS uses to calculate the delays associated with all of the retry attempts between the first and last retries in the backoff phase. You can use one of four retry-backoff functions.
The following diagram shows how each retry backoff function affects the delay associated with retries during the backoff phase: A delivery policy with the total number of retries set to 10, the minimum delay set to 5 seconds, and the maximum delay set to 260 seconds. The vertical axis represents the delay in seconds associated with each of the 10 retries. The horizontal axis represents the number of retries, from the first to the tenth attempt.