# Amazon SNS message archiving and replay for FIFO topics
<a name="fifo-message-archiving-replay"></a>

## What is message archiving and replay?
<a name="what-is-message-archiving-and-replay"></a>

Amazon SNS provides a no-code message archiving and replay feature, specifically designed for FIFO (First-In-First-Out) topics. This feature allows topic owners to store messages directly within the topic archive for up to 365 days and replay them to subscribers when needed. Message archiving and replay are essential for recovering lost messages and synchronizing applications across regions or systems by replicating states.

This functionality can be accessed through the Amazon API, SDK, Amazon CloudFormation, and Amazon Web Services Management Console.

**Key use cases**
+ **Message recovery** – Recover messages lost due to downstream application failures by replaying them to the subscriber’s endpoint.
+ **State replication** – Replicate the state of an existing system in a new environment by replaying messages starting from a specific timestamp.
+ **Error correction** – Resend missed messages during outages to ensure all events are processed correctly.

### Components of message archiving and replay
<a name="message-archiving-and-replay-components"></a>

Manage message archiving and replay for Amazon SNS FIFO topics, including setting retention periods, monitoring archived messages using CloudWatch, initiating replays through subscription attributes, and understanding the permissions required to modify and initiate replays.

**Message archiving**
+ The topic owner enables the archiving feature and sets the message retention period, which can be up to 365 days. For more, see [Amazon SNS message archiving for FIFO topic owners](message-archiving-and-replay-topic-owner.md)
+ CloudWatch metrics help monitor the archived messages.

**Message replay**
+ A subscriber initiates a replay, selecting the time window for the messages to be reprocessed to the subscribed endpoint. For more see, [Amazon SNS message replay for FIFO topic subscribers](message-archiving-and-replay-subscriber.md). 
+ You manage the replay through subscription attributes using the `ReplayPolicy` feature.

**Relevant permissions**
+ **`SetSubscriptionAttributes`** – Required to configure or modify replay settings using the `ReplayPolicy` attribute on a subscription.
+ **`Subscribe`** – Necessary to attach a new subscription and initiate replays.
+ **`GetTopicAttributes`** – Allows viewing the topic's properties, but replay initiation primarily revolves around subscription management.

# Amazon SNS message archiving for FIFO topic owners
<a name="message-archiving-and-replay-topic-owner"></a>

Message archiving provides the ability to archive a single copy of all messages published to your topic. You can store published messages within your topic by enabling the message archive policy on the topic, which enables message archiving for all subscriptions linked to that topic. Messages can be archived for a minimum of one day to a maximum of 365 days.

Additional charges apply when setting an archive policy. For pricing information, see [Amazon SNS pricing](https://aws.amazon.com/sns/pricing/).

## Create a message archive policy using the Amazon Web Services Management Console
<a name="message-archiving-and-replay-topic-console"></a>

Use this option to create a new message archive policy using the Amazon Web Services Management Console.

1. Sign in to the [Amazon SNS console](https://console.amazonaws.cn/sns/home).

1. Choose a topic or create a new one. To learn more about creating topics, see [Creating an Amazon SNS topic](sns-create-topic.md).
**Note**  
Amazon SNS message archiving and replay is only available for application-to-application (A2A) FIFO topics.

1. On the **Edit topic** page, expand the **Archive policy** section.

1. Enable the **Archive policy** feature, and enter the **number of days** for which you want to archive messages in the topic.

1. Choose **Save changes**.

**To view, edit, and deactivate a message archiving topic policy**
+ On the **Topic details** page, the **Retention policy** displays the status of the archive policy, including the number of days for which it is set. Select the **Archive policy** tab to view the following message archive details:
  + **Status** – The archive and replay status appears as **active** when an archive policy is applied. The archive and replay status appears as **inactive** when the archive policy is set to an empty JSON object.
  + **Message retention period** – The specified number of days for message retention.
  + **Archive start date** – The date from which subscribers can replay messages.
  + **JSON preview** – The JSON preview of the archive policy.
+ (Optional) To **edit** an archive policy, go to the topic summary page and choose **Edit**.
+ (Optional) To **deactivate** an archive policy, go to the topic summary page and choose **Edit**. Deactivate the **Archive Policy** and choose **Save changes**.
+ (Optional) To **delete** a topic with an archive policy, you must first deactivate the archive policy as previously described.
**Important**  
To avoid accidental message deletions, you can not delete a topic with an active message archive policy. The topic's message archive policy must be deactivated before the topic can be deleted. When you deactivate a message archive policy, Amazon SNS deletes all of the archived messages. When deleting a topic, subscriptions are removed, and any messages in transit may not be delivered.

## Create a message archive policy using the API
<a name="message-archiving-and-replay-topic-api"></a>

To create a message archive policy using the API, you need to add the attribute `ArchivePolicy` to your topic. You can set an `ArchivePolicy` using the API actions `CreateTopic` and `SetTopicAttributes`. `ArchivePolicy` has a single value, `MessageRetentionPeriod`, which represents the number of days Amazon SNS retains messages. To activate message archiving for your topic, set the `MessageRetentionPeriod` to an integer value greater than zero. For example, to retain messages in your archive for 30 days, set the `ArchivePolicy` to:

```
{
    "ArchivePolicy": {
        "MessageRetentionPeriod": "30"
    }
}
```

To disable message archiving for your topic, and clear the archive, unset the `ArchivePolicy`, as follows:

```
{}
```

## Create a message archive policy using the SDK
<a name="message-archiving-and-replay-topic-sdk"></a>

To use an Amazon SDK, you must configure it with your credentials. For more information, see [Shared `config` and `credentials` files](https://docs.amazonaws.cn/sdkref/latest/guide/file-format.html) in the * Amazon SDKs and Tools Reference Guide*.

The following code example shows how to set the `ArchivePolicy` for an Amazon SNS topic to retain all messages published to the topic for 30 days.

```
// Specify the ARN of the Amazon SNS topic to set the ArchivePolicy for.
String topicArn =
    "arn:aws-cn:sns:us-east-2:123456789012:MyArchiveTopic.fifo";

// Set the MessageRetentionPeriod to 30 days for the ArchivePolicy.
String archivePolicy =
    "{\"MessageRetentionPeriod\":\"30\"}";

// Set the ArchivePolicy for the Amazon SNS topic
SetTopicAttributesRequest request = new SetTopicAttributesRequest()
    .withTopicArn(topicArn)
    .withAttributeName("ArchivePolicy")
    .withAttributeValue(archivePolicy);
sns.setTopicAttributes(request);
```

## Create a message archive policy using Amazon CloudFormation
<a name="message-archiving-and-replay-topic-cfn"></a>

To create an archive policy using Amazon CloudFormation see [https://docs.amazonaws.cn/AWSCloudFormation/latest/UserGuide/aws-resource-sns-topic.html](https://docs.amazonaws.cn/AWSCloudFormation/latest/UserGuide/aws-resource-sns-topic.html) in the *Amazon CloudFormation User Guide*.

## Grant access to an encrypted archive
<a name="message-archiving-and-replay-topic-kms"></a>

Before a subscriber can begin replaying messages from an encrypted topic, you must complete the following steps. Because past messages are replayed, Amazon SNS needs to be provisioned `Decrypt` access to the KMS key that was used to encrypt the messages in the archive.

1. When you encrypt messages with a KMS key and store them within the topic, you must grant Amazon SNS the ability to decrypt these messages via Key Policy. For more, see [Grant decrypt permissions to Amazon SNS](#message-archiving-and-replay-topic-decrypt-permissions).

1. Enable Amazon KMS for Amazon SNS. For more, see [Configuring Amazon KMS permissions](sns-key-management.md#sns-what-permissions-for-sse).

**Important**  
When you add the new sections to your KMS key policy, do not change any existing sections in the policy. If encryption is enabled on a topic, and the KMS key is disabled or deleted, or the KMS key policy is not correctly configured for Amazon SNS, Amazon SNS cannot replay messages to your subscribers. 

### Grant decrypt permissions to Amazon SNS
<a name="message-archiving-and-replay-topic-decrypt-permissions"></a>

In order for Amazon SNS to access encrypted messages from within your topic’s archive and replay them to subscribed endpoints, you must enable the Amazon SNS service principle to decrypt these messages.

The following is an example policy that is required to allow the Amazon SNS service principal to decrypt stored messages during a replay of historical messages from within your topic.

```
{
    "Sid": "Allow SNS to decrypt archived messages",
    "Effect": "Allow",
    "Principal": {
        "Service": "sns.amazonaws.com"
    },
    "Action": [
        "kms:Decrypt",
        "kms:GenerateDataKey"
    ],
    "Resource": "*"
}
```

## Monitor message archive metrics using Amazon CloudWatch
<a name="message-archiving-and-replay-topic-cloudwatch"></a>

You can monitor archived messages using Amazon CloudWatch using the following metrics. To be notified of anomalies in your workloads and help avoid impact, you can configure Amazon CloudWatch alarms on these metrics. For more details, see [Logging and monitoring in Amazon SNS](sns-logging-monitoring.md).


| Metric | Description | 
| --- | --- | 
|  **ApproximateNumberOfMessagesArchived**  |  Provides the topic owner with the aggregate number of messages archived in the topic archive, at 60-minute resolution.  | 
|  **ApproximateNumberOfBytesArchived**   |  Provides the topic owner with the aggregate number of bytes archived, across all messages in the topic archive, at 60-minute resolution.  | 
|  **NumberOfMessagesArchiveProcessing**   |  Provides the topic owner with the number of messages saved to the topic archive during the interval in 1-minute resolution.  | 
|  **NumberOfBytesArchiveProcessing**  |  Provides the topic owner with the aggregate number of bytes saved to the topic archive during the interval in 1-minute resolution.  | 

The `GetTopicAttributes` API has a `BeginningArchiveTime` property, which represents the oldest timestamp at which a subscriber can start a replay. The following represents a sample response for this API action:

```
{
 "ArchivePolicy": {
    "MessageRetentionPeriod": "<integer>"
  },
  "BeginningArchiveTime": "<timestamp>",
  ...
}
```

# Amazon SNS message replay for FIFO topic subscribers
<a name="message-archiving-and-replay-subscriber"></a>

Amazon SNS replay allows topic subscribers to retrieve and redeliver archived messages from the topic data store to a subscribed endpoint.
+ Messages can be replayed immediately after the subscription is created.
+ A replayed message retains the same content, `MessageId`, and `Timestamp` as the original.
+ The message includes a `Replayed` attribute to indicate that it is a replayed message.
+ To replay only specific messages, apply a filter policy to your subscription.

For more on filtering messages, see [Filter replayed messages](#message-archiving-and-replay-subscription-filtering).

## Create a message replay policy using the Amazon Web Services Management Console
<a name="message-archiving-and-replay-replaying-console"></a>

Use this option to create a new replay policy using the Amazon Web Services Management Console.

1. Sign in to the [Amazon SNS console](https://console.amazonaws.cn/sns/home).

1. Choose a topic subscription or create a new one. To learn more about creating subscriptions, see [Creating a subscription to an Amazon SNS topic](sns-create-subscribe-endpoint-to-topic.md).

1. To initiate the message replay, go to the **Replay** drop-down and choose **Start replay**.

1. From the **Replay timeframe** modal, make the following selections:

   1. **Choose replay start date and time** – Choose the **date** (YYYY/MM/DD format) and **time** (24-hour hh:mm:ss format) from which you want to start replaying archived messages. The start time should be later than the beginning of the approximated archive time.

   1. **(Optional) Choose replay end date and time** – Choose the **date** (YYYY/MM/DD format) and **time** (24-hour hh:mm:ss format) when you want to stop replaying archived messages.

   1. Choose **Start replay**.

1. (Optional) To **stop** a message replay, go to the **Subscription details** page and choose **Stop replay** from the **Replay** drop-down.

1. (Optional) To **monitor** message replay metrics from within this workflow using CloudWatch, see [Monitor message replay metrics using Amazon CloudWatch](#message-archiving-and-replay-subscription-cloudwatch).

**To view and edit a message replay policy**

You can perform the following actions from the **Subscription details** page:
+ To **view** the message replay status, the **Replay status** field displays the following values:
  + **Completed** – The replay has successfully redelivered all messages, and is now delivering newly published messages.
  + **In progress** – The replay is currently replaying the selected messages.
  + **Failed** – The replay was unable to complete.
  + **Pending** – The default state while the replay initiates.
+ (Optional) To **modify** a message replay policy, go to the **Subscription details** page and choose **Start replay** from the **Replay** drop-down. Starting a replay will replace the existing replay.

## Add a replay policy to the subscription using the API
<a name="message-archiving-and-replay-subscription-api"></a>

To replay archived messages use the attribute `ReplayPolicy`. `ReplayPolicy` can be used with the `Subscribe` and `SetSubscriptionAttributes` API actions. This policy has the following values:
+ **`StartingPoint`** (Required) – Signals where to start replaying messages from.
+ **`EndingPoint`** (Optional) – Signals when to stop replaying messages. If `EndingPoint` is omitted, then the replay will continue until caught up to the current time.
+ **`PointType`** (Required) – Sets the type of starting and ending points. Currently, the supported value for `PointType` is `Timestamp`.

For example, to recover from a downstream failure and resend all messages for a two hour time period on October 1, 2023, use the `SetSubscriptionAttributes` API action to set a `ReplayPolicy` as follows:

```
{
  "PointType":"Timestamp", 
  "StartingPoint":"2023-10-01T10:00:00.000Z", 
  "EndingPoint":"2023-10-01T12:00:00.000Z"
}
```

To replay all messages sent to the topic as of October 1, 2023, and continue receiving all newly published messages to your topic, use the `SetSubscriptionAttributes` API action to set a `ReplayPolicy` on your subscription as follows:

```
{
  "PointType":"Timestamp",
  "StartingPoint":"2023-10-01T00:00:00.000Z"
}
```

To verify that a message has been replayed, the boolean attribute `Replayed` is added to each replayed message.

## Add a replay policy to the subscription using the SDK
<a name="message-replay-sdk"></a>

To use an Amazon SDK, you must configure it with your credentials. For more information, see [Shared `config` and `credentials` files](https://docs.amazonaws.cn/sdkref/latest/guide/file-format.html) in the * Amazon SDKs and Tools Reference Guide*.

The following code example shows how to set the `ReplayPolicy` on a subscription to redeliver messages from the Amazon SNS FIFO topic's archive for a 2-hour time window on October 1st 2023.

```
// Specify the ARN of the Amazon SNS subscription to initiate the ReplayPolicy on.
String subscriptionArn =
    "arn:aws-cn:sns:us-east-2:123456789012:MyArchiveTopic.fifo:1d2a3e9d-7f2f-447c-88ae-03f1c68294da";

// Set the ReplayPolicy to replay messages from the topic's archive 
// for a 2 hour time period on October 1st 2023 between 10am and 12pm UTC.
String replayPolicy =
    "{\"PointType\":\"Timestamp\",\"StartingPoint\":\"2023-10-01T10:00:00.000Z\",\"EndingPoint\":\"2023-10-01T12:00:00.000Z\"}";

// Set the ArchivePolicy for the Amazon SNS topic
SetSubscriptionAttributesRequest request = new SetSubscriptionAttributesRequest()
    .withSubscriptionArn(subscriptionArn)
    .withAttributeName("ReplayPolicy")
    .withAttributeValue(replayPolicy);
sns.setSubscriptionAttributes(request);
```

## Understanding the EndingPoint
<a name="message-archiving-and-replay-understanding-endpoint"></a>

When you apply a `ReplayPolicy` to an Amazon SNS subscription, the `EndingPoint` value is optional. If no `EndingPoint` is provided, the replay will start from the specified `StartingPoint` and continue until it catches up to the current time, including processing any newly published messages. Once caught up, the subscription will function as a regular subscription, receiving new messages as they are published.

If an `EndingPoint` is specified, the service will replay messages from the `StartingPoint` up to the `EndingPoint` and then stop. **This action effectively pauses the subscription.** While the subscription is paused, newly published messages will not be delivered to the subscribed endpoint.

To resume message delivery, apply a new `ReplayPolicy` without providing an `EndingPoint`, and set the `StartingPoint` to the desired point in time from which to continue receiving messages. For example, to resume a subscription from where a prior replay finished, set the new `StartingPoint` to the previously provided `EndingPoint`.

## Filter replayed messages
<a name="message-archiving-and-replay-subscription-filtering"></a>

Amazon SNS message filtering let's you control the replayed messages that Amazon SNS replays to your subscriber endpoint. When message filtering and message archiving are both enabled, Amazon SNS first retrieves the message from the topic’s data store, then applies the message against the subscription’s `FilterPolicy`. The message is delivered to the subscribed endpoint when there is a match, otherwise message is filtered out. For more information, see [Amazon SNS subscription filter policies](sns-subscription-filter-policies.md).

## Monitor message replay metrics using Amazon CloudWatch
<a name="message-archiving-and-replay-subscription-cloudwatch"></a>

You can monitor replay messages using Amazon CloudWatch using the following metrics. To be notified of anomalies in your workloads and help avoid impact, you can configure Amazon CloudWatch alarms on these metrics. For more details, see [Logging and monitoring in Amazon SNS](sns-logging-monitoring.md).


| Metric | Description | 
| --- | --- | 
|  **NumberOfReplayedNotificationsDelivered**  |  Provides the subscriber with the aggregate number of messages replayed from the topic archive, at 1-minute resolution.  | 
|  **NumberOfReplayedNotificationsFailed**   |  Provides the subscriber with the aggregate number of messages replayed that failed to deliver from the topic archive, at 1-minute resolution.  |