Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Using the warm-after setting in time series collections

Amazon OpenSearch Serverless NextGen time series collections use a combination of hot and warm storage. You use the index.warm.after index setting to specify the warm-after period, which is how long data in an index stays in hot storage before OpenSearch Serverless moves it to warm storage. You can specify the setting when you create an index or in an index template, and you can change it later by updating the index settings. Warm storage applies to data within an index, not to the index as a whole, so an index can contain both hot and warm data at the same time.

Specifying the warm-after period

You specify the warm-after period as a positive whole number followed by a unit suffix: d for days or h for hours (such as 7d or 36h). This setting is optional. If you don't specify it, the index uses a default warm-after period of 1d. Other units, such as minutes or seconds, aren't supported. The value must meet the following limits:

When OpenSearch Serverless applies the value to an index, it removes leading zeros. For example, a value of 007d is stored in the index settings as 7d.

OpenSearch Serverless resolves the warm-after period for an index in the following order of precedence:

The following OpenSearch REST API request creates an index named my-logs with a warm-after period of 7 days.

PUT /my-logs
{
  "settings": {
    "index": {
      "warm": {
        "after": "7d"
      }
    }
  }
}

If neither the create index request nor a matching index template specifies the setting, the index uses a warm-after period of 1d. To view the warm-after period of an index, send a GET /my-logs/_settings request.

Changing the warm-after period

To change the warm-after period of an existing index, update the index.warm.after setting. The following OpenSearch REST API request changes the warm-after period of the my-logs index to 3 days.

PUT /my-logs/_settings
{
  "index": {
    "warm": {
      "after": "3d"
    }
  }
}

The new value must meet the same limits as when you create an index. OpenSearch Serverless checks the new value against the current data retention period of the index. If you update index.data.retention and index.warm.after in the same request, OpenSearch Serverless checks the warm-after period against the new data retention period.

The change has the following effects:

You can't remove the index.warm.after setting from an index. A request that sets the value to null is rejected. To change the warm-after period, specify a new valid value.

Moving data to warm storage

OpenSearch Serverless moves data to warm storage in the order that the data was ingested. Older data in an index moves to warm storage first. Recently ingested data stays in hot storage. As a result, an index usually contains both hot and warm data.

You work with the index the same way regardless of which storage holds the data: you search, update, and delete documents through the index, and OpenSearch Serverless reads from both hot and warm storage.

OpenSearch Serverless stores the data of an index in underlying storage that it creates over time. The warm-after period is measured from when OpenSearch Serverless creates the underlying storage that holds the data. It isn't measured from when you ingest a document or from the value of a document's timestamp field. Data in the same underlying storage moves to warm storage together. Because of this, a document that you ingest after that storage is created can spend less than the full warm-after period in hot storage.

Resolving warm-after validation errors

If the index.warm.after value isn't valid, or the setting isn't supported for the target, OpenSearch Serverless rejects the request with an HTTP 400 error. The error message describes the cause. In the following messages, <value> is the value that you specified (without leading zeros, in the messages about maximum values), and <N> is the data retention period of the index in days.