Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

For similar capabilities to Amazon Timestream for LiveAnalytics, consider Amazon Timestream for InfluxDB. It offers simplified data ingestion and single-digit millisecond query response times for real-time analytics. Learn more here.

Size cluster functions independently with deployment configurations

An InfluxDB 3 Enterprise cluster does four things: it ingests writes, serves queries, runs processing engine plugins, and compacts data. Each of those has a different appetite for CPU and memory. Compaction is bursty and memory-hungry; query serving scales with concurrency; ingest scales with write throughput.

A deployment configuration lets you size each function separately. You declare groups of nodes, and each group gets its own instance type, its own node count, and its own set of functions. Because the groups can differ in size, the resulting cluster is heterogeneous, such as a large dedicated compactor alongside small query nodes instead of paying for the largest instance type across every node.

You set the shape with the --deployment-configuration parameter on create-db-cluster, and you change it later with the same parameter on update-db-cluster.

Node groups

A deployment configuration is a list of node groups. Every group declares three things:

The four functions map onto cluster behavior as follows:

Modes combine freely within the limits described in Shape rules and limits, so you can dedicate nodes narrowly. A group declared as ["INGEST"] with no QUERY is a write-only node: it accepts writes and does not serve queries, which keeps query load off your ingest path entirely. The one pairing the service requires is PROCESS with QUERY, because InfluxDB enables query serving implicitly whenever the processing engine is enabled.

Shape rules and limits

Timestream validates the shape before it provisions anything. A request that breaks any of the following rules fails with a validation error and no nodes are created.

Separately from the node limits above, a request itself has a fixed shape: a deployment configuration carries 1 to 10 node groups, each group declares 1 to 4 distinct modes and a count of at least 1, and no two groups may be identical in all three of mode set, instance type, and count.

Working out a valid shape

The limits interact, so it is easier to build a shape in the order the service checks it. Start from the compactor, then place ingest, then spend what is left on query:

These shapes are all valid, and show the range the limits allow:

And these are rejected:

Create a cluster with a deployment configuration

The following Amazon CLI command creates a five-node cluster: two nodes that both ingest and query, two query-only read nodes on a smaller instance type, and a dedicated compactor on a larger one. Compaction gets the compute it needs without enlarging the rest of the cluster.

aws timestream-influxdb create-db-cluster \
  --region us-east-1 \
  --name "my-heterogeneous-cluster" \
  --deployment-configuration '{
    "InfluxDBv3Enterprise": {
      "nodeGroups": [
        {"instanceModes": ["INGEST","QUERY"], "instanceType": "db.influx.large",   "count": 2},
        {"instanceModes": ["QUERY"],          "instanceType": "db.influx.medium",  "count": 2},
        {"instanceModes": ["COMPACT"],        "instanceType": "db.influx.2xlarge", "count": 1}
      ]
    }
  }' \
  --vpc-subnet-ids subnet-12345abc subnet-67890def \
  --vpc-security-group-ids sg-12345abc \
  --no-publicly-accessible

Smaller cluster with a shared compactor

At 3 or fewer non-compact nodes you can skip the dedicated compactor group and add COMPACT to a group that already ingests and queries:

aws timestream-influxdb create-db-cluster \
  --region us-east-1 \
  --name "my-small-cluster" \
  --deployment-configuration '{
    "InfluxDBv3Enterprise": {
      "nodeGroups": [
        {"instanceModes": ["INGEST","QUERY","COMPACT"], "instanceType": "db.influx.large", "count": 1},
        {"instanceModes": ["QUERY"],                    "instanceType": "db.influx.large", "count": 2}
      ]
    }
  }' \
  --vpc-subnet-ids subnet-12345abc subnet-67890def \
  --vpc-security-group-ids sg-12345abc \
  --no-publicly-accessible

Sharing a node between compaction and live traffic saves an instance, but compaction then competes with your reads and writes for CPU and memory. For production workloads that compact continuously, give the compactor its own group.

Reshape an existing cluster

To change a cluster's shape, submit the complete target shape to update-db-cluster. Timestream compares the shape you submit against the running cluster and then adds, removes, or resizes only what differs. A single call can add a group, remove a group, change a group's count, and change a group's instanceType.

This example resizes only the compactor, from db.influx.2xlarge to db.influx.4xlarge, by resubmitting the other two groups untouched:

aws timestream-influxdb update-db-cluster \
  --region us-east-1 \
  --db-cluster-id "abcdefgh12" \
  --deployment-configuration '{
    "InfluxDBv3Enterprise": {
      "nodeGroups": [
        {"instanceModes": ["INGEST","QUERY"], "instanceType": "db.influx.large",   "count": 2},
        {"instanceModes": ["QUERY"],          "instanceType": "db.influx.medium",  "count": 2},
        {"instanceModes": ["COMPACT"],        "instanceType": "db.influx.4xlarge", "count": 1}
      ]
    }
  }'

The same pattern covers the other reshaping operations:

Deployment configurations and parameter groups

Before deployment configurations, you expressed cluster topology as parameters on a DB parameter group: ingestQueryInstances, queryOnlyInstances, and dedicatedCompactor. A deployment configuration replaces those fields. The two mechanisms can coexist while you migrate, and Timestream resolves the overlap deterministically:

Read a cluster's shape and endpoints

get-db-cluster reports cluster shape in two fields:

For a node-by-node view, call list-db-instances-for-cluster. Each node reports its own instanceModes and dbInstanceType. Each node also reports a singular instanceMode; for InfluxDB 3 Enterprise nodes, read the plural instanceModes, because a node in a deployment configuration can perform several functions at once.

Both cluster endpoints keep working as you reshape the cluster, and Timestream updates their membership automatically:

For guidance on distributing client traffic evenly across the nodes behind an endpoint, see Managing DNS resolution for cluster endpoints.

Best practices