Working with concurrency scaling - Amazon Redshift
Concurrency scaling capabilitiesLimitations for concurrency scaling
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Working with concurrency scaling

With the Concurrency Scaling feature, you can support thousands of concurrent users and concurrent queries, with consistently fast query performance. When you turn on concurrency scaling, Amazon Redshift automatically adds additional cluster capacity to process an increase in both read and write queries. Users see the most current data, whether the queries run on the main cluster or a concurrency-scaling cluster.

You can manage which queries are sent to the concurrency-scaling cluster by configuring WLM queues. When you turn on concurrency scaling, eligible queries are sent to the concurrency-scaling cluster instead of waiting in a queue.

You're charged for concurrency-scaling clusters only for the time they're actively running queries. For more information about pricing, including how charges accrue and minimum charges, see Concurrency Scaling pricing.

Concurrency scaling capabilities

When you turn on concurrency scaling for a WLM queue, it works for read operations, such as dashboard queries. It also works for commonly used write operations, such as statements for data ingestion and processing.

Concurrency scaling capabilities for write operations

Concurrency scaling supports frequently used write operations, such as extract, transform, and load (ETL) statements. Concurrency scaling for write operations is especially useful when you want to maintain consistent response times when your cluster receives a large number of requests. It improves throughput for write operations contending for resources on the main cluster.

Concurrency scaling supports COPY, INSERT, DELETE, UPDATE, and CREATE TABLE AS (CTAS) statements. Additionally, concurrency scaling supports materialized-view refresh for MVs that do not use aggregations. Other data-manipulation language (DML) statements and data-definition language (DDL) statements aren't supported. When non-supported write statements, such as CREATE without TABLE AS, are included in an explicit transaction before the supported write statements, none of the write statements will run on concurrency-scaling clusters.

When you accrue credit for concurrency scaling, this credit accrual applies to both read and write operations.

Limitations for concurrency scaling

The following are limitations for using Amazon Redshift concurrency scaling:

  • It doesn't support queries on tables that use interleaved sort keys.

  • It doesn't support queries on temporary tables.

  • It doesn't support queries that access external resources that are protected by restrictive network or virtual private cloud (VPC) configurations.

  • It doesn't support queries that contain Python user-defined functions (UDFs) and Lambda UDFs.

  • It doesn't support queries that access system tables, PostgreSQL catalog tables, or no-backup tables.

  • It doesn’t support COPY or UNLOAD queries that access an external resource that has restrictive resource policies. External resources can include Amazon S3 buckets or DynamoDB tables. Policies can restrict access to a specific source VPC (aws:sourceVpc), source VPC endpoint (aws:sourceVpce), or source IP address (aws:SourceIp). In some cases, you might need to remove policies on an external resource. Doing this means that COPY or UNLOAD queries accessing the resource are sent to the concurrency-scaling cluster.

  • Amazon Redshift concurrency scaling for write operations is not supported for DDL operations, such as CREATE TABLE or ALTER TABLE.

  • It doesn't support ANALYZE for the COPY command.

  • It doesn't support write operations on a target table where DISTSTYLE is set to ALL.

  • It doesn't support COPY from the following file formats:

    • Parquet

    • ORC

  • It doesn't support write operations on tables with identity columns.

  • Amazon Redshift supports concurrency scaling for write operations on only Amazon Redshift RA3 nodes, specifically ra3.16xlarge, ra3.4xlarge, and ra3.xlplus. Concurrency scaling for write operations isn't supported on other node types.

  • Amazon Redshift concurrency scaling isn't supported for auto refresh for materialized views. For more information, see Streaming ingestion.

For more information about concurrency scaling, see the following topics in the Amazon Redshift Management Guide.