Working with concurrency scaling - Amazon Redshift
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Working with concurrency scaling

With the Concurrency Scaling feature, you can support thousands of concurrent users and concurrent queries, with consistently fast query performance. When you turn on concurrency scaling, Amazon Redshift automatically adds additional cluster capacity to process an increase in both read and write queries. Users see the most current data, whether the queries run on the main cluster or a concurrency-scaling cluster.

You can manage which queries are sent to the concurrency-scaling cluster by configuring WLM queues. When you turn on concurrency scaling, eligible queries are sent to the concurrency-scaling cluster instead of waiting in a queue.

You're charged for concurrency-scaling clusters only for the time they're actively running queries. For more information about pricing, including how charges accrue and minimum charges, see Concurrency Scaling pricing.

Concurrency scaling capabilities

When you turn on concurrency scaling for a WLM queue, it works for read operations, such as dashboard queries. It also works for commonly used write operations, such as statements for data ingestion and processing.

Concurrency scaling capabilities for write operations

Concurrency scaling supports frequently used write operations, such as extract, transform, and load (ETL) statements. Concurrency scaling for write operations is especially useful when you want to maintain consistent response times when your cluster receives a large number of requests. It improves throughput for write operations contending for resources on the main cluster.

Concurrency scaling supports COPY, INSERT, DELETE, UPDATE, and CREATE TABLE AS (CTAS) statements. Additionally, concurrency scaling supports materialized-view refresh for MVs that do not use aggregations. Other data-manipulation language (DML) statements and data-definition language (DDL) statements aren't supported. When non-supported write statements, such as CREATE without TABLE AS, are included in an explicit transaction before the supported write statements, none of the write statements will run on concurrency-scaling clusters.

When you accrue credit for concurrency scaling, this credit accrual applies to both read and write operations.

Limitations for concurrency scaling

The following are limitations for using Amazon Redshift concurrency scaling:

  • It doesn't support queries on tables that use interleaved sort keys.

  • It doesn't support queries on temporary tables.

  • It doesn't support queries that access external resources that are protected by restrictive network or virtual private cloud (VPC) configurations.

  • It doesn't support queries that contain Python user-defined functions (UDFs) and Lambda UDFs.

  • It doesn't support queries that access system tables, PostgreSQL catalog tables, or no-backup tables.

  • It doesn’t support COPY or UNLOAD queries that access an external resource when restrictive IAM policy permissions are in place. This includes permissions applied either to the resource, like an Amazon S3 bucket or DynamoDB table, or to the source. IAM sources can include the following:

    • aws:sourceVpc – A source VPC.

    • aws:sourceVpce – A source VPC endpoint.

    • aws:sourceIp – A source IP address.

    In some cases, you might need to remove permissions that restrict either the resource or the source, so that COPY and UNLOAD queries accessing the resource are sent to the concurrency-scaling cluster.

    For more information about resource policies, see Policy types in the Amazon Identity and Access Management user guide and Controlling access from VPC endpoints with bucket policies.

  • Amazon Redshift concurrency scaling for write operations is not supported for DDL operations, such as CREATE TABLE or ALTER TABLE.

  • It doesn't support ANALYZE for the COPY command.

  • It doesn't support write operations on a target table where DISTSTYLE is set to ALL.

  • It doesn't support COPY from the following file formats:

    • Parquet

    • ORC

  • It doesn't support write operations on tables with identity columns.

  • Amazon Redshift supports concurrency scaling for write operations on only Amazon Redshift RA3 nodes, specifically ra3.16xlarge, ra3.4xlarge, and ra3.xlplus. Concurrency scaling for write operations isn't supported on other node types.

For more information about concurrency scaling, see the following topics in the Amazon Redshift Management Guide.