Amazon MWAA Document History - Amazon Managed Workflows for Apache Airflow
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon MWAA Document History

The following table describes important additions to the Amazon MWAA service documentation, beginning in November 2020. To receive notifications about updates to this documentation, subscribe to the RSS feed.

ChangeDescriptionDate

Added a new environment class: mw1.micro

Amazon MWAA now provides a new environment class: mw1.micro.

November 19, 2024

Support for simpler method to access Apache Airflow REST API

Amazon MWAA now provides a simplified approach for interacting with the Apache Airflow REST API using Amazon credentials.

October 23, 2024

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.10.1. This update includes information on updated provider packages, and details about using Apache Airflow v2.10.1 on Amazon MWAA.

September 26, 2024

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.9.2. This update includes information on updated provider packages, and details about using Apache Airflow v2.9.2 on Amazon MWAA.

July 9, 2024

Amazon MWAA supports configuring a custom web server domain names

Amazon MWAA supports configuring a custom web server domain names for private environments with no internet access. This update includes the following new topic that describes setting up a new custom domain.

June 18, 2024

Amazon MWAA supports web server automatic scaling and the Apache Airflow REST API

Amazon MWAA now supports automatic scaling of web servers as well as the ability to access and use the Apache Airflow REST API.

May 16, 2024

Improved description of automatic scaling behavior

Updated the following topic to reflect the new Amazon MWAA automatic scaling behavior when workers pick up new tasks as Fargate workers downscale.

May 10, 2024

Support for larger instance sizes

Amazon MWAA now supports two larger instance size options for larger workloads: mw1.xlarge, and mw1.2xlarge

April 16, 2024

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.8.1. This update includes information on updated provider packages, and details about using Apache Airflow v2.8.1 on Amazon MWAA.

February 22, 2024

Support for shared Amazon VPC

Amazon MWAA supports cross-account environment creation for organizations using Amazon OpenSearch Service to manage Amazon MWAA resources using a central shared Amazon VPC in an owner account. As part of this launch, Amazon MWAA lets you choose to create, and manage, your own Amazon VPC endpoints.

November 15, 2023

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.7.2. This update includes information on updated provider packages, and details about using Apache Airflow v2.7.2 on Amazon MWAA.

November 6, 2023

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.6.3. This update includes information on updated provider packages, and details about using Apache Airflow v2.6.3 on Amazon MWAA,

August 9, 2023

Version deprecation information

Updated topic on version deprecation to include deprecation notices and timelines for Apache Airflow v2.0.2 and Apache Airflow v2.2.2.

July 31, 2023

New topics and use cases

Amazon MWAA supports minor version upgrades. This updates includes the following new topic that describes how to upgrade the environment and make sure your workflow resources are compatible with the version of Apache Airflow you are upgrading to:

June 5, 2023

New Regions

Amazon MWAA is now available in the Beijing and Ningxia Regions. For more information, see the following:

May 16, 2023

Updated topic

Updated customer managed IAM policies that grant a user full console and API access to Amazon MWAA. The update describes why you must provide permission for iam:PassRole in order to allow a user to pass roles to Amazon MWAA. Amazon MWAA uses these permissions to perform actions on a user's behalf.

April 12, 2023

New guidance

Updated topic on configuring Amazon Secrets Manager as a backend for Amazon MWAA to provide guidance on using lookup patterns. Using lookup patterns narrow the secrets that Apache Airflow searches for and reduce the number of API calls Amazon MWAA makes to Secrets Manager to retrieve connections and variables. This reduces the costs associated with using Secrets Manager as a backend.

April 12, 2023

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.5.1. This update includes information on updated provider packages, and details about using Apache Airflow v2.5.1 on Amazon MWAA,

April 11, 2023

New topics and use cases

Added a new topic on using a startup script with an Amazon MWAA environment. This topic descibes configuring a startup script for an existing environment, using it to install Linux runtimes, and setting environment variables.

April 3, 2023

Updated section on private web server access

Updated the following topic on private web server access. The update clarifies that, in environments with private web server access, you must use a Python wheel archive (.whl) to package, and install, dependencies.

February 24, 2023

Added information on deprecated Apache Airflow versions

Updated the Versions topic with new information on how Amazon MWAA managed deprecating Apache Airflow versions. Removed a section about upgrading to newer version of Apache Airflow, and a section that described changes between Apache Airflow v1 and Apache Airflow v2. For more information about migrating to a newversion of Apache Airflow, see the Amazon MWAA Migration Guide.

February 17, 2023

Fixes in Amazon MWAA container metrics

Updated the container metrics topic, and removed a set of erroneous metrics that did not exist under the Cluster dimension. Added an additional section that describes how you can evaluate the number of additional workers that an environment is utilizing at a given time by graphing the CPUUtilization or the MemoryUtilization metric for the AdditionalWorker component, and setting the statistics type to Sample Count.

January 20, 2023

New Apache Airflow version

Amazon MWAA now supports Apache Airflow v2.4.3. This update includes information on updated provider packages, details about using Apache Airflow v2.4.3 on Amazon MWAA, and consolidated information about which features are supported in each Apache Airflow version on Amazon MWAA.

January 5, 2023

Updated topic on service-linked role

Updated information about the service-linked role that Amazon MWAA uses to create and manage Amazon resources on your behalf, including information about how you can delete the service-linked role when you no longer need it. This includes an updated service-linked role permission policy that allows Amazon MWAA to publishe additional CloudWatch metrics under the AWS/MWAA namespace.

November 18, 2022

New topic on service metrics

Added new topic that describes service metrics emitted by Amazon MWAA under the AWS/MWAA namespace. These include Amazon ECS cluster metrics schedulers, workers, and web servers, Amazon SQS metrics for the queues that allow Amazon MWAA to decouple schedulers and workers, as well as Amazon RDS metrics for the metadata database.

November 18, 2022

New topic

Added new guidance on modifying a constraints file to specify new versions of provider packages to use with your Amazon MWAA environment.

November 18, 2022

Updated FAQ entry

Updated information related to Amazon MWAA's HIPAA eligibility.

November 15, 2022

New topic

Added new topic on using aws:SourceArn and aws:SourceAccount global condition context keys in an Amazon MWAA execution role trust policy, in order to prevent cross-service confused deputy.

October 21, 2022

New sample code

Added updated instructions and DAG code example that writes custom OS-level metrics to CloudWatch.

September 13, 2022

New sample code

Added updated instructions and a new Amazon Lambda Python code example that retrieves an Apache Airflow CLI token, then invokes a DAG in a specified Amazon MWAA environment.

September 12, 2022

New architectural diagrams

Added new architectural diagrams that demonstrate an Amazon MWAA environment with a public and private web server.

September 12, 2022

New sample code

Added updated instructions and a new DAG code example that retrieves an Apache Airflow CLI token, then invokes another DAG in a different Amazon MWAA environment.

August 16, 2022

New sample code

Added updated instructions and new DAG that queries an environment's Aurora PostgreSQL for metadata information, writes the result to CSV files and stores the files in Amazon S3.

August 12, 2022

New sample code

Added updated instructions and new DAG that refreshes an Amazon CodeArtifact token at runtime and stores the result in Amazon S3.

August 3, 2022

New sample code

Added updated instructions and DAG code sample for using the ECSOperator in Amazon MWAA.

July 26, 2022

New sample code

Added updated instructions and DAG code sample for using the SSHOperator in Amazon MWAA.

July 15, 2022

New sample code

Added new instructions and DAG code sample for using dbt Postgres with Amazon MWAA.

June 17, 2022

New topics and use cases

Added new instructions and DAG code sample for installing dependencies using Python wheel files for Amazon MWAA environments with public and private access.

May 13, 2022

New topics and use cases

Added new guidance on choosing which Apache Airflow metrics Amazon MWAA sends to CloudWatch.

April 19, 2022

New guides

Amazon MWAA offers a migration guide for migrating Apache Airflow workflows from self-managed deployments, as well as existing Amazon MWAA environments.

March 7, 2022

New topics and use cases

Added new security best practice for working with Apache Airflow, including a solution for detecting changes to the Apache Airflow user privileges.

February 18, 2022

New sample code

Added new code sample for creating timezone-aware DAGs using Pendulum, and clarified how to use a custom plugin to change the timezone in which Apache Airflow logs are created.

February 11, 2022

Apache Airflow v2.2.2 launch

Amazon Managed Workflows for Apache Airflow now supports Apache Airflow v2.2.2. Beginning with v2.2, Amazon MWAA will install Python packages and custom plugins directly on the Apache Airflow web server allowing you greater flexibility to manage your environments. For more information, see the following.

January 27, 2022

New tutorials

Added a new tutorial that demonstrates creating a new custom Apache Airflow role, and assigning the role to an Apache Airflow user mapped from IAM in order to limit the user's access to a subset of specified DAGs.

December 8, 2021

Fixes

Fixed a best practices recommendation for setting the value of scheduler.min_file_process_interval in order to optimize CPU usage. Added an IAM policy example granting access to Secrets Manager resources in the execution role. Added troubleshooting topic on using Secrets Manager condition keys.

November 22, 2021

New sample code

Added the following new code sample for modifying the time zone in which DAGs are processed using a custom plugin, and new troubleshooting topic for invoking the dags backfill Apache Airflow CLI command from within a bash operator.

November 1, 2021

Fixes

Fixed issues in the Amazon ECS operator code sample, and clarified the additional permissions required in the Amazon MWAA execution role to allow the environment to access Amazon ECS task log group in CloudWatch Logs.

October 26, 2021

New sample code

Added new code sample that queries the Aurora PostgreSQL database for information relevant to DAG runs and writes the results to CSV file stored on Amazon S3.

October 1, 2021

Fixes

Corrected information about how Amazon MWAA automatically syncs new and changed objects from your target Amazon S3 bucket to your schedulers and workers.

October 1, 2021

Now supported

Amazon MWAA now supports additional provider packages for Apache Airflow 2.0+. To learn more about supported packages, see the following:

September 24, 2021

New commands and procedures

Added additional guidance and Amazon CLI command examples for creating an Amazon S3 gateway endpoint when using an Amazon VPC without internet access:

September 24, 2021

New topics and use cases

Added the following changes:

September 19, 2021

New supported region

Amazon MWAA is now available in the following regions:

  • Asia Pacific (Mumbai) - ap-south-1

  • Asia Pacific (Seoul) - ap-northeast-2

  • Europe (London) - eu-west-2

  • Europe (Paris) - eu-west-3

  • Canada (Central) - ca-central-1

  • South America (São Paulo) - sa-east-1

For more information about region availability and service endpoints, see the following:

August 31, 2021

New topics and use cases

Added the following changes:

  • Updated the sample policies to allow Amazon MWAA to fetch account-level Amazon S3 settings (s3:GetAccountPublicAccessBlock) in Amazon MWAA execution role.

August 27, 2021

Fixes

Added the following changes:

August 27, 2021

New topics and use cases

Added the following changes:

August 20, 2021

New topics and use cases

Added the following changes:

August 13, 2021

New sample code

Added the following changes:

August 13, 2021

Fixes

Added the following changes:

August 13, 2021

New topics and use cases

Added the following changes:

August 6, 2021

New sample code

Added the following changes:

August 6, 2021

New topics and use cases

Added the following changes:

July 29, 2021

Fixes

Added the following changes:

July 29, 2021

Removed or moved topics

Added the following changes:

July 23, 2021

New guides

Added the following changes:

July 23, 2021

Fixes

Added the following changes:

July 23, 2021

New topics and use cases

Added the following changes:

July 16, 2021

New topics and use cases

Added the following changes:

July 9, 2021

New guides

Added the following changes:

July 9, 2021

New topics and use cases

Added the following changes:

July 2, 2021

New sample code

Added the following changes:

July 2, 2021

New topics and use cases

Added the following changes:

June 25, 2021

New sample code

Added the following changes:

June 25, 2021

New guides

Added the following changes:

June 25, 2021

New topics and use cases

Added the following changes:

June 18, 2021

New sample code

Added the following changes:

June 18, 2021

New topics and use cases

Added the following changes:

June 2, 2021

New sample code

Added the following changes:

June 2, 2021

New guides

Added the following changes:

June 2, 2021

Fixes

Added the following changes:

  • Added Apache Airflow VPC endpoints to the Amazon CloudFormation template in Option three: Creating a VPC network without Internet access to Create the VPC network.

June 2, 2021

Apache Airflow v2.0.2 launch

General availability launch of Apache Airflow v2.0.2.

May 26, 2021

New topics and use cases

Added the following changes:

May 14, 2021

Fixes

Added the following changes:

May 12, 2021

Removed or moved topics

Added the following changes:

May 10, 2021

New topics and use cases

Added the following changes:

May 10, 2021

Removed or moved topics

Added the following changes:

May 7, 2021

New topics and use cases

Added the following changes:

May 7, 2021

New topics and use cases

Added the following changes:

April 30, 2021

New sample code

Added the following changes:

April 30, 2021

New guides

Added the following changes:

April 30, 2021

Fixes

Added the following changes:

April 30, 2021

New topics and use cases

Added the following changes:

April 23, 2021

New sample code

Added the following changes:

April 23, 2021

New guides

Added the following changes:

April 23, 2021

New topics and use cases

Added the following changes:

April 16, 2021

New sample code

Added the following changes:

April 16, 2021

New topics and use cases

Added the following changes:

April 9, 2021

New sample code

Added the following changes:

April 9, 2021

Fixes

Added the following changes:

  • Oops! We've updated the format for a requirements.txt, and added an example that's compatible with Apache Airflow v1.10.12 in Installing Python dependencies.

March 31, 2021

New topics and use cases

Added the following changes:

March 26, 2021

New topics and use cases

Added the following changes:

March 19, 2021

New guides

Added the following changes:

March 19, 2021

New topics and use cases

Added the following changes:

March 12, 2021

New sample code

Added the following changes:

March 12, 2021

New guides

Added the following changes:

March 12, 2021

New topics and use cases

Added the following changes:

March 5, 2021

Now supported

Added the following changes:

  • Previously, backend_kwargs was not supported for Amazon Secrets Manager and you needed a workaround to override the Secrets Manager function call. Now, backend_kwargs is supported. See the Amazon Secrets Manager troubleshooting topic in Troubleshooting Amazon Managed Workflows for Apache Airflow.

March 4, 2021

Fixes

Added the following changes:

March 4, 2021

New topics and use cases

Added the following changes:

February 26, 2021

New topics and use cases

Added the following changes:

February 25, 2021

New tutorials

Added the following changes:

February 22, 2021

New topics and use cases

Added the following changes:

February 22, 2021

New sample code

Added the following changes:

February 22, 2021

New commands and procedures

Added the following changes:

February 22, 2021

New sample code

Added the following changes:

February 17, 2021

New commands and procedures

Added the following changes:

  • Added Amazon CLI commands to Working with DAGs on Amazon MWAA pages.

  • Apache Airflow doesn't support serialized DAGs in CLI commands. Since the CLI runs on the web server, which doesn't have plugins or requirements for security reasons, any MWAA environments with a plugins.zip or requirements.txt will not support these commands. Moved Apache Airflow list_dags and backfill commands to unsupported commands at Accessing Apache Airflow.

February 17, 2021

GitHub launch

User guide docs are now open source on GitHub. Choose "Edit this page on GitHub" on any page.

February 17, 2021

New topics and use cases

Added the following changes:

February 12, 2021

New topics and use cases

Added the following changes:

February 5, 2021

Removed or moved topics

Added the following changes:

February 4, 2021

Amazon MWAA CloudFormation

Update the parameters to create an environment at Amazon MWAA CloudFormation.

  • Remove SubnetList.

  • Remove TagList.

  • Add NetworkConfiguration.

  • Add TagMap.

  • Add create environment request examples.

February 4, 2021

New topics and use cases

Added the following changes:

January 29, 2021

Amazon MWAA launch

General availability launch of Amazon Managed Workflows for Apache Airflow.

  • User guide documentation

  • Amazon CloudFormation documentation

November 24, 2020