Amazon Fargate task maintenance FAQs - Amazon Elastic Container Service
What is Fargate task maintenance and retirement?What is in the task retirement notice?Can I change the task retirement wait time?Can I get task retirement notifications through other Amazon services?Can I change a task retirement after it is scheduled?Can I control the timing of a task replacement?How does Amazon ECS handle tasks that are part of a service?Can Amazon ECS automatically handle standalone tasks?
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Fargate task maintenance FAQs

What is Fargate task maintenance and retirement?

Amazon is responsible for maintaining the underlying infrastructure for Amazon Fargate. Amazon determines when a platform version revision needs to be replaced with a new revision. This is known as task retirement. Amazon sends a task retirement notification when a platform version revision is retired. We routinely update our supported platform versions to introduce a new revision containing updates to the Fargate runtime software and underlying dependencies such as the operating system and container runtime. Once a newer revision is made available, we retire the older revision in order to ensure all customer workloads run on the most up to date revision of the Fargate platform version. When a revision is retired, all tasks running on that revision are stopped.

Amazon ECS tasks can be categorized as either service tasks or standalone tasks. Service tasks are deployed as part of a service and controlled by the Amazon ECS schedule. For more information, see Amazon ECS services. Standalone tasks are tasks started by the Amazon ECS RunTask API, either directly or by an external scheduler such as scheduled tasks (which are started by Amazon EventBridge), Amazon Batch, or Amazon Step Functions.

For service tasks, you do not need to take any action unless you want to replace these tasks before Amazon does. When the Amazon ECS scheduler stops the tasks, it uses the minimum healthy percent and launches a new task in an attempt to maintain the desired count for the service. By default, the minimum healthy percent of a service is 100 percent, so a new task is started first before a task is stopped. Service tasks are routinely replaced in the same way when you scale the service, deploy configuration changes, or deploy task definition revisions. To prepare for the task retirement process, we recommend that you test your application behavior by simulating this scenario. You can do this by stopping an individual task in your service to test for resiliency.

For standalone task retirement, Amazon stops the task on or after the task retirement date. we don’t launch a replacement task when a task is stopped. If you need these tasks to continue to run, you need to stop the running tasks and launch a replacement task before the time indicated in the notification. Therefore, we recommend that customers monitor the state of standalone tasks and if required, implement logic to replace the stopped tasks.

When a task is stopped in any of the scenarios, you can run describe-tasks. The stoppedReason in the response is ECS is performing maintenance on the underlying infrastructure hosting the task.

Task maintenance applies when there is a new platform version revision needs to be replaced with a new revision. If there is an issue with an underlying Fargate host, Amazon ECS replaces the host without a task retirement notice.

What is in the task retirement notice?

The task retirement notifications are sent through Amazon Health Dashboard as well as through an email to the registered email address and includes the following information:

  • The task retirement date - The task is stopped on or after this date.

  • For standalone tasks, the IDs of the tasks.

  • For service tasks, the ID of the cluster where the service runs and the IDs of the service.

  • The next steps you need to take.

Typically, we send one notification each for service and standalone tasks in each Amazon Web Services Region. However, in certain cases you might receive more than one event for each task type, for example when there are too many tasks to be retired that will surpass limits in our notification mechanisms.

You can identify tasks scheduled for retirement in the following ways:

  • The Amazon Health Dashboard

    Amazon Health notifications can be sent through Amazon EventBridge to archival storage such as Amazon Simple Storage Service, take automated actions such as run an Amazon Lambda function, or other notification systems such as Amazon Simple Notification Service. For more information, see Monitoring Amazon Health events with Amazon EventBridge. For sample configuration to send notifications to Amazon Chime, Slack, or Microsoft Teams, see the Amazon Health Aware repository on GitHub.

    The following is a sample EventBridge event.

    {
    "version": "0",
    "id": "3c268027-f43c-0171-7425-1d799EXAMPLE",
    "detail-type": "AWS Health Event",
    "source": "aws.health",
    "account": "123456789012",
    "time": "2023-08-16T23:18:51Z",
    "region": "us-east-1",
    "resources": [
        "cluster/service",
        "cluster/service"
    ],
    "detail": {
        "eventArn": "arn:aws:health:us-east-1::event/ECS/AWS_ECS_TASK_PATCHING_RETIREMENT/AWS_ECS_TASK_PATCHING_RETIREMENT_test1",
        "service": "ECS",
        "eventScopeCode": "ACCOUNT_SPECIFIC",
        "communicationId": "7988399e2e6fb0b905ddc88e0e2de1fd17e4c9fa60349577446d95a18EXAMPLE",
        "lastUpdatedTime": "Wed, 16 Aug 2023 23:18:52 GMT",
        "eventRegion": "us-east-1",
        "eventTypeCode": "AWS_ECS_TASK_PATCHING_RETIREMENT",
        "eventTypeCategory": "scheduledChange",
        "startTime": "Wed, 16 Aug 2023 23:18:51 GMT",
        "endTime": "Fri, 18 Aug 2023 23:18:51 GMT",
        "eventDescription": [
            {
                "language": "en_US",
                "latestDescription": "\\nA software update has been deployed to Fargate which includes CVE patches or other critical patches. No action is required on your part. All new tasks launched automatically uses the latest software version. For existing tasks, your tasks need to be restarted in order for these updates to apply. Your tasks running as part of the following ECS Services will be automatically updated beginning Wed, 16 Aug 2023 23:18:51 GMT.\\n\\nAfter Wed, 16 Aug 2023 23:18:51 GMT, the ECS scheduler will gradually replace these tasks, respecting the deployment settings for your service. Typically, services should see little to no interruption during the update and no action is required. When AWS stops tasks, AWS uses the minimum healthy percent (1) and launches a new task in an attempt to maintain the desired count for the service. By default, the minimum healthy percent of a service is 100 percent, so a new task is started first before a task is stopped. Service tasks are routinely replaced in the same way when you scale the service or deploy configuration changes or deploy task definition revisions. If you would like to control the timing of this restart you can update the service before Wed, 16 Aug 2023 23:18:51 GMT, by running the update-service command from the ECS command-line interface specifying force-new-deployment for services using Rolling update deployment type. For example:\\n\\n$ aws ecs update-service -service service_name \\\n--cluster cluster_name -force-new-deployment\\n\\nFor services using Blue/Green deployment type with AWS CodeDeploy:\\nPlease refer to create-deployment document (2) and create new deployment using same task definition revision.\\n\\nFor further details on ECS deployment types, please refer to ECS Deployment Developer Guide (1).\\nFor further details on Fargate's update process, please refer to the AWS Fargate User Guide (3).\\nIf you have any questions or concerns, please contact AWS Support (4).\\n\\n(1) https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-types.html\\n(2) https://docs.aws.amazon.com/cli/latest/reference/deploy/create-deployment.html\\n(3) https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-maintenance.html\\n(4) https://aws.amazon.com/support\\n\\nA list of your affected resources(s) can be found in the 'Affected resources' tab in the 'Cluster/ Service' format in the AWS Health Dashboard. \\n\\n"
            }
        ],
        "affectedEntities": [
            {
                "entityValue": "cluster/service"
            },
            {
                "entityValue": "cluster/service"
            }
        ]
    }
}

  • Email

    An email is sent to the registered email for the Amazon Web Services account ID.

Can I change the task retirement wait time?

You can configure the time that Fargate starts the task retirement. For workloads that require immediate application of the updates, choose the immediate setting (0). When you need more control, for example, when a task can only be stopped during a certain window, configure the 7 day (7), or 14 day (14) option.

We recommend that you choose a shorter waiting period in order to pick up newer platform versions revisions sooner.

Configure the wait period by running put-account-setting-default or put-account-setting as the root user or an administrative user. Use the fargateTaskRetirementWaitPeriod option for the name and the value option set to one of the following values:

  • 0 - Amazon sends the notification, and immediately starts to retire the affected tasks.

  • 7 - Amazon sends the notification, and waits 7 calendar days before starting to retire the affected tasks.

  • 14 - Amazon sends the notification, and waits 14 calendar days before starting to retire the affected tasks.

The default is 7 days.

For more information, see, put-account-setting-default and put-account-setting in the Amazon Elastic Container Service API Reference.

For more information, see Amazon Fargate task retirement wait time.

Can I get task retirement notifications through other Amazon services?

Amazon sends a task retirement notification to the Amazon Health Dashboard and to the primary email contact on the Amazon Web Services account. The Amazon Health Dashboard provides a number of integrations into other Amazon services, including EventBridge. You can use EventBridge to automate the visibility of the notices (For example. forwarding the message to a ChatOps tool). For more information, see Solution overview: Capturing task retirement notifications.

Can I change a task retirement after it is scheduled?

No. The schedule is based off the task retirement wait time which has a default of 7 days. If you need more time, you can choose to configure the wait period to 14 days. For more information, see Can I change the task retirement wait time?. The change in this configuration applies to retirements that will be scheduled in the future. Currently scheduled retirements are not impacted. If you have any further concerns, contact Amazon Web Services Support.

Can I control the timing of a task replacement?

For services that use rolling deployment, you update the service using update-service with the force-deployment option before the retirement start time.

The following update-service example uses the force-deployment option.

aws ecs update-service —service service_name \ 
    --cluster cluster_name \
     --force-new-deployment

For services that use the blue/green deployment, you need to create a new deployment in Amazon CodeDeploy. For information about how to create the deployment, see create-deployment in the Amazon Command Line Interface Reference.

How does Amazon ECS handle tasks that are part of a service?

Amazon ECS gradually replaces affected tasks in your service when the Fargate retirement period starts. When Amazon ECS stops a task, it uses the service's minimum healthy percent and launches a new task to maintain the desired task count for the service. A new task is started before a task is stopped because the default minimum health percent is 100. Service tasks are routinely replaced in the same way when you scale the service, deploy configuration changes, or deploy task definition revisions. For more information about the minimum healthy percent, see Deployment configuration.

Can Amazon ECS automatically handle standalone tasks?

No. Amazon can't create a replacement task for standalone tasks which are started by RunTask, scheduled tasks (for example through EventBridge Scheduler), Amazon Batch, or Amazon Step Functions. Amazon ECS manages only tasks that are part of a service.