Run Amazon ECS or Fargate tasks with Step Functions
Learn how to integrate Step Functions with Amazon ECS or Fargate to run and manage tasks. In Amazon ECS, a task is the fundamental unit of computation. Tasks are defined by a task definition that specifies how a Docker container should be run, including the container image, CPU and memory limits, network configuration, and other parameters. This page lists the available Amazon ECS API actions and provides instructions on how to pass data to an Amazon ECS task using Step Functions.
To learn about integrating with Amazon services in Step Functions, see Integrating services and Passing parameters to a service API in Step Functions.
Key features of Optimized Amazon ECS/Fargate integration
-
The Run a Job (.sync) integration pattern is supported.
ecs:runTask
can return an HTTP 200 response, but have a non-emptyFailures
field as follows:Request Response: Return the response and do not fail the task. This is the same as no optimization.
Run a Job or Task Token: If a non-empty
Failures
field is encountered, the task is failed with anAmazonECS.Unknown
error.
Supported Amazon ECS/Fargate APIs
-
RunTask
starts a new task using the specified task definition.
Parameters in Step Functions are expressed in PascalCase
Even if the native service API is in camelCase, for example the API action startSyncExecution
, you specify parameters in PascalCase, such as: StateMachineArn
.
Passing Data to an Amazon ECS Task
To learn about integrating with Amazon services in Step Functions, see Integrating services and Passing parameters to a service API in Step Functions.
You can use overrides
to override the default
command for a container, and pass input to your Amazon ECS tasks. See ContainerOverride
. In the example, we have used JsonPath to pass
values to the Task
from the input to the Task
state.
The following includes a Task
state that runs an Amazon ECS task and waits for
it to complete.
{
"StartAt": "Run an ECS Task and wait for it to complete",
"States": {
"Run an ECS Task and wait for it to complete": {
"Type": "Task",
"Resource": "arn:aws:states:::ecs:runTask.sync",
"Parameters": {
"Cluster": "cluster-arn
",
"TaskDefinition": "job-id
",
"Overrides": {
"ContainerOverrides": [
{
"Name": "container-name
",
"Command.$": "$.commands"
}
]
}
},
"End": true
}
}
}
The "Command.$": "$.commands"
line in ContainerOverrides
passes the commands from the state input to the container.
For the previous example, each of the commands will be passed as a container override if the input to the execution is the following.
{
"commands": [
"test command 1",
"test command 2",
"test command 3"
]
}
The following includes a Task
state that runs an Amazon ECS task, and then
waits for the task token to be returned. See Wait for a Callback with Task Token.
{
"StartAt":"Manage ECS task",
"States":{
"Manage ECS task":{
"Type":"Task",
"Resource":"arn:aws:states:::ecs:runTask.waitForTaskToken",
"Parameters":{
"LaunchType":"FARGATE",
"Cluster":"cluster-arn
",
"TaskDefinition":"job-id
",
"Overrides":{
"ContainerOverrides":[
{
"Name":"container-name
",
"Environment":[
{
"Name":"TASK_TOKEN_ENV_VARIABLE",
"Value.$":"$$.Task.Token"
}
]
}
]
}
},
"End":true
}
}
}
IAM policies for calling Amazon ECS/Amazon Fargate
The following example templates show how Amazon Step Functions generates IAM policies based on the resources in your state machine definition. For more information, see How Step Functions generates IAM policies for integrated services and Discover service integration patterns in Step Functions.
Because the value for TaskId
is not known until the task is submitted, Step Functions
creates a more privileged "Resource": "*"
policy.
Note
You can only stop Amazon Elastic Container Service (Amazon ECS) tasks that were started by Step Functions, despite the
"*"
IAM policy.
If your scheduled Amazon ECS tasks require the use of a task execution role, a task role, or a task role override, then you must add iam:PassRole
permissions for each task
execution role, task role, or task role override to the CloudWatch Events IAM role of the calling entity, which in this case is Step Functions.