What is Step Functions?
With Amazon Step Functions, you can create workflows, also called state machines, to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning pipelines.
Step Functions is based on state machines and tasks. In Step Functions, state machines are called workflows, which are a series of event-driven steps. Each step in a workflow is called a state. For example, a Task state represents a unit of work that another Amazon service performs, such as calling another Amazon Web Service or API.
In the Step Functions' console, you can visualize, edit, and debug your application’s workflow. You can examine the state of each step in your workflow to make sure that your application runs in order and as expected.
Depending on your use case, you can have Step Functions call Amazon services, such as Lambda, to perform tasks. You can have Step Functions control Amazon services, such as Amazon Glue, to create extract, transform, and load workflows. You also can create long-running, automated workflows for applications that require human interaction.
For a complete list of Amazon regions where Step Functions is available, see the Amazon
Region Table
Learn how to use Step Functions
To learn how to use Step Functions, follow the interactive modules in the Amazon Step Functions Workshop
Topics
Choosing Standard or Express workflows
Step Functions has two workflow types:
Standard workflows have exactly-once workflow execution and can run for up to one year. This means that each step in a Standard workflow will execute exactly once.
Express workflows have at-least-once workflow execution and can run for up to five minutes. This means that one or more steps in an Express Workflow can potentially run more than once, while each step in the workflow executes at least once.
Executions are instances of running workflows.
Standard workflows are ideal for long-running, auditable workflows, as they show execution history and visual debugging.
Express workflows are ideal for high-event-rate workloads, such as streaming data processing and IoT data ingestion.
Standard workflows specifications
-
2,000 per second execution rate
-
4,000 per second state transition rate
-
Priced by state transition
-
Show execution history and visual debugging
-
Support all service integrations and patterns
Express workflows specifications
-
100,000 per second execution rate
-
Nearly unlimited state transition rate
-
Priced by number and duration of executions
-
Send execution history to Amazon CloudWatch
-
Show execution history and visual debugging based on the Log level enabled
-
Support all service integrations and most patterns
For more information pricing, see Step Functions pricing, see the following:
Integrating with other services
Step Functions integrates with multiple Amazon services. To call other Amazon services, you can use regular Amazon SDK integrations and Optimized integrations.
-
Amazon SDK integrations provide a way to call any Amazon service directly from your state machine, giving you access to thousands of API actions.
-
Optimized integrations provide custom options for using those services in your state machines.
To combine Step Functions with other services, you will use the following service integration patterns:
Request a response (default)
-
Call a service, and let Step Functions progress to the next state after it gets an HTTP response.
Run a job (.sync)
-
Call a service, and have Step Functions wait for a job to complete.
Wait for a callback with a task token (.waitForTaskToken)
-
Call a service with a task token, and have Step Functions wait until the task token returns with a callback.
The table below shows the available service integrations and service integration patterns for Step Functions.
Standard Workflows and Express Workflows support the same integrations but not the same integration patterns.
Optimized integrations pattern support is different for each integration.
Express Workflows do not support Run a Job (.sync) or Wait for Callback (.waitForTaskToken).
For more information, see Choosing Step Functions workflow type.
Example use cases for workflows
Step Functions manages your application's components and logic, so you can write less code and focus on building and updating your application quickly. This section describes typical use cases for working with Step Functions.
Use case #1: Function orchestration
You create a workflow that runs a group of Lambda functions (steps) in a specific order. One Lambda function's output passes to the next Lambda function's input. The last step in your workflow gives a result. With Step Functions, you can see how each step in your workflow interacts with one other, so you can make sure that each step performs its intended function.
For a tutorial that shows you how to create a state machine with a group of functions, see the following:
Use case #2: Branching
A customer requests a credit limit increase. Using a Choice
state, you can
have Step Functions make decisions based on the Choice
state’s input. If the
request is more than your customer’s pre-approved credit limit, you can have Step Functions
send your customer's request to a manager for sign-off. If the request is less than
your customer’s pre-approved credit limit, you can have Step Functions approve the request
automatically.
Use case #3: Error handling
Retry
In this use case, a customer requests a username. The first time, your customer’s
request is unsuccessful. Using a Retry
statement, you can have Step Functions
try your customer's request again. The second time, your customer’s request is
successful.
Catch
In a similar use case, a customer requests an unavailable username. Using a
Catch
statement, you have Step Functions suggest an available username. If your
customer takes the available username, you can have Step Functions go to the next step in
your workflow, which is to send a confirmation email. If your customer doesn’t take
the available username, you have Step Functions go to a different step in your workflow,
which is to start the sign-up process over.
For more detailed examples of Retry
and Catch
statements, see the following:
Use case #4: Human in the loop
Using a banking app, one of your customers sends money to a friend. Your customer waits for a confirmation email. With a callback and a task token, you have Step Functions tell Lambda to send your customer’s money and report back when your customer’s friend receives it. After Lambda reports back that your customer’s friend received the money, you can have Step Functions go to the next step in your workflow, which is to send your customer a confirmation email.
To see a sample project that shows a callback with a task token, see the following:
Use case #5: Parallel processing
A customer converts a video file into five different display resolutions, so
viewers can watch the video on multiple devices. Using a Parallel
state, Step Functions
inputs the video file, so Lambda can process it into the five display resolutions at
the same time.
Use case #6: Dynamic parallelism
A customer orders three items, and you need to prepare each item for delivery. You
check each item's availability, gather each item, and then package each item for
delivery. Using a Map
state, Step Functions has Lambda process each of your customer's items in
parallel. Once all of your customer's items are packaged for delivery, Step Functions goes to
the next step in your workflow, which is to send your customer a confirmation email
with tracking information.
To see a sample project that shows dynamic parallelism using a Map
state, see the following: