Iterating a Loop Using Lambda
In this tutorial, you implement a design pattern that uses a state machine and an Amazon Lambda function to iterate a loop a specific number of times.
Use this design pattern any time you need to keep track of the number of loops in a state machine. This implementation can help you break up large tasks or long-running executions into smaller chunks, or to end an execution after a specific number of events. You can use a similar implementation to periodically end and restart a long-running execution to avoid exceeding service quotas for Amazon Step Functions, Amazon Lambda, or other Amazon services.
Before you begin, go through the Creating a Step Functions state machine that uses Lambda tutorial to ensure you are familiar with using Lambda and Step Functions together.
Topics
Step 1: Create a Lambda Function to Iterate a Count
By using a Lambda function you can track the number of iterations of a loop in your
state machine. The following Lambda function receives input values for
count
, index
, and step
. It returns these values
with an updated index
and a Boolean value named continue
. The
Lambda function sets continue
to true
if the index
is less than count
.
Your state machine then implements a Choice
state that executes some
application logic if continue
is true
, or exits if it is
false
.
To create the Lambda function
-
Sign in to the Lambda console
, and then choose Create function. -
On the Create function page, choose Author from scratch.
-
In the Basic information section, configure your Lambda function, as follows:
-
For Function name, enter
Iterator
. -
For Runtime, choose Node.js 14.x.
-
In Change default execution role, choose Create a new role with basic Lambda permissions.
-
Choose Create function.
-
After your Lambda function is created, copy the function's Amazon Resource Name (ARN) displayed in the upper-right corner of the page. To copy the ARN, click
. The following is an example ARN:
arn:aws-cn:lambda:us-east-1:123456789012:function:Iterator
-
-
Copy the following code for the Lambda function into the Code source section of the
Iterator
page.exports.handler = function iterator (event, context, callback) { let index = event.iterator.index let step = event.iterator.step let count = event.iterator.count index = index + step callback(null, { index, step, count, continue: index < count }) }
This code accepts input values for
count
,index
, andstep
. It increments theindex
by the value ofstep
and returns these values, and the Booleancontinue
. The value ofcontinue
istrue
ifindex
is less thancount
. -
Choose Deploy.
Step 2: Test the Lambda Function
Run your Lambda function with numeric values to see it in operation. You can provide input values for your Lambda function that mimic an iteration, to see what output you get with specific input values.
To test your Lambda function
-
Choose Test.
-
In the Configure test event dialog box, enter
TestIterator
in the Event name box. -
Replace the example data with the following.
{ "Comment": "Test my Iterator function", "iterator": { "count": 10, "index": 5, "step": 1 } }
These values mimic what would come from your state machine during an iteration. The Lambda function will increment the index and return
continue
astrue
. When the index isn't less than thecount
, it returnscontinue
asfalse
. For this test, the index has already incremented to5
. The results should increment theindex
to6
and setcontinue
totrue
. -
Choose Create.
-
On the
Iterator
page, choose Test to test your Lambda function.The results of the test are displayed in the Execution results tab.
-
Choose the Execution results tab to see the output.
{ "index": 6, "step": 1, "count": 10, "continue": true }
Note
If you set
index
to9
for this test, theindex
increments to10
, andcontinue
isfalse
.
Step 3: Create a State Machine
Use the Step Functions console
-
ConfigureCount
– Sets the default values forcount
,index
, andstep
. -
Iterator
– References the Lambda function you created earlier, passing in the values configured inConfigureCount
. -
IsCountReached
– A choice state that either runs your sample work again or goes toDone
, based on a Boolean value returned from yourIterator
Lambda function. -
ExampleWork
– A stub for the work you want to accomplish in your execution. In this example, it's aPass
state. In an actual implementation, this would be aTask
state. -
Done
– The end state of your execution.
-
Open the Step Functions console
, and then choose Create a state machine. Important
Ensure that your state machine is under the same Amazon account and Region as the Lambda function you created earlier.
-
On the Choose authoring method page, choose Write your workflow in code.
-
For Type, retain the default selection, that is, Standard.
-
In the Definition pane, paste the following code, but replace the ARN of the Lambda function that you created earlier in the
Resource
field.{ "Comment": "Iterator State Machine Example", "StartAt": "ConfigureCount", "States": { "ConfigureCount": { "Type": "Pass", "Result": { "count": 10, "index": 0, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" }, "Iterator": { "Type": "Task", "Resource": "
arn:aws-cn:lambda:us-east-1:123456789012:function:Iterate
", "ResultPath": "$.iterator", "Next": "IsCountReached" }, "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "Done" }, "ExampleWork": { "Comment": "Your application logic, to run a specific number of times", "Type": "Pass", "Result": { "success": true }, "ResultPath": "$.result", "Next": "Iterator" }, "Done": { "Type": "Pass", "End": true } } }Be sure to update the ARN in the
Iterator
state above, so that it references the Lambda function that you created earlier. -
Use the graph in the Visual Workflow pane to check that your Amazon States Language code describes your state machine correctly.
This graph shows the logic expressed in the previous state machine code.
For more information about the Amazon States Language, see State Machine Structure.
If you don't see the graph, choose
in the Visual Workflow pane.
-
Choose Next.
-
Enter a Name for your state machine, such as
.IterateCount
Note
State machine, execution, and activity names must be 1–80 characters in length, must be unique for your account and Amazon Region, and must not contain any of the following:
-
Whitespace
-
Wildcard characters (
? *
) -
Bracket characters (
< > { } [ ]
) -
Special characters (
: ; , \ | ^ ~ $ # % & ` "
) -
Control characters (
\\u0000
-\\u001f
or\\u007f
-\\u009f
).
If your state machine is of type Express, you can provide the same name to multiple executions of the state machine. Step Functions generates a unique execution ARN for each Express state machine execution, even if multiple executions have the same name.
Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.
-
-
In Permissions, choose Create new role.
-
Choose Create state machine.
Step 4: Start a New Execution
After you create your state machine, you can start an execution.
-
On the IterateCount page, choose Start execution.
The Start execution dialog box is displayed.
-
(Optional) To identify your execution, you can specify a name for it in the Name box. By default, Step Functions generates a unique execution name automatically.
Note
Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.
-
Choose Start Execution.
A new execution of your state machine starts, showing your running execution.
The execution increments in steps, tracking the count using your Lambda function. On each iteration, it performs the example work referenced in the
ExampleWork
state in your state machine. -
(Optional) In the Details tab, view the Execution Status and the timestamps for Started and End Time of the execution.
When the count reaches the number specified in the
ConfigureCount
state in your state machine, the execution quits iterating and ends.