Iterating a Loop Using Lambda - Amazon Step Functions
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Iterating a Loop Using Lambda

In this tutorial, you implement a design pattern that uses a state machine and an Amazon Lambda function to iterate a loop a specific number of times.

Use this design pattern any time you need to keep track of the number of loops in a state machine. This implementation can help you break up large tasks or long-running executions into smaller chunks, or to end an execution after a specific number of events. You can use a similar implementation to periodically end and restart a long-running execution to avoid exceeding service quotas for Amazon Step Functions, Amazon Lambda, or other Amazon services.

Before you begin, go through the Creating a Step Functions state machine that uses Lambda tutorial to ensure you are familiar with using Lambda and Step Functions together.

Step 1: Create a Lambda Function to Iterate a Count

By using a Lambda function you can track the number of iterations of a loop in your state machine. The following Lambda function receives input values for count, index, and step. It returns these values with an updated index and a Boolean value named continue. The Lambda function sets continue to true if the index is less than count.

Your state machine then implements a Choice state that executes some application logic if continue is true, or exits if it is false.

To create the Lambda function

  1. Sign in to the Lambda console, and then choose Create function.

  2. On the Create function page, choose Author from scratch.

  3. In the Basic information section, configure your Lambda function, as follows:

    1. For Function name, enter Iterator.

    2. For Runtime, choose Node.js 14.x.

    3. In Change default execution role, choose Create a new role with basic Lambda permissions.

    4. Choose Create function.

    5. After your Lambda function is created, copy the function's Amazon Resource Name (ARN) displayed in the upper-right corner of the page. To copy the ARN, click 
                                        copy Amazon Resource Name
                                    . The following is an example ARN:

      arn:aws-cn:lambda:us-east-1:123456789012:function:Iterator
  4. Copy the following code for the Lambda function into the Code source section of the Iterator page.

    exports.handler = function iterator (event, context, callback) { let index = event.iterator.index let step = event.iterator.step let count = event.iterator.count index = index + step callback(null, { index, step, count, continue: index < count }) }

    This code accepts input values for count, index, and step. It increments the index by the value of step and returns these values, and the Boolean continue. The value of continue is true if index is less than count.

  5. Choose Deploy.

Step 2: Test the Lambda Function

Run your Lambda function with numeric values to see it in operation. You can provide input values for your Lambda function that mimic an iteration, to see what output you get with specific input values.

To test your Lambda function

  1. Choose Test.

  2. In the Configure test event dialog box, enter TestIterator in the Event name box.

  3. Replace the example data with the following.

    { "Comment": "Test my Iterator function", "iterator": { "count": 10, "index": 5, "step": 1 } }

    These values mimic what would come from your state machine during an iteration. The Lambda function will increment the index and return continue as true. When the index isn't less than the count, it returns continue as false. For this test, the index has already incremented to 5. The results should increment the index to 6 and set continue to true.

  4. Choose Create.

  5. On the Iterator page, choose Test to test your Lambda function.

    The results of the test are displayed in the Execution results tab.

  6. Choose the Execution results tab to see the output.

    { "index": 6, "step": 1, "count": 10, "continue": true }
    Note

    If you set index to 9 for this test, the index increments to 10, and continue is false.

Step 3: Create a State Machine

Use the Step Functions console to create a state machine with the following states to invoke the Lambda function that you created earlier in Step 1.

  • ConfigureCount – Sets the default values for count, index, and step.

  • Iterator – References the Lambda function you created earlier, passing in the values configured in ConfigureCount.

  • IsCountReached – A choice state that either runs your sample work again or goes to Done, based on a Boolean value returned from your Iterator Lambda function.

  • ExampleWork – A stub for the work you want to accomplish in your execution. In this example, it's a Pass state. In an actual implementation, this would be a Task state.

  • Done – The end state of your execution.

  1. Open the Step Functions console, and then choose Create a state machine.

    Important

    Ensure that your state machine is under the same Amazon account and Region as the Lambda function you created earlier.

  2. On the Choose authoring method page, choose Write your workflow in code.

  3. For Type, retain the default selection, that is, Standard.

  4. In the Definition pane, paste the following code, but replace the ARN of the Lambda function that you created earlier in the Resource field.

    { "Comment": "Iterator State Machine Example", "StartAt": "ConfigureCount", "States": { "ConfigureCount": { "Type": "Pass", "Result": { "count": 10, "index": 0, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" }, "Iterator": { "Type": "Task", "Resource": "arn:aws-cn:lambda:us-east-1:123456789012:function:Iterate", "ResultPath": "$.iterator", "Next": "IsCountReached" }, "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "Done" }, "ExampleWork": { "Comment": "Your application logic, to run a specific number of times", "Type": "Pass", "Result": { "success": true }, "ResultPath": "$.result", "Next": "Iterator" }, "Done": { "Type": "Pass", "End": true } } }

    Be sure to update the ARN in the Iterator state above, so that it references the Lambda function that you created earlier.

  5. Use the graph in the Visual Workflow pane to check that your Amazon States Language code describes your state machine correctly.

    This graph shows the logic expressed in the previous state machine code.

    
              State machine workflow

    For more information about the Amazon States Language, see State Machine Structure.

    If you don't see the graph, choose 
       refresh
    in the Visual Workflow pane.

  6. Choose Next.

  7. Enter a Name for your state machine, such as IterateCount.

    Note

    State machine, execution, and activity names must be 1–80 characters in length, must be unique for your account and Amazon Region, and must not contain any of the following:

    • Whitespace

    • Wildcard characters (? *)

    • Bracket characters (< > { } [ ])

    • Special characters (: ; , \ | ^ ~ $ # % & ` ")

    • Control characters (\\u0000 - \\u001f or \\u007f - \\u009f).

    If your state machine is of type Express, you can provide the same name to multiple executions of the state machine. Step Functions generates a unique execution ARN for each Express state machine execution, even if multiple executions have the same name.

    Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.

  8. In Permissions, choose Create new role.

  9. Choose Create state machine.

Step 4: Start a New Execution

After you create your state machine, you can start an execution.

  1. On the IterateCount page, choose Start execution.

    The Start execution dialog box is displayed.

  2. (Optional) To identify your execution, you can specify a name for it in the Name box. By default, Step Functions generates a unique execution name automatically.

    Note

    Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.

  3. Choose Start Execution.

    A new execution of your state machine starts, showing your running execution.

    
                            State machine execution

    The execution increments in steps, tracking the count using your Lambda function. On each iteration, it performs the example work referenced in the ExampleWork state in your state machine.

  4. (Optional) In the Details tab, view the Execution Status and the timestamps for Started and End Time of the execution.

    When the count reaches the number specified in the ConfigureCount state in your state machine, the execution quits iterating and ends.

    
                            State machine execution complete