Start an Amazon Glue job with Step Functions - Amazon Step Functions
Supported APIsIAM policies
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Start an Amazon Glue job with Step Functions

Learn to use Step Functions to start a job run on Amazon Glue. This page lists the supported API actions and provides an example Task state to start a Amazon Glue job.

To learn about integrating with Amazon services in Step Functions, see Integrating services and Passing parameters to a service API in Step Functions.

Key features of Optimized Amazon Glue integration

  • The Run a Job (.sync) integration pattern is available.

  • The JobName field is extracted from the request and inserted into the response, which normally only contains JobRunID.

The following includes a Task state that starts an Amazon Glue job.

"Glue StartJobRun": {
      "Type": "Task",
      "Resource": "arn:aws:states:::glue:startJobRun.sync",
      "Parameters": {
        "JobName": "GlueJob-JTrRO5l98qMG"
      },
      "Next": "ValidateOutput"
    },
Parameters in Step Functions are expressed in PascalCase

Even if the native service API is in camelCase, for example the API action startSyncExecution, you specify parameters in PascalCase, such as: StateMachineArn.

Supported Amazon Glue APIs

IAM policies for calling Amazon Glue

The following example templates show how Amazon Step Functions generates IAM policies based on the resources in your state machine definition. For more information, see How Step Functions generates IAM policies for integrated services and Discover service integration patterns in Step Functions.

Amazon Glue does not have resource-based control.

Run a Job (.sync)
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "glue:StartJobRun",
                "glue:GetJobRun",
                "glue:GetJobRuns",
                "glue:BatchStopJobRun"
            ],
            "Resource": "*"
        }
    ]
}
Request Response and Callback (.waitForTaskToken)
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "glue:StartJobRun"
            ],
            "Resource": "*"
        }
    ]
}