Tutorial: Get Started Using the Amazon A2I API - Amazon SageMaker
Create a Private Work TeamCreate a Human Review WorkflowCreate a Human Loop
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Tutorial: Get Started Using the Amazon A2I API

This tutorial explains the API operations you can use to get started using Amazon A2I.

To use a Jupyter Notebook to run these operations, select a Jupyter Notebook from Use Cases and Examples Using Amazon A2I and use Use SageMaker Notebook Instance with Amazon A2I Jupyter Notebook to learn how to use it in a SageMaker notebook instance.

To learn more about the API operations you can use with Amazon A2I, see Use APIs in Amazon Augmented AI.

Create a Private Work Team

You can create a private work team and add yourself as a worker so that you can preview Amazon A2I.

If you are not familiar with Amazon Cognito, we recommend that you use the SageMaker console to create a private workforce and add yourself as a private worker. For instructions, see Step 1: Create a Work Team.

If you are familiar with Amazon Cognito, you can use the following instructions to create a private work team using the SageMaker API. After you create a work team, note the work team ARN (WorkteamArn).

To learn more about the private workforce and other available configurations, see Use a Private Workforce.

Create a private workforce

If you have not created a private workforce, you can do so using an Amazon Cognito user pool. Make sure that you have added yourself to this user pool. You can create a private work team using the Amazon SDK for Python (Boto3) create_workforce function. For other language-specific SDKs, refer to the list in CreateWorkforce.

    
    response = client.create_workforce(
        CognitoConfig={
            "UserPool": "Pool_ID",
            "ClientId": "app-client-id"
        },
        WorkforceName="workforce-name"
    )
Create a private work team

After you have created a private workforce in the Amazon Region to configure and start your human loop, you can create a private work team using the Amazon SDK for Python (Boto3) create_workteam function. For other language-specific SDKs, refer to the list in CreateWorkteam.


    response = client.create_workteam(
        WorkteamName="work-team-name",
        WorkforceName= "workforce-name",
        MemberDefinitions=[
            {
                "CognitoMemberDefinition": {
                    "UserPool": "<aws-region>_ID",
                    "UserGroup": "user-group",
                    "ClientId": "app-client-id"
                },
            }
        ]
    )

Access your work team ARN as follows:


    workteamArn = response["WorkteamArn"]
List private work teams in your account

If you have already created a private work team, you can list all work teams in a given Amazon Region in your account using the Amazon SDK for Python (Boto3) list_workteams function. For other language-specific SDKs, refer to the list in ListWorkteams. 


    response = client.list_workteams()

If you have numerous work teams in your account, you may want to use MaxResults, SortBy, and NameContains to filter your results.

Create a Human Review Workflow

You can create a human review workflow using the Amazon A2I CreateFlowDefinition operation. Before you create your human review workflow, you need to create a human task UI. You can do this with the CreateHumanTaskUi operation.

If you are using Amazon A2I with the Amazon Textract or Amazon Rekognition integrations, you can specify activation conditions using a JSON.

Create a Human Task UI

If you are creating a human review workflow to be used with Amazon Textract or Amazon Rekognition integrations, you need to use and modify pre-made worker task template. For all custom integrations, you can use your own custom worker task template. Use the following table to learn how to create a human task UI using a worker task template for the two built-in integrations. Replace the template with your own to customize this request.

Amazon Textract – Key-value pair extraction

To learn more about this template, see Custom Template Example for Amazon Textract.

Amazon Rekognition – Image moderation

To learn more about this template, see Custom Template Example for Amazon Rekognition.

Custom Integration

The following is an example template that can be used in a custom integration. This template is used in this notebook, demonstrating a custom integration with Amazon Comprehend.

Using the template specified above, you can create a template using the Amazon SDK for Python (Boto3) create_human_task_ui function. For other language-specific SDKs, refer to the list in CreateHumanTaskUi. 

    
    response = client.create_human_task_ui(
        HumanTaskUiName="human-task-ui-name",
        UiTemplate={
            "Content": template
        }
    )

This response element contains the human task UI ARN. Save this as follows:


    humanTaskUiArn = response["HumanTaskUiArn"]

Create JSON to specify activation conditions

For Amazon Textract and Amazon Rekognition built-in integrations, you can save activation conditions in a JSON object and use this in your CreateFlowDefinition request.

Next, select a tab to see example activation conditions you can use for these built-in integrations. For additional information about activation condition options, see JSON Schema for Human Loop Activation Conditions in Amazon Augmented AI.

Amazon Textract – Key-value pair extraction

This example specifies conditions for specific keys (such as Mail address) in the document. If Amazon Textract's confidence falls outside of the thresholds set here, the document is sent to a human for review, with the specific keys that initiated the human loop prompted to the worker.


      import json  

      humanLoopActivationConditions = json.dumps(
        {
            "Conditions": [
                {
                  "Or": [
                    
                    {
                        "ConditionType": "ImportantFormKeyConfidenceCheck",
                        "ConditionParameters": {
                            "ImportantFormKey": "Mail address",
                            "ImportantFormKeyAliases": ["Mail Address:","Mail address:", "Mailing Add:","Mailing Addresses"],
                            "KeyValueBlockConfidenceLessThan": 100,
                            "WordBlockConfidenceLessThan": 100
                        }
                    },
                    {
                        "ConditionType": "MissingImportantFormKey",
                        "ConditionParameters": {
                            "ImportantFormKey": "Mail address",
                            "ImportantFormKeyAliases": ["Mail Address:","Mail address:","Mailing Add:","Mailing Addresses"]
                        }
                    },
                    {
                        "ConditionType": "ImportantFormKeyConfidenceCheck",
                        "ConditionParameters": {
                            "ImportantFormKey": "Phone Number",
                            "ImportantFormKeyAliases": ["Phone number:", "Phone No.:", "Number:"],
                            "KeyValueBlockConfidenceLessThan": 100,
                            "WordBlockConfidenceLessThan": 100
                        }
                    },
                    {
                      "ConditionType": "ImportantFormKeyConfidenceCheck",
                      "ConditionParameters": {
                        "ImportantFormKey": "*",
                        "KeyValueBlockConfidenceLessThan": 100,
                        "WordBlockConfidenceLessThan": 100
                      }
                    },
                    {
                      "ConditionType": "ImportantFormKeyConfidenceCheck",
                      "ConditionParameters": {
                        "ImportantFormKey": "*",
                        "KeyValueBlockConfidenceGreaterThan": 0,
                        "WordBlockConfidenceGreaterThan": 0
                      }
                    }
            ]
        }
            ]
        }
    )
Amazon Rekognition – Image moderation

The human loop activation conditions used here are tailored towards Amazon Rekognition content moderation; they are based on the confidence thresholds for the Suggestive and Female Swimwear Or Underwear moderation labels.


        import json  

        humanLoopActivationConditions = json.dumps(
        {
            "Conditions": [
                {
                  "Or": [
                    {
                        "ConditionType": "ModerationLabelConfidenceCheck",
                        "ConditionParameters": {
                            "ModerationLabelName": "Suggestive",
                            "ConfidenceLessThan": 98
                        }
                    },
                    {
                        "ConditionType": "ModerationLabelConfidenceCheck",
                        "ConditionParameters": {
                            "ModerationLabelName": "Female Swimwear Or Underwear",
                            "ConfidenceGreaterThan": 98
                        }
                    }
                  ]
               }
            ]
        }
    )

Create a human review workflow

This section gives an example of the CreateFlowDefinition Amazon SDK for Python (Boto3) request using the resources created in the previous sections. For other language-specific SDKs, refer to the list in CreateFlowDefinition. Use the tabs in the following table to see the requests to create a human review workflow for Amazon Textract and Amazon Rekognition built-in integrations.

Amazon Textract – Key-value pair extraction

If you use the built-in integration with Amazon Textract, you must specify "AWS/Textract/AnalyzeDocument/Forms/V1" for "AwsManagedHumanLoopRequestSource" in HumanLoopRequestSource. 


    response = client.create_flow_definition(
        FlowDefinitionName="human-review-workflow-name",
        HumanLoopRequestSource={
            "AwsManagedHumanLoopRequestSource": "AWS/Textract/AnalyzeDocument/Forms/V1"
        }, 
        HumanLoopActivationConfig={
            "HumanLoopActivationConditionsConfig": {
                "HumanLoopActivationConditions": humanLoopActivationConditions
            }
        },
        HumanLoopConfig={
            "WorkteamArn": workteamArn,
            "HumanTaskUiArn": humanTaskUiArn,
            "TaskTitle": "Document entry review",
            "TaskDescription": "Review the document and instructions. Complete the task",
            "TaskCount": 1,
            "TaskAvailabilityLifetimeInSeconds": 43200,
            "TaskTimeLimitInSeconds": 3600,
            "TaskKeywords": [
                "document review",
            ],
        },
        OutputConfig={
            "S3OutputPath": "s3://DOC-EXAMPLE-BUCKET/prefix/",
        },
        RoleArn="arn:aws:iam::<account-number>:role/<role-name>",
        Tags=[
            {
                "Key": "string",
                "Value": "string"
            },
        ]
    )
Amazon Rekognition – Image moderation

If you use the built-in integration with Amazon Rekognition, you must specify "AWS/Rekognition/DetectModerationLabels/Image/V3" for "AwsManagedHumanLoopRequestSource" in HumanLoopRequestSource.


    response = client.create_flow_definition(
        FlowDefinitionName="human-review-workflow-name",
        HumanLoopRequestSource={
            "AwsManagedHumanLoopRequestSource": "AWS/Rekognition/DetectModerationLabels/Image/V3"
        }, 
        HumanLoopActivationConfig={
            "HumanLoopActivationConditionsConfig": {
                "HumanLoopActivationConditions": humanLoopActivationConditions
            }
        },
        HumanLoopConfig={
            "WorkteamArn": workteamArn,
            "HumanTaskUiArn": humanTaskUiArn,
            "TaskTitle": "Image content moderation",
            "TaskDescription": "Review the image and instructions. Complete the task",
            "TaskCount": 1,
            "TaskAvailabilityLifetimeInSeconds": 43200,
            "TaskTimeLimitInSeconds": 3600,
            "TaskKeywords": [
                "content moderation",
            ],
        },
        OutputConfig={
            "S3OutputPath": "s3://DOC-EXAMPLE-BUCKET/prefix/",
        },
        RoleArn="arn:aws:iam::<account-number>:role/<role-name>",
        Tags=[
            {
                "Key": "string",
                "Value": "string"
            },
        ]
    )
Custom Integration

If you use a custom integration, exclude the following parameters: HumanLoopRequestSource, HumanLoopActivationConfig.


    response = client.create_flow_definition(
        FlowDefinitionName="human-review-workflow-name",
        HumanLoopConfig={
            "WorkteamArn": workteamArn,
            "HumanTaskUiArn": humanTaskUiArn,
            "TaskTitle": "Image content moderation",
            "TaskDescription": "Review the image and instructions. Complete the task",
            "TaskCount": 1,
            "TaskAvailabilityLifetimeInSeconds": 43200,
            "TaskTimeLimitInSeconds": 3600,
            "TaskKeywords": [
                "content moderation",
            ],
        },
        OutputConfig={
            "S3OutputPath": "s3://DOC-EXAMPLE-BUCKET/prefix/",
        },
        RoleArn="arn:aws:iam::<account-number>:role/<role-name>",
        Tags=[
            {
                "Key": "string",
                "Value": "string"
            },
        ]
    )

After you create a human review workflow, you can retrieve the flow definition ARN from the response:


    humanReviewWorkflowArn = response["FlowDefinitionArn"]

Create a Human Loop

The API operation you use to start a human loop depends on the Amazon A2I integration you use.

  • If you use the Amazon Textract built-in integration, you use the AnalyzeDocument operation.

  • If you use the Amazon Rekognition built-in integration, you use the DetectModerationLabels operation.

  • If you use a custom integration, you use the StartHumanLoop operation.

Select your task type in the following table to see example requests for Amazon Textract and Amazon Rekognition using the Amazon SDK for Python (Boto3).

Amazon Textract – Key-value pair extraction

The following example uses the Amazon SDK for Python (Boto3) to call analyze_document in us-west-2. Replace the italicized red text with your resources. Include the DataAttributes parameter if you are using the Amazon Mechanical Turk workforce. For more information, see the analyze_document documention in the Amazon SDK for Python (Boto) API Reference.


   response = client.analyze_document(
         Document={"S3Object": {"Bucket": "AWSDOC-EXAMPLE-BUCKET", "Name": "document-name.pdf"},
         HumanLoopConfig={
            "FlowDefinitionArn":"arn:aws:sagemaker:us-west-2:111122223333:flow-definition/flow-definition-name",
            "HumanLoopName":"human-loop-name",
            "DataAttributes" : {ContentClassifiers:["FreeOfPersonallyIdentifiableInformation"|"FreeOfAdultContent"]}
         }
         FeatureTypes=["FORMS"]
    )

Human loops are only created if Amazon Textract's confidence for document analysis task meets the activation conditions you specified in your human review workflow. You can check the response element to determine if a human loop has been created. To see everything included in this response, see HumanLoopActivationOutput.


    if "HumanLoopArn" in analyzeDocumentResponse["HumanLoopActivationOutput"]:
        # A human loop has been started!
        print(f"A human loop has been started with ARN: {analyzeDocumentResponse["HumanLoopActivationOutput"]["HumanLoopArn"]}"
Amazon Rekognition – Image moderation

The following example uses the Amazon SDK for Python (Boto3) to call detect_moderation_labels in us-west-2. Replace the italicized red text with your resources. Include the DataAttributes parameter if you are using the Amazon Mechanical Turk workforce. For more information, see the detect_moderation_labels documention in the Amazon SDK for Python (Boto) API Reference.


   response = client.detect_moderation_labels(
            Image={"S3Object":{"Bucket": "AWSDOC-EXAMPLE-BUCKET", "Name": "image-name.png"}},
            HumanLoopConfig={
               "FlowDefinitionArn":"arn:aws:sagemaker:us-west-2:111122223333:flow-definition/flow-definition-name",
               "HumanLoopName":"human-loop-name",
               "DataAttributes":{ContentClassifiers:["FreeOfPersonallyIdentifiableInformation"|"FreeOfAdultContent"]}
             }
    )

Human loops are only created if Amazon Rekognition's confidence for an image moderation task meets the activation conditions you specified in your human review workflow. You can check the response element to determine if a human loop has been created. To see everything included in this response, see HumanLoopActivationOutput.


    if "HumanLoopArn" in response["HumanLoopActivationOutput"]:
        # A human loop has been started!
        print(f"A human loop has been started with ARN: {response["HumanLoopActivationOutput"]["HumanLoopArn"]}")
Custom Integration

The following example uses the Amazon SDK for Python (Boto3) to call start_human_loop in us-west-2. Replace the italicized red text with your resources. Include the DataAttributes parameter if you are using the Amazon Mechanical Turk workforce. For more information, see the start_human_loop documention in the Amazon SDK for Python (Boto) API Reference.


   response = client.start_human_loop(
        HumanLoopName= "human-loop-name",
        FlowDefinitionArn= "arn:aws:sagemaker:us-west-2:111122223333:flow-definition/flow-definition-name",
        HumanLoopInput={"InputContent": inputContentJson},
        DataAttributes={"ContentClassifiers":["FreeOfPersonallyIdentifiableInformation"|"FreeOfAdultContent"]}
   )

This example stores input content in the variable inputContentJson. Assume that the input content contains two elements: a text blurb and sentiment (such as Positive, Negative, or Neutral), and it is formatted as follows:


    inputContent = {
        "initialValue": sentiment,
         "taskObject": blurb
     }

The keys initialValue and taskObject must correspond to the keys used in the liquid elements of the worker task template. Refer to the custom template in Create a Human Task UI to see an example.

To create inputContentJson, do the following: 


    import json
    
    inputContentJson = json.dumps(inputContent)

A human loop starts each time you call start_human_loop. To check the status of your human loop, use describe_human_loop: 


    human_loop_info = a2i.describe_human_loop(HumanLoopName="human_loop_name")
    print(f"HumanLoop Status: {resp["HumanLoopStatus"]}")
    print(f"HumanLoop Output Destination: {resp["HumanLoopOutput"]}")