Reading from Adobe Marketo Engage entities - Amazon Glue
Partitioning queries
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Reading from Adobe Marketo Engage entities

Prerequisite

A Adobe Marketo Engage object you would like to read from. You will need the object name such as leads or activities or customobjects. The following tables shows the supported entities.

Supported entities for source (synchronous):

Entity Can be filtered Supports limit Supports Order by Supports Select * Supports partitioning
leads Yes Yes No Yes No
activities Yes Yes No Yes No
customobjects Yes Yes No Yes No

Supported entities for source (asynchronous):

Entity Can be filtered Supports limit Supports Order by Supports Select * Supports partitioning
leads Yes No No Yes Yes
activities Yes No No Yes No
customobjects Yes No No Yes Yes

Example:

adobe-marketo-engage_read = glueContext.create_dynamic_frame.from_options(
    connection_type="adobe-marketo-engage",
    connection_options={
        "connectionName": "connectionName",
        "ENTITY_NAME": "leads",
        "API_VERSION": "v2",
        "INSTANCE_URL": "https://539-t**-6**.mktorest.com"
    }

Adobe Marketo Engage entity and field details:

Entities with static metadata:

Entity Field Data type Support operators
activities sinceDatetime (only supported in synchronous) DateTime >= (only for synchronous mode)
createdAt (only supported in asynchronous) DateTime between (only for asynchronous mode)
activitiesTypeId Integer =
adobe-marketo-engageGUID Long = (only for synchronous mode)
leadId Long N/A
activityDate DateTime N/A
campaignId Long N/A
primaryAttributeValueId Integer N/A
primaryAttributeValue String N/A
attributes String N/A

Entities with dynamic metadata:

For the following entities, Adobe Marketo Engage provides endpoints to fetch metadata dynamically, so that operator support is captured at the datatype level for each entity.

Entity Data type Supported operators
leads Integer = (only for synchronous mode)
DateTime between (only for asynchronous mode)
String = (only for synchronous mode)
Long N/A
Boolean N/A
Date N/A
Float N/A
customobjects Integer N/A
DateTime between (only for asynchronous mode)
String = (only for synchronous mode)
Date N/A
Long N/A
Boolean N/A
Float N/A

Partitioning queries

You can provide the additional Spark options PARTITION_FIELD, LOWER_BOUND, UPPER_BOUND, and NUM_PARTITIONS if you want to utilize concurrency in Spark. With these parameters, the original query would be split into NUM_PARTITIONS number of sub-queries that can be executed by Spark tasks concurrently.

  • PARTITION_FIELD: the name of the field to be used to partition the query.

  • LOWER_BOUND: an inclusive lower bound value of the chosen partition field.

    For the DateTime field, we accept the value in ISO format.

    Example of valid value:

    "2024-07-01T00:00:00.000Z"

  • UPPER_BOUND: an exclusive upper bound value of the chosen partition field.

  • NUM_PARTITIONS: the number of partitions.

The following table describes the entity partitioning field support details:

Entity name Partitioning fields Data type
leads createdAt DateTime
updateAt DateTime
customobjects updatedAt DateTime

Example:

adobe-marketo-engage_read = glueContext.create_dynamic_frame.from_options(
    connection_type="adobe-marketo-engage",
    connection_options={
        "connectionName": "connectionName",
        "ENTITY_NAME": "leads",
        "API_VERSION": "v1",
        "PARTITION_FIELD": "createdAt"
        "LOWER_BOUND": "2024-07-01T00:00:00.000Z"
        "UPPER_BOUND": "2024-07-02T00:00:00.000Z"
        "NUM_PARTITIONS": "10"
    }