ResolveChoice class
Resolves a choice type within a DynamicFrame
.
Example
We recommend that you use the DynamicFrame.resolveChoice()
method to handle fields that contain multiple types in a DynamicFrame
. To view a code example, see Example: Use resolveChoice to handle a column that contains multiple types.
Methods
__call__(frame, specs = none, choice = "", transformation_ctx = "", info = "", stageThreshold = 0, totalThreshold = 0)
Provides information for resolving ambiguous types within a DynamicFrame
. It
returns the resulting DynamicFrame
.
-
frame
– TheDynamicFrame
in which to resolve the choice type (required). -
specs
– A list of specific ambiguities to resolve, each in the form of a tuple:(path, action)
. Thepath
value identifies a specific ambiguous element, and theaction
value identifies the corresponding resolution.You can only use one of the
spec
andchoice
parameters. If thespec
parameter is notNone
, then thechoice
parameter must be an empty string. Conversely, if thechoice
is not an empty string, then thespec
parameter must beNone
. If neither parameter is provided, Amazon Glue tries to parse the schema and use it to resolve ambiguities.You can specify one of the following resolution strategies in the
action
portion of aspecs
tuple:-
cast
– Allows you to specify a type to cast to (for example,cast:int
). -
make_cols
– Resolves a potential ambiguity by flattening the data. For example, ifcolumnA
could be anint
or astring
, the resolution is to produce two columns namedcolumnA_int
andcolumnA_string
in the resultingDynamicFrame
. -
make_struct
– Resolves a potential ambiguity by using a struct to represent the data. For example, if data in a column could be anint
or astring
, using themake_struct
action produces a column of structures in the resultingDynamicFrame
with each containing both anint
and astring
. -
project
– Resolves a potential ambiguity by retaining only values of a specified type in the resultingDynamicFrame
. For example, if data in aChoiceType
column could be anint
or astring
, specifying aproject:string
action drops values from the resultingDynamicFrame
that are not typestring
.
If the
path
identifies an array, place empty square brackets after the name of the array to avoid ambiguity. For example, suppose you are working with data structured as follows:"myList": [ { "price": 100.00 }, { "price": "$100.00" } ]
You can select the numeric rather than the string version of the price by setting the
path
to"myList[].price"
, and setting theaction
to"cast:double"
. -
-
choice
– The default resolution action if thespecs
parameter isNone
. If thespecs
parameter is notNone
, then this must not be set to anything but an empty string.In addition to the
specs
actions previously described, this argument also supports the following action:-
MATCH_CATALOG
– Attempts to cast eachChoiceType
to the corresponding type in the specified Data Catalog table.
-
-
database
– The Amazon Glue Data Catalog database to use with theMATCH_CATALOG
choice (required forMATCH_CATALOG
). -
table_name
– The Amazon Glue Data Catalog table name to use with theMATCH_CATALOG
action (required forMATCH_CATALOG
). -
transformation_ctx
– A unique string that is used to identify state information (optional). -
info
– A string associated with errors in the transformation (optional). -
stageThreshold
– The maximum number of errors that can occur in the transformation before it errors out (optional). The default is zero. -
totalThreshold
– The maximum number of errors that can occur overall before processing errors out (optional). The default is zero.
apply(cls, *args, **kwargs)
Inherited from GlueTransform
apply.
name(cls)
Inherited from GlueTransform
name.
describeArgs(cls)
Inherited from GlueTransform
describeArgs.
describeReturn(cls)
Inherited from GlueTransform
describeReturn.
describeTransform(cls)
Inherited from GlueTransform
describeTransform.
describeErrors(cls)
Inherited from GlueTransform
describeErrors.
describe(cls)
Inherited from GlueTransform
describe.