Programming Amazon Glue ETL scripts in Scala
You can find Scala code examples and utilities for Amazon Glue in the Amazon Glue samples repository
Amazon Glue supports an extension of the PySpark Scala dialect for scripting extract, transform, and load (ETL) jobs. The following sections describe how to use the Amazon Glue Scala library and the Amazon Glue API in ETL scripts, and provide reference documentation for the library.
Contents
- Using Scala
- Scala script example
- Scala API list
- com.amazonaws.services.glue
- com.amazonaws.services.glue.ml
- com.amazonaws.services.glue.dq
- com.amazonaws.services.glue.types
- com.amazonaws.services.glue.util
- ChoiceOption
- DataSink
- DataSource trait
- DynamicFrame
- DynamicFrame class
- errorsCount
- applyMapping
- assertErrorThreshold
- Count
- dropField
- dropFields
- dropNulls
- errorsAsDynamicFrame
- Filter
- getName
- getNumPartitions
- getSchemaIfComputed
- isSchemaComputed
- javaToPython
- Join
- Map
- mergeDynamicFrames
- printSchema
- recomputeSchema
- Relationalize
- renameField
- Repartition
- resolveChoice
- Schema
- selectField
- selectFields
- Show
- SimplifyDDBJson
- Spigot
- splitFields
- Def splitRows
- stageErrorsCount
- toDF
- Unbox
- Unnest
- unnestDDBJson
- withFrameSchema
- Def withName
- withTransformationContext
- DynamicFrame object
- DynamicFrame class
- DynamicRecord
- GlueContext
- MappingSpec
- ResolveSpec
- ArrayNode
- BinaryNode
- BooleanNode
- ByteNode
- DateNode
- DecimalNode
- DoubleNode
- DynamicNode
- EvaluateDataQuality
- FloatNode
- FillMissingValues
- FindMatches
- FindIncrementalMatches
- IntegerNode
- LongNode
- MapLikeNode
- MapNode
- NullNode
- ObjectNode
- ScalarNode
- ShortNode
- StringNode
- TimestampNode
- GlueArgParser
- Job