Class: Aws::Glue::Types::SparkSQL

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Specifies a transform where you enter a SQL query using Spark SQL syntax to transform the data. The output is a single DynamicFrame.

See Also:

Constant Summary collapse

SENSITIVE = 
[]

Instance Attribute Summary collapse

Instance Attribute Details

#inputs ⇒ Array<String>

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

Returns:

  • (Array<String>) 


19691
19692
19693
19694
19695
19696
19697
19698
19699
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 19691

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#name ⇒ String

The name of the transform node.

Returns:

  • (String) 


19691
19692
19693
19694
19695
19696
19697
19698
19699
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 19691

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#output_schemas ⇒ Array<Types::GlueSchema>

Specifies the data schema for the SparkSQL transform.

Returns:



19691
19692
19693
19694
19695
19696
19697
19698
19699
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 19691

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#sql_aliases ⇒ Array<Types::SqlAlias>

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

Returns:



19691
19692
19693
19694
19695
19696
19697
19698
19699
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 19691

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#sql_query ⇒ String

A SQL query that must use Spark SQL syntax and return a single data set.

Returns:

  • (String) 


19691
19692
19693
19694
19695
19696
19697
19698
19699
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 19691

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end