Class Classification
The classification within a EMR Containers application configuration.
Inheritance
Namespace: Amazon.CDK.AWS.StepFunctions.Tasks
Assembly: Amazon.CDK.Lib.dll
Syntax (csharp)
public class Classification : DeputyBase
Syntax (vb)
Public Class Classification
Inherits DeputyBase
Remarks
Class can be extended to add other classifications. For example, new Classification('xxx-yyy');
ExampleMetadata: infused
Examples
new EmrContainersStartJobRun(this, "EMR Containers Start Job Run", new EmrContainersStartJobRunProps {
VirtualCluster = VirtualClusterInput.FromVirtualClusterId("de92jdei2910fwedz"),
ReleaseLabel = ReleaseLabel.EMR_6_2_0,
JobName = "EMR-Containers-Job",
JobDriver = new JobDriver {
SparkSubmitJobDriver = new SparkSubmitJobDriver {
EntryPoint = TaskInput.FromText("local:///usr/lib/spark/examples/src/main/python/pi.py")
}
},
ApplicationConfig = new [] { new ApplicationConfiguration {
Classification = Classification.SPARK_DEFAULTS,
Properties = new Dictionary<string, string> {
{ "spark.executor.instances", "1" },
{ "spark.executor.memory", "512M" }
}
} }
});
Synopsis
Constructors
Classification(ByRefValue) | Used by jsii to construct an instance of this class from a Javascript-owned object reference |
Classification(DeputyBase.DeputyProps) | Used by jsii to construct an instance of this class from DeputyProps |
Classification(String) | Creates a new Classification. |
Properties
ClassificationStatement | A literal string in case a new EMR classification is released, if not already defined. |
SPARK | Sets the maximizeResourceAllocation property to true or false. |
SPARK_DEFAULTS | Sets values in the spark-defaults.conf file. |
SPARK_ENV | Sets values in the spark-env.sh file. |
SPARK_HIVE_SITE | Sets values in the hive-site.xml for Spark. |
SPARK_LOG4J | Sets values in the log4j.properties file. |
SPARK_METRICS | Sets values in the metrics.properties file. |
Constructors
Classification(ByRefValue)
Used by jsii to construct an instance of this class from a Javascript-owned object reference
protected Classification(ByRefValue reference)
Parameters
- reference Amazon.JSII.Runtime.Deputy.ByRefValue
The Javascript-owned object reference
Classification(DeputyBase.DeputyProps)
Used by jsii to construct an instance of this class from DeputyProps
protected Classification(DeputyBase.DeputyProps props)
Parameters
- props Amazon.JSII.Runtime.Deputy.DeputyBase.DeputyProps
The deputy props
Classification(String)
Creates a new Classification.
public Classification(string classificationStatement)
Parameters
- classificationStatement System.String
A literal string in case a new EMR classification is released, if not already defined.
Properties
ClassificationStatement
A literal string in case a new EMR classification is released, if not already defined.
public virtual string ClassificationStatement { get; }
Property Value
System.String
SPARK
Sets the maximizeResourceAllocation property to true or false.
public static Classification SPARK { get; }
Property Value
Remarks
When true, Amazon EMR automatically configures spark-defaults properties based on cluster hardware configuration.
For more info:
SPARK_DEFAULTS
Sets values in the spark-defaults.conf file.
public static Classification SPARK_DEFAULTS { get; }
Property Value
Remarks
For more info:
See: https://spark.apache.org/docs/latest/configuration.html
SPARK_ENV
Sets values in the spark-env.sh file.
public static Classification SPARK_ENV { get; }
Property Value
Remarks
For more info:
See: https://spark.apache.org/docs/latest/configuration.html#environment-variables
SPARK_HIVE_SITE
Sets values in the hive-site.xml for Spark.
public static Classification SPARK_HIVE_SITE { get; }
Property Value
SPARK_LOG4J
Sets values in the log4j.properties file.
public static Classification SPARK_LOG4J { get; }
Property Value
Remarks
For more settings and info:
See: https://github.com/apache/spark/blob/master/conf/log4j.properties.template
SPARK_METRICS
Sets values in the metrics.properties file.
public static Classification SPARK_METRICS { get; }
Property Value
Remarks
For more settings and info:
See: https://github.com/apache/spark/blob/master/conf/metrics.properties.template