Class JobProps
(experimental) Construction properties for {@link Job}.
Inheritance
Implements
Namespace: Amazon.CDK.AWS.Glue
Assembly: Amazon.CDK.AWS.Glue.dll
Syntax (csharp)
public class JobProps : Object, IJobProps
Syntax (vb)
Public Class JobProps
Inherits Object
Implements IJobProps
Remarks
Stability: Experimental
ExampleMetadata: infused
Examples
Bucket bucket;
new Job(this, "PythonShellJob", new JobProps {
Executable = JobExecutable.PythonShell(new PythonShellExecutableProps {
GlueVersion = GlueVersion.V1_0,
PythonVersion = PythonVersion.THREE,
Script = Code.FromBucket(bucket, "script.py")
}),
Description = "an example Python Shell job"
});
Synopsis
Constructors
JobProps() |
Properties
Connections | (experimental) The {@link Connection}s used for this job. |
ContinuousLogging | (experimental) Enables continuous logging with the specified props. |
DefaultArguments | (experimental) The default arguments for this job, specified as name-value pairs. |
Description | (experimental) The description of the job. |
EnableProfilingMetrics | (experimental) Enables the collection of metrics for job profiling. |
Executable | (experimental) The job's executable properties. |
JobName | (experimental) The name of the job. |
MaxCapacity | (experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. |
MaxConcurrentRuns | (experimental) The maximum number of concurrent runs allowed for the job. |
MaxRetries | (experimental) The maximum number of times to retry this job after a job run fails. |
NotifyDelayAfter | (experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification. |
Role | (experimental) The IAM role assumed by Glue to run this job. |
SecurityConfiguration | (experimental) The {@link SecurityConfiguration} to use for this job. |
SparkUI | (experimental) Enables the Spark UI debugging and monitoring with the specified props. |
Tags | (experimental) The tags to add to the resources on which the job runs. |
Timeout | (experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. |
WorkerCount | (experimental) The number of workers of a defined {@link WorkerType} that are allocated when a job runs. |
WorkerType | (experimental) The type of predefined worker that is allocated when a job runs. |
Constructors
JobProps()
public JobProps()
Properties
Connections
(experimental) The {@link Connection}s used for this job.
public IConnection[] Connections { get; set; }
Property Value
Remarks
Connections are used to connect to other AWS Service or resources within a VPC.
Default: [] - no connections are added to the job
Stability: Experimental
ContinuousLogging
(experimental) Enables continuous logging with the specified props.
public IContinuousLoggingProps ContinuousLogging { get; set; }
Property Value
Remarks
Default: - continuous logging is disabled.
Stability: Experimental
See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
DefaultArguments
(experimental) The default arguments for this job, specified as name-value pairs.
public IDictionary<string, string> DefaultArguments { get; set; }
Property Value
System.Collections.Generic.IDictionary<System.String, System.String>
Remarks
Default: - no arguments
Stability: Experimental
See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters
Description
(experimental) The description of the job.
public string Description { get; set; }
Property Value
System.String
Remarks
Default: - no value
Stability: Experimental
EnableProfilingMetrics
(experimental) Enables the collection of metrics for job profiling.
public Nullable<bool> EnableProfilingMetrics { get; set; }
Property Value
System.Nullable<System.Boolean>
Remarks
Default: - no profiling metrics emitted.
Stability: Experimental
See: --enable-metrics
at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
Executable
(experimental) The job's executable properties.
public JobExecutable Executable { get; set; }
Property Value
Remarks
Stability: Experimental
JobName
(experimental) The name of the job.
public string JobName { get; set; }
Property Value
System.String
Remarks
Default: - a name is automatically generated
Stability: Experimental
MaxCapacity
(experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
public Nullable<double> MaxCapacity { get; set; }
Property Value
System.Nullable<System.Double>
Remarks
Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.
Default: - 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell
Stability: Experimental
MaxConcurrentRuns
(experimental) The maximum number of concurrent runs allowed for the job.
public Nullable<double> MaxConcurrentRuns { get; set; }
Property Value
System.Nullable<System.Double>
Remarks
An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
Default: 1
Stability: Experimental
MaxRetries
(experimental) The maximum number of times to retry this job after a job run fails.
public Nullable<double> MaxRetries { get; set; }
Property Value
System.Nullable<System.Double>
Remarks
Default: 0
Stability: Experimental
NotifyDelayAfter
(experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification.
public Duration NotifyDelayAfter { get; set; }
Property Value
Remarks
Default: - no delay notifications
Stability: Experimental
Role
(experimental) The IAM role assumed by Glue to run this job.
public IRole Role { get; set; }
Property Value
Remarks
If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
Default: - a role is automatically generated
Stability: Experimental
See: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html
SecurityConfiguration
(experimental) The {@link SecurityConfiguration} to use for this job.
public ISecurityConfiguration SecurityConfiguration { get; set; }
Property Value
Remarks
Default: - no security configuration.
Stability: Experimental
SparkUI
(experimental) Enables the Spark UI debugging and monitoring with the specified props.
public ISparkUIProps SparkUI { get; set; }
Property Value
Remarks
Default: - Spark UI debugging and monitoring is disabled.
Stability: Experimental
See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
Tags
(experimental) The tags to add to the resources on which the job runs.
public IDictionary<string, string> Tags { get; set; }
Property Value
System.Collections.Generic.IDictionary<System.String, System.String>
Remarks
Default: {} - no tags
Stability: Experimental
Timeout
(experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
public Duration Timeout { get; set; }
Property Value
Remarks
Default: cdk.Duration.hours(48)
Stability: Experimental
WorkerCount
(experimental) The number of workers of a defined {@link WorkerType} that are allocated when a job runs.
public Nullable<double> WorkerCount { get; set; }
Property Value
System.Nullable<System.Double>
Remarks
Default: - differs based on specific Glue version/worker type
Stability: Experimental
WorkerType
(experimental) The type of predefined worker that is allocated when a job runs.
public WorkerType WorkerType { get; set; }
Property Value
Remarks
Default: - differs based on specific Glue version
Stability: Experimental