Amazon Glue API code examples using Amazon SDKs - Amazon Glue
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon Glue API code examples using Amazon SDKs

The following code examples show how to use Amazon Glue with an Amazon software development kit (SDK).

Actions are code excerpts from larger programs and must be run in context. While actions show you how to call individual service functions, you can see actions in context in their related scenarios and cross-service examples.

Scenarios are code examples that show you how to accomplish a specific task by calling multiple functions within the same service.

For a complete list of Amazon SDK developer guides and code examples, see Using this service with an Amazon SDK. This topic also includes information about getting started and details about previous SDK versions.

Get started

The following code examples show how to get started using Amazon Glue.

.NET
Amazon SDK for .NET
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

namespace GlueActions; public class HelloGlue { private static ILogger logger = null!; static async Task Main(string[] args) { // Set up dependency injection for AWS Glue. using var host = Host.CreateDefaultBuilder(args) .ConfigureLogging(logging => logging.AddFilter("System", LogLevel.Debug) .AddFilter<DebugLoggerProvider>("Microsoft", LogLevel.Information) .AddFilter<ConsoleLoggerProvider>("Microsoft", LogLevel.Trace)) .ConfigureServices((_, services) => services.AddAWSService<IAmazonGlue>() .AddTransient<GlueWrapper>() ) .Build(); logger = LoggerFactory.Create(builder => { builder.AddConsole(); }) .CreateLogger<HelloGlue>(); var glueClient = host.Services.GetRequiredService<IAmazonGlue>(); var request = new ListJobsRequest(); var jobNames = new List<string>(); do { var response = await glueClient.ListJobsAsync(request); jobNames.AddRange(response.JobNames); request.NextToken = response.NextToken; } while (request.NextToken is not null); Console.Clear(); Console.WriteLine("Hello, Glue. Let's list your existing Glue Jobs:"); if (jobNames.Count == 0) { Console.WriteLine("You don't have any AWS Glue jobs."); } else { jobNames.ForEach(Console.WriteLine); } } }
  • For API details, see ListJobs in Amazon SDK for .NET API Reference.

C++
SDK for C++
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

Code for the CMakeLists.txt CMake file.

# Set the minimum required version of CMake for this project. cmake_minimum_required(VERSION 3.13) # Set the AWS service components used by this project. set(SERVICE_COMPONENTS glue) # Set this project's name. project("hello_glue") # Set the C++ standard to use to build this target. # At least C++ 11 is required for the AWS SDK for C++. set(CMAKE_CXX_STANDARD 11) # Use the MSVC variable to determine if this is a Windows build. set(WINDOWS_BUILD ${MSVC}) if (WINDOWS_BUILD) # Set the location where CMake can find the installed libraries for the AWS SDK. string(REPLACE ";" "/aws-cpp-sdk-all;" SYSTEM_MODULE_PATH "${CMAKE_SYSTEM_PREFIX_PATH}/aws-cpp-sdk-all") list(APPEND CMAKE_PREFIX_PATH ${SYSTEM_MODULE_PATH}) endif () # Find the AWS SDK for C++ package. find_package(AWSSDK REQUIRED COMPONENTS ${SERVICE_COMPONENTS}) if (WINDOWS_BUILD AND AWSSDK_INSTALL_AS_SHARED_LIBS) # Copy relevant AWS SDK for C++ libraries into the current binary directory for running and debugging. # set(BIN_SUB_DIR "/Debug") # if you are building from the command line you may need to uncomment this # and set the proper subdirectory to the executables' location. AWSSDK_CPY_DYN_LIBS(SERVICE_COMPONENTS "" ${CMAKE_CURRENT_BINARY_DIR}${BIN_SUB_DIR}) endif () add_executable(${PROJECT_NAME} hello_glue.cpp) target_link_libraries(${PROJECT_NAME} ${AWSSDK_LINK_LIBRARIES})

Code for the hello_glue.cpp source file.

#include <aws/core/Aws.h> #include <aws/glue/GlueClient.h> #include <aws/glue/model/ListJobsRequest.h> #include <iostream> /* * A "Hello Glue" starter application which initializes an AWS Glue client and lists the * AWS Glue job definitions. * * main function * * Usage: 'hello_glue' * */ int main(int argc, char **argv) { Aws::SDKOptions options; // Optionally change the log level for debugging. // options.loggingOptions.logLevel = Utils::Logging::LogLevel::Debug; Aws::InitAPI(options); // Should only be called once. int result = 0; { Aws::Client::ClientConfiguration clientConfig; // Optional: Set to the AWS Region (overrides config file). // clientConfig.region = "us-east-1"; Aws::Glue::GlueClient glueClient(clientConfig); std::vector<Aws::String> jobs; Aws::String nextToken; // Used for pagination. do { Aws::Glue::Model::ListJobsRequest listJobsRequest; if (!nextToken.empty()) { listJobsRequest.SetNextToken(nextToken); } Aws::Glue::Model::ListJobsOutcome listRunsOutcome = glueClient.ListJobs( listJobsRequest); if (listRunsOutcome.IsSuccess()) { const std::vector<Aws::String> &jobNames = listRunsOutcome.GetResult().GetJobNames(); jobs.insert(jobs.end(), jobNames.begin(), jobNames.end()); nextToken = listRunsOutcome.GetResult().GetNextToken(); } else { std::cerr << "Error listing jobs. " << listRunsOutcome.GetError().GetMessage() << std::endl; result = 1; break; } } while (!nextToken.empty()); std::cout << "Your account has " << jobs.size() << " jobs." << std::endl; for (size_t i = 0; i < jobs.size(); ++i) { std::cout << " " << i + 1 << ". " << jobs[i] << std::endl; } } Aws::ShutdownAPI(options); // Should only be called once. return result; }
  • For API details, see ListJobs in Amazon SDK for C++ API Reference.

Java
SDK for Java 2.x
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

package com.example.glue; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.glue.GlueClient; import software.amazon.awssdk.services.glue.model.ListJobsRequest; import software.amazon.awssdk.services.glue.model.ListJobsResponse; import java.util.List; public class HelloGlue { public static void main(String[] args) { GlueClient glueClient = GlueClient.builder() .region(Region.US_EAST_1) .build(); listJobs(glueClient); } public static void listJobs(GlueClient glueClient) { ListJobsRequest request = ListJobsRequest.builder() .maxResults(10) .build(); ListJobsResponse response = glueClient.listJobs(request); List<String> jobList = response.jobNames(); jobList.forEach(job -> { System.out.println("Job Name: " + job); }); } }
  • For API details, see ListJobs in Amazon SDK for Java 2.x API Reference.

JavaScript
SDK for JavaScript (v3)
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

import { ListJobsCommand, GlueClient } from "@aws-sdk/client-glue"; const client = new GlueClient({}); export const main = async () => { const command = new ListJobsCommand({}); const { JobNames } = await client.send(command); const formattedJobNames = JobNames.join("\n"); console.log("Job names: "); console.log(formattedJobNames); return JobNames; };
  • For API details, see ListJobs in Amazon SDK for JavaScript API Reference.

Python
SDK for Python (Boto3)
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

import boto3 from botocore.exceptions import ClientError def hello_glue(): """ Lists the job definitions in your AWS Glue account, using the AWS SDK for Python (Boto3). """ try: # Create the Glue client glue = boto3.client("glue") # List the jobs, limiting the results to 10 per page paginator = glue.get_paginator("get_jobs") response_iterator = paginator.paginate( PaginationConfig={"MaxItems": 10, "PageSize": 10} ) # Print the job names print("Here are the jobs in your account:") for page in response_iterator: for job in page["Jobs"]: print(f"\t{job['Name']}") except ClientError as e: print(f"Error: {e}") if __name__ == "__main__": hello_glue()
  • For API details, see ListJobs in Amazon SDK for Python (Boto3) API Reference.

Ruby
SDK for Ruby
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

require 'aws-sdk-glue' require 'logger' # GlueManager is a class responsible for managing AWS Glue operations # such as listing all Glue jobs in the current AWS account. class GlueManager def initialize(client) @client = client @logger = Logger.new($stdout) end # Lists and prints all Glue jobs in the current AWS account. def list_jobs @logger.info('Here are the Glue jobs in your account:') paginator = @client.get_jobs(max_results: 10) jobs = [] paginator.each_page do |page| jobs.concat(page.jobs) end if jobs.empty? @logger.info("You don't have any Glue jobs.") else jobs.each do |job| @logger.info("- #{job.name}") end end end end if $PROGRAM_NAME == __FILE__ glue_client = Aws::Glue::Client.new manager = GlueManager.new(glue_client) manager.list_jobs end
  • For API details, see ListJobs in Amazon SDK for Ruby API Reference.

Rust
SDK for Rust
Note

There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository.

let mut list_jobs = glue.list_jobs().into_paginator().send(); while let Some(list_jobs_output) = list_jobs.next().await { match list_jobs_output { Ok(list_jobs) => { let names = list_jobs.job_names(); info!(?names, "Found these jobs") } Err(err) => return Err(GlueMvpError::from_glue_sdk(err)), } }
  • For API details, see ListJobs in Amazon SDK for Rust API reference.