Listing object keys programmatically
In Amazon S3, keys can be listed by prefix. You can choose a common prefix for the names of related keys and mark these keys with a special character that delimits hierarchy. You can then use the list operation to select and browse keys hierarchically. This is similar to how files are stored in directories within a file system.
Amazon S3 exposes a list operation that lets you enumerate the keys contained in a bucket. Keys
are selected for listing by bucket and prefix. For example, consider a bucket named
"dictionary
" that contains a key for every English word. You might make a
call to list all the keys in that bucket that start with the letter "q". List results are
always returned in UTF-8 binary order.
Both the SOAP and REST list operations return an XML document that contains the names of matching keys and information about the object identified by each key.
Note
SOAP support over HTTP is deprecated, but SOAP is still available over HTTPS. New Amazon S3 features are not supported for SOAP. Instead of using SOAP, we recommend that you use either the REST API or the Amazon SDKs.
Groups of keys that share a prefix terminated by a special delimiter can be rolled up by that common prefix for the purposes of listing. This enables applications to organize and browse their keys hierarchically, much like how you would organize your files into directories in a file system.
For example, to extend the dictionary bucket to contain more than just English words, you
might form keys by prefixing each word with its language and a delimiter, such as
"French/logical
". Using this naming scheme and the hierarchical listing
feature, you could retrieve a list of only French words. You could also browse the top-level
list of available languages without having to iterate through all the lexicographically
intervening keys. For more information about this aspect of listing, see Organizing objects using prefixes.
REST API
If your application requires it, you can send REST requests directly. You can send a GET request to return some or all of the objects in a bucket or you can use selection criteria to return a subset of the objects in a bucket. For more information, see GET Bucket (List Objects) Version 2 in the Amazon Simple Storage Service API Reference.
List implementation efficiency
List performance is not substantially affected by the total number of keys in your
bucket. It's also not affected by the presence or absence of the prefix
,
marker
, maxkeys
, or delimiter
arguments.
Iterating through multipage results
As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, the Amazon S3 API supports pagination to split them into multiple responses. Each list keys response returns a page of up to 1,000 keys with an indicator indicating if the response is truncated. You send a series of list keys requests until you have received all the keys. Amazon SDK wrapper libraries provide the same pagination.
Examples
The following code examples show how to use ListObjectsV2
.
- .NET
-
- Amazon SDK for .NET
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. /// <summary> /// Shows how to list the objects in an Amazon S3 bucket. /// </summary> /// <param name="client">An initialized Amazon S3 client object.</param> /// <param name="bucketName">The name of the bucket for which to list /// the contents.</param> /// <returns>A boolean value indicating the success or failure of the /// copy operation.</returns> public static async Task<bool> ListBucketContentsAsync(IAmazonS3 client, string bucketName) { try { var request = new ListObjectsV2Request { BucketName = bucketName, MaxKeys = 5, }; Console.WriteLine("--------------------------------------"); Console.WriteLine($"Listing the contents of {bucketName}:"); Console.WriteLine("--------------------------------------"); ListObjectsV2Response response; do { response = await client.ListObjectsV2Async(request); response.S3Objects .ForEach(obj => Console.WriteLine($"{obj.Key,-35}{obj.LastModified.ToShortDateString(),10}{obj.Size,10}")); // If the response is truncated, set the request ContinuationToken // from the NextContinuationToken property of the response. request.ContinuationToken = response.NextContinuationToken; } while (response.IsTruncated); return true; } catch (AmazonS3Exception ex) { Console.WriteLine($"Error encountered on server. Message:'{ex.Message}' getting list of objects."); return false; } }
List objects with a paginator.
using System; using System.Threading.Tasks; using Amazon.S3; using Amazon.S3.Model; /// <summary> /// The following example lists objects in an Amazon Simple Storage /// Service (Amazon S3) bucket. /// </summary> public class ListObjectsPaginator { private const string BucketName = "doc-example-bucket"; public static async Task Main() { IAmazonS3 s3Client = new AmazonS3Client(); Console.WriteLine($"Listing the objects contained in {BucketName}:\n"); await ListingObjectsAsync(s3Client, BucketName); } /// <summary> /// This method uses a paginator to retrieve the list of objects in an /// an Amazon S3 bucket. /// </summary> /// <param name="client">An Amazon S3 client object.</param> /// <param name="bucketName">The name of the S3 bucket whose objects /// you want to list.</param> public static async Task ListingObjectsAsync(IAmazonS3 client, string bucketName) { var listObjectsV2Paginator = client.Paginators.ListObjectsV2(new ListObjectsV2Request { BucketName = bucketName, }); await foreach (var response in listObjectsV2Paginator.Responses) { Console.WriteLine($"HttpStatusCode: {response.HttpStatusCode}"); Console.WriteLine($"Number of Keys: {response.KeyCount}"); foreach (var entry in response.S3Objects) { Console.WriteLine($"Key = {entry.Key} Size = {entry.Size}"); } } } }
-
For API details, see ListObjectsV2 in Amazon SDK for .NET API Reference.
-
- Bash
-
- Amazon CLI with Bash script
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. ############################################################################### # function errecho # # This function outputs everything sent to it to STDERR (standard error output). ############################################################################### function errecho() { printf "%s\n" "$*" 1>&2 } ############################################################################### # function list_items_in_bucket # # This function displays a list of the files in the bucket with each file's # size. The function uses the --query parameter to retrieve only the key and # size fields from the Contents collection. # # Parameters: # $1 - The name of the bucket. # # Returns: # The list of files in text format. # And: # 0 - If successful. # 1 - If it fails. ############################################################################### function list_items_in_bucket() { local bucket_name=$1 local response response=$(aws s3api list-objects \ --bucket "$bucket_name" \ --output text \ --query 'Contents[].{Key: Key, Size: Size}') # shellcheck disable=SC2181 if [[ ${?} -eq 0 ]]; then echo "$response" else errecho "ERROR: AWS reports s3api list-objects operation failed.\n$response" return 1 fi }
-
For API details, see ListObjectsV2 in Amazon CLI Command Reference.
-
- C++
-
- SDK for C++
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. bool AwsDoc::S3::ListObjects(const Aws::String &bucketName, const Aws::Client::ClientConfiguration &clientConfig) { Aws::S3::S3Client s3_client(clientConfig); Aws::S3::Model::ListObjectsV2Request request; request.WithBucket(bucketName); Aws::String continuationToken; // Used for pagination. Aws::Vector<Aws::S3::Model::Object> allObjects; do { if (!continuationToken.empty()) { request.SetContinuationToken(continuationToken); } auto outcome = s3_client.ListObjectsV2(request); if (!outcome.IsSuccess()) { std::cerr << "Error: ListObjects: " << outcome.GetError().GetMessage() << std::endl; return false; } else { Aws::Vector<Aws::S3::Model::Object> objects = outcome.GetResult().GetContents(); allObjects.insert(allObjects.end(), objects.begin(), objects.end()); continuationToken = outcome.GetResult().GetNextContinuationToken(); } } while (!continuationToken.empty()); std::cout << allObjects.size() << " object(s) found:" << std::endl; for (const auto &object: allObjects) { std::cout << " " << object.GetKey() << std::endl; } return true; }
-
For API details, see ListObjectsV2 in Amazon SDK for C++ API Reference.
-
- CLI
-
- Amazon CLI
-
The following example uses the
list-objects
command to display the names of all the objects in the specified bucket:aws s3api list-objects --bucket text-content --query 'Contents[].{Key: Key, Size: Size}'
The example uses the
--query
argument to filter the output oflist-objects
down to the key value and size for each objectFor more information about objects, see Working with Amazon S3 Objects in the Amazon S3 Developer Guide.
-
For API details, see ListObjectsV2
in Amazon CLI Command Reference.
-
- Go
-
- SDK for Go V2
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. // BucketBasics encapsulates the Amazon Simple Storage Service (Amazon S3) actions // used in the examples. // It contains S3Client, an Amazon S3 service client that is used to perform bucket // and object actions. type BucketBasics struct { S3Client *s3.Client } // ListObjects lists the objects in a bucket. func (basics BucketBasics) ListObjects(bucketName string) ([]types.Object, error) { result, err := basics.S3Client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{ Bucket: aws.String(bucketName), }) var contents []types.Object if err != nil { log.Printf("Couldn't list objects in bucket %v. Here's why: %v\n", bucketName, err) } else { contents = result.Contents } return contents, err }
-
For API details, see ListObjectsV2
in Amazon SDK for Go API Reference.
-
- Java
-
- SDK for Java 2.x
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.ListObjectsRequest; import software.amazon.awssdk.services.s3.model.ListObjectsResponse; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.model.S3Object; import java.util.List; /** * Before running this Java V2 code example, set up your development * environment, including your credentials. * * For more information, see the following documentation topic: * * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html */ public class ListObjects { public static void main(String[] args) { final String usage = """ Usage: <bucketName>\s Where: bucketName - The Amazon S3 bucket from which objects are read.\s """; if (args.length != 1) { System.out.println(usage); System.exit(1); } String bucketName = args[0]; Region region = Region.US_EAST_1; S3Client s3 = S3Client.builder() .region(region) .build(); listBucketObjects(s3, bucketName); s3.close(); } public static void listBucketObjects(S3Client s3, String bucketName) { try { ListObjectsRequest listObjects = ListObjectsRequest .builder() .bucket(bucketName) .build(); ListObjectsResponse res = s3.listObjects(listObjects); List<S3Object> objects = res.contents(); for (S3Object myValue : objects) { System.out.print("\n The name of the key is " + myValue.key()); System.out.print("\n The object is " + calKb(myValue.size()) + " KBs"); System.out.print("\n The owner is " + myValue.owner()); } } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); } } // convert bytes to kbs. private static long calKb(Long val) { return val / 1024; } }
List objects using pagination.
import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.services.s3.model.ListObjectsV2Request; import software.amazon.awssdk.services.s3.model.S3Exception; import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable; public class ListObjectsPaginated { public static void main(String[] args) { final String usage = """ Usage: <bucketName>\s Where: bucketName - The Amazon S3 bucket from which objects are read.\s """; if (args.length != 1) { System.out.println(usage); System.exit(1); } String bucketName = args[0]; Region region = Region.US_EAST_1; S3Client s3 = S3Client.builder() .region(region) .build(); listBucketObjects(s3, bucketName); s3.close(); } public static void listBucketObjects(S3Client s3, String bucketName) { try { ListObjectsV2Request listReq = ListObjectsV2Request.builder() .bucket(bucketName) .maxKeys(1) .build(); ListObjectsV2Iterable listRes = s3.listObjectsV2Paginator(listReq); listRes.stream() .flatMap(r -> r.contents().stream()) .forEach(content -> System.out.println(" Key: " + content.key() + " size = " + content.size())); } catch (S3Exception e) { System.err.println(e.awsErrorDetails().errorMessage()); System.exit(1); } } }
-
For API details, see ListObjectsV2 in Amazon SDK for Java 2.x API Reference.
-
- JavaScript
-
- SDK for JavaScript (v3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. List all of the objects in your bucket. If there is more than one object, IsTruncated and NextContinuationToken will be used to iterate over the full list.
import { S3Client, // This command supersedes the ListObjectsCommand and is the recommended way to list objects. ListObjectsV2Command, } from "@aws-sdk/client-s3"; const client = new S3Client({}); export const main = async () => { const command = new ListObjectsV2Command({ Bucket: "my-bucket", // The default and maximum number of keys returned is 1000. This limits it to // one for demonstration purposes. MaxKeys: 1, }); try { let isTruncated = true; console.log("Your bucket contains the following objects:\n"); let contents = ""; while (isTruncated) { const { Contents, IsTruncated, NextContinuationToken } = await client.send(command); const contentsList = Contents.map((c) => ` • ${c.Key}`).join("\n"); contents += contentsList + "\n"; isTruncated = IsTruncated; command.input.ContinuationToken = NextContinuationToken; } console.log(contents); } catch (err) { console.error(err); } };
-
For API details, see ListObjectsV2 in Amazon SDK for JavaScript API Reference.
-
- Kotlin
-
- SDK for Kotlin
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. suspend fun listBucketObjects(bucketName: String) { val request = ListObjectsRequest { bucket = bucketName } S3Client { region = "us-east-1" }.use { s3 -> val response = s3.listObjects(request) response.contents?.forEach { myObject -> println("The name of the key is ${myObject.key}") println("The object is ${myObject.size?.let { calKb(it) }} KBs") println("The owner is ${myObject.owner}") } } } private fun calKb(intValue: Long): Long { return intValue / 1024 }
-
For API details, see ListObjectsV2
in Amazon SDK for Kotlin API reference.
-
- PHP
-
- SDK for PHP
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. List objects in a bucket.
$s3client = new Aws\S3\S3Client(['region' => 'us-west-2']); try { $contents = $this->s3client->listObjectsV2([ 'Bucket' => $this->bucketName, ]); echo "The contents of your bucket are: \n"; foreach ($contents['Contents'] as $content) { echo $content['Key'] . "\n"; } } catch (Exception $exception) { echo "Failed to list objects in $this->bucketName with error: " . $exception->getMessage(); exit("Please fix error with listing objects before continuing."); }
-
For API details, see ListObjectsV2 in Amazon SDK for PHP API Reference.
-
- PowerShell
-
- Tools for PowerShell
-
Example 1: This command retrieves the information about all of the items in the bucket "test-files".
Get-S3Object -BucketName test-files
Example 2: This command retrieves the information about the item "sample.txt" from bucket "test-files".
Get-S3Object -BucketName test-files -Key sample.txt
Example 3: This command retrieves the information about all items with the prefix "sample" from bucket "test-files".
Get-S3Object -BucketName test-files -KeyPrefix sample
-
For API details, see ListObjectsV2
in Amazon Tools for PowerShell Cmdlet Reference.
-
- Python
-
- SDK for Python (Boto3)
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. class ObjectWrapper: """Encapsulates S3 object actions.""" def __init__(self, s3_object): """ :param s3_object: A Boto3 Object resource. This is a high-level resource in Boto3 that wraps object actions in a class-like structure. """ self.object = s3_object self.key = self.object.key @staticmethod def list(bucket, prefix=None): """ Lists the objects in a bucket, optionally filtered by a prefix. :param bucket: The bucket to query. This is a Boto3 Bucket resource. :param prefix: When specified, only objects that start with this prefix are listed. :return: The list of objects. """ try: if not prefix: objects = list(bucket.objects.all()) else: objects = list(bucket.objects.filter(Prefix=prefix)) logger.info( "Got objects %s from bucket '%s'", [o.key for o in objects], bucket.name ) except ClientError: logger.exception("Couldn't get objects for bucket '%s'.", bucket.name) raise else: return objects
-
For API details, see ListObjectsV2 in Amazon SDK for Python (Boto3) API Reference.
-
- Ruby
-
- SDK for Ruby
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. require "aws-sdk-s3" # Wraps Amazon S3 bucket actions. class BucketListObjectsWrapper attr_reader :bucket # @param bucket [Aws::S3::Bucket] An existing Amazon S3 bucket. def initialize(bucket) @bucket = bucket end # Lists object in a bucket. # # @param max_objects [Integer] The maximum number of objects to list. # @return [Integer] The number of objects listed. def list_objects(max_objects) count = 0 puts "The objects in #{@bucket.name} are:" @bucket.objects.each do |obj| puts "\t#{obj.key}" count += 1 break if count == max_objects end count rescue Aws::Errors::ServiceError => e puts "Couldn't list objects in bucket #{bucket.name}. Here's why: #{e.message}" 0 end end # Example usage: def run_demo bucket_name = "doc-example-bucket" wrapper = BucketListObjectsWrapper.new(Aws::S3::Bucket.new(bucket_name)) count = wrapper.list_objects(25) puts "Listed #{count} objects." end run_demo if $PROGRAM_NAME == __FILE__
-
For API details, see ListObjectsV2 in Amazon SDK for Ruby API Reference.
-
- Rust
-
- SDK for Rust
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. pub async fn list_objects(client: &Client, bucket: &str) -> Result<(), Error> { let mut response = client .list_objects_v2() .bucket(bucket.to_owned()) .max_keys(10) // In this example, go 10 at a time. .into_paginator() .send(); while let Some(result) = response.next().await { match result { Ok(output) => { for object in output.contents() { println!(" - {}", object.key().unwrap_or("Unknown")); } } Err(err) => { eprintln!("{err:?}") } } } Ok(()) }
-
For API details, see ListObjectsV2
in Amazon SDK for Rust API reference.
-
- SAP ABAP
-
- SDK for SAP ABAP
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. TRY. oo_result = lo_s3->listobjectsv2( " oo_result is returned for testing purposes. " iv_bucket = iv_bucket_name ). MESSAGE 'Retrieved list of objects in S3 bucket.' TYPE 'I'. CATCH /aws1/cx_s3_nosuchbucket. MESSAGE 'Bucket does not exist.' TYPE 'E'. ENDTRY.
-
For API details, see ListObjectsV2 in Amazon SDK for SAP ABAP API reference.
-
- Swift
-
- SDK for Swift
-
Note
This is prerelease documentation for an SDK in preview release. It is subject to change.
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Amazon Code Examples Repository
. public func listBucketFiles(bucket: String) async throws -> [String] { let input = ListObjectsV2Input( bucket: bucket ) let output = try await client.listObjectsV2(input: input) var names: [String] = [] guard let objList = output.contents else { return [] } for obj in objList { if let objName = obj.key { names.append(objName) } } return names }
-
For API details, see ListObjectsV2
in Amazon SDK for Swift API reference.
-