Querying metadata tables - Amazon Simple Storage Service
Query pricing considerations
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Querying metadata tables

Your Amazon S3 Metadata tables are stored in an Amazon managed S3 table bucket, which provides storage that's optimized for tabular data. To query your metadata, you can integrate your table bucket with Amazon SageMaker Lakehouse. This integration, which uses the Amazon Glue Data Catalog and Amazon Lake Formation, allows Amazon analytics services to automatically discover and access your table data.

After your table bucket is integrated with the Amazon Glue Data Catalog, you can directly query your metadata tables with Amazon analytics services such as Amazon Athena, Amazon EMR, and Amazon Redshift. You can also create interactive dashboards with your query data by using Amazon QuickSight.

For more information about integrating your Amazon managed S3 table bucket with Amazon SageMaker Lakehouse, see Using Amazon S3 Tables with Amazon analytics services.

You can also query your metadata tables with Apache Spark, Apache Trino, and any other application that supports the Apache Iceberg format by using the Amazon Glue Iceberg REST endpoint, Amazon S3 Tables Iceberg REST endpoint, or the Amazon S3 Tables Catalog for Apache Iceberg client catalog. For more information about accessing your metadata tables, see Accessing table data.

You can analyze your metadata tables with any query engine that supports the Apache Iceberg format. For example, you can query your metadata tables to do the following:

  • Discover storage usage patterns and trends

  • Audit Amazon Key Management Service (Amazon KMS) encryption key usage across your objects

  • Search for objects by user-defined metadata and object tags

  • Understand object metadata changes over time

  • Learn when objects are updated or deleted, including the Amazon Web Services account ID or IP address that made the request

You can also join S3 managed metadata tables and custom metadata tables, allowing you to query across multiple datasets.

Query pricing considerations

Additional pricing applies for running queries on your metadata tables. For more information, see pricing information for the query engine that you're using.

For information on making your queries more cost effective, see Optimizing metadata table query performance.

Topics