Considerations for using Iceberg with Spark - Amazon EMR
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Considerations for using Iceberg with Spark

  • Amazon EMR 6.5.0 does not support Iceberg running on Amazon EMR on EKS by default. An Amazon EMR 6.5.0 custom image is available so that you can pass --jars local:///usr/share/aws/iceberg/lib/iceberg-spark3-runtime.jar as a spark-submit parameter to create Iceberg tables on Amazon EMR on EKS. For more information, see Submit a Spark workload in Amazon EMR using a custom image in the Amazon EMR on EKS Development Guide. You can also contact Amazon Web Services Support for assistance. Starting with Amazon EMR 6.6.0, Iceberg is supported on Amazon EMR on EKS.

  • When using Amazon Glue as a catalog for Iceberg, make sure the database in which you are creating a table exists in Amazon Glue. If you are using services such as Amazon Lake Formation and you're unable to load the catalog, make sure you have proper access to the service to execute the command.