Amazon EMR 6.9.0 - Hive release notes - Amazon EMR
Hive changes Known issues
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon EMR 6.9.0 - Hive release notes

Amazon EMR 6.9.0 - Hive changes

Type Description
Upgrade Upgrade Jetty to 9.4.48.v20220622
Upgrade Support for Hadoop 3.3.3
Feature Amazon EMR Hive integration with Lake Formation for interactive workloads using GCSC API.
Feature Amazon EMR Hive integration with Iceberg.
Improvement Enable SSL in HiveServer2 when in-transit encryption is enabled using Amazon EMR security configurations.
Improvement Enable Hive EMRFS Amazon S3 optimized committer by default. For more information see, Enabling Hive EMRFS S3 optimized committer.
Improvement Add HiveHBaseTableInputFormatV2 that inherits only mapred version of InputFormat to fix SPARK-34210. Set hive.hbase.inputformat.v2 to true to use it.
Improvement Wait for TezAM to launch in background with hive.cli.tez.session.async instead of terminating it and launching new immediately. Use hive.emr.cli.tez.session.open.timeout to set this timeout in seconds.
Improvement Add option hive.conf.restricted.list.append to append comma-separated configs to existing restricted config list hive.conf.restricted.list.
Improvement Clearer error message when Hive query fails because location is not defined for database.
Backport HIVE-24484: Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
Backport HIVE-22398: Remove YARN queue management via ShimLoader.
Backport HIVE-23190: LLAP: modify IndexCache to pass filesystem object to TezSpillRecord.
Backport HIVE-22185: HADOOP-15832 will cause problems with tests using MiniYarn clusters.
Backport HIVE-21670: Replacing mockito-all with mockito-core dependency.
Backport HIVE-24542: Prepare Guava for Upgrades.
Backport HIVE-23751: QTest: Override #mkdirs() method in ProxyFileSystem to align after HADOOP-16582.
Backport HIVE-21603: Java 11 preparation: update powermock version.
Backport HIVE-24083: hcatalog error in Hadoop 3.3.0: authentication type needed.
Backport HIVE-24282: Show columns shouldn't sort output columns unless explicitly mentioned.
Backport HIVE-20656: Sensible defaults: Map aggregation memory configs are too aggressive.
Backport HIVE-25443: Arrow SerDe cannot serialize/deserialize complex data types when there are more than 1024 values
Backport HIVE-19792: Upgrade orc to 1.5.2 and enable decimal_64 schema evolution tests.
Backport HIVE-20437: Handle schema evolution from float, double, and decimal.
Backport HIVE-21987: Hive is unable to read Parquet int32 annotated with decimal.
Backport HIVE-20038: Update queries on non-bucketed and partitioned tables throws NPE.

Amazon EMR 6.9.0 - Hive known issues

  • With Amazon EMR 6.6.0 through 6.9.x, INSERT queries with dynamic partition and an ORDER BY or SORT BY clause will always have two reducers. This issue is caused by OSS change HIVE-20703, which puts dynamic sort partition optimization under cost-based decision. If your workload doesn't require sorting of dynamic partitions, we recommend that you set the hive.optimize.sort.dynamic.partition.threshold property to -1 to disable the new feature and get the correctly calculated number of reducers. This issue is fixed in OSS Hive as part of HIVE-22269 and is fixed in Amazon EMR 6.10.0.