Amazon EMR 6.7.0 - Hive release notes - Amazon EMR
Hive changesKnown issues
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Amazon EMR 6.7.0 - Hive release notes

Amazon EMR 6.7.0 - Hive changes

Type Description
Feature Amazon EMR Hive integration with LakeFormation.
Feature Additional audit logging for Hive EMRFS Amazon S3 optimized committer. Hive config: hive.blobstore.output-committer.logging, default: false
Feature Deleted target directory on insert overwrite with empty select result to an unpartitioned table/static partition to behave similarly to Hive 2.x. Hive config: hive.emr.iow.clean.target.dir, default: false
Bug Fixed intermittent query failure when using Hive EMRFS Amazon S3 optimized committer with partition bucket sorting.
Upgrade Hive version upgraded to 3.1.3. Refer to Apache Hive 3.1.3 release notes for more details.
Upgrade Upgraded Parquet to 1.12.2.
Backport HIVE-20065: Metastore should not rely on jackson 1.x
Backport HIVE-20071: Migrate to jackson 2.x and prevent usage
Backport HIVE-20607: TxnHandler should use PreparedStatement to execute direct SQL queries
Backport HIVE-20740: Remove global lock in ObjectStore.setConf method
Backport HIVE-20961: Retire NVL implementation
Backport HIVE-22059: hive-exec jar doesn't contain (fasterxml) jackson library
Backport HIVE-22351: Fix incorrect threaded ObjectStore usage in TestObjectStore
Backport HIVE-23534: NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message
Backport HIVE-24048: Harmonise Jackson components to version 2.10.latest - Hive
Backport HIVE-24768: Use jackson-bom everywhere for version replacement
Backport HIVE-24816: Upgrade jackson to 2.10.5.1 or 2.11.0+ due to CVE-2020-25649
Backport HIVE-25971: Tez task shutdown getting delayed due to cached thread pool not closed
Backport HIVE-26036: NPE caused by getMTable() in ObjectStore

Amazon EMR 6.7.0 - Hive known issues

  • Queries with windowing functions on the same column as join may lead to invalid transformations as reported in HIVE-25278 and cause incorrect results or query failures. A workaround would be to disable CBO at the query level for such queries. The fix will be available in an Amazon EMR release following 6.7.0. For more information, contact Amazon support.

  • With Amazon EMR 6.6.0 through 6.9.x, INSERT queries with dynamic partition and an ORDER BY or SORT BY clause will always have two reducers. This issue is caused by OSS change HIVE-20703, which puts dynamic sort partition optimization under cost-based decision. If your workload doesn't require sorting of dynamic partitions, we recommend that you set the hive.optimize.sort.dynamic.partition.threshold property to -1 to disable the new feature and get the correctly calculated number of reducers. This issue is fixed in OSS Hive as part of HIVE-22269 and is fixed in Amazon EMR 6.10.0.