Amazon EMR release 4.4.0

• Application versions in Amazon EMR 7.x releases

• Application versions in Amazon EMR 6.x releases

• Application versions in Amazon EMR 5.x releases

• Application versions in Amazon EMR 4.x releases

Features

• Added HCatalog 1.0.0

• Added Sqoop-Sandbox 1.4.6

• Upgraded to Presto 0.136

• Upgraded to Zeppelin 0.5.6

• Upgraded to Mahout 0.11.1

• Enabled dynamicResourceAllocation by default.

• Added a table of all configuration classifications for the release. For more information, see the Configuration Classifications table in Configuring Applications.

Known issues resolved from previous releases

• Fixed an issue where the maximizeResourceAllocation setting would not reserve enough memory for YARN ApplicationMaster daemons.

• Fixed an issue encountered with a custom DNS. If any entries in resolve.conf precede the custom entries provided, then the custom entries are not resolvable. This behavior was affected by clusters in a VPC where the default VPC name server is inserted as the top entry in resolve.conf.

• Fixed an issue where the default Python moved to version 2.7 and boto was not installed for that version.

• Fixed an issue where YARN containers and Spark applications would generate a unique Ganglia round robin database (rrd) file, which resulted in the first disk attached to the instance filling up. Because of this fix, YARN container level metrics have been disabled and Spark application level metrics have been disabled.

• Fixed an issue in log pusher where it would delete all empty log folders. The effect was that the Hive CLI was not able to log because log pusher was removing the empty user folder under /var/log/hive.

• Fixed an issue affecting Hive imports, which affected partitioning and resulted in an error during import.

• Fixed an issue where EMRFS and s3-dist-cp did not properly handle bucket names that contain periods.

• Changed a behavior in EMRFS so that in versioning-enabled buckets the _$folder$ marker file is not continuously created, which may contribute to improved performance for versioning-enabled buckets.

• Changed the behavior in EMRFS such that it does not use instruction files except for cases where client-side encryption is enabled. If you want to delete instruction files while using client-side encryption, you can set the emrfs-site.xml property, fs.s3.cse.cryptoStorageMode.deleteInstructionFiles.enabled, to true.

• Changed YARN log aggregation to retain logs at the aggregation destination for two days. The default destination is your cluster HDFS storage. If you want to change this duration, change the value of yarn.log-aggregation.retain-seconds using the yarn-site configuration classification when you create your cluster. As always, you can save your application logs to Amazon S3 using the log-uri parameter when you create your cluster.

Patches Applied

• HIVE-9655

• HIVE-9183

• HADOOP-12810