Hive 2.3.9.0 - 2201 (EEP 8.1.0) Release Notes
Hive Version | 2.3.9.0 |
Release Date | January 2022 |
HPE Version Interoperability | See Hive and HCatalog Support Matrix and Ecosystem Support Matrix and EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/hive |
GitHub Release Tag | 2.3.9.0-eep-810 |
Maven Artifacts | https://repository.mapr.com/maven/ |
Package Names | Navigate to http://package.ezmeral.hpe.com/releases/MEP/, and select your EEP(MEP) and OS to view the list of package names. |
ODBC/JDBC Drivers |
Hive 2.3.9 works with the following HPE Hive drivers:
For additional driver information, see Connecting to HiveServer2. |
Feature support
The following list describes support of various components and functionality with Hive 2.3.9 - 2201:
-
Supports Hive-2.3.9 on Tez-0.9.2 For more information, see Tez 0.9.2 - 2201 (EEP 8.1.0) Release Notes.
-
Does not support Hive on Spark. You cannot use Spark as a query engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
-
Does not support HDFS encryption in Hive tables.
-
Does not support LLAP with Hive-2.3.9 because Apache Slider is not an HPE supported ecosystem component.
-
Starting from Hive 2.1, Hive must run the
schematool
command as an initialization step.
Changes in default security configuration
-
Starting from EEP-8.1.0, Hive supports SCRAM token and SCRAM-SHA-256 authentication in HPE Ezmeral Data Fabric.
Table 1. Description of hive.delegation.token.authentication
# Property Data Type Default value Description 1 hive.delegation.token.authentication String DIGEST Delegation token authentication method. Possible values are DIGEST, SCRAM To configure SCRAM token and SCRAM-SHA-256 authentication, set the following property on HIVE_HOME/conf/hive-site.xml file:<property> <name>hive.delegation.token.authentication</name> <value>SCRAM</value> </property>
Execute
MAPR_HOME/server/configure.sh -R
script on a newly installed and MapR-SASL or KERBEROS secured cluster to automatically configure the following authentications:- For a FIPS enabled cluster, Hive configures
hive.delegation.token.authentication=SCRAM
authentication. - For a non-FIPS cluster if you configure Hadoop with
hadoop.security.token.authentication.method=SCRAM
authentication, Hive configures the SCRAM authentication. - For other clusters, Hive configures
hive.delegation.token.authentication=DIGEST
authentication.
For non-secure clusters, Hive configures
hive.delegation.token.authentication=DIGEST
authentication.When you upgrade Hive, the upgrade does not update the value of the set
hive.delegation.token.authentication
property.Manually set the value of
hive.delegation.token.authentication
property when you change the cluster settings from FIPS to non-FIPS or from non-FIPS to FIPS. - For a FIPS enabled cluster, Hive configures
New in This Release
Hive 2.3.9.0 - 2201 introduces the following enhancements or HPE platform-specific behavior changes:
- Starting from EEP-8.1.0, Hive supports FIPS and SCRAM SASL.
- Beginning with EEP 8.1.0, JAR artifacts for
Hive use four digits instead of three digits. For
example:
If your application includes a Hive dependency in thehive-service-rpc-2.3.9.0-mapr-SNAPSHOT.jar hive-llap-ext-client-2.3.9.0-mapr-SNAPSHOT.jar hive-exec-2.3.9.0-mapr-SNAPSHOT.jar hive-beeline-2.3.9.0-mapr-SNAPSHOT.jar
pom.xml
file, you must update the JAR artifact before using Hive 2.3.9 - 2201.The Hive package name on package.ezmeral.hpe.com continues to use two digits, and the Hive root folder continues to use three digits.
- Improved
Describe table
operator in terms of fetching statistics of partitions. Starting from EEP 8.0.0, you can fetch the partition information using thedescribe
command withformatted
orextended
statements.Configure thehive.describe.partitionedtable.ignore.stats
property to change the behaviour of fetching statistics of partitions. It is set to the default value offalse
.<property> <name>hive.describe.partitionedtable.ignore.stats</name> <value>false</value> <description>Enables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description> </property>
<property> <name>hive.describe.partitionedtable.ignore.stats</name> <value>true</value> <description>Disables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description> </property
- Hive supports symbolic links on file system. See Hive Features in HPE Ezmeral Data Fabric.
Fixes
GitHub Commit Number | Data (YYYY-MM-DD) | HPE Fix Number and Description |
a6d17888e2 | 2022-01-31 | EEP-HIVE-1153 : FAILURE! - in org.apache.hive.conftool.SslDefaultTest |
5d4dcca0d9 | 2022-01-31 | EEP-HIVE-1151 : Build Hive ECO EEP 8.1.0 components with DF v 6.2.0 |
be5e7ce3a7 | 2022-01-27 | EEP-HIVE-1148 : Cannot run HiveMetastore service on non-fips cluster for EEP-8.1.0 |
bc913e56b1 | 2022-01-27 | EEP-HIVE-1147 : JDBC connection failed if working with different connections on FIPS enabled clusters |
9dc2d25f98 | 2022-01-24 | EEP-HIVE-1141 : Hive services cannot communicate between FIPS / non-FIPS nodes |
e527acd2dd | 2022-01-18 | EEP-HIVE-1135: HiveServer2 will fail if default provider will not support FIPS |
3498c725a6 | 2022-01-06 | EEP-HIVE-1065: CVE-2021-37136, CVE-2021-37137, WS-2020-0408, CVE-2021-21290: netty-*-4.1.55.Final.jar |
e82cca2f60 | 2022-01-05 | EEP-HIVE-1062: CVE-2016-5007, CVE-2016-9878 ,CVE-2018-1271, CVE-2018-1272, CVE-2020-5421: spring-*-3.2.16.RELEASE.jar |
e2ae1f021e | 2022-01-05 | EEP-HIVE-1068: CVE-2020-9480: spark-network-common_2.11-2.3.0.jar, CVE-2018-17190: spark-core_2.11-2.3.0.jar |
110f14340f | 2022-01-04 | EEP-HIVE-1117: Update log4j v2 to the latest available (to 2.17+) |
5cd9fcc2d9 | 2022-01-04 | EEP-HIVE-1116: Hive returns an incorrect number of columns |
ae3e3d49e8 | 2022-01-04 | Revert "MAPR-HIVE-930: Cannot run join with Order by and Limit clause specified at the same time" |
8b01f203da | 2021-12-30 | EEP-HIVE-1119: com.fasterxml.jackson.annotation.JsonFormat.empty()Lcom/fasterxml/jackson/annotation/JsonFormat |
250a524cbe | 2021-12-27 | EEP-HIVE-1099 : [FIPS] HS2 connection PAM + SSL doesn't work with SBD configuration when FIPS enabled |
da9feb7041 | 2021-12-24 | EEP-HIVE-1064: CVE-2021-30639 ; CVE-2021-33037: tomcat-coyote-10.0.4.jar |
3288fbbea0 | 2021-12-24 | EEP-HIVE-1059: CVE-2019-10172, CVE-2019-10202: jackson-mapper-asl-1.9.13.jar, jackson-mapper-asl-1.9.2.jar |
70be9aeb2c | 2021-12-24 | EEP-HIVE-1056: CVE-2021-35515, CVE-2021-35516, CVE-2021-35517, CVE-2021-36090: commons-compress-1.20.jar |
2a7a346e3e | 2021-12-24 | EEP-HIVE-1055: CVE fixes of bcprov-jdk15on-1.52.jar |
2e9596dc8a | 2021-12-24 | EEP-HIVE-1054: WS-2021-0419: gson-2.2.4.jar |
63a06a47d7 | 2021-12-22 | EEP-HIVE-1115 : Unrecognized VM option UseGCLogFileRotation |
3dd5c4f710 | 2021-12-21 | EEP-HIVE-1088: [Hive-2.3.9] Hive on Tez engine + native S3/OPAL Unable to load AWS credentials from any provider in the chain |
6e06d411a3 | 2021-12-21 | EEP-HIVE-1097: CVE-2021-44228 - Log4j vulnerability |
facbb602ee | 2021-12-21 | EEP-HIVE-1091 : Add SCRAM-SASL to Hive |
158d663d48 | 2021-12-07 | EEP-HIVE-1087: CAST gives NULL values during insert when vectorization enabled. |
1e74e4cccb | 2021-12-02 | MAPR-HIVE-1090 : TLSv1.2 SSLContext not available |
904758aef6 | 2021-11-25 | MAPR-HIVE-1086 : Upgrade Jetty to 9.4.44.v20210927 |
04c60792f4 | 2021-11-24 | MAPR-HIVE-1074 : Beeline fails to connect to HiveServer2 with java.lang.NoSuchFieldError: BCFKS error on Core 6.2.0/EEP 8.1.0 |
2851a75533 | 2021-11-22 | MAPR-HIVE-1071 : Fix SslDefaultTest |
c6e21ea94d | 2021-11-22 | MAPR-HIVE-1069 : HIVE-1069 Update hbase version to 1.4.13.0-eep-810-SNAPSHOT |
62c6ca2825 | 2021-11-18 | MAPR-HIVE-1053 : Relative path in absolute URI: slider reads hdfs-site.xml from hadoop-hdfs.jar |
35f8178503 | 2021-11-18 | MAPR-HIVE-1036 : java.lang.NoClassDefFoundError: org/apache/commons/digester/Digester |
e7b42cb8b1 | 2021-11-16 | MAPR-HIVE-1026 : Migrate to python 3 in LLAP package.py file |
b925b5c99d | 2021-11-16 | MAPR-HIVE-1025 : LLAP server expects tez.tar.gz archive in MapR FS |
7fa0949cde | 2021-11-16 | MAPR-HIVE-1033 : Add MapR slider dependency to Hive |
b7cf317ce4 | 2021-11-11 | MAPR-HIVE-1031 : logError: command not found if any error happens during configuring Hive |
c2c8453683 | 2021-11-11 | MAPR-HIVE-1024 : Replace deprecated AuthMethod.DIGEST with AuthMethod.TOKEN in HadoopThriftAuthBridge25Sasl |
806f29ca93 | 2021-11-10 | DFDEVOPS-2081 : Configure Jenkins job for hive-2.3.9 mep-8.1.0 |
4364d7d1c4 | 2021-11-10 | MAPR-HIVE-1022 : Update Hive version to 2.3.9.0-eep-810-SNAPSHOT |
3212a89673 | 2021-11-09 | MAPR-HIVE-1021 : Upgrade mapr-core version to 7.0.0.0-mapr-SNAPSHOT |
d300f64a22 | 2021-11-04 | MAPR-HIVE-831 : Move Hive to 4 digits in jar artefacts |
bca47dbbea | 2021-11-02 | MAPR-HIVE-975: Customer request to investigate temporary hive session files cleanup improvements |
304849fc9a | 2021-11-01 | MAPR-HIVE-1018 : Update hadoop version to 2.7.6.200-eep-810-SNAPSHOT |
7c8fe3cc80 | 2021-11-01 | MAPR-HIVE-1012 : Hiveserver2 could not start on cluster with enabled FIPS |
1ccc9d7b5a | 2021-10-13 | MAPR-HIVE-1002 : Hive-2.3 does not remove old compressed logs |
c26ff7e273 | 2021-10-11 | MAPR-HIVE-1016 : Update Conjars repository URL to secure |
4bf0c7aadc | 2021-10-11 | MAPR-HIVE-1015 : Configure repositories for Jenkins job |
69d55d6809 | 2021-10-11 | MAPR-HIVE-997 : ConfigureShInsecureTest hangs up |
92a25a685c | 2021-10-11 | MAPR-HIVE-1013 : Update calcite version to 1.10.0-eep |
16c0ba4efd | 2021-10-11 | MAPR-HIVE-1014 : Could not transfer artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde |
GitHub Commit Number | Data (YYYY-MM-DD) | HPE Fix Number and Description |
95ae2bebc9 | 2021-12-21 | HIVE-17774: compaction may start with 0 splits and fail |
ed4b16c0fa | 2021-10-08 | HIVE-16820 : TezTask may not shut down correctly before submit (Sergey Shelukhin, reviewed by Siddharth Seth) |
794c971152 | 2021-10-28 | HIVE-20072 : Write access being requested when performing select on a table |
Known Issues and Limitations
- HIVE-1089: Hive on MapReduce engine does not support data
insertion into Versioned buckets.You must use Unversioned buckets on MaprReduce engine
in S3 file system.
Hewlett Packard Enterprise recommends using Hive on Tez engine for full S3 file system support.
-
HIVE-19502: Unable to insert values into table stored by JdbcStorageHandler
-
HIVE-19286: NPE in MERGE operator on MR mode
-
HIVE-760: [Hive-2.3] Could not start hive-metastore on Centos 8 MetaException(message:Version information not found in metastore)
Starting in EEP 7.0.0, use the MySQL driver with MariaDB.<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property>
Some SELECT queries can be converted to a single FETCH task minimizing latency. Currently, the query should be single sourced and should not have a subquery or any aggregations or distincts (which incurs RS), lateral views and joins:none : disable hive.fetch.task.conversion minimal : SELECT star, filter on partition columns, LIMIT only more : SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
Resolved Issues
- MAPR-TEZ-172 fixes a HIVE-789 known issue from Hive 2.3.8 - 2104 in this release.