Hive 2.3.9 - 2110 (EEP 8.0.0) Release Notes
Hive Version | 2.3.9 |
Release Date | October 2021 |
HPE Version Interoperability | See Hive and HCatalog Support Matrix and Ecosystem Support Matrix and EEP Components and OS Support. |
Source on GitHub | https://github.com/mapr/hive |
GitHub Release Tag | 2.3.9-eep-2110 |
Maven Artifacts | https://repository.mapr.com/maven/ |
Package Names | Navigate to http://package.ezmeral.hpe.com/releases/MEP/, and select your EEP(MEP) and OS to view the list of package names. |
ODBC/JDBC Drivers |
Hive 2.3.9 works with the following HPE Hive drivers:
For additional driver information, see Connecting to HiveServer2. |
Feature support
The following list describes support of various components and functionality with Hive 2.3.9 - 2110:
-
Supports Hive-2.3.9 on Tez-0.9.2 For more information, see Tez 0.9.2 - 2110 (EEP 8.0.0) Release Notes.
-
Does not support Hive on Spark. You cannot use Spark as a query engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
-
Does not support HDFS encryption in Hive tables.
-
Does not support LLAP with Hive-2.3.9 because Apache Slider is not an HPE supported ecosystem component.
-
Starting from Hive 2.1, Hive must run the
schematool
command as an initialization step.
Changes in default security configuration
- None.
New in This Release
Hive 2.3.9 - 2110 introduces the following enhancements or HPE platform-specific behavior changes:
- Improved
Describe table
operator in terms of fetching statistics of partitions. Starting from EEP 8.0.0, you can fetch the partition information using thedescribe
command withformatted
orextended
statements.Configure thehive.describe.partitionedtable.ignore.stats
property to change the behaviour of fetching statistics of partitions. It is set to the default value offalse
.<property> <name>hive.describe.partitionedtable.ignore.stats</name> <value>false</value> <description>Enables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description> </property>
<property> <name>hive.describe.partitionedtable.ignore.stats</name> <value>true</value> <description>Disables partitioned table stats collection for 'DESCRIBE FORMATTED' or 'DESCRIBE EXTENDED' commands</description> </property
- Hive supports symbolic links on file system. See Hive Features in HPE Ezmeral Data Fabric.
Fixes
GitHub Commit Number | Data (YYYY-MM-DD) | HPE Fix Number and Description |
2549e5350a | 2021-09-17 | MAPR-HIVE-994: Non mapr user unable to read SSL configuration from XML files on Core 7.0 |
e28a1edd63 | 2021-09-14 | MAPR-HIVE-1007: Permission denied to hbase temp files while running hcat jobs from other user |
c2864e3d06 | 2021-09-08 | MAPR-HIVE-999 : Make mapr-db jar with provided scope |
705bab5a12 | 2021-09-06 | MAPR-HIVE-998 : Update htrace version to 4.2.0-eep-incubating |
46862cf932 | 2021-09-03 | MAPR-HIVE-990 : mapr-security-web jar should be taken from the cluster |
c4aed1b675 | 2021-09-03 | MAPR-HIVE-995 : Upadate pig vesion to 0.17.0.0-eep-SNAPSHOT |
7e012fa650 | 2021-09-03 | MAPR-HIVE-993 : Update hbase version to 1.4.13.0-eep-SNAPSHOT |
1a886bbe78 | 2021-09-03 | MAPR-HIVE-992 : Update tez version to 0.9.2.0-eep-SNAPSHOT |
1439299a5c | 2021-09-03 | MAPR-HIVE-991 : Update hadoop version to 2.7.6.0-eep-800-SNAPSHOT |
09724c6192 | 2021-09-03 | MAPR-HIVE-987 : Update the maven artifact version strings to eep |
6dbd5a3ac8 | 2021-08-25 | MAPR-HIVE-981: [symlink functionality] Implement LOAD DATA INPATH functionality from symlinks with relative path |
29c2c6f3ab | 2021-08-18 | MAPR-HIVE-979: [symlink functionality] cannot insert in the external table based on symlinked directory |
1073cecb48 | 2021-08-17 | MAPR-HIVE-977 : Downgrade jackson to v2.11.1 or to 2.11.3 to be consistent with core version |
13ec2228e5 | 2021-08-17 | MAPR-HIVE-976 : Update tez version from 0.9.2-mapr-SNAPSHOT to 0.9.2.0-mapr-SNAPSHOT for development artifacts |
bfc754f1ca | 2021-08-11 | MAPR-HIVE-973 : FAILURE! - in org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2 |
7add0d42af | 2021-08-09 | MAPR-HIVE-972 : Replace Apache htrace-4.2.0-incubating with 4.2.0-mapr-incubating dependency |
ad1eeb8b20 | 2021-08-09 | MAPR-HIVE-971 : Exclude htrace-3.1 |
bcddd3cc00 | 2021-08-09 | MAPR-HIVE-969: Add possibility to run MR jobs against source files that are symlinks to original data |
74302e7f0d | 2021-08-09 | MAPR-HIVE-970 : Update hadoop version to 2.7.6.0-mapr-720-SNAPSHOT |
f6953ff0a8 | 2021-08-05 | MAPR-HIVE-968: Add possibility to run TEZ jobs against source files that are symlinks to original data |
8186a55978 | 2021-07-29 | MAPR-HIVE-880: Add possibility to distinguish file/dir links during Hive DML/DDL operations |
10779d1b56 | 2021-07-29 | MAPR-HIVE-960 : CVE-2012-5783 vulnerability in commons-httpclient |
0be9010773 | 2021-07-29 | MAPR-HIVE-959 : Update derbyclient and derbynet to most feasible version |
d0825bf854 | 2021-07-29 | MAPR-HIVE-963 : CVE-2020-13956,WS-2017-3734 vulnerabilities in httpclient |
7982b4be51 | 2021-07-29 | MAPR-HIVE-962 : WS-2019-0379: commons-codec vulnerability |
d70a411074 | 2021-07-29 | MAPR-HIVE-953 : FAILURE! - in org.apache.hive.hcatalog.templeton.TestCustomHeadersE2e |
88056bb162 | 2021-07-29 | MAPR-HIVE-952 : FAILURE! - in org.apache.hadoop.hive.ql.io.parquet.TestVectorizedColumnReader |
054a0b73b2 | 2021-07-29 | MAPR-HIVE-896 : CVE-2020-17521 vulnerability in Groovy |
5b44866042 | 2021-07-29 | MAPR-HIVE-927 : NPE thrown from XmlUtil by Hive Client |
2e446568b2 | 2021-07-29 | MAPR-HIVE-858 : WARNING: Illegal reflective access org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer |
b3520d3c85 | 2021-07-29 | MAPR-HIVE-950 : HiveVersionInfo.getShortVersion returns wrong version |
9b4c3ad944 | 2021-06-08 | MAPR-HIVE-949 : Make SchemaEvolution class behavior the same as in Orc-1.5.12 |
9b75385d22 | 2021-06-06 | MAPR-HIVE-894 : CVE-2020-13955 vulnerability in Calcite |
2a7526fdea | 2021-05-25 | MAPR-HIVE-947 : org.apache.hadoop.hive.ql.exec.tez.TezTask at creating session |
a64ce8195c | 2021-05-23 | MAPR-HIVE-945 : FAILURE! - in org.apache.hadoop.hive.maprdb.json.MapRDbJsonFetchByIdOptimizerPositiveTest |
4215a9ab82 | 2021-05-23 | MAPR-HIVE-944 : FAILURE! - in org.apache.hadoop.hive.hbase.TestHBaseSerDe |
30655745a5 | 2021-05-23 | MAPR-HIVE-943 : Fix org.apache.hadoop.hive.cli.TestCliDriverMethods |
bfda9cc9a8 | 2021-05-23 | MAPR-HIVE-942 : FAILURE! - in org.apache.hadoop.hive.accumulo.predicate.TestAccumuloPredicateHandler |
GitHub Commit Number | Data (YYYY-MM-DD) | HPE Fix Number and Description |
3a7b6db040 | 2021-09-15 | HIVE-24965: Describe table partition stats fetch should be configurable |
1d1bb4c2fd | 2021-09-15 | HIVE-22453: Describe table unnecessarily fetches partitions |
96b37ab69c | 2021-09-14 | HIVE-23756 : Added more constraints to the package.jdo file |
40cdec9dd2 | 2021-09-12 | HIVE-24177 : hive mapjoin throws udf class not found |
606c3c240b | 2021-09-08 | HIVE-17659 : get_token thrift call fails for DBTokenStore in remote HMS mode (Vihang Karajgaonkar, reviewed by Aihua Xu) |
a2416a115c | 2021-09-06 | HIVE-25054: Upgrade `jodd-core` dependency to get rid of CVE-2018-21234 (Abhay Chennagiri, reviewed by Jesus Camacho Rodriguez) |
e593c8cb4e | 2021-08-18 | HIVE-17824 : msck repair table should drop the missing partitions from metastore (Janaki Lahorani, reviewed by Peter Vary, Alexander Kolbasov and Vihang Karajgaonkar) |
3591ea65fa | 2021-08-18 | HIVE-16143: Improve msck repair batching (Vihang Karajgaonkar, reviewed by Sahil Takiar & Aihua Xu) |
fab9a7603a | 2021-07-29 | HIVE-19228: Remove commons-httpclient 3.x usage (Janaki Lahorani reviewed by Aihua Xu) |
0ffeae33b1 | 2021-06-13 | HIVE-21200: Vectorization: date column throwing java.lang.UnsupportedOperationException for parquet (#2276) |
c6300400bd | 2021-06-13 | HIVE-24608: Switch back to get_table in HMS client for Hive 2.3.x (#2080) |
0518323174 | 2021-06-13 | HIVE-18147 : Tests can fail with java.net.BindException: Address already in use (Janaki Lahorani, reviewed by Andrew Sherman and Vihang Karajgaonkar) |
d6766f34fb | 2021-06-13 | HIVE-21563 : Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce |
a3477edb7f | 2021-06-13 | HIVE-24797: Disable validate default values when parsing Avro schemas (#1994) |
1fc7585a2e | 2021-06-08 | ORC-437: Make acid schema checks case insensitive |
9120da5c4f | 2021-05-31 | HIVE-21075 : Metastore: Drop partition performance downgrade with Postgres DB |
39d42ddf12 | 2021-05-31 | HIVE-9447: Metastore: inefficient Oracle query for removing unused column descriptors when add/drop table/partition (Selina Zhang reviewed by Ashutosh Chauhan, Adam Szita) |
1ccb218119 | 2021-05-22 | HIVE-21085: Materialized views registry starts non-external tez session (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
dea7190511 | 2021-05-22 | HIVE-19691: Start SessionState in materialized views registry (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
09b4ca437f | 2021-05-22 | HIVE-17853: RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout (Chris Drome, reviewed by Mithun Radhakrishnan) |
b8902a7bb8 | 2021-05-22 | HIVE-23534: NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message (Stamatis Zampetakis, reviewed by Jesus Camacho Rodriguez) |
11db00d681 | 2021-05-22 | HIVE-18494: Regression: from HIVE-18069, the metastore directsql is getting disabled (Jesus Camacho Rodriguez, reviewed by Gopal V) |
1920988b66 | 2021-05-22 | HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu) (addendum) |
e5ed2cb9ed | 2021-05-22 | HIVE-18069: MetaStoreDirectSql to get tables has misplaced comma (Jesus Camacho Rodriguez, reviewed by Aihua Xu) |
9b506546a4 | 2021-05-22 | HIVE-15436: Enhancing metastore APIs to retrieve only materialized views (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan) |
051002d23a | 2021-05-22 | HIVE-6990 : Direct SQL fails when the explicit schema setting is different from the default on (Bing Li, Sergey Shelukhin via Ashutosh Chauhan) |
Known Issues and Limitations
-
HIVE-19502 Unable to insert values into table stored by JdbcStorageHandler
-
HIVE-19286 NPE in MERGE operator on MR mode
-
HIVE-760 [Hive-2.3] Could not start hive-metastore on Centos 8 MetaException(message:Version information not found in metastore)
Starting in MEP 7.0.0, use the MySQL driver with MariaDB.<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property>
Some SELECT queries can be converted to a single FETCH task minimizing latency. Currently, the query should be single sourced and should not have a subquery or any aggregations or distincts (which incurs RS), lateral views and joins:none : disable hive.fetch.task.conversion minimal : SELECT star, filter on partition columns, LIMIT only more : SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
Resolved Issues
- MAPR-TEZ-172 fixes a HIVE-789 known issue from Hive 2.3.8 - 2104 in this release.