Drill 1.16.1.500-2305 (EEP 8.1.1) Release Notes
The notes below relate specifically to the HPE Ezmeral Data Fabric Distribution for Apache Drill. You may also be interested in the Apache Drill homepage and the Apache Drill release notes:
| Version | 1.16.1.500 |
| Release Date | May 2023 |
| HPE Version Interoperability | See Component Versions for Released EEPs and EEP Components and OS Support. |
| Package Names | See Package Names for Ecosystem Packs (EEPs). |
New in This Release
Drill 1.16.1.500-2305 introduces the following enhancements or HPE platform-specific behavior changes:
- Separate storage in HBase for query profiles and storage plugin information. See Configuring HBase Persistent Storage Tables.
Fixes
This HPE release includes the following fixes on the base release:
| Commit | Date (YYYY-MM-DD) | Comment |
| 38c9c314a9 | 2023-05-05 | MD-6438: Update version of shaded guava to 31.1-jre |
| df338e7671 | 2023-05-03 | MD-6439: Update jackson version to 2.12.7 and jackson databind 2.12.7.1 |
| 2572e8a8e2 | 2023-05-03 | MD-6437: Update protobuf-java to 3.21.12 |
| 33da89f90f | 2023-04-28 | MD-6426: Fix endless retrying zk set data for a large query |
| 2cfc983fbb | 2023-04-20 | MD-6435: Update jetty to 9.4.51.v20230217 |
| c7bac76d3f | 2023-04-19 | MD-6429: remove conjars repo |
| 8a69b8c69b | 2023-03-10 | MD-6412: Update commons-text to 1.10.0 due to CVE |
| b1abff4981 | 2022-09-09 | MD-6371: Enable mapr maven profile by default |
| 2531a80da4 | 2022-09-08 | MD-6370: Exclude Xalan dependency |
| e20c4ca438 | 2022-05-04 | MD-6272: Update Drill to use Antlr4 version 4.9.3 |
| 53181b0213 | 2022-05-02 | MD-6312: Hide warning about an illegal reflective access operation while connecting via sqlline |
| 88bfe5171c | 2022-04-21 | DRILL-8122: Change kafka metadata obtaining due to KAFKA-5697 (#2456) |
| 7f7fe93ce3 | 2022-04-12 | MD-6304: add requested file from DRILL-7204 |
| 64b78f7f04 | 2022-04-12 | DRILL-7619: Fixed link to the metrics endpoint |
| 8933f37e56 | 2022-04-12 | DRILL-7582: Moved Drillbits REST API communication to the back end layer |
| 2b4db26be7 | 2022-04-11 | MD-6264: Cannot verify certificate when ssl enabled on DOY (#581) |
| c9eef515ca | 2022-04-11 | MD-6271: drill couldn't start when FIPS enabled on Rocky linux (#580) |
| 59c28b849f | 2022-04-07 | MD-6306: Remove spring transitive dependencies (#583) |
Known Issues
- During the Drill-on-YARN installation, the system fails to upload the Drill archive
because the
/user/drilldirectory does not exist. If you install and try to start Drill-on-YARN (version 1.16.1.400 in EEP-8.1.0 or 1.16.1.500 in EEP-8.1.1 on Core 6.2.0 or 7.0.0) using themapr-drill-yarnpackage, the system returns the following messages:/opt/mapr/drill/drill-1.16.1/bin/drill-on-yarn.sh start WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by javassist.util.proxy.SecurityActions (file:/opt/mapr/drill/drill-1.16.1/jars/3rdparty/javassist-3.24.0-GA.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) WARNING: Please consider reporting this to the maintainers of javassist.util.proxy.SecurityActions WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Connecting to DFS... Connected. 2023-06-20 02:28:59,1379 ERROR JniCommon fc/jni_MapRClient.cc:816 Thread: 1438881 Mismatch found for java and native libraries java build version 6.2.0.0.20200915234957.GA, native build version 6.2.0.0.20200909000740.GA java patch version $Id: mapr-version: 6.2.0.0.20200915234957.GA ccd6754df227770285, native patch version $Id: mapr-version: 6.2.0.0.20200909000740.GA a40a31acab7f5e88e1 Uploading /opt/mapr/drill/drill-1.16.1/drill.tar.gz to /user/drill/drill.tar.gz ... Failed. Failed to upload Drill archive Caused by: Failed to create DFS directory: /user/drill Caused by: Could not create FileClient err: 0 Caused by: Could not create FileClient err: 0To resolve this issue, complete the following steps:- Install Drill-on-YARN, as described in Installing Drill to Run Under YARN, but do not configure or start Drill-on-YARN.
- Create a file named
recreate_archive.shwith the following information:TIPYou can create and run this file in any location you choose.read -p "This operation will recreate drill.tar.gz in drill home directory. Continue? " -n 1 -r echo if [[ ! $REPLY =~ ^[Yy]$ ]] then exit 1 fi drillHome="/opt/mapr/drill/drill-$(cat /opt/mapr/drill/drillversion)" hadoopHome="/opt/mapr/hadoop/hadoop-$(cat /opt/mapr/hadoop/hadoopversion)" hbaseJar="$(ls /opt/mapr/lib/mapr-hbase-*-mapr.jar)" maprWebJar="$(ls /opt/mapr/lib/mapr-security*-mapr.jar)" maprdbJar="$(ls /opt/mapr/lib/maprdb-[0-9].[0-9].[0-9].[0-9]-mapr.jar)" mapredJar="$(ls /opt/mapr/lib/maprdb-mapreduce-*-mapr.jar)" maprfsJar="$(ls /opt/mapr/lib/maprfs-[0-9].[0-9].[0-9].[0-9]-mapr.jar)" jerseyClientJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-client-*.jar)" jerseyCoreJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-core-*.jar)" echo "Drop old mapr jars from ${drillHome}/jars/3rdparty/" rm -f ${drillHome}/jars/3rdparty/mapr-hbase* rm -f ${drillHome}/jars/3rdparty/mapr-security-web-* rm -f ${drillHome}/jars/3rdparty/maprdb-* rm -f ${drillHome}/jars/3rdparty/maprfs-* echo "Copy new jars from /opt/mapr/lib/" cp ${hbaseJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${hbaseJar} has been copied" cp ${maprWebJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprWebJar} has been copied" cp ${maprdbJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprdbJar} has been copied" cp ${mapredJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${mapredJar} has been copied" cp ${maprfsJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprfsJar} has been copied" echo "Copy jersey jars from hadoop for timeline client" cp ${jerseyClientJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${jerseyClientJar} has been copied" cp ${jerseyCoreJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${jerseyCoreJar} has been copied" if [ -f ${drillHome}/drill.tar.gz ]; then rm -f ${drillHome}/drill.tar.gz echo "${drillHome}/drill.tar.gz has been dropped" fi tempDir=drill-$(date +%s) mkdir /tmp/$tempDir [ $? == 0 ] && echo "Created temporary directory ${tempDir}" cd /tmp/$tempDir mkdir drill [ $? == 0 ] && echo "Created drill directory" cp -r ${drillHome}/* ./drill/ echo "${drillHome} copied to drill directory" tar -czf drill.tar.gz ./drill echo "Created new drill archive" cp drill.tar.gz ${drillHome}/ echo "drill.tar.gz copied to ${drillHome}" rm -rf /tmp/$tempDir echo -e "\033[0;32mDONE.\033[0m" - Issue the following command to run the
script:
sh recreate_archive.shWhen the script runs, a prompt appears:This operation will recreate drill.tar.gz in drill home directory. Continue?Reply with
yto allow the script to update the JAR files in Drill-on-YARN.You should see the following output:Drop old mapr jars from /opt/mapr/drill/drill-1.16.1/jars/3rdparty/ Copy new jars from /opt/mapr/lib/ /opt/mapr/lib/mapr-hbase-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/mapr-security-web-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprdb-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprdb-mapreduce-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprfs-6.2.0.0-mapr.jar has been copied Copy jersey jars from hadoop for timeline client /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-client-1.19.jar has been copied /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-core-1.19.jar has been copied /opt/mapr/drill/drill-1.16.1/drill.tar.gz has been dropped Created temporary directory drill-1656424210 Created drill directory /opt/mapr/drill/drill-1.16.1 copied to drill directory Created new drill archive drill.tar.gz copied to /opt/mapr/drill/drill-1.16.1 DONE. - Configure Drill to run under YARN and start Drill-on-YARN, as described in Configuring Drill to Run Under YARN.
- Due to Drill version changes (3-digit to 4-digit), you cannot upgrade from Drill in
EEP 7.0.0 (Drill 1.16.1) to Drill in
EEP 7.0.1 (Drill 1.16.1.5) or later. You
must perform a new installation of Drill. Alternatively, if you are running Drill on
CentOS or RHEL, you can issue the following command as a workaround to upgrade Drill:
rpm -Uv --<old-package> <path/to/packages>/*.rpmSee Upgrading Drill.
Limitations
- Older versions of Drill, such as Drill 1.10.0, supported the HBase plug-in, but Drill versions 1.11.0 through 1.16.0.x do not support queries on HBase tables.