Drill 1.16.1.400-2201 (EEP 8.1.0) Release Notes

The notes below relate specifically to the HPE Ezmeral Data Fabric Distribution for Apache Drill. You may also be interested in the Apache Drill homepage and the Apache Drill release notes:

Version 1.16.1.400
Release Date January 2022
HPE Version Interoperability See Component Versions for Released EEPs and EEP Components and OS Support.
Package Names See Package Names for Ecosystem Packs (EEPs).

New in This Release

Drill 1.16.1.400-2201 introduces the following enhancements or HPE platform-specific behavior changes:

  • None

Fixes

This HPE release includes the following fixes on the base release:
Commit Date (YYYY-MM-DD) Comment
33a7d24ea2 2022-01-10 MD-6251: Secondary index error when reading from mapr-db json (#575)
8365703194 2021-12-30 MD-5516: Add https support for Drill-on-YARN (#566)
d492882d29 2021-12-30 MD-6191: Allow partition pruning with dynamic CURRENT_DATE operator.
a6ab48aede 2021-12-20 MD-6196: Medial CVE fixes (jquery, bootstrap, data tables)
5ee9b79204 2021-12-20 MD-6196: Medial CVE fixes (junit, bcpkix-jdk15on)
4f92206395 2021-12-08 MD-6202: Excluding transitive netty dependency from zookeeper
aeb91d399c 2021-12-07 MD-6161: fix skipping of types written not in uppercase (#565)
cbbceb728b 2021-11-26 DRILL-8009: DrillConnectionImpl#isValid() doesn't correspond JDBC API
a5b79c9e67 2021-11-21 MD-6196: Fix the CVEs with the high severity
bddeab869d 2021-11-16 DRILL-7586: Fix loading incorrect version of commons-lang3
ec1051c0ec 2021-11-09 MD-6182: Fixed problem with starting drill on the fips cluster (#564)

Known Issues

  • During the Drill-on-YARN installation, the system fails to upload the Drill archive because the /user/drill directory does not exist. If you install and try to start Drill-on-YARN (version 1.16.1.400 in EEP-8.1.0 on Core 6.2.0 or 7.0.0) using the mapr-drill-yarn package, the system returns the following messages:
    /opt/mapr/drill/drill-1.16.1/bin/drill-on-yarn.sh start

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by javassist.util.proxy.SecurityActions (file:/opt/mapr/drill/drill-1.16.1/jars/3rdparty/javassist-3.24.0-GA.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of javassist.util.proxy.SecurityActions
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Connecting to DFS... Connected.
2022-06-20 02:28:59,1379 ERROR JniCommon fc/jni_MapRClient.cc:816 Thread: 1438881 Mismatch found for java and native libraries java build version 6.2.0.0.20200915234957.GA, native build version 6.2.0.0.20200909000740.GA java patch version $Id: mapr-version: 6.2.0.0.20200915234957.GA ccd6754df227770285, native patch version $Id: mapr-version: 6.2.0.0.20200909000740.GA a40a31acab7f5e88e1
Uploading /opt/mapr/drill/drill-1.16.1/drill.tar.gz to /user/drill/drill.tar.gz ... Failed.
Failed to upload Drill archive
  Caused by: Failed to create DFS directory: /user/drill
  Caused by: Could not create FileClient err: 0
  Caused by: Could not create FileClient err: 0
    To resolve this issue, complete the following steps:
    1. Install Drill-on-YARN, as described in Installing Drill to Run Under YARN, but do not configure or start Drill-on-YARN.
    2. Create a file named recreate_archive.sh with the following information:
      TIP
      You can create and run this file in any location you choose.
      read -p "This operation will recreate drill.tar.gz in drill home directory. Continue? " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]
then
        exit 1
fi

drillHome="/opt/mapr/drill/drill-$(cat /opt/mapr/drill/drillversion)"
hadoopHome="/opt/mapr/hadoop/hadoop-$(cat /opt/mapr/hadoop/hadoopversion)"

hbaseJar="$(ls /opt/mapr/lib/mapr-hbase-*-mapr.jar)"
maprWebJar="$(ls /opt/mapr/lib/mapr-security*-mapr.jar)"
maprdbJar="$(ls /opt/mapr/lib/maprdb-[0-9].[0-9].[0-9].[0-9]-mapr.jar)"
mapredJar="$(ls /opt/mapr/lib/maprdb-mapreduce-*-mapr.jar)"
maprfsJar="$(ls /opt/mapr/lib/maprfs-[0-9].[0-9].[0-9].[0-9]-mapr.jar)"
jerseyClientJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-client-*.jar)"
jerseyCoreJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-core-*.jar)"

echo "Drop old mapr jars from ${drillHome}/jars/3rdparty/"
rm -f ${drillHome}/jars/3rdparty/mapr-hbase*
rm -f ${drillHome}/jars/3rdparty/mapr-security-web-*
rm -f ${drillHome}/jars/3rdparty/maprdb-*
rm -f ${drillHome}/jars/3rdparty/maprfs-*

echo "Copy new jars from /opt/mapr/lib/"
cp ${hbaseJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${hbaseJar} has been copied"
cp ${maprWebJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${maprWebJar} has been copied"
cp ${maprdbJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${maprdbJar} has been copied"
cp ${mapredJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${mapredJar} has been copied"
cp ${maprfsJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${maprfsJar} has been copied"

echo "Copy jersey jars from hadoop for timeline client"
cp ${jerseyClientJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${jerseyClientJar} has been copied"
cp ${jerseyCoreJar} ${drillHome}/jars/3rdparty/
[ $? == 0 ] && echo "${jerseyCoreJar} has been copied"

if [ -f ${drillHome}/drill.tar.gz ]; then
	rm -f ${drillHome}/drill.tar.gz
	echo "${drillHome}/drill.tar.gz has been dropped"
fi

tempDir=drill-$(date +%s)

mkdir /tmp/$tempDir
[ $? == 0 ] && echo "Created temporary directory ${tempDir}"

cd /tmp/$tempDir

mkdir drill
[ $? == 0 ] && echo "Created drill directory"

cp -r ${drillHome}/* ./drill/
echo "${drillHome} copied to drill directory"

tar -czf drill.tar.gz ./drill
echo "Created new drill archive"

cp drill.tar.gz ${drillHome}/
echo "drill.tar.gz copied to ${drillHome}"

rm -rf /tmp/$tempDir
echo -e "\033[0;32mDONE.\033[0m"
    3. Issue the following command to run the script:
      sh recreate_archive.sh
      When the script runs, a prompt appears:
      This operation will recreate drill.tar.gz in drill home directory. Continue?

      Reply with y to allow the script to update the JAR files in Drill-on-YARN.

      You should see the following output:
      Drop old mapr jars from /opt/mapr/drill/drill-1.16.1/jars/3rdparty/
Copy new jars from /opt/mapr/lib/
/opt/mapr/lib/mapr-hbase-6.2.0.0-mapr.jar has been copied
/opt/mapr/lib/mapr-security-web-6.2.0.0-mapr.jar has been copied
/opt/mapr/lib/maprdb-6.2.0.0-mapr.jar has been copied
/opt/mapr/lib/maprdb-mapreduce-6.2.0.0-mapr.jar has been copied
/opt/mapr/lib/maprfs-6.2.0.0-mapr.jar has been copied
Copy jersey jars from hadoop for timeline client
/opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-client-1.19.jar has been copied
/opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-core-1.19.jar has been copied
/opt/mapr/drill/drill-1.16.1/drill.tar.gz has been dropped
Created temporary directory drill-1656424210
Created drill directory
/opt/mapr/drill/drill-1.16.1 copied to drill directory
Created new drill archive
drill.tar.gz copied to /opt/mapr/drill/drill-1.16.1
DONE.
    4. Configure Drill to run under YARN and start Drill-on-YARN, as described in Configuring Drill to Run Under YARN.
  • Due to Drill version changes (3-digit to 4-digit), you cannot upgrade from Drill in EEP 7.0.0 (Drill 1.16.1) to Drill in EEP 7.0.1 (Drill 1.16.1.5) or later. You must perform a new installation of Drill. Alternatively, if you are running Drill on CentOS or RHEL, you can issue the following command as a workaround to upgrade Drill: 
    rpm -Uv --<old-package> <path/to/packages>/*.rpm

    See Upgrading Drill.

Limitations

  • Older versions of Drill, such as Drill 1.10.0, supported the HBase plug-in, but Drill versions 1.11.0 through 1.16.0.x do not support queries on HBase tables.