Before You Configure Drill to Run Under YARN
Provides the steps required to repackage the Drill-on-YARN archive.
During Drill-on-YARN installation, the system fails to upload the Drill archive because JNI
finds a mismatch for Java and native libraries. If you install and try to start Drill-on-YARN
using the mapr-drill-yarn
package, the system returns an error.
To prevent this issue from occurring, complete the following steps to repackage the
Drill-on-YARN archive:
- Install Drill-on-YARN, as described in Installing Drill to Run Under YARN, but do not configure or start Drill-on-YARN.
- Create a file named
recreate_archive.sh
with the following information:TIPYou can create and run this file in any location you choose.read -p "This operation will recreate drill.tar.gz in drill home directory. Continue? " -n 1 -r echo if [[ ! $REPLY =~ ^[Yy]$ ]] then exit 1 fi drillHome="/opt/mapr/drill/drill-$(cat /opt/mapr/drill/drillversion)" hadoopHome="/opt/mapr/hadoop/hadoop-$(cat /opt/mapr/hadoop/hadoopversion)" hbaseJar="$(ls /opt/mapr/lib/mapr-hbase-*-mapr.jar)" maprWebJar="$(ls /opt/mapr/lib/mapr-security*-mapr.jar)" maprdbJar="$(ls /opt/mapr/lib/maprdb-[0-9].[0-9].[0-9].[0-9]-mapr.jar)" mapredJar="$(ls /opt/mapr/lib/maprdb-mapreduce-*-mapr.jar)" maprfsJar="$(ls /opt/mapr/lib/maprfs-[0-9].[0-9].[0-9].[0-9]-mapr.jar)" jerseyClientJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-client-*.jar)" jerseyCoreJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-core-*.jar)" echo "Drop old mapr jars from ${drillHome}/jars/3rdparty/" rm -f ${drillHome}/jars/3rdparty/mapr-hbase* rm -f ${drillHome}/jars/3rdparty/mapr-security-web-* rm -f ${drillHome}/jars/3rdparty/maprdb-* rm -f ${drillHome}/jars/3rdparty/maprfs-* echo "Copy new jars from /opt/mapr/lib/" cp ${hbaseJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${hbaseJar} has been copied" cp ${maprWebJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprWebJar} has been copied" cp ${maprdbJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprdbJar} has been copied" cp ${mapredJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${mapredJar} has been copied" cp ${maprfsJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${maprfsJar} has been copied" echo "Copy jersey jars from hadoop for timeline client" cp ${jerseyClientJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${jerseyClientJar} has been copied" cp ${jerseyCoreJar} ${drillHome}/jars/3rdparty/ [ $? == 0 ] && echo "${jerseyCoreJar} has been copied" if [ -f ${drillHome}/drill.tar.gz ]; then rm -f ${drillHome}/drill.tar.gz echo "${drillHome}/drill.tar.gz has been dropped" fi tempDir=drill-$(date +%s) mkdir /tmp/$tempDir [ $? == 0 ] && echo "Created temporary directory ${tempDir}" cd /tmp/$tempDir mkdir drill [ $? == 0 ] && echo "Created drill directory" cp -r ${drillHome}/* ./drill/ echo "${drillHome} copied to drill directory" tar -czf drill.tar.gz ./drill echo "Created new drill archive" cp drill.tar.gz ${drillHome}/ echo "drill.tar.gz copied to ${drillHome}" rm -rf /tmp/$tempDir echo -e "\033[0;32mDONE.\033[0m"
- Issue the following command to run the
script:
sh recreate_archive.sh
When the script runs, a prompt appears:This operation will recreate drill.tar.gz in drill home directory. Continue?
Reply with
y
to allow the script to update the JAR files in Drill-on-YARN.You should see output similar to the following (versions may differ):Drop old mapr jars from /opt/mapr/drill/drill-1.16.1/jars/3rdparty/ Copy new jars from /opt/mapr/lib/ /opt/mapr/lib/mapr-hbase-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/mapr-security-web-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprdb-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprdb-mapreduce-6.2.0.0-mapr.jar has been copied /opt/mapr/lib/maprfs-6.2.0.0-mapr.jar has been copied Copy jersey jars from hadoop for timeline client /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-client-1.19.jar has been copied /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-core-1.19.jar has been copied /opt/mapr/drill/drill-1.16.1/drill.tar.gz has been dropped Created temporary directory drill-1656424210 Created drill directory /opt/mapr/drill/drill-1.16.1 copied to drill directory Created new drill archive drill.tar.gz copied to /opt/mapr/drill/drill-1.16.1 DONE.
- Continue to Step 1: Configure Drill.