Before You Configure Drill to Run Under YARN

Provides the steps required to repackage the Drill-on-YARN archive.

During Drill-on-YARN installation, the system fails to upload the Drill archive because JNI finds a mismatch for Java and native libraries. If you install and try to start Drill-on-YARN using the mapr-drill-yarn package, the system returns an error.

To prevent this issue from occurring, complete the following steps to repackage the Drill-on-YARN archive:
  1. Install Drill-on-YARN, as described in Installing Drill to Run Under YARN, but do not configure or start Drill-on-YARN.
  2. Create a file named recreate_archive.sh with the following information:
    TIP
    You can create and run this file in any location you choose.
    read -p "This operation will recreate drill.tar.gz in drill home directory. Continue? " -n 1 -r
    echo
    if [[ ! $REPLY =~ ^[Yy]$ ]]
    then
            exit 1
    fi
    
    drillHome="/opt/mapr/drill/drill-$(cat /opt/mapr/drill/drillversion)"
    hadoopHome="/opt/mapr/hadoop/hadoop-$(cat /opt/mapr/hadoop/hadoopversion)"
    
    hbaseJar="$(ls /opt/mapr/lib/mapr-hbase-*-mapr.jar)"
    maprWebJar="$(ls /opt/mapr/lib/mapr-security*-mapr.jar)"
    maprdbJar="$(ls /opt/mapr/lib/maprdb-[0-9].[0-9].[0-9].[0-9]-mapr.jar)"
    mapredJar="$(ls /opt/mapr/lib/maprdb-mapreduce-*-mapr.jar)"
    maprfsJar="$(ls /opt/mapr/lib/maprfs-[0-9].[0-9].[0-9].[0-9]-mapr.jar)"
    jerseyClientJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-client-*.jar)"
    jerseyCoreJar="$(ls ${hadoopHome}/share/hadoop/yarn/lib/jersey-core-*.jar)"
    
    echo "Drop old mapr jars from ${drillHome}/jars/3rdparty/"
    rm -f ${drillHome}/jars/3rdparty/mapr-hbase*
    rm -f ${drillHome}/jars/3rdparty/mapr-security-web-*
    rm -f ${drillHome}/jars/3rdparty/maprdb-*
    rm -f ${drillHome}/jars/3rdparty/maprfs-*
    
    echo "Copy new jars from /opt/mapr/lib/"
    cp ${hbaseJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${hbaseJar} has been copied"
    cp ${maprWebJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${maprWebJar} has been copied"
    cp ${maprdbJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${maprdbJar} has been copied"
    cp ${mapredJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${mapredJar} has been copied"
    cp ${maprfsJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${maprfsJar} has been copied"
    
    echo "Copy jersey jars from hadoop for timeline client"
    cp ${jerseyClientJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${jerseyClientJar} has been copied"
    cp ${jerseyCoreJar} ${drillHome}/jars/3rdparty/
    [ $? == 0 ] && echo "${jerseyCoreJar} has been copied"
    
    if [ -f ${drillHome}/drill.tar.gz ]; then
    	rm -f ${drillHome}/drill.tar.gz
    	echo "${drillHome}/drill.tar.gz has been dropped"
    fi
    
    tempDir=drill-$(date +%s)
    
    mkdir /tmp/$tempDir
    [ $? == 0 ] && echo "Created temporary directory ${tempDir}"
    
    cd /tmp/$tempDir
    
    mkdir drill
    [ $? == 0 ] && echo "Created drill directory"
    
    cp -r ${drillHome}/* ./drill/
    echo "${drillHome} copied to drill directory"
    
    tar -czf drill.tar.gz ./drill
    echo "Created new drill archive"
    
    cp drill.tar.gz ${drillHome}/
    echo "drill.tar.gz copied to ${drillHome}"
    
    rm -rf /tmp/$tempDir
    echo -e "\033[0;32mDONE.\033[0m"
  3. Issue the following command to run the script:
    sh recreate_archive.sh
    When the script runs, a prompt appears:
    This operation will recreate drill.tar.gz in drill home directory. Continue?

    Reply with y to allow the script to update the JAR files in Drill-on-YARN.

    You should see output similar to the following (versions may differ):
    Drop old mapr jars from /opt/mapr/drill/drill-1.16.1/jars/3rdparty/
    Copy new jars from /opt/mapr/lib/
    /opt/mapr/lib/mapr-hbase-6.2.0.0-mapr.jar has been copied
    /opt/mapr/lib/mapr-security-web-6.2.0.0-mapr.jar has been copied
    /opt/mapr/lib/maprdb-6.2.0.0-mapr.jar has been copied
    /opt/mapr/lib/maprdb-mapreduce-6.2.0.0-mapr.jar has been copied
    /opt/mapr/lib/maprfs-6.2.0.0-mapr.jar has been copied
    Copy jersey jars from hadoop for timeline client
    /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-client-1.19.jar has been copied
    /opt/mapr/hadoop/hadoop-2.7.6/share/hadoop/yarn/lib/jersey-core-1.19.jar has been copied
    /opt/mapr/drill/drill-1.16.1/drill.tar.gz has been dropped
    Created temporary directory drill-1656424210
    Created drill directory
    /opt/mapr/drill/drill-1.16.1 copied to drill directory
    Created new drill archive
    drill.tar.gz copied to /opt/mapr/drill/drill-1.16.1
    DONE.
  4. Continue to Step 1: Configure Drill.