Configuring Spark Thrift Server with MapR-SASL

Describes how to enable and start the Spark Thrift server on all nodes.

You can configure Spark Thrift server to use MapR-SASL for its communications with various components on a secure Data Fabric cluster. Minimal configuration is required.
NOTE
Starting in EEP 4.0, for secure clusters, you can skip the steps outlined in this section. For new installs done through the 6.0 Installer, the installer enables this configuration. For manual installs and upgrades, running configure.sh -R enables these settings.

To manually enable MapR-SASL authentication on a non-secure cluster or in versions earlier than EEP 4.0:

  1. Verify that the hive.server2.authentication property in hive-site.xml is set to the value, MAPRSASL. 
    <property>
     <name>hive.server2.authentication</name>
     <value>MAPRSASL</value>
</property>
  2. Restart Spark Thrift server to apply this change. sbin is in your Spark directory at /opt/mapr/spark/spark-<spark_version>/.
    IMPORTANT
    The Data Fabric administrative user (generally, the account named mapr) should start the Spark Thrift server. Then, process identifier (PID) files will be owned by this user, and impersonation support (where applicable) will function correctly.
    ./sbin/stop-thriftserver.sh
./sbin/start-thriftserver.sh

Bringing up the Spark Thrift server on every node

When you start and stop Warden after enabling Spark or after running configure.sh or after installing a patch, Spark starts only on one (1) node and not on all nodes. This happens because by default, the Warden configuration file for Spark has the value 1 instead of all. For example:

# grep services /opt/mapr/conf/conf.d/warden.spark-thriftserver.conf
services=spark-thriftserver:1:cldb
To fix this issue permanently:
  1. Modify /opt/mapr/spark/spark-2.4.0/warden/warden.spark-thriftserver.conf and change 1 to all: 
    # grep services /opt/mapr/spark/spark-2.4.0/warden/warden.spark-thriftserver.conf
            services=spark-thriftserver:all:cldb
  2. Run /opt/mapr/server/configure.sh -R.
The change is then propogated to the /opt/mapr/conf/conf.d/warden.spark-thriftserver.conf file.