Security with Spark on YARN

Starting in the EEP 4.0 release, for secure clusters, you no longer need to manually configure your cluster to enable Spark security features. Using the Data Fabric installer for new installations or running configure.sh -R for manual installs and upgrades automatically enables security features on secure clusters. See Spark configure.sh for details, including instructions on how to avoid enabling security features on secure clusters.

When running Spark applications on a secure cluster, you must pass the -Dmapr_sec_enabled flag to Spark. For secure clusters, this flag is set in spark-env.sh. For situations where your Spark application does not invoke this script, e.g., a Spark web service, you must manually pass the flag.