Using Spark on YARN

This section includes information about using Spark on YARN in a data-fabric cluster.

For a simple test of your Spark installation, run the following command as the mapr user:

  • On Spark 2.0.1 or later:
    /opt/mapr/spark/spark-<version>/bin/run-example --master yarn --deploy-mode client SparkPi 10
  • On Spark 1.6.1:
    MASTER=yarn-client /opt/mapr/spark/spark-<version>/bin/run-example org.apache.spark.examples.SparkPi 10
NOTE These commands will fail if it is run as the root user.

For more information about running Spark applications, see the Apache Spark documentation.