Sample Spark Applications

This topic describes how to locate the sample Spark Applications to run it using Spark Operator.

Some ready-to-use sample Spark Applications are built into the container image. These applications are located at /opt/mapr/spark/spark-[version]/jars/spark-examples_[full-version].jar and should be referenced using the local schema. You may also build your own applications and make them available in /opt/mapr.

If the Spark Application is located elsewhere, then modify the mainApplicationFile field to point to that storage and interface.

The Spark Operator supports the http://, maprfs:///, dtap://, and S3:// interfaces.

Here are some use-cases for invoking a custom Spark job via the Spark Operator:

• Build an image, place the .jar in /apps/, and then reference it as follows:

  mainApplicationFile: "local:///apps/my_app.jar"

• If the application is stored and shared from HPE Ezmeral Data Fabric, then you can reference it by specifying maprfs:/// or S3:// in mainApplicationFIle. For example:

  maprfs:///path/to/my_app.jar
  S3://path/to/my_app.jar

• If the application is stored on the web server, then set mainApplicationFIle as follows:

  http://host:port/path/to/my_app.jar