Submitting Spark Applications Using spark-submit

To install the Helm chart for spark-client for Apache Spark 3.x.x in an existing tenant namespace, run:

helm install <spark-client-name> ./<path-to-spark-client-chart> -n <tenant-namespace>

To install the Helm chart for spark-client for Apache Spark 3.x.x in a new tenant namespace, run:

helm install <spark-client-name> ./<path-to-spark-client-chart> -n <tenant-namespace> --create-namespace

To install the Helm chart for spark-client for Apache Spark 2.x.x in an existing namespace, run

helm install <spark-client-name> ./<path-to-spark-client-chart> -n <tenant-namespace> /
--set image.imageName=<spark-client-image-name> /
--set image.tag=<spark-client-imagetag>

See Spark Images.

When your spark-client pod is up and running, perform the following steps:

1. Determine the gateway FQDN and port to SSH into the client pod. Run:

   kubectl describe svc spark-client-spark-client-chart -n t1 | grep gateway\

2. SSH into that FQDN and port, and authenticate as a tenant member AD user.
3. When you are in the spark-client pod, run maprlogin password and enter the AD password again.
4. Locate the Spark binary of your choice under /opt/mapr/spark/spark-<version>/bin. See Interoperability Matrix for Spark.
5. Submit the Spark applications. See Launching Applications with spark-submit.

Run the following command to delete the spark-client Helm chart.

helm delete <spark-client-name> -n <tenant-namespace>

For example:

helm delete spark-client -n sampletenant