Managing Spark Applications

This topic describes how to view and manage Spark applications using HPE Ezmeral Runtime Enterprise new UI.

About this task

View and manage the status of all the Spark applications and scheduled Spark applications.

Procedure

Click the Spark icon on the left navigation bar or click View All in HPE Ezmeral Runtime Enterprise new UI home page.
To view actions you can perform on Applications and Scheduled Applications tab, click the menu icon in the Actions column.

View Details:

To view the details of an application, including CPU, memory usage, and events and logs of the pods, select View Details.

To access the Spark History Server and view and monitor the applications, click Spark Web UI in the top right of the Application Detail screen.
NOTE To view the Spark Web UI link, ensure you have installed the Spark History Server in your tenant.

View YAML:

To view the YAML file and see the configuration details, select View YAML.

Edit YAML:

To open an editor to change the application configuration using a YAML in the GUI, click Edit YAML. You can use the editor to add the extra configuration options not available through application wizard. To apply the changes, click Update Application. To cancel the changes, click Discard Changes.

View Logs:

To view the Spark driver pod logs, select View Logs.

Edit:

To change application configurations and resubmit the application, select Edit.

You can use the editor to add the extra configuration options not available through application wizard.

You can update all the application parameters except name, cluster, tenant, Spark version, and type using Edit. Use Clone to update the parameters and create a new application.

You can update the schedule of the scheduled Spark application by using the Edit.

To open an editor to change the application configuration using YAML in the GUI, click Edit YAML in Review step. To apply the changes, click Save Changes. To cancel the changes, click Discard Changes.

To schedule the Spark application, select Schedule or select Clone.

NOTE Using Edit to resubmit an application will remove pods and logs of the previous application run.

Clone:

To create a new Spark application with the similar configuration as an existing Spark application, select Clone. You can update any application parameters and submit it as a new application.

NOTE
If you enter the same name as the current Spark application in the same tenant and configure the scheduling details on Schedule Application step, it will create a new scheduled Spark application.

Submitting an application in the same tenant with same name and application type as an existing application will remove pods and logs of the previous application run.

Schedule:

To schedule the application, click Schedule. You can view this application in the Scheduled Applications tab. To learn more about the Schedule Application step, see Creating Spark Applications.

Suspend:

To stop the application from running at its scheduled time, select Suspend from the Actions menu in the Scheduled Applications tab.

Resume:

To restart the schedule of the suspended applications, select Resume from Actions menu on Scheduled Applications tab.

Delete:

To delete the Spark application, select Delete.
Delete multiple Spark applications at once:
1. To select multiple applications, click the check box besides Application Name in the table.
2. Click Delete on the top right pane of the table.
To display the Spark applications according to the clusters, tenants, and status, click Filter icon.
To select the columns to display on your applications table, click Columns icon.