Submitting a Spark Wordcount Application
Provides an end-to-end example for creating and submitting a wordcount Spark Application in HPE Ezmeral Unified Analytics Software.
Prerequisites
- Sign in to HPE Ezmeral Unified Analytics Software.
- Download the
wordcount.yaml
file from the wordcount folder.
About this task
The wordcount Spark application counts the number of occurrences of each unique word in the
wordcount.txt
input file.
- In HPE Ezmeral Unified Analytics Software, use
one the following methods to go to Spark Applications:
- In the left navigation bar, click the Analytics icon and click Spark Applications.
- In the left navigation bar, click the Tools & Frameworks icon. On the Analytics tab, click Open in the Spark Operator tile.
- Click Create Application on the Spark Applications screen.
Navigate through each step within the Create Spark Application
wizard:
-
Application Details: Choose Upload YAML.
- YAML File
-
Click Select File to upload the downloaded
wordcount.yaml
file from your local system. The fields in the wizard are populated with the information from YAML.
- Name:
- Update the application name as
username-word-count
.NOTEThe application name must be unique. - Description:
- Enter the application description. For example: This application counts words in a text file.
-
Configure Spark Application: The fields in this wizard are
populated with the information from YAML.
- Click Dependencies. The wordcount application does not require any additional dependencies.
- Click Driver Configuration. When boxes in this wizard are
left blank, default values are set. The default values are as follows:
-
Number of Cores: 1
-
Core Limit: unlimited
-
Memory: 1g
-
- Click Executor Configuration. When boxes in this wizard are
left blank, default values are set. The default values are as follows:
-
Number of Executors: 1
-
Number of Cores per Executor: 1
-
Core Limit per Executor: unlimited
-
Memory per Executor: 1g
-
- Click Schedule Application. If you want to schedule a Spark application, see Creating Spark Applications for details.
- Click Review. To view the application configuration, click Edit YAML. To apply the changes, click Save Changes. To cancel the changes, click Discard Changes. You can also click the pencil icon in each section to navigate to the specific step to change the application configuration.
-
- Click Create Spark Application on the bottom right of the Review step.
Results
You can also view the logs to check the output of the wordcount application. To see the
logs, click the menu icon in the Actions
column of the username-word-count
application, and click View
Logs.