Submitting a Spark Wordcount Application

Provides an end-to-end example for creating and submitting a wordcount Spark Application in HPE Ezmeral Unified Analytics Software.

Prerequisites

  • Sign in to HPE Ezmeral Unified Analytics Software.
  • Download the wordcount.yaml file from the wordcount folder.

About this task

The wordcount Spark application counts the number of occurrences of each unique word in the wordcount.txt input file.

  1. In HPE Ezmeral Unified Analytics Software, use one the following methods to go to Spark Applications:
    • In the left navigation bar, click the Analytics icon and click Spark Applications.
    • In the left navigation bar, click the Tools & Frameworks icon. On the Analytics tab, click Open in the Spark Operator tile.
  2. Click Create Application on the Spark Applications screen. Navigate through each step within the Create Spark Application wizard:
    1. Application Details: Choose Upload YAML.

      YAML File

      Click Select File to upload the downloaded wordcount.yaml file from your local system. The fields in the wizard are populated with the information from YAML.

      Name:
      Update the application name as username-word-count.
      NOTE
      The application name must be unique.
      Description:
      Enter the application description. For example: This application counts words in a text file.
    2. Configure Spark Application: The fields in this wizard are populated with the information from YAML.

    3. Click Dependencies. The wordcount application does not require any additional dependencies.
    4. Click Driver Configuration. When boxes in this wizard are left blank, default values are set. The default values are as follows:
      • Number of Cores: 1

      • Core Limit: unlimited

      • Memory: 1g

    5. Click Executor Configuration. When boxes in this wizard are left blank, default values are set. The default values are as follows:
      • Number of Executors: 1

      • Number of Cores per Executor: 1

      • Core Limit per Executor: unlimited

      • Memory per Executor: 1g

    6. Click Schedule Application. If you want to schedule a Spark application, see Creating Spark Applications for details.
    7. Click Review. To view the application configuration, click Edit YAML. To apply the changes, click Save Changes. To cancel the changes, click Discard Changes. You can also click the pencil icon in each section to navigate to the specific step to change the application configuration.
  3. Click Create Spark Application on the bottom right of the Review step.

Results

The wordcount Spark application is created and submitted. You can view it on the Spark Applications screen.

You can also view the logs to check the output of the wordcount application. To see the logs, click the menu icon in the Actions column of the username-word-count application, and click View Logs.