Spark Prerequisites

This topic describes the prerequisites to run Spark Applications on Kubernetes clusters in HPE Ezmeral Runtime Enterprise.

  • Install HPE Ezmeral Runtime Enterprise.
  • Apply the license to run Apache Spark Applications. See Product Licensing and License Tab.
  • Log in to HPE Ezmeral Runtime Enterprise as a Platform Administrator. To learn more about the users and their roles in HPE Ezmeral Runtime Enterprise, see Users and Roles.
  • Configure Kubernetes cluster with one Master host and at least two Worker hosts. To submit a simple Spark Application, a single worker host must have a combined total of at least 3 vCPU cores and at least 4GB of RAM (recommended). The amount of RAM required depends on the number of Spark jobs running concurrently.
  • (Optional) Configure AD/LDAP server to use the autoticketgenerator feature. See Spark Security.
  • HPE Ezmeral Runtime Enterprise only supports the Spark operator on Kubernetes clusters created within the deployment. Spark Operator is not supported on imported Kubernetes clusters.