Livy Overview

This topic provides the overview for Apache Livy on HPE Ezmeral Runtime Enterprise.

Livy is an HTTP server that allows you to launch Spark applications and submit code statements using the REST API. Livy can be launched as a tenant service in HPE Ezmeral Runtime Enterprise. See Apache Livy Documentation.

Figure 1. Using Apache Livy to Submit Spark Applications

Support for the Apache Livy Operator

HPE Ezmeral Runtime Enterprise integrates Apache Livy. You can only install one version of Livy in one tenant. You cannot simultaneously run Spark 2.x.x and Spark 3.x.x applications in the same tenant using Livy.

Learn more about supported Livy versions at Interoperability Matrix for Spark.

The new Livy features:
  1. Authentication and Impersonation
  2. Livy User Interface (UI)
  3. Spark History Server
  4. Livy Session Recovery
  5. Support for Hive Metastore
NOTE Livy does not support open-source Spark images on HPE Ezmeral Runtime Enterprise.