Interface to Livy

Airflow provides an interface to Livy by using the Providers.

See Provider examples of Livy at <airflow_home>/build/env/lib/python3.9/site-packages/airflow/providers/ezmeral/livy/example_dags/.

Hooks

EzLivyHook
Python path: airflow.providers.ezmeral.livy.hooks.ezlivy.EzLivyHook
Description: Apache Livy hook on HPE Ezmeral Data Fabric through the REST API.
API: The same API as Apache LivyHook.

Operators

EzLivyOperator
Python path: airflow.providers.ezmeral.livy.operators.ezlivy.EzLivyOperator
Description: Wraps the Livy batch REST API, enabling to submit a Spark application to the underlying cluster.
API: The same API as Apache LivyOperator.

Sensors

EzLivySensor
Python path: airflow.providers.ezmeral.livy.sensors.ezlivy.EzLivySensor
Description: Monitors Livy sessions for termination.
API: The same API as Apache LivySensor.