Interface to Spark
Airflow provides an interface to Spark by using the Providers.
See Provider examples of Spark at <airflow_home>/build/env/lib/python3.9/site-packages/airflow/providers/ezmeral/spark/example_dags/.
Hooks
- EzSparkSubmitHook
- Python path: airflow.providers.ezmeral.spark.hooks.ezspark_submit
- EzSparkSqlHook
- Python path: airflow.providers.ezmeral.spark.hooks.ezspark_sql
- EzSparkJDBCHook
- Python path: airflow.providers.ezmeral.spark.hooks.ezspark_jdbc
Operators
- EzSparkSubmitOperator
- Python path: airflow.providers.ezmeral.spark.operators.ezspark_submit
- EzSparkSqlOperator
- Python path: airflow.providers.ezmeral.spark.operators.ezspark_sql
- EzSparkJDBCOperator
- Python path: airflow.providers.ezmeral.spark.operators.ezspark_jdbc