Interface to Hive

Airflow provides an interface to Hive by using the Providers.

See Provider examples of Hive at <airflow_home>/build/env/lib/python3.9/site-packages/airflow/providers/ezmeral/hive/example_dags/.

Hooks

All hooks have the same API as Apache Hive Hooks.

EzHiveCliHook
Python path: airflow.providers.ezmeral.hive.hooks.ezhive.EzHiveCliHook
Description: Provides access to Hive using Hive client.
EzHiveMetastoreHook
Python path: airflow.providers.ezmeral.hive.hooks.ezhive.EzHiveMetastoreHook
Description: Provides access to Hive Metastore using HMSClient.
EzHiveServer2Hook
Python path: airflow.providers.ezmeral.hive.hooks.ezhive. EzHiveServer2Hook
Description: Provides access to HiveServer using PyHive.

Operators

EzHiveOperator
Python path: airflow.providers.ezmeral.hive.operators.ezhive.EzHiveOperator
Description: Executes HiveQL code or Hive script in a specific Hive database.
API: The same API as Apache Hive Operator.
EzHiveStatsCollectionOperator
Python path: airflow.providers.ezmeral.hive.operators.ezhive_stats.EzHiveStatsCollectionOperator
Description: Gathers partition statistics using a dynamically generated Presto query and inserts the statistics into a MySQL table.
API: The same API as Apache Hive Stats Operator.

Sensors

EzHivePartitionSensor
Python path: airflow.providers.ezmeral.hive.sensors.ezhive_partition.EzHivePartitionSensor
Description: Waits for a partition to show up in Hive.
API: The same API as Apache Hive Partition Sensor.
EzMetastorePartitionSensor
Python path: airflow.providers.ezmeral.hive.sensors.ezmetastore_partition.EzMetastorePartitionSensor
Description: An alternative to the HivePartitionSensor that talks directly to the MySQL database.
API: The same API as Apache Hive Metastore Partition Sensor.
EzNamedHivePartitionSensor
Python path: airflow.providers.ezmeral.hive.sensors.eznamed_hive_partition.EzNamedHivePartitionSensor
Description: Waits for a set of partitions to show up in Hive.
API: The same API as Apache Hive Named Partition Sensor.

Transfers

EzHiveToMySqlOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_hive_to_mysql.EzHiveToMySqlOperator
Description: Moves data from Hive to MySQL. Use this operator to move small amount of data as data first loads into the memory and then into the MySQL.
API: The same API as Apache HiveToMySqlOperator.
EzHiveToSambaOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_hive_to_samba.EzHiveToSambaOperator
Description: Moves data from Hive to Samba. Executes HiveQL code in a specific Hive database and loads the results of the query as a CSV to a Samba location.
API: The same API as Apache HiveToSambaOperator.
EzMsSqlToHiveOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_mssql_to_hive.EzMsSqlToHiveOperator
Description: Moves data from Microsoft SQL Server to Hive.
API: The same API as Apache MsSqlToHiveOperator.
EzMySqlToHiveOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_mysql_to_hive.EzMySqlToHiveOperator
Description: Moves data from MySQL to Hive.
API: The same API as Apache MySqlToHiveOperator.
EzS3ToHiveOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_s3_to_hive.EzS3ToHiveOperato
Description: Moves data from Amazon S3 to Hive.
API: The same API as Apache S3ToHiveOperator.
EzVerticaToHiveOperator
Python path: airflow.providers.ezmeral.hive.transfers.ez_vertica_to_hive.EzVerticaToHiveOperator
Description: Moves data from Vertica to Hive.
API: The same API as Apache VerticaToHiveOperator.