Interface to the File System

Airflow provides an interface to the file system by using the Providers.

See Provider examples of the file system at <airflow_home>/build/env/lib/python3.9/site-packages/airflow/providers/ezmeral/fs/example_dag/.

Hooks

EZFSHook
Python path: airflow.providers.ezmeral.fs.hooks.ezfs.EZFSHook
Description: Interacts with file system.

API::class airflow.providers.ezmeral.fs.hooks.ezfs.EZFSHook(classpath: list = '/opt/mapr/lib/*')

The classpath parameter provides a path for Java and binaries libraries to access the file system Java client.

WebEZFSHook
Python path: airflow.providers.ezmeral.fs.hooks.web_ezfs.WebEZFSHook
Description: Interacts with file system through HttpFS.
API: The same API as Apache Web HDFS.

Sensors

EzfsSensor
Python path: airflow.providers.ezmeral.fs.sensors.ezfs.EzfsSensor
Description: Waits for a file or folder to show up in the file system.

API: The same API as Apache HDFS Sensor.

EzfsRegexSensor
Python path: airflow.providers.ezmeral.fs.sensors.ezfs.EzfsRegexSensor
Description:Waits for matching files by matching on regex.
API: The same API as Apache HDFS Regex sensor.
EzfsFolderSensor
Python path: airflow.providers.ezmeral.fs.sensors.ezfs.EzfsFolderSensor
Description:Waits for a non-empty directory.
API: The same API as Apache HDFSFolder sensor.
WebEzfsSensor
Python path: airflow.providers.ezmeral.fs.sensors.web_ezfs.WebEzfsSensor
Description: Waits for a file or folder to show up in the file system.
API: The same API as Apache WebHDFS sensor.