Configuring HBase Persistent Storage Tables
Describes how to configure Drill to persist query profile data to a table that is unaffected by the TTL duration.
- Drill 1.16.0.500 (EEP-8.1.1)
- Drill 1.20.2 (EEP 9.0.0 on Core 7.1.0) and later
- Query profile data
- State information for storage plugins
- State information for ALTER SYSTEM settings
Data persists in the HBase table for the duration of the configured TTL, after which the data is automatically removed to free up space.
You can configure the system to persist query profiles to another table that does or does not have a TTL configured. Query profiles persist in the table until the query profiles are deleted or the TTL expires.
The following section describes how to configure HBase persistent storage.
Persistent Storage Configurations
Configure HBase persistent storage through the
sys.store.provider.hbase.table
option in the
/opt/mapr/drill/drill-<version>/conf/drill-override.conf
or
/opt/mapr/drill/drill-<version>/conf/drill-distrib.conf
file.
hbase.security.authentication
property from the
configuration.- Default Persistent Storage Configuration
-
The following example configuration shows the default persistent storage configuration:
drill.exec.sys.store.provider: { class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider", hbase: { table: "drill_store", config: { "hbase.zookeeper.quorum": "node1.com,node2.com,node3.com", "hbase.zookeeper.property.clientPort": "5181", "hbase.security.authentication": "maprsasl" } } }
With this configuration, the HBase table
drill_store
stores query profile data and state information for storage plugins and ALTER SYSTEM settings until they expire based on the set TTL. - Query Profile Persistent Storage Configuration
-
If you want to store query profile data in a separate table, add the
drill.exec.sys.store.provider.hbase.blob.table
property to the persistent storage configuration, as shown in the following example:drill.exec.sys.store.provider: { class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider", hbase: { blob.table: "drill_blob_store", table: "drill_store", config: { "hbase.zookeeper.quorum": "node1.com,node2.com,node3.com", "hbase.zookeeper.property.clientPort": "5181", "hbase.security.authentication": "maprsasl" } } }
With this configuration, the Hbase tabledrill_blob_store
stores query profiles only. The HBase tabledrill_store
stores storage plugins and ALTER SYSTEM settings.IMPORTANTDrill cannot access query profile data stored inhbase.table
(drill_store
) after you configureblob.table
(drill_blob_store
).