Configuring HBase Persistent Storage Tables

Describes how to configure Drill to persist query profile data to a table that is unaffected by the TTL duration.

You can configure HBase persistent storage tables that do not have a TTL in the following versions of Drill:
  • Drill 1.16.0.500 (EEP-8.1.1)
  • Drill 1.20.2 (EEP 9.0.0 on Core 7.1.0) and later
By default, the Drill HBase persistent store persists the following information in one HBase table:
  • Query profile data
  • State information for storage plugins
  • State information for ALTER SYSTEM settings

Data persists in the HBase table for the duration of the configured TTL, after which the data is automatically removed to free up space.

You can configure the system to persist query profiles to another table that does or does not have a TTL configured. Query profiles persist in the table until the query profiles are deleted or the TTL expires.

The following section describes how to configure HBase persistent storage.

Persistent Storage Configurations

Configure HBase persistent storage through the sys.store.provider.hbase.table option in the /opt/mapr/drill/drill-<version>/conf/drill-override.conf or /opt/mapr/drill/drill-<version>/conf/drill-distrib.conf file.

The following sections provide HBase persistent storage configuration options.
IMPORTANT
For insecure Data Fabric clusters (maprsasl disabled), exclude the hbase.security.authentication property from the configuration.

Default Persistent Storage Configuration

The following example configuration shows the default persistent storage configuration:

drill.exec.sys.store.provider: {
    class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
    hbase: {
      table: "drill_store",
      config: {
      "hbase.zookeeper.quorum": "node1.com,node2.com,node3.com",
      "hbase.zookeeper.property.clientPort": "5181",
      "hbase.security.authentication": "maprsasl"
      }
    }
}

With this configuration, the HBase table drill_store stores query profile data and state information for storage plugins and ALTER SYSTEM settings until they expire based on the set TTL.

Query Profile Persistent Storage Configuration
If you want to store query profile data in a separate table, add the drill.exec.sys.store.provider.hbase.blob.table property to the persistent storage configuration, as shown in the following example:
drill.exec.sys.store.provider: {
    class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
    hbase: {
      blob.table: "drill_blob_store",
      table: "drill_store",
      config: {
      "hbase.zookeeper.quorum": "node1.com,node2.com,node3.com",
      "hbase.zookeeper.property.clientPort": "5181",
      "hbase.security.authentication": "maprsasl"
      }
    }
}
With this configuration, the Hbase table drill_blob_store stores query profiles only. The HBase table drill_store stores storage plugins and ALTER SYSTEM settings.
IMPORTANT
Drill cannot access query profile data stored in hbase.table (drill_store) after you configure blob.table (drill_blob_store).