Iceberg Connection Parameters
List of Iceberg connection parameters, descriptions, default values, and supported data types.
The following sections list the required and optional Iceberg connection parameters.
IMPORTANT
- Currently, Iceberg cannot use MAPRSASL to authenticate to an HPE Ezmeral Data Fabric cluster when the catalog type is hadoop; however, you can use the hive catalog type to connect Iceberg to an HPE Ezmeral Data Fabric cluster.
Required Connection Parameters
The following table lists the required connection parameters:
Parameter | Description | Default Value | Data Type | Possible Values |
---|---|---|---|---|
Name | Provide a unique name for the Iceberg data source connection. | |||
Iceberg Catalog Type | The catalog type for Iceberg tables. | hive | STRING | possibleValues(hive, hadoop) |
Iceberg File Format | The storage file format for Iceberg tables. | PARQUET | STRING | possibleValues(PARQUET, ORC) |
Iceberg Compression Codec | The compression codec to use when writing files. The available values are NONE, SNAPPY, GZIP, LZ4, and ZSTD | GZIP | STRING | possibleValues(NONE, SNAPPY, GZIP, LZ4, ZSTD) |
Iceberg Catalog Cached Catalog Num | The number of Iceberg catalogs to cache, This property is required if the iceberg.catalog.type is hadoop | 10 | INTEGER | |
Iceberg Max Partitions Per Writer | The Maximum number of partitions handled per writer. | 100 | INTEGER | |
Iceberg Minimum Assigned Split Weight | A decimal value in the range (0, 1] used as a minimum for weights assigned to each split | 0.05 | DOUBLE | |
Hive Metastore | The type of Hive metastore to use | thrift | STRING | possibleValues(thrift, file, glue) |
Hive Metastore Catalog Dir | Hive file-based metastore catalog directory | STRING | ||
Hive Metastore Uri | Hive metastore URIs (comma separated). | STRING | ||
Hive Metastore Service Principal | The Kerberos principal of the Hive metastore service | STRING | ||
Hive Metastore Client Principal | The Kerberos principal that Presto will use when connecting to the Hive metastore service. | STRING | ||
Hive Metastore Client Keytab | Hive metastore client keytab location. | FILEPATH | ||
Hive Hdfs Presto Principal | The Kerberos principal that presto will use when connecting to HDFS | STRING | ||
Hive Hdfs Presto Keytab | HDFS client keytab location | FILEPATH | ||
Security Config File | Config file where rules are defined | STRING | ||
Security Refresh Period | Time after which rules will be refreshed from the file. | DURATION | Min(1ms) | |
Enable Local Snapshot Table | Enables local copy of database table for accelerated query performance | TRUE | BOOLEAN |
Optional Connection Parameters
The following table lists the optional connection parameters:
Parameter | Description | Default Value | Data Type | Possible Values |
---|---|---|---|---|
Iceberg Hadoop Config Resources | The path(s) for Hadoop configuration resources. | FILEPATH | ||
Iceberg Catalog Warehouse | The catalog warehouse root path for Iceberg tables. | STRING | ||
Hive Metastore User | Hive file-based metastore username for file access | presto | STRING | |
Hive Metastore Glue Region | AWS region of the Glue Catalog. | STRING | ||
Hive Metastore Glue Endpoint Url | Glue API endpoint URL | STRING | ||
Hive Metastore Glue Pin Client To Current Region | Should the Glue client be pinned to the current EC2 region | FALSE | BOOLEAN | |
Hive Metastore Glue Max Connections | Max number of concurrent connections to Glue | 5 | INTEGER | Min(1) |
Hive Metastore Glue Max Error Retries | Maximum number of error retries for the Glue client | 10 | INTEGER | Min(0) |
Hive Metastore Glue Default Warehouse Dir | Hive Glue metastore default warehouse directory | STRING | ||
Hive Metastore Glue Catalogid | The ID of the Glue Catalog in which the metadata database resides. | STRING | ||
Hive Metastore Glue Partitions Segments | Number of segments for partitioned Glue tables. | 5 | INTEGER | Min(1), Max(10) |
Hive Metastore Glue Get Partition Threads | Number of threads for parallel partition fetches from Glue. | 20 | INTEGER | Min(1) |
Hive Metastore Glue Iam Role | ARN of an IAM role to assume when connecting to the Glue Catalog. | STRING | ||
Hive Metastore Glue Aws Access Key | AWS access key to use to connect to the Glue Catalog. If specified along with hive.metastore.glue.aws-secret-key, this parameter takes precedence over hive.metastore.glue.iam-role. | STRING | ||
Hive Metastore Glue Aws Secret Key | AWS secret key to use to connect to the Glue Catalog. If specified along with hive.metastore.glue.aws-access-key, this parameter takes precedence over hive.metastore.glue.iam-role. | STRING | ||
Hive Metastore Username | Username for accessing the Hive metastore | STRING | ||
Hive Metastore Load Balancing Enabled | Enable load balancing between multiple Metastore instances | FALSE | BOOLEAN | |
Hive Insert Overwrite Immutable Partitions Enabled | When enabled, insertion query will overwrite existing partitions when partitions are immutable. This config only takes effect with hive.immutable-partitions set to true | FALSE | BOOLEAN | |
Hive Create Empty Bucket Files For Temporary Table | Create empty files when there is no data for temporary table buckets | FALSE | BOOLEAN | |
Hive Enable Parquet Batch Reader Verification | enable optimized parquet reader | FALSE | BOOLEAN | |
Hive Create Empty Bucket Files For Temporary Table | Create empty files when there is no data for temporary table buckets | FALSE | BOOLEAN | |
Hive Min Bucket Count To Not Ignore Table Bucketing | Ignore table bucketing when table bucket count is less than the value specified, otherwise, it is controlled by property hive.ignore-table-bucketing | 0 | INTEGER | |
Hive Partition Statistics Based Optimization Enabled | Enables partition statistics based optimization, including partition pruning and predicate stripping | FALSE | BOOLEAN | |
Hive Experimental Optimized Partition Update Serialization Enabled | Serialize PartitionUpdate objects using binary SMILE encoding and compress with the ZSTD compression | FALSE | BOOLEAN | |
Hive Materialized View Missing Partitions Threshold | Materialized views with missing partitions more than this threshold falls back to the base tables at read time | 100 | INTEGER | |
Hive S3select Pushdown Max Connections | The maximum number of client connections allowed for those operations from worker nodes | 500 | INTEGER | Min(1) |
Hive Temporary Staging Directory Enabled | Should use (if possible) temporary staging directory for write operations | TRUE | BOOLEAN | |
Hive Temporary Staging Directory Path | Location of temporary staging directory for write operations. Use ${USER} placeholder to use different location for each user. | /tmp/presto-${USER} | STRING | |
Hive Temporary Table Storage Format | The default file format used when creating new tables. | ORC | STRING | possibleValues(ORC, DWRF, PARQUET, AVRO, RCBINARY, RCTEXT, SEQUENCEFILE, JSON, TEXTFILE, CSV, PAGEFILE) |
Hive Temporary Table Compression Codec | The compression codec to use when writing files for temporary tables | SNAPPY | STRING | possibleValues(NONE, SNAPPY, LZ4, ZSTD, GZIP) |
Hive Use Pagefile For Hive Unsupported Type | Automatically switch to PAGEFILE format for materialized exchange when encountering unsupported types | TRUE | BOOLEAN | |
Hive Parquet Pushdown Filter Enabled | Enable complex filter pushdown for Parquet | FALSE | BOOLEAN | |
Hive Range Filters On Subscripts Enabled | enable pushdown of range filters on subscripts (a[2] = 5) into ORC column readers | FALSE | BOOLEAN | |
Hive Adaptive Filter Reordering Enabled | Enable adaptive filter reordering | TRUE | BOOLEAN | |
Hive Parquet Batch Read Optimization Enabled | Is Parquet batch read optimization enabled | FALSE | BOOLEAN | |
Hive Enable Parquet Dereference Pushdown | Is dereference pushdown expression pushdown into Parquet reader enabled | FALSE | BOOLEAN | |
Hive Max Metadata Updater Threads | Maximum number of metadata updated threads | 100 | INTEGER | Min(1) |
Hive Partial_aggregation_pushdown_enabled | enable partial aggregation pushdown | FALSE | BOOLEAN | |
Hive Manifest Verification Enabled | Enable verification of file names and sizes in manifest / partition parameters | FALSE | BOOLEAN | |
Hive Undo Metastore Operations Enabled | Enable undo metastore operations | TRUE | BOOLEAN | |
Hive Verbose Runtime Stats Enabled | Enable tracking all runtime stats. Note that this may affect query performance | FALSE | BOOLEAN | |
Hive Prefer Manifests To List Files | Prefer to fetch the list of file names and sizes from manifests rather than storage | FALSE | BOOLEAN | |
Hive Partition Lease Duration | Partition lease duration | 0.00s | DURATION | |
Hive Size Based Split Weights Enabled | Enable estimating split weights based on size in bytes | TRUE | BOOLEAN | |
Hive Minimum Assigned Split Weight | Minimum weight that a split can be assigned when size based split weights are enabled | 0.05 | DOUBLE | Min(0, inclusive=false), Max(1) |
Hive Use Record Page Source For Custom Split | Use record page source for custom split. By default, true. Used to query MOR tables in Hudi. | TRUE | BOOLEAN | |
Hive Split Loader Concurrency | Number of maximum concurrent threads per split source | 4 | INTEGER | Min(1) |
Hive Domain Compaction Threshold | Maximum ranges to allow in a tuple domain without compacting it | 100 | INTEGER | Min(1) |
Hive Max Concurrent File Renames | Maximum concurrent file renames | 20 | INTEGER | |
Hive Max Concurrent Zero Row File Creations | Maximum number of zero row file creations | 20 | INTEGER | Min(1) |
Hive Recursive Directories | Enable reading data from subdirectories of table or partition locations. If disabled, subdirectories are ignored. | FALSE | BOOLEAN | |
Hive User Defined Type Encoding Enabled | Enable user defined type | FALSE | BOOLEAN | |
Hive Loose Memory Accounting Enabled | When enabled relaxes memory accounting for queries violating memory limits to run that previously honored memory thresholds | FALSE | BOOLEAN | |
Hive Max Outstanding Splits Size | Maximum amount of memory allowed for split buffering for each table scan in a query, before the query is failed | 256MB | DATASIZE | Min(1MB) |
Hive Max Split Iterator Threads | Maximum number of iterator threads | 1000 | INTEGER | |
Hive Allow Corrupt Writes For Testing | Allow Hive connector to write data even when data will likely be corrupt | FALSE | BOOLEAN | |
Hive Create Empty Bucket Files | Should empty files be created for buckets that have no data? | TRUE | BOOLEAN | |
Hive Max Partitions Per Writers | Maximum number of partitions per writer | 100 | INTEGER | Min(1) |
Hive Write Validation Threads | Number of threads used for verifying data after a write | 16 | INTEGER | |
Hive Orc Tiny Stripe Threshold | ORC: Threshold below which an ORC stripe or file will read in its entirety | 8MB | DATASIZE | |
Hive Orc Lazy Read Small Ranges | ORC read small disk ranges lazily | TRUE | BOOLEAN | |
Hive Orc Bloom Filters Enabled | ORC: Enable bloom filters for predicate pushdown | FALSE | BOOLEAN | |
Hive Orc Default Bloom Filter Fpp | ORC Bloom filter false positive probability | 0.05 | DOUBLE | |
Hive Orc Optimized Writer Enabled | Experimental: ORC: Enable optimized writer | TRUE | BOOLEAN | |
Hive Orc Writer Validation Percentage | Percentage of ORC files to validate after write by re-reading the whole file | 0 | DOUBLE | Min(0.0), Max(100.0) |
Hive Orc Writer Validation Mode | Level of detail in ORC validation. Lower levels require more memory | BOTH | STRING | possibleValues(HASHED, DETAILED, BOTH) |
Hive Rcfile Optimized Writer Enabled | Experimental: RCFile: Enable optimized writer | TRUE | BOOLEAN | |
Hive Assume Canonical Partition Keys | Assume canonical parition keys? | FALSE | BOOLEAN | |
Hive Parquet Fail On Corrupted Statistics | Fail when scanning Parquet files with corrupted statistics | TRUE | BOOLEAN | |
Hive Parquet Max Read Block Size | Parquet: Maximum size of a block to read | 16MB | DATASIZE | |
Hive Optimize Mismatched Bucket Count | Enable optimization to avoid shuffle when bucket count is compatible but not the same | FALSE | BOOLEAN | |
Hive Zstd Jni Decompression Enabled | use JNI based zstd decompression for reading ORC files | FALSE | BOOLEAN | |
Hive File Status Cache Size | Hive file status cache size | 0 | LONG | |
Hive File Status Cache Expire Time | Hive file status cache : expiry time | 0.00s | DURATION | |
Hive Per Transaction Metastore Cache Maximum Size | Maximum number of metastore data objects in the Hive metastore cache per transaction | 1000 | INTEGER | Min(1) |
Hive Metastore Refresh Interval | Asynchronously refresh cached metastore data after access if it is older than this but is not yet expired, allowing subsequent accesses to see fresh data. | 0.00s | DURATION | |
Hive Metastore Cache Maximum Size | Maximum number of metastore data objects in the Hive metastore cache | 10000 | INTEGER | Min(1) |
Hive Metastore Refresh Max Threads | Maximum threads used to refresh cached metastore data | 100 | INTEGER | Min(1) |
Hive Partition Versioning Enabled | FALSE | BOOLEAN | ||
Hive Metastore Impersonation Enabled | Should Presto user be impersonated when communicating with Hive Metastore | FALSE | BOOLEAN | |
Hive Partition Cache Validation Percentage | Percentage of partition cache validation | 0 | DOUBLE | Min(0.0), Max(100.0) |
Hive Metastore Thrift Client Socks Proxy | metastore thrift client socks proxy | STRING | ||
Hive Metastore Timeout | Timeout for Hive metastore requests | 10.00s | DURATION | |
Hive Dfs Verify Checksum | Verify checksum for data consistency | TRUE | BOOLEAN | |
Hive Metastore Cache Ttl | Duration how long cached metastore data should be considered valid | 0.00s | DURATION | Min(0ms) |
Hive Metastore Recording Path | metastore recording path | STRING | ||
Hive Replay Metastore Recording | replay metastore recording | FALSE | BOOLEAN | |
Hive Metastore Recoding Duration | Metastore recording duration | 0.00m | DURATION | |
Hive Dfs Require Hadoop Native | hadoop native is required? | TRUE | BOOLEAN | |
Hive Metastore Cache Scope | Metastore cache scope | ALL | STRING | possibleValues(ALL, PARTITION) |
Hive Metastore Authentication Type | Hive metastore authentication type. | NONE | STRING | possibleValues(NONE, KERBEROS) |
Hive Hdfs Authentication Type | HDFS authentication type. | NONE | STRING | possibleValues(NONE, KERBEROS) |
Hive Hdfs Impersonation Enabled | Should Presto user be impersonated when communicating with HDFS | FALSE | BOOLEAN | |
Hive Hdfs Wire Encryption Enabled | Should be turned on when HDFS wire encryption is enabled | FALSE | BOOLEAN | |
Hive Skip Target Cleanup On Rollback | Skip deletion of target directories when a metastore operation fails and the write mode is DIRECT_TO_TARGET_NEW_DIRECTORY | FALSE | BOOLEAN | |
Hive Bucket Execution | Enable bucket-aware execution: only use a single worker per bucket | TRUE | BOOLEAN | |
Hive Bucket Function Type For Exchange | Hash function type for exchange | HIVE_COMPATIBLE | STRING | possibleValues(HIVE_COMPATIBLE, PRESTO_NATIVE) |
Hive Ignore Unreadable Partition | Ignore unreadable partitions and report as warnings instead of failing the query | FALSE | BOOLEAN | |
Hive Max Buckets For Grouped Execution | Maximum number of buckets to run with grouped execution | 1000000 | INTEGER | |
Hive Sorted Write To Temp Path Enabled | Enable writing temp files to temp path when writing to bucketed sorted tables | FALSE | BOOLEAN | |
Hive Sorted Write Temp Path Subdirectory Count | Number of directories per partition for temp files generated by writing sorted table | 10 | INTEGER | |
Hive Fs Cache Max Size | Hadoop FileSystem cache size | 1000 | INTEGER | |
Hive Non Managed Table Writes Enabled | Enable writes to non-managed (external) tables | FALSE | BOOLEAN | |
Hive Non Managed Table Creates Enabled | Enable non-managed (external) table creates | TRUE | BOOLEAN | |
Hive Table Statistics Enabled | Enable use of table statistics | TRUE | BOOLEAN | |
Hive Partition Statistics Sample Size | Specifies the number of partitions to analyze when computing table statistics. | 100 | INTEGER | Min(1) |
Hive Ignore Corrupted Statistics | Ignore corrupted statistics rather than failing | FALSE | BOOLEAN | |
Hive Collect Column Statistics On Write | Enables automatic column level statistics collection on write | FALSE | BOOLEAN | |
Hive S3select Pushdown Enabled | Enable query pushdown to AWS S3 Select service | FALSE | BOOLEAN | |
Hive Max Initial Splits | Max initial splits | 200 | INTEGER | |
Hive Max Initial Split Size | Max initial split size | null | DATASIZE | |
Hive Writer Sort Buffer Size | Write sort buffer size | 64MB | DATASIZE | Min(1MB), Max(1GB) |
Hive Node Selection Strategy | Node affinity selection strategy | NO_PREFERENCE | STRING | possibleValues(HARD_AFFINITY, SOFT_AFFINITY, NO_PREFERENCE) |
Hive Max Split Size | Max split size | 64MB | DATASIZE | |
Hive Max Partitions Per Scan | Maximum allowed partitions for a single table scan | 100000 | INTEGER | Min(1) |
Hive Max Outstanding Splits | Target number of buffered splits for each table scan in a query, before the scheduler tries to pause itself | 1000 | INTEGER | Min(1) |
Hive Metastore Partition Batch Size Min | hive metastore : min batch size for partitions | 10 | INTEGER | Min(1) |
Hive Metastore Partition Batch Size Max | hive metastore : max batch size for partitions | 100 | INTEGER | Min(1) |
Hive Config Resources | An optional comma-separated list of HDFS configuration files | [] | FILEPATH | |
Hive Dfs Ipc Ping Interval | The client will send ping when the interval is passed without receiving bytes | 10.00s | DURATION | |
Hive Dfs Timeout | DFS timeout | 60.00s | DURATION | Min(1ms) |
Hive Dfs Connect Timeout | DFS connection timeout | 500.00ms | DURATION | Min(1ms) |
Hive Dfs Connect Max Retries | DFS - max retries in case of connection issue | 5 | INTEGER | Min(0) |
Hive Storage Format | The default file format used when creating new tables. | ORC | STRING | possibleValues(ORC, DWRF, PARQUET, AVRO, RCBINARY, RCTEXT, SEQUENCEFILE, JSON, TEXTFILE, CSV, PAGEFILE) |
Hive Compression Codec | The compression codec to use when writing files | GZIP | STRING | possibleValues(NONE, SNAPPY, LZ4, ZSTD, GZIP) |
Hive Orc Compression Codec | The preferred compression codec to use when writing ORC and DWRF files | GZIP | STRING | possibleValues(NONE, SNAPPY, LZ4, ZSTD, GZIP) |
Hive Respect Table Format | Should new partitions be written using the existing table format or the default PrestoDB format? | TRUE | BOOLEAN | |
Hive Immutable Partitions | Can new data be inserted into existing partitions? | FALSE | BOOLEAN | |
Hive Max Open Sort Files | Maximum number of writer temporary files to read in one pass | 50 | INTEGER | Min(2), Max(1000) |
Hive Dfs Domain Socket Path | This is a path in the filesystem that allows the client and the DataNodes to communicate. | null | STRING | |
Hive S3 File System Type | s3 file system type | PRESTO | STRING | possibleValues(PRESTO, EMRFS, HADOOP_DEFAULT) |
Hive S3 Use Instance Credentials | Use the EC2 metadata service to retrieve API credentials (defaults to true). This works with IAM roles in EC2. | FALSE | BOOLEAN | |
Hive S3 Encryption Materials Provider | Use a custom encryption materials provider for S3 data encryption | STRING | ||
Hive S3 Multipart Min File Size | Minimum file size for an S3 multipart upload | 16MB | DATASIZE | |
Hive S3 Multipart Min Part Size | Minimum part size for an S3 multipart upload | 5MB | DATASIZE | |
Hive S3 Pin Client To Current Region | Pin S3 requests to the same region as the EC2 instance where Presto is running | FALSE | BOOLEAN | |
Hive S3 Upload Acl Type | Canned ACL type for S3 uploads | PRIVATE | STRING | possibleValues(AUTHENTICATED_READ, AWS_EXEC_READ, BUCKET_OWNER_FULL_CONTROL, BUCKET_OWNER_READ, LOG_DELIVERY_WRITE, PRIVATE, PUBLIC_READ, PUBLIC_READ_WRITE) |
Hive S3 User Agent Prefix | The user agent prefix to use for S3 calls | STRING | ||
Hive S3 Skip Glacier Objects | Ignore Glacier objects rather than failing the query. This will skip data that may be expected to be part of the table or partition | FALSE | BOOLEAN | |
Hive S3 Sse Enabled | Use S3 server-side encryption | FALSE | BOOLEAN | |
Hive S3 Sse Type | The type of key management for S3 server-side encryption | S3 | STRING | possibleValues(S3, KMS) |
Hive S3 Max Client Retries | Maximum number of read attempts to retry | 5 | INTEGER | Min(0) |
Hive S3 Max Error Retries | Maximum number of error retries, set on the S3 client | 10 | INTEGER | Min(0) |
Hive S3 Max Backoff Time | Use exponential backoff starting at 1 second up to this maximum value when communicating with S3 | 10.00m | DURATION | Min(1s) |
Hive S3 Max Retry Time | Maximum time to retry communicating with S3 | 10.00m | DURATION | Min(1ms) |
Hive S3 Connect Timeout | The default timeout for creating new connections. | 5.00s | DURATION | Min(1ms) |
Hive S3 Socket Timeout | The default timeout for reading from a connected socket. | 5.00s | DURATION | Min(1ms) |
Hive S3 Max Connections | Sets the maximum number of allowed open HTTP connections | 500 | INTEGER | Min(1) |
Hive S3 Staging Directory | Local staging directory for data written to S3. | STRING | ||
Hive S3 Aws Access Key | Default AWS access key to use. | STRING | ||
Hive S3 Aws Secret Key | Default AWS secret key to use. | STRING | ||
Hive S3 Endpoint | The S3 storage endpoint server. | STRING | ||
Hive S3 Storage Class | The S3 storage class to use when writing the data. | STANDARD | STRING | possibleValues(STANDARD, INTELLIGENT_TIERING) |
Hive S3 Signer Type | Specify a different signer type for S3-compatible storage | STRING | possibleValues(S3SignerType, AWS3SignerType, AWS4SignerType, AWSS3V4SignerType, CloudFrontSignerType, QueryStringSignerType) | |
Hive S3 Path Style Access | Use path-style access for all requests to the S3-compatible storage | FALSE | BOOLEAN | |
Hive S3 Iam Role | IAM role to assume | STRING | ||
Hive S3 Iam Role Session Name | AWS STS session name when IAM role to assume to access S3 buckets | presto-session | STRING | |
Hive S3 Ssl Enabled | Use HTTPS to communicate with the S3 API | TRUE | BOOLEAN | |
Hive S3 Kms Key Id | If set, use S3 client-side encryption and use the AWS KMS to store encryption keys and use the value of this property as the KMS Key ID for newly created objects | STRING | ||
Hive S3 Sse Kms Key Id | The KMS Key ID to use for S3 server-side encryption with KMS-managed keys | STRING | ||
Hive Gcs Json Key File Path | JSON key file used to access Google Cloud Storage | FILEPATH | ||
Hive Gcs Use Access Token | Use client-provided OAuth token to access Google Cloud Storage | FALSE | BOOLEAN | |
Hive Orc Use Column Names | Access ORC columns using names from the file | FALSE | BOOLEAN | |
Hive Orc Max Merge Distance | ORC: Maximum size of gap between two reads to merge into a single read | 1MB | DATASIZE | |
Hive Orc Max Buffer Size | ORC: Maximum size of a single read | 8MB | DATASIZE | |
Hive Orc Stream Buffer Size | ORC: Size of buffer for streaming reads | 8MB | DATASIZE | |
Hive Orc Max Read Block Size | ORC: Soft max size of Presto blocks produced by ORC reader | 16MB | DATASIZE | |
Hive Rcfile Writer Validate | Validate RCFile after write by re-reading the whole file | FALSE | BOOLEAN | |
Hive Text Max Line Length | Maximum line length for text files | 100MB | DATASIZE | Min(1B), Max(1GB) |
Hive Parquet Use Column Names | Access Parquet columns using names from the file | FALSE | BOOLEAN | |
Hive File Status Cache Tables | The tables that have file status cache enabled. Setting to '*' includes all tables | STRING | ||
Hive Skip Deletion For Alter | Skip deletion of old partition data when a partition is deleted and then inserted in the same transaction | FALSE | BOOLEAN | |
Hive Sorted Writing | Enable writing to bucketed sorted tables | TRUE | BOOLEAN | |
Hive Ignore Table Bucketing | Ignore table bucketing to enable reading from unbucketed partitions | FALSE | BOOLEAN | |
Hive Temporary Table Schema | Schema where to create temporary tables | default | STRING | |
Hive Pushdown Filter Enabled | Experimental: enable complex filter pushdown | FALSE | BOOLEAN | |
Hive Pagefile Writer Stripe Max Size | PAGEFILE: Max stripe size | 24MB | DATASIZE | |
Hive File_renaming_enabled | enable file renaming | FALSE | BOOLEAN | |
Hive Partial_aggregation_pushdown_for_variable_length_datatypes_enabled | enable partial aggregation pushdown for variable length datatypes | FALSE | BOOLEAN | |
Hive Time Zone | Sets the default time zone | STRING | ||
Hive Orc Writer Stripe Min Size | ORC: Min stripe size | 32MB | DATASIZE | |
Hive Orc Writer Stripe Max Size | ORC: Max stripe size | 64MB | DATASIZE | |
Hive Orc Writer Stripe Max Rows | ORC: Max stripe row count | 10000000 | INTEGER | |
Hive Orc Writer Row Group Max Rows | ORC : Max rows in row group | 10000 | INTEGER | |
Hive Orc Writer Dictionary Max Memory | ORC: Max dictionary memory | 16MB | DATASIZE | |
Hive Orc Writer String Statistics Limit | ORC: Maximum size of string statistics; drop if exceeding | 64B | DATASIZE | |
Hive Orc Writer Stream Layout Type | ORC: Stream layout type | BY_COLUMN_SIZE | STRING | possibleValues(BY_STREAM_SIZE, BY_COLUMN_SIZE) |
Hive Orc Writer Dwrf Stripe Cache Mode | Describes content of the DWRF stripe metadata cache. | INDEX_AND_FOOTER | STRING | possibleValues (NONE, INDEX, FOOTER, INDEX_AND_FOOTER) |
Hive Orc Writer Max Compression Buffer Size | ORC : Max compression buffer size | 256kB | DATASIZE | |
Hive Orc Writer Dwrf Stripe Cache Enabled | DWRF stripe cache enabled? | FALSE | BOOLEAN | |
Hive Orc Writer Dwrf Stripe Cache Max Size | DWRF stripe cache max size | 8MB | DATASIZE | |
Hive Parquet Optimized Writer Enabled | Parquet: Optimized writer enabled? | FALSE | BOOLEAN | |
Hive Parquet Writer Block Size | Parquet: Writer block size | 134217728B | DATASIZE | |
Hive Parquet Writer Page Size | Parquet: Writer page size | 1048576B | DATASIZE | |
Hive Allow Add Column | Allow Hive connector to add column | FALSE | BOOLEAN | |
Hive Allow Drop Column | Allow Hive connector to drop column | FALSE | BOOLEAN | |
Hive Allow Drop Table | Allow Hive connector to drop table | FALSE | BOOLEAN | |
Hive Allow Rename Table | Allow Hive connector to rename table | FALSE | BOOLEAN | |
Hive Allow Rename Column | Allow Hive connector to rename column | FALSE | BOOLEAN | |
Hive Security | The type of access control to use | legacy | STRING | possibleValues(legacy, file, read-only, sql-standard) |
Generic Cache Table Ttl | TTL for cache table expiry in minutes | 1440 | INTEGER |