Hive Logging (Hive 2.3 and Later)

For Hive 2.3 and later starting with EEP 6.3.0, this topic describes the folder structure of the Hive logs and includes details about the log-file contents and how log files are installed in multinode installations.

Hive Log Folder Structure

Table 1 shows the Hive log folder structure:
Table 1. Hive Logs
Folder or File Description 
${HIVE_HOME}/log
 Root folder for all Hive logs 
hive-${ADMIN_USER}-hiveserver2-${HOSTNAME}.out
hive-${ADMIN_USER}-metastore-${HOSTNAME}.out
init_derby_db_${TIMESTAMP}.log
 Each service has a separate file for logging 
${HIVE_HOME}/log/${ADMIN_USER}
 Root folder for admin cluster logs 
${ADMIN_USER}-hiveserver2-${HOSTNAME}.log
${ADMIN_USER}-metastore-${HOSTNAME}.log
${ADMIN_USER}-cli-${HOSTNAME}.log
 Cluster admin log files 
${HIVE_HOME}/log/${ADMIN_USER}/webhcat/
 Root folder for webHcat logs 
webhcat.log
webhcat-console.log
webhcat-console-error.log
 WebHcat log files 
${HIVE_HOME}/log/${OTHER_USER}
 Root folder for a user other than the admin user 
${OTHER_USER}-cli-${HOSTNAME}.log
 CLI log for a user other than the admin user
In Table 1:
This element Represents
${HIVE_HOME} The Hive home folder, which is usually /opt/mapr/hive/hive.
${ADMIN_USER} The admin user of a cluster that runs HiveServer2 and HiveMetastore daemons. Usually, this is the mapr user.
${HOSTNAME} The name of the host where the daemon runs.
${TIMESTAMP} The date and time of log creation.
${OTHER_USER} A user other than the admin user.
${OTHER_USER}-cli-${HOSTNAME}.log The log file that is created when ${OTHER_USER} launches the Hive CLI.

Content of Log Files

Table 2 shows the content of the log files:
Table 2. Log File Contents
File Name Description
hive-${ADMIN_USER}-hiveserver2-${HOSTNAME}.out Contains information about when the HiveServer2 daemon was started and the PID of the file.
hive-${ADMIN_USER}-metastore-${HOSTNAME}.out Contains information about when the HiveMetastore daemon was started and the PID of the file.
${ADMIN_USER}-hiveserver2-${HOSTNAME}.log Contains information from the HiveServer2 daemons. This file also contains the job progress.
${ADMIN_USER}-metastore-${HOSTNAME}.log Contains information from the HiveMetastore daemons.
${ADMIN_USER}-cli-${HOSTNAME}.log Created when a user runs the Hive CLI over the ${ADMIN_USER}. This file contains the job progress.
${OTHER_USER}-cli-${HOSTNAME}.log Created when a user runs the Hive CLI over the ${OTHER_USER}. This file also contains the job progress.
init_derby_db_${TIMESTAMP}.log Created if and only if Hive was configured for Apache Derby through cluster installation.

Log Files in a Multinode Hive Installation

Table 3 shows a Hive multinode installation (that is, Hive packages installed on different nodes):
Table 3. Cluster Configuration
Hive Metastore HiveServer2 HiveWebHCat
node1 ✓
node2 ✓
node3 ✓
See Table 4 for the log configurations:
Table 4. Log Configuration on Hive Multi-Node Installation
File or Folder Name node1 node2 node3
hive-${ADMIN_USER}-metastore- ${HOSTNAME}.out ✓
${ADMIN_USER}-metastore- ${HOSTNAME}.log ✓
hive-${ADMIN_USER}-hiveserver2- ${HOSTNAME}.out ✓
${ADMIN_USER}-hiveserver2- ${HOSTNAME}.log ✓
/webhcat/ ✓
webhcat.log ✓
webhcat-console.log ✓
webhcat-console-error.log ✓
/${ADMIN_USER}/ ✓ ✓ ✓