Logging Options
In a HPE Data Fabric cluster, the logging option that you configure defines how the logs are stored and accessed:
- Centralized logging. The logs are written to local volumes on the file system.
- YARN log aggregation. The logs are written to the local file system, and then the container logs from each node are aggregated and stored on the file system.
-
Local logging. The log files for each job or application are written to the local
file system. This method is the default behavior for MapReduce version 2 (MRv2)
applications and non-MapReduce applications.
- For MRv2 or other applications that run on YARN, the logs are written to the
following directory on the local file system:
/opt/mapr/hadoop/hadoop-2.x.x/logs/userlogs/
- For MRv2 or other applications that run on YARN, the logs are written to the
following directory on the local file system:
The logging options that you can choose from are determined by the type of jobs or applications that you run:
| Type of Job or Application | Available Logging Options |
|---|---|
| MRv2 |
|
|
YARN applications (non-MapReduce) |
|
If you enable centralized logging for MRv2, the MapReduce applications uses centralized logging, while the other YARN applications in the cluster use local logging.
NOTE
Select a logging option that stores the logs on the file system for the following
reasons: - Prevent job or application failures due to a lack of space on the local file system for logs.
-
Prevent the loss or inaccessibility of logs due to node failure. Logs stored in a local volume are two-way replicated.