Using the Command Line to View Logs for Completed Applications
Describes how to view logs from the CLI.
Procedure
-
Use the
maprcli job linklogs
command to create centralized logs for completed applications. For example, you can run the followingmaprcli job linklogs
command to create centralized logs forapplication_1434605941718_0001:
maprcli job linklogs -jobid application_1434605941718_0001 -todir /logsdir
The centralized log directory contains symbolic links that are organized by hostname and containerID. -
To determine where the logs are located, run the following command on the
directory that contains the symlinks to the log files for a specific container:
hadoop mfs -ls <todir>/<applicationID>/hosts/<hostName>/<containerID>
For example, if you specifiedlogsdir
as the directory, you might issue a command similar to the following example. The system then displays the location of the log files:hadoop mfs -ls /logsdir/application_1434605941718_0001/hosts/qa-node178.qa.lab/container_e02_1434605941718_0001_01_000003 Found 1 items lrwxrwxrwx U U U 3 root root 138 2015-06-18 05:50 0 /logsdir/application_1434605941718_0001/hosts/qa-node178.qa.lab/container_e02_1434605941718_0001_01_000003 -> ../../../../var/mapr/local/qa-node178.qa.lab/logs/yarn/userlogs/application_1434605941718_0001/container_e02_1434605941718_0001_01_000003 p 2068.40.262432 qa-node178.qa.lab:5660 qa-node175.qa.lab:5660
The link location appears after the arrow. -
To determine the types of log files that are available for this container and the
path to each available log file, run the following command:
hadoop fs -ls <link location>
For example:hadoop fs -ls ../../../../var/mapr/local/qa-node178.qa.lab/logs/yarn/userlogs/application_1434605941718_0001/container_e02_1434605941718_0001_01_000003 -rw-r----- 2 root root 2337 2015-06-18 05:48 ../../../../var/mapr/local/qa-node178.qa.lab/logs/yarn/userlogs/application_1434605941718_0001/container_e02_1434605941718_0001_01_000003/syslog
In this example, the path to the syslog is the only one that is displayed in the output. However, the stdout or stderr may also be available depending on what is generated by the application. -
Run one of the following options to view the contents of a log file: